We Keep Coding

Django: IOError [Errno 2] No such file or directory while Reading JSON file path using python

I am getting No such file or directory error , while trying to read JSON file from my local django app.
Please Note i tried most of the method mentioned on Stackoverflow. so think before saying duplicate...
here is my code,
Try 1:
json_data = os.path.join(BASE_DIR, STATIC_URL, "utils/config/my_json")
data = open(json_data,'r')
Try 2:
json_data = os.path.join(STATIC_URL, "utils/config/my_json")
data = open(json_data,'r')
Try 3:
json_data = os.path.join(BASE_DIR, STATIC_URL, "utils/config/my_json")
with open(json_data,'r') as file:
print file
I am getting this error,
IOError at /my_url/
[Errno 2] No such file or directory: '/static/utils/config/my_json'
file url after performing os.path.join() is '/static/'
where is the issue in my code? how to read JSON in my case?

can you try the following:
json_data = os.path.join(BASE_DIR, 'static', "utils/config/my_json")
data = open(json_data,'r')
Kindly make a folder named static in the BASE_DIR directory.
I guess since the STATIC_URL starts with / it searches in the root directory

You're using STATIC_URL where you should be using STATIC_ROOT. Reading a file from disk has nothing to do with its URL.
Note first that the static dir probably already has BASE_DIR prepended. Also note, this is not really a static file at all and probably should just be in your code directory instead.

Serve protected media files with django

I'd like Django to serve some media files (e.g. user-uploaded files) only for logged-in users. Since my site is quite low-traffic, I think I will keep things simple and do not use django-sendfile to tell Nginx when to serve a file. Instead I'll let Django/Gunicorn do the job. To me this seems a lot simpler and for a low traffic site this maybe more secure.
But what is the best way to organize the file storage location? Media files are all stored below MEDIA_ROOT and this directory is served by Nginx in production. If I upload my files to MEDIA_ROOT/protected/ I have to tell Nginx not to serve the files in the subdirectory protected.
But is this a good idea? It seems a litte risky to me to allow Nginx access /media/ in the first place and then protect the subdirectory /media/protected/. Wouldn't it be better not to use a subdirectory of MEDIA_ROOT to store protected files?
But if I try something like this quick-and-dirty in my model:
upload_to='../protected/documents/%Y/%m/'
Django complains:
SuspiciousFileOperation at /admin/core/document/add/
The joined path (/home/me/projects/project/protected/documents/2016/09/test.file) is located outside of the base path component (/home/me/projects/project/media)
So I thing it is not good practice to "leave" the MEDIA_ROOT.
What is the best solution to store and serve protected media files?

Serving media files ( that may be large files) from view directly is not good. You can use sendfile extension available in nginx server; a sample nginx configuration is like below.
location /projects/project/media/{
# this path is not public
internal;
# absolute path
alias /projects/project/media/;
}
change your view to
#login_required
def serve_protected_document(request, file):
document = get_object_or_404(ProtectedDocument, file="protected/documents/" + file)
# Split the elements of the path
path, file_name = os.path.split(file)
response = HttpResponse()
response["Content-Disposition"] = "attachment; filename=" + file_name
# nginx uses this path to serve the file
response["X-Accel-Redirect"] = document.name # path to file
return response
Link: More details on configuring sendfile extension on nginx is here

I now came up with the following solution:
I have this in my Django settings:
MEDIA_ROOT = "/projects/project/media/"
MEDIA_URL = "/media/
In my models I do either:
document = models.FileField(upload_to="public/documents")
or
document = models.FileField(upload_to="protected/documents")
This way, I now have the two subdirectories 'public' and 'protected' in my media files directory.
Nginx or Djangos development server only serves the files in the 'public' subdirectory.
For Djangos development server:
if os.environ["ENVIRONMENT_TYPE"] == 'development':
urlpatterns += static(settings.MEDIA_URL + "public/", document_root=settings.MEDIA_ROOT + "public/")
And for Nginx (used in production):
location /media/public/ {
alias /projects/project/media/public/;
}
When I want to serve a protected document, I do the following:
In urls.py:
url(r'^media/protected/documents/(?P<file>.*)$', core.views.serve_protected_document, name='serve_protected_document'),
And in views.py:
#login_required()
def serve_protected_document(request, file):
document = get_object_or_404(ProtectedDocument, file="protected/documents/" + file)
# Split the elements of the path
path, file_name = os.path.split(file)
response = FileResponse(document.file,)
response["Content-Disposition"] = "attachment; filename=" + file_name
return response
I would appreciate any comments! Are there better ways to implement this?

Django 1.8 Cache busting + Amazon S3

I´ve been doing some research, and I have found articles explaining how to use Django's(1.8) cache busting, but they don´t explain my situation.
I am using S3 (and it works) with the following setting in settings.py:
STATICFILES_STORAGE= 'pms.s3utils.StaticRootS3BotoStorage'
In order to use cache busting the docs say I have to set: STATICFILES_STORAGE='django.contrib.staticfiles.storage.ManifestStaticFilesStorage'
I don´t know what setting to use in order to use both S3 and cache busting.
Thanks!

So I finally found a workaround.
In order to be able to upload my files to 2 different folders (static and uploads) in my S3 bucket I have this in my settings.py file:
STATICFILES_STORAGE = 'myapp.s3utils.StaticRootS3BotoStorage'
DEFAULT_FILE_STORAGE = 'myapp.s3utils.MediaRootS3BotoStorage'
And this in the myapp.s3utils.py file:
from storages.backends.s3boto import S3BotoStorage
StaticRootS3BotoStorage = lambda: S3BotoStorage(location='static')
MediaRootS3BotoStorage = lambda: S3BotoStorage(location='uploads')
But I couldn´t use Django´s cache busting.
The solution was to change my myapp.s3utils.py file to:
from storages.backends.s3boto import S3BotoStorage
from django.contrib.staticfiles.storage import ManifestFilesMixin
class CustomS3Storage(ManifestFilesMixin, S3BotoStorage):
pass
StaticRootS3BotoStorage = lambda: CustomS3Storage(location='static')
MediaRootS3BotoStorage = lambda: S3BotoStorage(location='uploads')
Basically it adds the ManiFestfilesMixin, which allows you to use cache busting.
As a side note, you can see that I am only using cache busting for the static files, but not for the uploads files. That is why the MediaRootS3BotoStorage calls the S3BotoStorage class instead of the CustomS3Storage.
I do it this way because the uploads files are not stored in my server (the static files are), they are stored directly in the S3 bucket, so when I ran the collectstatic they are not on my server, so I don´t have to add the hash to the names.

Save Image generated with Reportlab in my MEDIA folder (in Amazon S3)

I implemented this library for generate barcodes images (http://kennethngedo.wordpress.com/2014/02/07/how-to-generate-barcode-in-django-using-reportlab/)
Everything works fine, the image is generated correctly, BUT... the image is created in a folder outside the project, and such I'm using Heroku for Production, I can't access to the image.
I'm using this Django structure (http://django-skel.readthedocs.org/en/latest/) specially adapted to work on Heroku with Amazon S3.
Do you know guys how can I upload the generated image on my Media folder on Amazon?
This is my Views.py where the image is created and saved:
from random import randint
from reportlab.lib.units import mm
from reportlab.graphics.barcode import *
from reportlab.graphics.shapes import Drawing, String
from django.shortcuts import render_to_response
class MyBarcodeDrawing(Drawing):
def __init__(self, text_value, *args, **kw):
barcode = createBarcodeDrawing('Code128', value=text_value, barHeight=10*mm, humanReadable=True)
Drawing.__init__(self,barcode.width,barcode.height,*args,**kw)
self.add(barcode, name='barcode')
def barcode_generator(barcode_value):
text = barcode_value
filename = "nightology_barcode_" + barcode_value
path_to_save = "media/barcodes/"
b = MyBarcodeDrawing(text)
b.save(formats=['gif','pdf'],outDir=path_to_save,fnRoot=filename)
barcodePicUrl = "barcode/"+ filename + ".gif"
return barcodePicUrl
I hope somebody could help me on this... I'll really appreciate.
Thanks!

I had a similar Problem but without the Amazon S3 part. For me it was really easy to create a new File in the media folder. I could simply use default_storage to get the path to the media folder:
from django.core.files.storage import default_storage
import os
# Get the path to the barcodes folder in the media folder
# check if the folder exists and create it if not
folderPath = default_storage.path('barcodes')
if not default_storage.exists('barcodes'):
os.mkdir(folderPath)
Since django-skel seems to be using a django-storage backend for Amazon S3 this should also work for your setup. If the Amazon S3 storage backend is not the default storage backend, you might have to use the S3Storage class directly.

How to open a file on app engine patch?

I tried to read a file in a view like this:
def foo(request):
f = open('foo.txt', 'r')
data = f.read()
return HttpResponse(data)
I tried to place the foo.txt in almost every folder in the project but it still returns
[Errno 2] No such file or directory:
'foo.txt'
So does anybody knows how to open a file in app engine patch? Where should i place the files i wish to open? many thanks.
I'm using app-engine-patch 1.1beta1

In App Engine, patch or otherwise, you should be able to open (read-only) any file that gets uploaded with your app's sources. Is 'foo.txt' in the same directory as the py file? Does it get uploaded (what does your app.yaml say?)?

Put './' in front of your file path:
f = open('./foo.txt')
If you don't, it will still work in App Engine Launcher 1.3.4, which could be confusing, but once you upload it, you'll get an error.
Also it seems that you shouldn't mention the file (or its dir) you want to access in app.yaml. I'm including css, js and html in my app this way.

You should try f = open('./foo.txt', 'r')