We Keep Coding

How to access uploaded images via URL in Django?

I'm building a simple blog style web app for a friend. I am using CK editor and mostly stock Django 4.1. I'm stumped in how in what would be the best approach to let the creator to upload and serve images in their posts. CK editor allows you to add images via URL and I'd like for it to be possible to the creator to upload their files to the backend and would like to know if there are best practices when it comes to this.
Right now I have this:
def editor_upload(request):
if request.method == 'GET':
form = UploadFileForm
path = os.path.join(settings.STATIC_ROOT, 'files')
files = os.listdir(path)
return render(request, 'upload.html', {"form": form, "files": files})
if request.method == 'POST':
form = UploadFileForm(request.POST, request.FILES)
file = request.FILES['file']
title = str(file)
if form.is_valid():
path = os.path.join(settings.STATIC_ROOT, 'files')
fs = FileSystemStorage(location=path)
fs.save(title, file)
return redirect('/editor/upload')
def open_file(request, file_name):
print(file_name)
return render(request, 'file.html', {"file": file_name})
in editor_upload, the user can see what files there are in the specified folder (but cannot access them yet) and there is the logic to upload new pictures to the folder. The problem comes now: I don't know how to properly access them.
In open_file, the request receives the name of the file and gives it to the HTML templating. This does not work because the template looks into the project folder and not into the root static folder. I must admit I'm a bit lost with how the static folders in Django works because I've been mostly utilizing just the project static folder. Also, I don't think this would work for CK editor to grab the image.
Any ideas on how I could solve this? I also tried doing it by uploading the image to a database but I had kind of the same problem. I need to somehow generate an URL to pass into CK editor. Many thanks!

How to display progress bar for multi file upload form in Django?

I'm developing an application that requires users to upload a massive directory of multiple files/file types/subfolders. It will be handling 10's possibly 100s of GB per user, so having a progress bar displayed while uploading will be very helpful in informing the user the progress of the massive files.
My current setup is based on the following answer to another question: https://stackoverflow.com/a/52016594/15739035
Here is a simplified setup:
models.py:
class Feed(models.Model):
user=models.ForeignKey(User, on_delete=models.CASCADE)
text=models.TextField(blank=False, max_length=500)
class FeedFile(models.Model):
file = models.FileField(upload_to="files/%Y/%m/%d")
feed = models.ForeignKey(Feed, on_delete=models.CASCADE)
forms.py:
class FeedModelForm(forms.ModelForm):
class Meta:
model = Feed
fields = ['text']
class FileModelForm(forms.ModelForm):
class Meta:
model = FeedFile
fields = ['file']
widgets = {
'file': ClearableFileInput(attrs={'multiple': True}),
}
# widget is important to upload multiple files
views.py:
def upload_files_view(request):
user = request.user
if request.method == 'POST':
form = FeedModelForm(request.POST)
file_form = FileModelForm(request.POST, request.FILES)
files = request.FILES.getlist('file') # field name in model
if form.is_valid() and file_form.is_valid():
feed_instance = form.save(commit=False)
feed_instance.user = user
feed_instance.save()
total_files = len(files) # Get the number of files in 'files'
files_count = 0 # Use to show user the progress out of 'total_files'
for f in files:
file_instance = FeedFile(file=f, user=use, feed=feed_instance)
file_instance.save()
progress = f"{total_files}/{files_count}"
# render(request, 'some_folder/upload_progress.html', {'file_form': file_form, 'form': form, 'progress': progress}) # somehow refresh the page showing 'progress' in an html tag?
files_count += 1
return redirect('upload_success_page')
else:
form = FeedModelForm()
file_form = FileModelForm()
return render(request, 'some_folder/file_upload.html', {'file_form': file_form, 'form': form})
My end goal is to get the total file count (like the above views.py file shows), detect what file is currently being uploaded, and return that value to a template for the user to monitor somehow. (You can see what I've attempted above in the views.py file, however this method doesn't work for a number of reasons.)
My question:
How do I get this information while the files are being uploaded, and send it to the template/page the user is viewing?
Here is an example of the Google Drive folder upload progress bar, a folder of 1185 files is being uploaded and the current progress is 11:
In terms of research I've done, I've found many similar questions about using Ajax. All of the other StackExchagne questions are either really old or don't have an answer and weren't much help. This article seemed promising, however it appears that the code is unfinished, and I couldn't get it to work: https://anshu-dev.medium.com/file-upload-progress-bar-using-django-and-ajax-ba4eb7482d9c
In my opinion the ideal method would be entirely using Django views.py, however I'm not entirely sure if or how this would be possible.
I really appreciate any help I can get, thank you for your time!

In Django how to convert an uploaded pdf file to an image file and save to the corresponding column in database?

I am creating an HTML template to show the cover of a pdf file(first page or user can choose one). I want Django to create the cover image automatically without extra upload.
The pdf file is uploaded using Django Modelform. Here is the structure of my code
models.py
class Pdffile(models.Model):
pdf = models.FileField(upload_to='pdfdirectory/')
filename = models.CharField(max_length=20)
pagenumforcover = models.IntegerField()
coverpage = models.FileField(upload_to='coverdirectory/')
form.py
class PdffileForm(ModelForm):
class Meta:
model = Pdffile
fields = (
'pdf',
'filename',
'pagenumforcover',
)
views.py
def upload(request):
if request.method == 'POST':
form = PdffileForm(request.POST, request.FILES)
if form.is_valid():
form.save()
return redirect('pdffilelist')
else:
form = PdffileForm()
return render(request, "uploadform.html", {'form': form})
def pdfcover(request, pk):
thispdf = get_object_or_404(Pdffile, pk=pk)
return render(request, 'pdfcover.html', {'thispdf': thispdf})
In the 'pdfcover.html', I want to use the Django template language so I can render different HTML for different uploaded pdf files. That's why I want to save the image file to the same column as the pdf file.
I am new to Python, new to Django, and obviously new to stack overflow. I have tried pdf2image and PyPDF2 and I believe they all could work however I just cannot find the right code. If you guys enlighten me I will be thankful.

In the pdf2image package there is a function called convert_from_path.
This is the description inside the package of what each of the parameters of the function does.
Parameters:
pdf_path -> Path to the PDF that you want to convert
dpi -> Image quality in DPI (default 200)
output_folder -> Write the resulting images to a folder (instead of directly in memory)
first_page -> First page to process
last_page -> Last page to process before stopping
fmt -> Output image format
jpegopt -> jpeg options `quality`, `progressive`, and `optimize` (only for jpeg format)
thread_count -> How many threads we are allowed to spawn for processing
userpw -> PDF's password
use_cropbox -> Use cropbox instead of mediabox
strict -> When a Syntax Error is thrown, it will be raised as an Exception
transparent -> Output with a transparent background instead of a white one.
single_file -> Uses the -singlefile option from pdftoppm/pdftocairo
output_file -> What is the output filename or generator
poppler_path -> Path to look for poppler binaries
grayscale -> Output grayscale image(s)
size -> Size of the resulting image(s), uses the Pillow (width, height) standard
paths_only -> Don't load image(s), return paths instead (requires output_folder)
use_pdftocairo -> Use pdftocairo instead of pdftoppm, may help performance
timeout -> Raise PDFPopplerTimeoutError after the given time
Because convert_from_path is designed to be able to turn every page in a pdf into an image the function returns an array of Image objects.
If you set the output_folder parameter each image will be saved to that location from the base directory. output_folder must be a full path in this case e.g. 'path/from/root/to/output_folder'. If you don't set it the images won't be saved when converted, only in memory.
By default if you do not set the output_file parameter the function will generate a random formatted filename such as 0a15a918-59ba-4f15-90f0-2ed5fbd0c36c-1.ext. Although if you do set a filename, because this filename is used for converting multiple pdf pages, if your output_file was 'file_name' then each file would be named starting from 'file_name0001-1.ext'.
Beware that if you set output_file and output_folder and try converting two different pdfs the second pdf will overwrite the image files of the first if they are in the same directory.
Here is some code modelled around yours in the question. This code assumes you have pdf2image installed.
I've added a built-in validator on the pdf FileField because else the code will crash if anything else but a pdf is uploaded.
validators=[FileExtensionValidator(allowed_extensions=['pdf'])]
I also created three constants for the upload directories and file format. If you need to change any of them then the rest of the code can remain the same.
COVER_PAGE_DIRECTORY = 'coverdirectory/'
PDF_DIRECTORY = 'pdfdirectory/'
COVER_PAGE_FORMAT = 'jpg'
Also I'm assuming you have the default settings setup for saving files.
settings.py
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
models.py
from django.core.validators import FileExtensionValidator
from django.db.models.signals import post_save
from pdf2image import convert_from_path
from django.conf import settings
import os
COVER_PAGE_DIRECTORY = 'coverdirectory/'
PDF_DIRECTORY = 'pdfdirectory/'
COVER_PAGE_FORMAT = 'jpg'
# this function is used to rename the pdf to the name specified by filename field
def set_pdf_file_name(instance, filename):
return os.path.join(PDF_DIRECTORY, '{}.pdf'.format(instance.filename))
# not used in this example
def set_cover_file_name(instance, filename):
return os.path.join(COVER_PAGE_DIRECTORY, '{}.{}'.format(instance.filename, COVER_PAGE_FORMAT))
class Pdffile(models.Model):
# validator checks file is pdf when form submitted
pdf = models.FileField(
upload_to=set_pdf_file_name,
validators=[FileExtensionValidator(allowed_extensions=['pdf'])]
)
filename = models.CharField(max_length=20)
pagenumforcover = models.IntegerField()
coverpage = models.FileField(upload_to=set_cover_file_name)
def convert_pdf_to_image(sender, instance, created, **kwargs):
if created:
# check if COVER_PAGE_DIRECTORY exists, create it if it doesn't
# have to do this because of setting coverpage attribute of instance programmatically
cover_page_dir = os.path.join(settings.MEDIA_ROOT, COVER_PAGE_DIRECTORY)
if not os.path.exists(cover_page_dir):
os.mkdir(cover_page_dir)
# convert page cover (in this case) to jpg and save
cover_page_image = convert_from_path(
pdf_path=instance.pdf.path,
dpi=200,
first_page=instance.pagenumforcover,
last_page=instance.pagenumforcover,
fmt=COVER_PAGE_FORMAT,
output_folder=cover_page_dir,
)[0]
# get name of pdf_file
pdf_filename, extension = os.path.splitext(os.path.basename(instance.pdf.name))
new_cover_page_path = '{}.{}'.format(os.path.join(cover_page_dir, pdf_filename), COVER_PAGE_FORMAT)
# rename the file that was saved to be the same as the pdf file
os.rename(cover_page_image.filename, new_cover_page_path)
# get the relative path to the cover page to store in model
new_cover_page_path_relative = '{}.{}'.format(os.path.join(COVER_PAGE_DIRECTORY, pdf_filename), COVER_PAGE_FORMAT)
instance.coverpage = new_cover_page_path_relative
# call save on the model instance to update database record
instance.save()
post_save.connect(convert_pdf_to_image, sender=Pdffile)
convert_pdf_to_image is a function that runs on the post_save signal of the Pdffile model. It gets run after your PdffileForm gets saved in your upload view so that we can create the cover image file from the saved pdf file.
cover_page_image = convert_from_path(
pdf_path=instance.pdf.path,
dpi=200,
first_page=instance.pagenumforcover,
last_page=instance.pagenumforcover,
fmt=COVER_PAGE_FORMAT,
output_folder=cover_page_dir,
)[0]
Changing dpi will change the quality of the image. In order to only convert one page the first_page and last_page parameters are the same. Because the result is an array we grab the first and only element in the list inside cover_page_image in this case.
Minor change to your upload view.
views.py
def upload(request):
form = PdffileForm()
if request.method == 'POST':
form = PdffileForm(request.POST, request.FILES)
# if form is not valid then form data will be sent back to view to show error message
if form.is_valid():
form.save()
return redirect('pdffilelist')
return render(request, "uploadform.html", {'form': form})
I don't know what your upload.html file looks like but I used the following which will work with the code provided.
upload.html
<h1>Upload PDF</h1>
<form method="POST" enctype="multipart/form-data">
{% csrf_token %}
{{ form.as_p }}
<button type="submit">Upload</button>
</form>
With an example pdf
Uploaded through the form
The resulting database record
The resulting file locations once uploaded
Final note:
Because FileFields have code to ensure that existing files don't get overwritten, The code
# get name of pdf_file
pdf_filename, extension = os.path.splitext(os.path.basename(instance.pdf.name))
new_cover_page_path = '{}.{}'.format(os.path.join(cover_page_dir, pdf_filename), COVER_PAGE_FORMAT)
# rename file to be the same as the pdf file
os.rename(cover_page_image.filename, new_cover_page_path)
# get the relative path to the cover page to store in model
new_cover_page_path_relative = '{}.{}'.format(os.path.join(COVER_PAGE_DIRECTORY, pdf_filename), COVER_PAGE_FORMAT)
instance.coverpage = new_cover_page_path_relative
ensures the pdf FileField filename is used to name the cover page because it is almost completely unique.

I used the explanation here, and everything works fine, except when from admin panel I oped the saved Pdffile object and try to change the pagenumforcover to another integer and then save it then it won't generate the new coverpage

File upload not working after changing models format

I have a model class similar to following -
class Document(models.Model):
docfile = models.FileField(upload_to='documents/%Y/%M/%D')
Everything is working fine and files are uploaded successfully based on directory structure.
Now I don't want to upload files in this format but simply all files in one folder so I changed the logic ..
class Document(models.Model):
docfile = models.FileField(upload_to='documents')
Now It is not uploading the files and throwing error. Maybe I need to run some command but I do not know what ??
Please suggest something
Edit1:
Ok .. I found that the actual problem lies somewhere else.
I have a view like this - (please ignore the bad spacing but that is fine in actual code)
def lists(request):
// Problematic Code Start
path = settings.MEDIA_URL + 'upload/location.txt'
f = open(path, 'w')
myfile = File(f)
myfile.write('Hello World')
myfile.closed
f.closed
// Problematic Code ends
# Handle file upload
if request.method == 'POST':
form = DocumentForm(request.POST, request.FILES)
if form.is_valid():
filename = Document(docfile = request.FILES['docfile'])
filename.save()
# Redirect to the document list after POST
return HttpResponseRedirect(reverse('sdm:lists'))
#return render_to_response(reverse('sdm:lists'))
else:
form = DocumentForm() # A empty, unbound form
# Load documents for the list page
documents = Document.objects.all()
# Render list page with the documents and the form
return render_to_response(
'sdm/lists.html',
{'documents': documents, 'form': form},
context_instance=RequestContext(request)
)
When I remove the problematic code , everything works fine. (ignore the purpose of this weird code, actual interest is something bigger)
MEDIA_URL=/media/
Here is the error:
IOError at /sdm/lists
[Errno 2] No such file or directory: '/media/upload/location.txt'
Although File Exists and all permissions are www-data:www-data with 755

"problematic" code indeed - whoever wrote this should find another job. This code is wrong in more than one way (using MEDIA_URL instead of MEDIA_ROOT - which is the cause of the IOError you get - and also badly misusing Python's dead simple file objects) and totally useless, and looks like a leftover of someone programming by accident. To make a long story short : just remove it and you'll be fine.

How to unzip a zip file in django , which contains .shp , .prj , .shx and .dbf , thus being able to upload in the database?

I have uploaded a zip file using django in a local directory . How do I unzip it , and store it in 3 different files ?
Ok . Let me explain my problem a bit more in detail .
def upload(request):
if request.method == 'POST':
form = UploadForm(request.POST, request.FILES)
if form.is_valid():
form.handle(request.FILES['file_obj'])
#form.save() # if a modelform
#form.cleaned_data['user'] = request.user
z = zipfile.ZipFile('file_obj')
for files in z.namelist():
file(files,'wb').write(z.read(files))
z.close()
return render_to_response('uploaded.html', RequestContext(request,{}))
else:
form = UploadForm()
return render_to_response('upload.html', RequestContext(request,{'form': form})
)
This is my upload form , which is supposed to work . file_obj contains the uploaded zip file . But it doesnt give any output .

It doesnt look like you're actually opening the upload, but a file in the current directory called 'file_obj'. You want something more like
z = zipfile.ZipFile(request.FILES['file_obj'])
also I might be wrong, but i don't think you need the form.handle() call at all, at least I've never used it, but i might stand corrected.
Also, you want to be very careful here, as you're writing out to the filenames contained in the zipfile, which for all you know could be absolute paths somewhere. You need to be very careful with that. In case I didn't say it, be careful with user created content, it might be malicious.