This question already has answers here:
Save uploaded image to S3 with Django
(2 answers)
Closed 1 year ago.
My problem is that images stored in media folder are not transferring to S3 Bucket. I tested with other file from request and the file did transfer, so I assume settings.py must be OK.
From views.py ->
This works:
if request.method == 'POST':
imageFile = request.FILES['images']
upload = Upload(file=imageFile)
upload.save()
image_url = upload.file.url
print(image_url)
This does not work:
for i in os.listdir(folder):
f = os.path.join(conf_settings.MEDIA_ROOT,company, i)
upload = Upload(file=f)
upload.save()
No error but it just does not work.
This also does not work:
for i in os.listdir(folder):
with open(os.path.join(folder, i)) as f:
upload = Upload(file=f)
upload.save()
>The error I am getting is:
>
>Exception Value:
>'_io.TextIOWrapper' object has no attribute '_committed'
>
>at upload.save()
This is my storage_backend.py
from django.conf import settings
from storages.backends.s3boto3 import S3Boto3Storage
class MediaStorage(S3Boto3Storage):
location = 'media'
default_acl = 'public-read'
file_overwrite = True
This is my model.py
class Upload(models.Model):
uploaded_at = models.DateTimeField(auto_now_add=True)
file = models.FileField()
I am uploading a .ZIP file with images. Unzipping it and saving them to media folder, then I want to upload from media folder to S3 Bucket. This operation fails.
The file in request.FILES is the Zip file, which I am using to test that all settings.py for AWS should be correct because it does transfer correctly.
I believe my issue has to do with the way I am reading the file and passing it.
So after many hours....this actually worked. Although the transfer is a bit slow, im sure there must be a better way.
https://stackoverflow.com/a/53260957/11116189
I am creating an HTML template to show the cover of a pdf file(first page or user can choose one). I want Django to create the cover image automatically without extra upload.
The pdf file is uploaded using Django Modelform. Here is the structure of my code
models.py
class Pdffile(models.Model):
pdf = models.FileField(upload_to='pdfdirectory/')
filename = models.CharField(max_length=20)
pagenumforcover = models.IntegerField()
coverpage = models.FileField(upload_to='coverdirectory/')
form.py
class PdffileForm(ModelForm):
class Meta:
model = Pdffile
fields = (
'pdf',
'filename',
'pagenumforcover',
)
views.py
def upload(request):
if request.method == 'POST':
form = PdffileForm(request.POST, request.FILES)
if form.is_valid():
form.save()
return redirect('pdffilelist')
else:
form = PdffileForm()
return render(request, "uploadform.html", {'form': form})
def pdfcover(request, pk):
thispdf = get_object_or_404(Pdffile, pk=pk)
return render(request, 'pdfcover.html', {'thispdf': thispdf})
In the 'pdfcover.html', I want to use the Django template language so I can render different HTML for different uploaded pdf files. That's why I want to save the image file to the same column as the pdf file.
I am new to Python, new to Django, and obviously new to stack overflow. I have tried pdf2image and PyPDF2 and I believe they all could work however I just cannot find the right code. If you guys enlighten me I will be thankful.
In the pdf2image package there is a function called convert_from_path.
This is the description inside the package of what each of the parameters of the function does.
Parameters:
pdf_path -> Path to the PDF that you want to convert
dpi -> Image quality in DPI (default 200)
output_folder -> Write the resulting images to a folder (instead of directly in memory)
first_page -> First page to process
last_page -> Last page to process before stopping
fmt -> Output image format
jpegopt -> jpeg options `quality`, `progressive`, and `optimize` (only for jpeg format)
thread_count -> How many threads we are allowed to spawn for processing
userpw -> PDF's password
use_cropbox -> Use cropbox instead of mediabox
strict -> When a Syntax Error is thrown, it will be raised as an Exception
transparent -> Output with a transparent background instead of a white one.
single_file -> Uses the -singlefile option from pdftoppm/pdftocairo
output_file -> What is the output filename or generator
poppler_path -> Path to look for poppler binaries
grayscale -> Output grayscale image(s)
size -> Size of the resulting image(s), uses the Pillow (width, height) standard
paths_only -> Don't load image(s), return paths instead (requires output_folder)
use_pdftocairo -> Use pdftocairo instead of pdftoppm, may help performance
timeout -> Raise PDFPopplerTimeoutError after the given time
Because convert_from_path is designed to be able to turn every page in a pdf into an image the function returns an array of Image objects.
If you set the output_folder parameter each image will be saved to that location from the base directory. output_folder must be a full path in this case e.g. 'path/from/root/to/output_folder'. If you don't set it the images won't be saved when converted, only in memory.
By default if you do not set the output_file parameter the function will generate a random formatted filename such as 0a15a918-59ba-4f15-90f0-2ed5fbd0c36c-1.ext. Although if you do set a filename, because this filename is used for converting multiple pdf pages, if your output_file was 'file_name' then each file would be named starting from 'file_name0001-1.ext'.
Beware that if you set output_file and output_folder and try converting two different pdfs the second pdf will overwrite the image files of the first if they are in the same directory.
Here is some code modelled around yours in the question. This code assumes you have pdf2image installed.
I've added a built-in validator on the pdf FileField because else the code will crash if anything else but a pdf is uploaded.
validators=[FileExtensionValidator(allowed_extensions=['pdf'])]
I also created three constants for the upload directories and file format. If you need to change any of them then the rest of the code can remain the same.
COVER_PAGE_DIRECTORY = 'coverdirectory/'
PDF_DIRECTORY = 'pdfdirectory/'
COVER_PAGE_FORMAT = 'jpg'
Also I'm assuming you have the default settings setup for saving files.
settings.py
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
models.py
from django.core.validators import FileExtensionValidator
from django.db.models.signals import post_save
from pdf2image import convert_from_path
from django.conf import settings
import os
COVER_PAGE_DIRECTORY = 'coverdirectory/'
PDF_DIRECTORY = 'pdfdirectory/'
COVER_PAGE_FORMAT = 'jpg'
# this function is used to rename the pdf to the name specified by filename field
def set_pdf_file_name(instance, filename):
return os.path.join(PDF_DIRECTORY, '{}.pdf'.format(instance.filename))
# not used in this example
def set_cover_file_name(instance, filename):
return os.path.join(COVER_PAGE_DIRECTORY, '{}.{}'.format(instance.filename, COVER_PAGE_FORMAT))
class Pdffile(models.Model):
# validator checks file is pdf when form submitted
pdf = models.FileField(
upload_to=set_pdf_file_name,
validators=[FileExtensionValidator(allowed_extensions=['pdf'])]
)
filename = models.CharField(max_length=20)
pagenumforcover = models.IntegerField()
coverpage = models.FileField(upload_to=set_cover_file_name)
def convert_pdf_to_image(sender, instance, created, **kwargs):
if created:
# check if COVER_PAGE_DIRECTORY exists, create it if it doesn't
# have to do this because of setting coverpage attribute of instance programmatically
cover_page_dir = os.path.join(settings.MEDIA_ROOT, COVER_PAGE_DIRECTORY)
if not os.path.exists(cover_page_dir):
os.mkdir(cover_page_dir)
# convert page cover (in this case) to jpg and save
cover_page_image = convert_from_path(
pdf_path=instance.pdf.path,
dpi=200,
first_page=instance.pagenumforcover,
last_page=instance.pagenumforcover,
fmt=COVER_PAGE_FORMAT,
output_folder=cover_page_dir,
)[0]
# get name of pdf_file
pdf_filename, extension = os.path.splitext(os.path.basename(instance.pdf.name))
new_cover_page_path = '{}.{}'.format(os.path.join(cover_page_dir, pdf_filename), COVER_PAGE_FORMAT)
# rename the file that was saved to be the same as the pdf file
os.rename(cover_page_image.filename, new_cover_page_path)
# get the relative path to the cover page to store in model
new_cover_page_path_relative = '{}.{}'.format(os.path.join(COVER_PAGE_DIRECTORY, pdf_filename), COVER_PAGE_FORMAT)
instance.coverpage = new_cover_page_path_relative
# call save on the model instance to update database record
instance.save()
post_save.connect(convert_pdf_to_image, sender=Pdffile)
convert_pdf_to_image is a function that runs on the post_save signal of the Pdffile model. It gets run after your PdffileForm gets saved in your upload view so that we can create the cover image file from the saved pdf file.
cover_page_image = convert_from_path(
pdf_path=instance.pdf.path,
dpi=200,
first_page=instance.pagenumforcover,
last_page=instance.pagenumforcover,
fmt=COVER_PAGE_FORMAT,
output_folder=cover_page_dir,
)[0]
Changing dpi will change the quality of the image. In order to only convert one page the first_page and last_page parameters are the same. Because the result is an array we grab the first and only element in the list inside cover_page_image in this case.
Minor change to your upload view.
views.py
def upload(request):
form = PdffileForm()
if request.method == 'POST':
form = PdffileForm(request.POST, request.FILES)
# if form is not valid then form data will be sent back to view to show error message
if form.is_valid():
form.save()
return redirect('pdffilelist')
return render(request, "uploadform.html", {'form': form})
I don't know what your upload.html file looks like but I used the following which will work with the code provided.
upload.html
<h1>Upload PDF</h1>
<form method="POST" enctype="multipart/form-data">
{% csrf_token %}
{{ form.as_p }}
<button type="submit">Upload</button>
</form>
With an example pdf
Uploaded through the form
The resulting database record
The resulting file locations once uploaded
Final note:
Because FileFields have code to ensure that existing files don't get overwritten, The code
# get name of pdf_file
pdf_filename, extension = os.path.splitext(os.path.basename(instance.pdf.name))
new_cover_page_path = '{}.{}'.format(os.path.join(cover_page_dir, pdf_filename), COVER_PAGE_FORMAT)
# rename file to be the same as the pdf file
os.rename(cover_page_image.filename, new_cover_page_path)
# get the relative path to the cover page to store in model
new_cover_page_path_relative = '{}.{}'.format(os.path.join(COVER_PAGE_DIRECTORY, pdf_filename), COVER_PAGE_FORMAT)
instance.coverpage = new_cover_page_path_relative
ensures the pdf FileField filename is used to name the cover page because it is almost completely unique.
I used the explanation here, and everything works fine, except when from admin panel I oped the saved Pdffile object and try to change the pagenumforcover to another integer and then save it then it won't generate the new coverpage
My Project structure:
myproject
--*
--*
--media
--*
--*
I want to allow users to access to my media directory: I want them to be able to read and download all files in my media directory and I want them to be able to write files to my media directory.
How can I accomplish this using Django rest framework?
Assume that there are 2 files in my media directory: I want to return the following JSON object as a response to a GET request:
{
file1: link_to_example1.txt
}
{
file2: link_to_example2.txt
}
How do I do this -- what should my app's model.py, views.py and maybe serializers.py look like?
We can start with simple thing like this. Every time user upload a file, one record is created in the model. If you plan to add/remove file without using your system, you may need a method to sync the latest folder content with your model's records.
class MediaFile(models.Model):
media_file = models.FileField()
uploaded = models.DateTimeField(auto_now_add=True)
class MediaFileSerializer(serializers.ListSerializer):
class Meta:
model = MediaFile
fields = ('media_file',)
class MediaViewSet(viewsets.ViewSet):
"""
A simple ViewSet for listing or retrieving MediaList.
"""
serializer_class = MediaFileSerializer
Then you can study how to upload file with Django and DRF.
I have a website, that lets user upload files. These files are attached to a node, which ID is part of the upload request. Since the same file might be attached to different nodes, Django will rename the file by adding a hash to the filename. Thus if a user downloads a previously uploaded file, it won't have the original filename.
Is it possible to create a subdirectory (named after the node ID) inside the media folder a file is uploaded? The closest solution I found was to change the System Storage of the FileField, but this is static for all files of that one model. Or is there another, better way to solve the problem with duplicate files?
Model:
class Attachment(models.Model):
node = models.IntegerField(default=-1)
file = models.FileField(upload_to=".")
View:
def file_upload(request):
if request.method == "POST":
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
instance = Attachment(file=request.FILES["file"], node_id=request.POST["node_id"])
instance.save()
return HttpResponse(instance.file.url)
Yes, take a look at the documentation on upload_to.
You could do something like this, which includes the node id (defined as an integer in your model in the upload_to path:
def attachment_path_with_node(instance, filename):
return "attachments/{}/{}".format(instance.node, filename)
class Attachment(models.Model):
node = models.IntegerField(default=-1)
file = models.FileField(upload_to=attachment_path_with_node)
Also path can be further customized like this:
document = models.FileField(upload_to='documents/%Y/%m/%d/')
which would upload to: MEDIA_ROOT/documents/2020/12/22/.
See more at https://simpleisbetterthancomplex.com/tutorial/2016/08/01/how-to-upload-files-with-django.html
I'm trying to learn how to upload files in Django and use them in templates.
My goal is to create post form with multiuplad field but im starting with simple things. Im trying to uplad single files and add link to those files in tempalate. Thats what i have done so far:
model:
class Adds(models.Model):
author = models.ForeignKey(User, verbose_name=u"Autor postu", blank=True, null=True)
image = models.FileField(upload_to='photos/')
def get_absolute_url(self):
return settings.MEDIA_ROOT+"\%s" % self.image.name
I've added function get_absolute_url to get url with MEDIA_ROOT added. Its not working because MEDIA_ROOT path part is with "\" and the res is separated with "/". Thats first question how to make this function work properly.
I am adding links like this {{plik.get_absolute_url}}
but i read about url function so i tryed somethin like this {{plik.image.url}}but it returns url without MEDIA_ROOT part so only "upladed_to_name/file_name".
My MEDIA_ROOT in settings:
MEDIA_ROOT= os.path.join(os.path.dirname(BASE_DIR), "static_env", "my_static", "media")
In shell i have somethin like this:
In [1]: from voiceManaging.models import Pracownik
In [2]: i=Pracownik.objects.get(pk=1)
In [3]: i.zdjecie.url
Out[3]: 'photos/mama_0SADn31.jpg'
In [4]: i.zdjecie.path
Out[4]: u'E:\\Django\\Projects\\KCKAPP\\static_env\\media\\photos\\mama_0SADn31.
jpg'
Because you're using the default FileSystemStorage class, providing you have defined a MEDIA_URL in your settings the storage class can build a URL for your uploaded files using that setting & the corresponding file name using the .url attribute.
So using your example;
class Adds(models.Model):
author = models.ForeignKey(User, blank=True, null=True)
image = models.FileField(upload_to='photos')
To illustrate the attributes & what they each do for the FieldField take a look at this;
>>> add = Adds.objects.get(author="Mark")
>>> add.image
<FileField: mark.jpg>
>>> car.photo.name
'photos/mark.jpg'
>>> add.image.path
'/media/photos/mark.jpg'
>>> add.image.url
'http://media.example.com/photos/mark.jpg'
So MEDIA_URL should be a URL where your web server is configured to serve the files stored in MEDIA_ROOT.