Large Uploads in Django - python

I am working on a cookiecutter-django project that requires users to be able to upload files in the .5-1gb size range. From what I understand when uploading files that are larger then 2.5Mbs django uses the TemporaryFileUploadHandler to first upload the file to the tmp directory and then to then transfer them to the media directory. However when I try to upload a larger file my site just hangs, I don't see any files come into the tmp directory, and I see no errors in the logs.
Is there some setting I need to turn on to instruct django to use the TemporaryFileUploadHandler? Or is there some way to possibly figure out where the site might be getting hung up? I am currently building out this project locally so I have full control over my environment.
I am using Python 3.4.5 and django 1.10
UPDATE 1
I have tested uploading files of 5mb, 10mb, 20mb, and 30mb they all upload fine. The file I am having issues with is 873mb.
UPDATE 2
After more debugging I have found that when up loading a large file to my site it gets as far as the get_form method on my view class. After that I am not sure what is called. I have been slowly adding logs at each step in the hopes of figuring out where it is stuck.

Related

How to fix image is not showing in heroku deployed django app

I have a Blog created with Django 3.x. When I run the project on my local server then it works fine. So, I deployed my project to heroku and here is the link bblog-shovon.herokuapp.com/. Then the project was working fine. My project git link https://git.heroku.com/bblog-shovon.gitThen I created a post and then I gave it a thumbnail and finally everything was fine. But, After sometime when i revisit the site then every image were gone. I didn't found them. And when I tried to open the image on another tab then it shows this Error message screenshot. When I am writing this question then I created a new post and after publishing this this posts image is working. But, older image is not working.
Then, How can i fix this problem? Please help me!!
Heroku flushes the filesystem to the latest commit. So you will have to use third-party plugins such as AWSBucket, etc.
With Heroku the filesystem is not persisted so you will lose media files occasionally. Use S3 or some other provider to store your user-uploaded media (also your static files if you want, otherwise use whitenoise). Django-storages makes it very easy.

How To Keep My File Paths Congruent for My Django/Python Application

I am developing a Django app on my Personal Computer. I have my files uploaded to a private repository on GitHub and I am using that to pull them down to my production server.
My question is this
I have file paths set inside of my application on my PC to file locations that are on my PC. But I also have those same files being uploaded and pulled onto my Production Server, but then needing to change the file path on my Production Server side application files in order to match the actual location on my Production Server
I have searched far and wide and cannot seem to find a proper solution to this. Is there some type of way I can write out the paths so that when I use an sys.path.append I don't have to give out the complete path, but rather a local one, so that it matches on both servers?
Possible Findings
Is there an issue if I just use sys.path.append('./<directory>')? This is, of course, assuming that the Python file is running within that current directory.

How to Access text files uploaded to Heroku that is running with Python Script

apologies upfront. I am an extreme newbie and this is probably a very easy question. After much trial and error, I set up an app on Heroku that runs a python script that scrapes data off of a website and stores it in a text file. (I may switch the output to a .csv file). The script and app are running on a Heroku Scheduler, so the scraping takes place on a schedule and the data automatically gets written to the file that is on the Heroku platform. I simply want to download the particular output file occasionally so that I can look at it. (Part of the data that is scraped is being tweeted on a twitter bot that is part of the script.)
(Not sure that this is relevant but I uploaded everything through Git.)
Many thank in advance.
You can run this command heroku run cat path/to/file.txt, but keep in mind that Heroku uses ephemeral storage, so you don't have any guarantee that your file will be there.
For example, Heroku restarts your dynos every 24 hours or so. After that you won't have that file anymore. The general practice is to store files on some external storage provider like Amazon S3.
Not just ephemeral, but immutable, which means you can't write to the file system. You'll have to put the file in something like S3, or just put the data into a database like Postgres.

Django site not updating files immediately with Filezilla but fine with putty/nano?

I have spent 18 hours trying to find an answer to this, and so I hope my question is ok.
I have set up shared hosting on A Small Orange, and have installed django using a virtual environment.
I'm following a tutorial playing with formatting. When I update a .py file that controls the format of the page (the file is admin.py within the /home/user/website/myproject/myapp folder) it doesn't update on the site immediately.
I've been back and forth with tech support and so far have:
Disabled nginx (or as much as possible)
Stopped browser caching
Stopped storing py file as pyc files
Set my django cache setting to dummy cache as per the docs
I've noticed that if I update files using Filezilla - they dont update for 15-30 minutes (even though the updated file is shown correctly in filezilla). If i use Putty to get a command line, and then update a file using nano, it updates instantly...
Why is it only ftp that experiences a delay?

A upload file is 0byte on django

My webserver symbolic linked another server. But a uploaded file was 0 byte when I uploaded file. It was work well before (I used just web server directory)
I don't have any idea dealing with this problem. Does anybody have a trouble like this problem on NFS?
In addition, It works well if I use on Python console.
Thanks,
I found a problem. It was django problem with NFS.
If you have same trouble, check this URLs.
http://www.mail-archive.com/django-users#googlegroups.com/msg55892.html
https://code.djangoproject.com/ticket/9400

Categories