I am implementing a way to restrict file upload on Django 1.8 running python 3.4
Basically, I want to check the MIMEType of a file when they upload using mimetype. However, when I manipulate the file name from bad_image.exe to bad_image.exe.jpg, the mimetype is still image/jpeg. This could still result in a malicious attack.
Is there a way to actually implement this? I tried magic too but it still does not work.
You're right, rename a file .exe to .exe.jpg and content_type output (image/jpeg).
But using python-magic if properly check the file as checking their headers and not the extension, so the output was in my test:
PE32 executable (GUI) Intel 80386, for MS Windows
Even so I think the headers may be modified... hope that helps.
EDIT: In my test use cleaning a specific attribute
class UploadFileForm(forms.ModelForm):
class Meta:
model = FileTestUpload
fields = ('title','file')
def clean_file(self):
f = self.cleaned_data.get("file", False)
ftype = magic.from_buffer(f.read()) # InMemoryUploadedFile
print ftype
return f
It is too simple but it was just to test.
Related
The following python code successfully fills out a pdf form:
import pypdftk
data_dict = {key:value pairs}
PDF_PATH = 'form.pdf' #form to be filled out in same folder as the file executing this code
out_file = 'out_file.pdf' #completed pdf file
generated_pdf = pypdftk.fill_form(
pdf_path = PDF_PATH,
datas = data_dict,
out_file = out_file,
)
However, the same code used in my django project results in the following error message:
Error: Unable to find file.
Error: Failed to open PDF file:
form.pdf
Errors encountered. No output created.
Done. Input errors, so no output created.
... REMAINDER OF TRACEBACK EXCLUDED FOR BREVITY IF YOU WANT TO SEE IT I'LL POST...
raise subprocess.CalledProcessError(retcode, cmd, output=output) output=output) df fill_form C:\Users\Home\AppData\Local\Temp\tmpbqq__7c4 output out_file.pdf flatten
subprocess.CalledProcessError: Command 'pdftk l_a_r.pdf fill_form C:\Users\Home\AppData\Local\Temp\tmpbqq_0 87495_7c4 output out_file.pdf flatten'
returned non-zero exit status 1.
pypdftk is installed in the virtual environment the project is running in.
The pdftk server is added as a windows path variable.
In the above example, and every other time this has happened the temp file referenced at the end of the error message contains all of the expected data in XML.
I've tried the following combinations of code to try to make this work:
Running the exact above code within a view function, with the pdf form to be filled in the same folder as the views.py file:
import pypdftk
def filler_view(request):
form = MyForm()
if request.method =='POST':
#code to successfully populate dictionary data_dict with form data
PDF_PATH = 'form.pdf' #form to be filled out in same folder as the file executing this code
out_file = 'out_file.pdf #completed pdf file
generated_pdf = pypdftk.fill_form(
pdf_path = PDF_PATH,
datas = data_dict,
out_file = out_file,
)
return render(request, 'success.html')
Storing the code and file in a folder and importing to call the relevant function within the view:
-appFolder
-pfd_filler_folder
-form.pdf
-form_filler.py
-views.py
views.py
from appFolder.pdf_filler_folder import form_filler as f
def filler_view(request):
form = MyForm()
if request.method =='POST':
#code to successfully populate dictionary data_dict with form data
f.fill_form(data_dict, 'output.pdf')
form_filler.py:
import pypdftk
def fill_form(data_dict, out_file):
PDF_PATH = 'form.pdf'
generated_pdf = pypdftk.fill_form(
pdf_path = PDF_PATH,
datas = data_dict,
out_file = out_file,
)
Running both of the above with the full path from c:\... of the form.pdf file.
I've also verified that I can successfully fill a form with the executing .py file and the form.pdf file in same folder on two storage drives and from within the django project itself, when not being executed by the django project. pdftk finds the forms.py with no problems at all in this circumstance.
I believe that the file not found error message is key, as it seems to refer to the pdf form I'm trying to fill out. I've spent from 1500 till 1800 researching this, and I haven't managed to get it to work, although I am lead to believe that my error message indicates a missing parameter in the cl execution command. I'm not sure what this would be, as all parameters seem present and correct.
Interestingly enough, a friend of mine is experiencing the same error message just in windows. I'm aware that pdftk can sometimes be touchy in windows, and I think there's probably a nuance I'm missing here.
The outcome I'd like is to fill out a pdf form from within my django project, with data obtained from a form through a post request.
I'd welcome either someone enlightening me as to why pdftk is struggling to either see or use the form file whilst being used from within my django project and pointing me in the right direction
I'm aware that there are alternatives to using pdftk, but pdftk is the simplest, and honestly pypdftk is the only library I've found to reliably work with python to fill out pdf forms so far in Windows. I don't want to go down the route of generating my own replica form and populating it with data, but I'm aware that that is also an option.
Question answered just now on Reddit:
When in Django, it is either wsgi.py or manage.py which is ultimately responsible for what goes on. On that basis, placing the form.pdf file in the same folder as wsgy.py solved the problem and the code now runs as intended, with an unbound form POSTing data back to a view, and a pdf form being filled out and a duplicate saved with said data. Hope that helps anyone else who comes up against this!
I'm having trouble in data migrating tables that contains FieldFile object. The problem here is that I don't have the files yet. I use a special method to create ObjectFile.
def create_object_file(file):
return = ObjectFile.objects.create(
name=name,
content_type=file.content_type,
extension=extension,
uploaded_by=object_user,
md5sum=file.md5sum,
file=''
)
File is a FieldFile. I need to register it without a file since I don't have the files yet.
I've tried to import a FieldFile from django.db.models thinking of instanciate it in order to attribute it to file. But it misses chunks now.
I presume that one solution would have been using raw sql to populate new database.
So, this solution is not a real solution to the problem: how to migrate data file without files?
I managed to have some of the files I have to work on and I did what I had to, which is quite simple:
from django.core.files import File
def create_source_file(file, object_file):
media = os.path.join(SOURCE_FILES, file.path)
if not os.path.exists(media):
return
of = ObjectFile(
name=name,
content_type=file.file_type,
extension=extension.lower(),
md5sum=file.md5sum,
)
with open(media, 'rb') as m:
of.file.save(file.name.lower(), File(m))
of.save()
I am trying to send an image using Asana's API but it just attaches a blank file.
This is the code I have been using.
client.attachments.create_on_task(task_id=123456789,file_content="Url_of_file",file_name='Name_of_File',file_content_type="image/jpeg")
I have tried using different file formats like .txt and .png but for some reason the Asana API is blocking my requests.It just posts a black image on Asana.
I have tried to convert the file to base64 as well but it still doesn't work
In this original documentation(below), it shows that we need to pass two arguments;one for the file's content and the other for file itself ('file').
def create_on_task(self, task_id, file_content, file_name, file_content_type=None, **options):
"""Upload an attachment for a task. Accepts a file object or string, file name, and optional file Content-Type"""
path = '/tasks/%d/attachments' % (task_id)
return self.client.request('post', path, files=[('file', (file_name, file_content, file_content_type))], **options)
But when I am trying to pass the arguments for file and file content it shows me an error.
Can somebody please help me with this?
Another user on the Asana development forum had the same problem using the Curl API. (https://forum.asana.com/t/sending-file-with-api/16897/2). Apparently it has something to do with the 'multipart form upload'.
Looking through another thread here on stackoverflow (How to send a "multipart/form-data" with requests in python?), it seemed the file object had to be read in binary was all.
so the parameters would be:
client.attachments.create_on_task(task_id=<task id here>,file_content=open(filename_with_path, 'rb'),file_name='Name_of_File',file_content_type="image/jpeg")
How to restrict the size of file being uploaded.
I am using django 1.1 with apache.
Can I use apache for this and show some html error page if say size is bigger then 100MB.
Thanks.
I mean before uploading the file
On client side it isn't possible...
I suggest to write a custom upload handlers and to override receive_data_chunk.
Example: QuotaUploadHandler
You can do this in javascript in most recent browsers, using the File API: http://www.w3.org/TR/FileAPI/
For example (using jquery):
var TYPES = ['image/jpeg', 'image/jpg', 'image.png'];
var file = $('#my_file_input')[0].files[0];
var size = file.size || file.fileSize;
var type = file.type;
if (size > MAX_BYTES) {
alert('Error: file too large');
} else if (TYPES.indexOf(type) < 0) {
alert('Error: file not a JPG or PNG');
} else {
// proceed with file upload
}
No need for Java or Flash. Of course, you'll still need some sort of checking on the server for users who disable javascript.
apache has a server setting for max file size..(also dont forget max post size). I do not believe apache can show an error page on its own, you can probably use python for that.
unfortunetly I know nothing obout python (yet) so I can't really help you beyond that.
I know php can do that easily so I'm sure there is a method for python.
If you want to get the file size before uploading begins you will need to use Flash or a Java applet.
Writing a custom upload handler is the best approach. I think something like the following would work (untested). It terminates the upload as early as possible.
from django.conf import settings
from django.core.files.uploadhandler import FileUploadHandler, StopUpload
class MaxSizeUploadHandler(FileUploadHandler):
"""
This test upload handler terminates the connection for
files bigger than settings.MAX_UPLOAD_SIZE
"""
def __init__(self, request=None):
super(MaxSizeUploadHandler, self).__init__(request)
def handle_raw_input(self, input_data, META, content_length, boundary, encoding=None):
if content_length > settings.MAX_UPLOAD_SIZE:
raise StopUpload(connection_reset=True)
How would one go about testing a Pylons controller (using Nose Tests) that takes a file upload as a POST parameter?
Like this:
class TestUploadController(TestController):
// ....
def test_upload_files(self):
""" Check that upload of text file works. """
files = [("Filedata", "filename.txt", "contents of the file")]
res = self.app.post("/my/upload/path", upload_files = files)
Uploading file usually requires authenticated user so you may also need to pass "extra_environ" argument to self.app.post() to circumvent that.
See paste.fixture documentation for details on the arguments accepted by self.app.post()