I need to upload and process a CSV file from a form in a Google App Engine application based on Webapp2 (Python) I understand I could use blobstore to temporary store the file but I am curious to know if there is a way to process the file without having to store it at all.
If you need to upload a file via webapp2 with an HTML form, the first thing you need to do is change HTML form enctype attribute to multipart/form-data, so the code snippet seems like:
<form action="/emails" class="form-horizontal" enctype="multipart/form-data" method="post">
<input multiple id="file" name="attachments" type="file">
</form>
In python code, you can read the file directly via request.POST, here's a sample code snippet:
class UploadHandler(BaseHandler):
def post(self):
attachments = self.request.POST.getall('attachments')
_attachments = [{'content': f.file.read(),
'filename': f.filename} for f in attachments]
The content of uploaded files is in self.request.POST in your handler, so you can get that content (assuming e.g the field for the uploaded file is named 'foo') with e.g
content = self.request.POST.multi['foo'].file.read()
So now you have the content as a string -- process it as you wish. This does of course assume the thing will fit in memory (no multi-megabyte uploads!-)...
Related
So I send a Form to my Flask App and wish to receive the Form image data input and pass it to another function.
Example of form:
<form action="https://example.com/api/accept_form" method="POST" enctype="multipart/form-data">
<label for="fileupload"><input type="file" name="fileupload" value="fileupload" id="fileupload" accept="image/*">
Select a file to upload</label>
<br><button id="submit">Post to Instagram</button>
</form>
On the flask app:
def image_function(image):
#DO SOMETHING WITH THE IMAGE
#main.route("/api/accept_form", methods=["POST"])
def manipulate_image():
image = request.form["fileupload"]
image_function(image)
I can see how the image data looks using the print command:
request.files["fileupload"] = <FileStorage: '666650673.jpg' ('image/jpeg')>
request.files["fileupload"].read() = b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x00\x00H\x00H\x00\x00\xff\xe1\x00 and so on ...'
How do I pass the image to another function in the Flask App as if it was the original .jpg submitted, called from the Flask Apps current directory using "./666650673.jpg"?
References:
https://linuxhint.com/python-string-decode-method/
https://stackoverflow.com/a/2324133/14843373
https://github.com/williballenthin/python-evtx/issues/43
My mistake, as #teddybearsuicide pointed out was that I was passing a file handle and not a file pointer.
Maybe there is a better way but I just saved it locally on the EC2 based on this
Solution
request.files["image"].save("./imageToSave.jpg")
And called it as you would any file in your directory.
image = "./imageToSave.jpg"
I would prefer not to have had to save it so if anyone knows how to pass the file handle directly to achieve the same effect that would be great.
I want to upload python file through django form and read the function available inside and use it for processing.
Till now what i had done is:
Taken file from the user and saved in the media folder.
taken function name (so that i can use it if required for calling funtion)
Index.py
<form method="POST" enctype="multipart/form-data" action="/result">Enter function name
{% csrf_token %}
<input type="text" name="functionname"><br>
Upload your .py file here
<input type="file" name="functionfile">
<input type="submit" value="Submit">
</form>
views.py
def result(request):
if request.method == 'POST':
functionname=request.POST['functionname']
functionfile=request.FILES['functionfile']
fs= FileSystemStorage()
modulename=fs.save(functionfile.name,functionfile)
url=fs.url(modulename)
print(url)
return render(request,'result.html')
I don't have any clue how to use that the function of the uploaded file in the backend
Desired result would be something like.
for eg. example.py file contains a function
def add(data):
p=10+data
return p
i upload a example.py file
suppose in background i have d = 100
django calls result=add(d)
print the result
Any reference or resource will also be helpful.
Thanks
An simple approach would be use the normal file upload in django get the data in an action, launch a subprocess using docker which has python3 in it on the server side where you are running django application.
Why docker ?
It is safer to run even malicious code inside the docker container rather than on the server machine itself.
Other Ideas :
You can run the code and get result through online API's.
REF : https://github.com/saikat007/Online-Compiler-using-Django-python/blob/master/src/ide/views.py
Google App Engine documentation makes it appear very simple to get the contents of an uploaded file (self.request.get('user_file')), but while I can do this with old_dev_appserver.py, I cannot get it with the current dev_appserver.py (v1.9.2).
Here's a simple example that generates the incoming data:
<form name="myform" action="http://localhost:8080/sync?cmd=tester" enctype="multipart/form-data" method="post">
Username: <input type="file" name="user_file" />
<input type="submit" value="Submit" />
</form>
In old_dev_appserver.py, one could get the file in GAE via self.request.get('user_file'), but not with the latest (1.9.2) dev_appserver.py
WebApp2 says "Uploaded files are available as cgi.FieldStorage (see the cgi module) instances directly in request.POST." But, request.POST is empty, and cgi.FieldStorage() does not contain 'user_file' either.
Strangely, if I print out self.request.params, I do see an element in the UnicodeMultiDict that is (u'user_file', FieldStorage(u'user_file', u'myfile.ico')). But when I try to get that element, either via named access or just iterating over params, I cannot get it. Even if I do a Len(self.request.params) I get one less than what I see, and the 'user_file' element is missing. If I do this with old_dev_appserver, the Len(self.request.params) is correct.
How do I get user_file?
Figured it out (finally!). I had been logging elements of the request (params, body, etc.) as I sorted through other issues. But you can only instantiate a cgi.FieldStorage element once, so by the time I got to the actual self.request.get('user_file'), it was already gone.
Classic case in which removing the debugging actually makes it work.
I'm looking to use Flask to host a single-page website that would allow users to upload a CSV that would be parsed and put into a database. All of the database shenanigans are complete (through SQLalchemy in another Python script) and I've got everything worked out once a script has access to the CSV, I just need help getting it there.
Here's the scenario:
1. User directs browser at URL (probably something like
http://xxx.xxx.xxx.xxx/upload/)
2. User chooses CSV to upload
3. User presses upload
4. File is uploaded and processed, but user is sent to a thank you page while our
script is still working on the CSV (so that their disconnect doesn't cause the
script to abort).
It's totally cool if the CSV is left on the server (in fact, it's probably preferred since we'd have a backup in case processing went awry)
I think what I want is a daemon that listens on a socket, but I'm not really experienced with this and don't know where to start getting it configured or setting up Flask.
If you think some framework other than Flask would be easier, definitely let me know, I'm not tied to Flask, I've just read that it's pretty easy to set up!
Thank you very much!!
Here is a (very slightly simplified) example of handling file uploading in web.py based on a cook book example (the Flash example, which I have less experience with, looks even easier):
import web
urls = ('/', 'Upload')
class Upload:
def GET(self):
web.header("Content-Type","text/html; charset=utf-8")
return """
<form method="POST" enctype="multipart/form-data" action="">
<input type="file" name="myfile" />
<br/>
<input type="submit" />
"""
def POST(self):
x = web.input(myfile={})
filedir = '/uploads' # change this to the directory you want to store the file in.
if 'myfile' in x: # to check if the file-object is created
filepath=x.myfile.filename.replace('\\','/') # replaces the windows-style slashes with linux ones.
filename=filepath.split('/')[-1] # splits the and chooses the last part (the filename with extension)
fout = open(filedir +'/'+ filename,'wb') # creates the file where the uploaded file should be stored
fout.write(x.myfile.file.read()) # writes the uploaded file to the newly created file.
fout.close() # closes the file, upload complete.
raise web.seeother('/')
if __name__ == "__main__":
app = web.application(urls, globals())
app.run()
This renders a upload form, and then (on POST) reads the uploaded file and saves it to a designated path.
I have created a test form which will ask users to enter a name and upload the image file:
<html lang="en">
<head>
<title>Testing image upload</title>
</head>
<body>
<form action="/services/upload" method="POST" enctype="multipart/form-data">
File Description: <input name='fdesc' type='text'><br>
File name: <input type="file" name="fname"><br>
<div><input type="submit"></div>
</form>
</body>
</html>
i need to get the file uploaded by the user and store it on my local PC. can this be done in python ? please let me know.
mod_python includes the FieldStorage class which allows you access to uploaded form data. In order to use it, you'd put something like the following in your Python script:
req.form = FieldStorage(req)
description = req.form['fdesc']
Since fdesc is a text input, description will be a string (more precisely, a StringField, which you can treat as a string).
file_field = req.form['fname']
Since fname is a file input, file_field will not be a string (or StringField), but rather a Field object which allows you access to the file data. The attribute file_field.file is a file-like object which you can use to read the file's contents, for example like so:
for line in file_field.file:
# process the line
You could use this to copy the file's data somewhere of your choosing, for example.
file_field.filename is the name of the file as provided by the client. Other useful attributes are listed in the documentation I linked to.
Maybie the minimal http cgi upload recipe and it's comments are helpful for you.
Hey David i got it working, i did it this way:
filename = request.FILES['fname']
destination = open('%s/%s'%(/tmp/,fileName), 'wb+')
for chunk in filename.chunks():
destination.write(chunk)
destination.close()
file = open('%s/%s'%(/tmp/,fileName),"rb").read()
Thanks for the help guys.