We Keep Coding

How to take txt file as input into python (flask)

Apologies in advance if this is easy but I cant figure it out.
I have a flask app which takes a txt file as an input on a webpage. I then want to run some python scripts on it and then export it back to the website user. I'm pretty much there but I just cant figure out how to get the txt file into python.
HTML
<div class="file_input">
<form action="." method="POST">
<input type="file" name = "txt_file">
<input type="submit" name="submit_form">
</form>
</div>
Python (Flask)
#app.route('/landing')
def landing():
return render_template('index.html',
PageTitle = "Landing page")
#app.route('/', methods = ['POST'])
def test():
text_file = request.form['txt_file']
book_io = open(text_file, "r", encoding = "UTF-8")
book_str = book_io.read()
book_str
return book_str
This isn't working as it's just taking the inputted text file as a string and then printing out this string.
I basically need to know how to import the text from the file into python, but I cant figure out how.
Thanks a lot in advance!!

Exporting TextField in Django - Python model to PDF using ReportLab - line break becomes unprintable characters

I have a model attribute that saves user's HTML Textarea input. Because when the user hit enter in the Textarea, the form submitted, I have written a small Javascript code to input the "\n" automatically, rendering line break.
When I try to export this attribute to PDF using ReportLab, I receive non-printable characters that looks like 2 black rectangles. When I try to copy them to textpad, it becomes "n".
here is the code from the model
class Package(models.Model):
content = models.TextField()
here is the JavaScript and Html for the specific field:
<form action="" role="form" method="post">{% csrf_token %}
<textarea class="form-control" autofocus name="content">{{content}}</textarea>
<br/>
</form>
<script type="text/javascript">
$('textarea').keypress(function(event) {
if (event.which == 13) {
event.preventDefault();
var s = $(this).val();
$(this).val(s+"\n");
}
});​
</script>
This is the Reportlab code from views.py:
from reportlab.pdfgen import canvas
from reportlab.platypus import Spacer
from reportlab.lib.pagesizes import letter
def create(pdf):
packet = StringIO.StringIO()
can = canvas.Canvas(packet, pagesize=letter)
p_content = str(package.content)
textobject = can.beginText(0.5*inch, 4.75*inch)
for line in p_content:
textobject.textOut(line)
if line=="\n":
textobject.textLines(p_content)
can.save()
Please help. I cannot seem to figure this out.
Many thanks.

I have received outside help so I will just post it here for anyone who is interested.
The problem is this line of codes:
for line in p_content:
textobject.textOut(line)
if line=="\n":
textobject.textLines(p_content)
can.save()
The think is i PRINT out before I do a check of the escape character. The correct code should be
for c in p_content:
if c == '\n':
textobject.textLine()
elif c == '\r':
pass # do nothing
else:
textobject.textOut(c)

Reading a file uploaded via CGI in Python

I am having trouble analyzing a file that is uploaded to my server. Ideally, what would happen is that the user would upload a .csv and I would run some numeric integration (hence the import numpy as np) from data in the file and return the result. To test the cgi part before I did any analysis, I made the python script below. My problem is that message always appears blank, so I get a blank html page displayed in browser at the end.
Clearly I am not reading the file correctly (which means I can not analyze it), but I have n o idea what I am doing wrong. My own searches indicate that what I have below should work.
#!/usr/bin/python
#---------------------------------------------
#=============================================
#---------------------------------------------
#imports
#import csv
#import time as tm
#import numpy as np
#import os
import cgi, cgitb
cgitb.enable()
form = cgi.FieldStorage()
#The variables
#httpopen=""
#httpclose=""
message="meow"
#get the fileitem
fileitem=form['userfile']
if fileitem.file:
#yay...we got a file
message=fileitem.file.readline()
print """\
Content-Type: text/html\n
<html><body>
<p>%s</p>
</body></html>
""" % (message,)
And here is the html form that gives the upload:
<html>
<body>
<form enctype="multipart/form-data"
action="capacity_rewrite.py" method="post">
<p>File: <input type="file" name="userfile" /></p>
<p><input type="submit" value="Upload" /></p>
</form>
</body>
</html>
Thanks,

You're only reading one line from the uploaded file. Perhaps that first line is a blank line? Try fileitem.file.read() instead, which will read the entire file into a string. Don't do that for large files as you may run out of memory, but it should help in understanding your test.

How to properly decode Quoted Printable encoding in Django HTML Template

I have a Google app engine in python form submit that POSTS text to a server, and the text gets encoded with the encoding Quoted Printables.
My code for POSTing is this:
<form action={{ upload_url }} method="post" enctype="multipart/form-data">
<div class="sigle-form"><textarea name="body" rows="5"></textarea></div>
<div class="sigle-form"><input name="file" type="file" /></div>
</form>
Then the result of the fetching self.request.get('body') will be encoded with the encoding Quoted Printables. I store this in text DB.textProperty() and later sends the text to a HTML template using Django. When i write out the variable using {{ body }}, the result is written with Quoted printable encoding, and it does not seem that there is a way of decoding this in the Django HTML template.
Is there any way of encoding the text in the body thats sent on another way than with Quoted Printables? If not, how to decode this encoding in the Django HTML template?
The result for submiting the text "ÅØÆ" is encoded to " xdjG ", so the sum of the Quoted Prinables are somehow added togheter as well. This happens when more than one special character are present in the encoded text. An ordinary "ø" is encoded to =F8.
EDIT: I get this problem only in production, and this thread seems to talk about the same problem.
If anyone else here on Stack Overflow are doing form submit with blobs and åæøè characters, please respond to this thread on how you have solved it!

Ok, after two days working with this issue i finally resolved it. Its seemingly a bug with Google App Engine that makes the encoding f'ed up in production. When in production the text is sometimes encoded with Quoted Printable encoded, and some other times encoded with base64 encoding. Weird. Here is my solution:
postBody = self.request.get('body')
postBody = postBody.encode('iso-8859-1')
DEBUG = os.environ['SERVER_SOFTWARE'].startswith('Dev')
if DEBUG:
r.body = postBody
else:
postBody += "=" * ((4 - len(postBody) % 4) % 4)
b64 = base64.urlsafe_b64decode(postBody)
Though the resulting b64 can't be stored in the data storage because it's not ascii encoded
'ascii' codec can't decode byte 0xe5 in position 5: ordinal not in range(128)

I solved a similar problem by using the Python quopri module to decode the string before passing it to an HTML template.
import quopri
body = quopri.decodestring(body)
This seems to be something to do with the multipart/form-data enctype. Quotable printable encoding is applied to the textarea input, which is then, in my case, submitted via a blobstore upload link. The blobstore returns the text to my upload handler still in encoded form.

Not sure what Quoted Printables are but have you tried safe?
{{ body|safe }}
https://docs.djangoproject.com/en/dev/ref/templates/builtins/?from=olddocs#safe

Problems with Python in Google App Engine - UTF-8 and ASCII

So for the past few days I've been trying to learn Python in App Engine. However, I've been encountering a number of problems with ASCII and UTF encoding. The freshest issue is as follows:
I have the following piece of code of a simplistic chatroom from the book 'Code in the Cloud'
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
import datetime
# START: MainPage
class ChatMessage(object):
def __init__(self, user, msg):
self.user = user
self.message = msg
self.time = datetime.datetime.now()
def __str__(self):
return "%s (%s): %s" % (self.user, self.time, self.message)
Messages = []
class ChatRoomPage(webapp.RequestHandler):
def get(self):
self.response.headers["Content-Type"] = "text/html"
self.response.out.write("""
<html>
<head>
<title>MarkCC's AppEngine Chat Room</title>
</head>
<body>
<h1>Welcome to MarkCC's AppEngine Chat Room</h1>
<p>(Current time is %s)</p>
""" % (datetime.datetime.now()))
# Output the set of chat messages
global Messages
for msg in Messages:
self.response.out.write("<p>%s</p>" % msg)
self.response.out.write("""
<form action="" method="post">
<div><b>Name:</b>
<textarea name="name" rows="1" cols="20"></textarea></div>
<p><b>Message</b></p>
<div><textarea name="message" rows="5" cols="60"></textarea></div>
<div><input type="submit" value="Send ChatMessage"></input></div>
</form>
</body>
</html>
""")
# END: MainPage
# START: PostHandler
def post(self):
chatter = self.request.get("name")
msg = self.request.get("message")
global Messages
Messages.append(ChatMessage(chatter, msg))
# Now that we've added the message to the chat, we'll redirect
# to the root page, which will make the user's browser refresh to
# show the chat including their new message.
self.redirect('/')
# END: PostHandler
# START: Frame
chatapp = webapp.WSGIApplication([('/', ChatRoomPage)])
def main():
run_wsgi_app(chatapp)
if __name__ == "__main__":
main()
# END: Frame
It works ok in English. However, the moment I add some non-standard characters all sorts of problems start
First of all, in order for the thing to be actually able to display characters in HTML I add meta tag - charset=UTF-8" etc
Curiously, if you enter non-standard letters, the program processes them nicely, and displays them with no issues. However, it fails to load if I enter any non-ascii letters to the web layout iteself withing the script. I figured out that adding utf-8 encoding line would work. So I added (# -- coding: utf-8 --). This was not enough. Of course I forgot to save the file in UTF-8 format. Upon that the program started running.
That would be the good end to the story, alas....
It doesn't work
Long story short this code:
# -*- coding: utf-8 -*-
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
import datetime
# START: MainPage
class ChatMessage(object):
def __init__(self, user, msg):
self.user = user
self.message = msg
self.time = datetime.datetime.now()
def __str__(self):
return "%s (%s): %s" % (self.user, self.time, self.message)
Messages = []
class ChatRoomPage(webapp.RequestHandler):
def get(self):
self.response.headers["Content-Type"] = "text/html"
self.response.out.write("""
<html>
<head>
<title>Witaj w pokoju czatu MarkCC w App Engine</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<h1>Witaj w pokoju czatu MarkCC w App Engine</h1>
<p>(Dokladny czas Twojego logowania to: %s)</p>
""" % (datetime.datetime.now()))
# Output the set of chat messages
global Messages
for msg in Messages:
self.response.out.write("<p>%s</p>" % msg)
self.response.out.write("""
<form action="" method="post">
<div><b>Twój Nick:</b>
<textarea name="name" rows="1" cols="20"></textarea></div>
<p><b>Twoja Wiadomość</b></p>
<div><textarea name="message" rows="5" cols="60"></textarea></div>
<div><input type="submit" value="Send ChatMessage"></input></div>
</form>
</body>
</html>
""")
# END: MainPage
# START: PostHandler
def post(self):
chatter = self.request.get(u"name")
msg = self.request.get(u"message")
global Messages
Messages.append(ChatMessage(chatter, msg))
# Now that we've added the message to the chat, we'll redirect
# to the root page, which will make the user's browser refresh to
# show the chat including their new message.
self.redirect('/')
# END: PostHandler
# START: Frame
chatapp = webapp.WSGIApplication([('/', ChatRoomPage)])
def main():
run_wsgi_app(chatapp)
if __name__ == "__main__":
main()
# END: Frame
Fails to process anything I write in the chat application when it's running. It loads but the moment I enter my message (even using only standard characters) I receive
File "D:\Python25\lib\StringIO.py", line 270, in getvalue
self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 64: ordinal not in range(128)
error message. In other words, if I want to be able to use any characters within the application I cannot put non-English ones in my interface. Or the other way round, I can use non-English characters within the app only if I don't encode the file in utf-8. How to make it all work together?

Your strings contain unicode characters, but they're not unicode strings, they're byte strings. You need to prefix each one with u (as in u"foo") in order to make them into unicode strings. If you ensure all your strings are Unicode strings, you should eliminate that error.
You should also specify the encoding in the Content-Type header rather than a meta tag, like this:
self.response.headers['Content-Type'] = 'text/html; charset=UTF-8'
Note your life would be a lot easier if you used a templating system instead of writing HTML inline with your Python code.

#Thomas K.
Thank you for your guidance here. Thanks to you I was able to come up with, maybe - as you said - a little roudabout solution - so the credit for the answer should go to you. The following line of code:
Messages.append(ChatMessage(chatter, msg))
Should look like this:
Messages.append(ChatMessage(chatter.encode( "utf-8" ), msg.encode( "utf-8" )))
Basically I have to encode all the utf-8 string to ascii.