I have some code which is supposed to render a file from svn into an email
command = 'svn cat -r {} "{}{}"'.format(svn_revision, svn_repo, filename)
file_content = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE).stdout.read()
context = {
'file_content': file_content,
}
print file_content
email = get_template('file_email/file_email.txt').render(Context(context))
print email
The email template is (for now) just:
{{file_content|safe}}
The first print statement dumps the file as expected.
The second print statement prints nothing.
The template is rendered, if I put words around the tag then they show up.
It only happens for some files (others work fine) and it seems to be an encoding problem. Notepad++ believes the offending files are encoded "UCS-2 Little Endian" whereas the ones that work are "UTF-8 without BOM".
How can I reliably dump a file, regardless of encoding, to a template?
Related
The following python code successfully fills out a pdf form:
import pypdftk
data_dict = {key:value pairs}
PDF_PATH = 'form.pdf' #form to be filled out in same folder as the file executing this code
out_file = 'out_file.pdf' #completed pdf file
generated_pdf = pypdftk.fill_form(
pdf_path = PDF_PATH,
datas = data_dict,
out_file = out_file,
)
However, the same code used in my django project results in the following error message:
Error: Unable to find file.
Error: Failed to open PDF file:
form.pdf
Errors encountered. No output created.
Done. Input errors, so no output created.
... REMAINDER OF TRACEBACK EXCLUDED FOR BREVITY IF YOU WANT TO SEE IT I'LL POST...
raise subprocess.CalledProcessError(retcode, cmd, output=output) output=output) df fill_form C:\Users\Home\AppData\Local\Temp\tmpbqq__7c4 output out_file.pdf flatten
subprocess.CalledProcessError: Command 'pdftk l_a_r.pdf fill_form C:\Users\Home\AppData\Local\Temp\tmpbqq_0 87495_7c4 output out_file.pdf flatten'
returned non-zero exit status 1.
pypdftk is installed in the virtual environment the project is running in.
The pdftk server is added as a windows path variable.
In the above example, and every other time this has happened the temp file referenced at the end of the error message contains all of the expected data in XML.
I've tried the following combinations of code to try to make this work:
Running the exact above code within a view function, with the pdf form to be filled in the same folder as the views.py file:
import pypdftk
def filler_view(request):
form = MyForm()
if request.method =='POST':
#code to successfully populate dictionary data_dict with form data
PDF_PATH = 'form.pdf' #form to be filled out in same folder as the file executing this code
out_file = 'out_file.pdf #completed pdf file
generated_pdf = pypdftk.fill_form(
pdf_path = PDF_PATH,
datas = data_dict,
out_file = out_file,
)
return render(request, 'success.html')
Storing the code and file in a folder and importing to call the relevant function within the view:
-appFolder
-pfd_filler_folder
-form.pdf
-form_filler.py
-views.py
views.py
from appFolder.pdf_filler_folder import form_filler as f
def filler_view(request):
form = MyForm()
if request.method =='POST':
#code to successfully populate dictionary data_dict with form data
f.fill_form(data_dict, 'output.pdf')
form_filler.py:
import pypdftk
def fill_form(data_dict, out_file):
PDF_PATH = 'form.pdf'
generated_pdf = pypdftk.fill_form(
pdf_path = PDF_PATH,
datas = data_dict,
out_file = out_file,
)
Running both of the above with the full path from c:\... of the form.pdf file.
I've also verified that I can successfully fill a form with the executing .py file and the form.pdf file in same folder on two storage drives and from within the django project itself, when not being executed by the django project. pdftk finds the forms.py with no problems at all in this circumstance.
I believe that the file not found error message is key, as it seems to refer to the pdf form I'm trying to fill out. I've spent from 1500 till 1800 researching this, and I haven't managed to get it to work, although I am lead to believe that my error message indicates a missing parameter in the cl execution command. I'm not sure what this would be, as all parameters seem present and correct.
Interestingly enough, a friend of mine is experiencing the same error message just in windows. I'm aware that pdftk can sometimes be touchy in windows, and I think there's probably a nuance I'm missing here.
The outcome I'd like is to fill out a pdf form from within my django project, with data obtained from a form through a post request.
I'd welcome either someone enlightening me as to why pdftk is struggling to either see or use the form file whilst being used from within my django project and pointing me in the right direction
I'm aware that there are alternatives to using pdftk, but pdftk is the simplest, and honestly pypdftk is the only library I've found to reliably work with python to fill out pdf forms so far in Windows. I don't want to go down the route of generating my own replica form and populating it with data, but I'm aware that that is also an option.
Question answered just now on Reddit:
When in Django, it is either wsgi.py or manage.py which is ultimately responsible for what goes on. On that basis, placing the form.pdf file in the same folder as wsgy.py solved the problem and the code now runs as intended, with an unbound form POSTing data back to a view, and a pdf form being filled out and a duplicate saved with said data. Hope that helps anyone else who comes up against this!
I have a python server for mp3 streaming. Using Twisted Matrix library.
If I try to access the file a.mp3 it works normally.
But this file, for example 衝.mp3 doesn't work, it says "File not found".
This file name as URL escaped is %E8%A1%9D.mp3 but it can't access it.
If I try to access it using unicode instead of the symbol, like this \u885d.mp3 it still says "File not found".
Here is the code, notice that I had to put request.path = request.path.replace('%20', ' ') because that's the only way it can access a file that has spaces in the path. That shouldn't be the normal behaviour I believe.
class playMP3(Resource):
isLeaf = True
def render_GET(self, request):
this=urlparse.urlparse(request.path)#scheme,netloc,path,query
root,ext=os.path.splitext(this.path)
filename=os.path.basename(request.path)
fileFolder=request.path.replace(filename,"")
self.serverRoot=os.getcwd()
print (request.path)
if ext==".mp3":
request.path = request.path.replace('%20', ' ')
thisFile=File(self.serverRoot+request.path)
return File.render_GET(thisFile,request)
resource = playMP3()
factory = Site(resource)
reactor.listenTCP(8880, factory)
reactor.run()
I also tried to put request.path = urllib.unquote(request.path) but instead of decoding it to 衝.mp3 it becomes ÞíØ.mp3. Weird.
I am reading text from two different .txt files and concatenating them together. Then add that to a body of the email through by using webbrowser.
One text file is English characters (ascii) and the other Japanese (UTF-8). The text will display fine if I write it to a text file. But if I use webbrowser to insert the text into an email body the Japanese text displays as question marks.
I have tried running the script on multiple machines that have different mail clients as their defaults. Initially I thought maybe that was the issue, but that does not appear to be. Thunderbird and Mail (MacOSX) display question marks.
Hello. Today is 2014-05-09
????????????????2014-05-09????
I have looked at similar issues around on SO but they have not solved the issue.
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in
position 20: ordinal not in
range(128)
Japanese in python function
Printing out Japanese (Chinese) characters
python utf-8 japanese
Is there a way to have the Japanese (UTF-8) display in the body of an email created with webbrowser in python? I could use the email functionality but the requirement is the script needs to open the default mail client and insert all the information.
The code and text files I am using are below. I have simplified it to focus on the issue.
email-template.txt
Hello. Today is {{date}}
email-template-jp.txt
こんにちは。今日は {{date}} です。
Python Script
#
# -*- coding: utf-8 -*-
#
import sys
import re
import os
import glob
import webbrowser
import codecs,sys
sys.stdout = codecs.getwriter('utf8')(sys.stdout)
# vars
date_range = sys.argv[1:][0]
email_template_en = "email-template.txt"
email_template_jp = "email-template-jp.txt"
email_to_send = "email-to-send.txt" # finished email is saved here
# Default values for the composed email that will be opened
mail_list = "test#test.com"
cc_list = "test1#test.com, test2#test.com"
subject = "Email Subject"
# Open email templates and insert the date from the parameters sent in
try:
f_en = open(email_template_en, "r")
f_jp = codecs.open(email_template_jp, "r", "UTF-8")
try:
email_content_en = f_en.read()
email_content_jp = f_jp.read()
email_en = re.sub(r'{{date}}', date_range, email_content_en)
email_jp = re.sub(r'{{date}}', date_range, email_content_jp).encode("UTF-8")
# this throws an error
# UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 26: ordinal not in range(128)
# email_en_jp = (email_en + email_jp).encode("UTF-8")
email_en_jp = (email_en + email_jp)
finally:
f_en.close()
f_jp.close()
pass
except Exception, e:
raise e
# Open the default mail client and fill in all the information
try:
f = open(email_to_send, "w")
try:
f.write(email_en_jp)
# Does not send Japanese text to the mail client. But will write to the .txt file fine. Unsure why.
webbrowser.open("mailto:%s?subject=%s&cc=%s&body=%s" %(mail_list, subject, cc_list, email_en_jp), new=1) # open mail client with prefilled info
finally:
f.close()
pass
except Exception, e:
raise e
edit: Forgot to add I am using Python 2.7.1
EDIT 2: Found a workable solution after all.
Replace your webbrowser call with this.
import subprocess
[... other code ...]
arg = "mailto:%s?subject=%s&cc=%s&body=%s" % (mail_list, subject, cc_list, email_en_jp)
subprocess.call(["open", arg])
This will open your default email client on MacOS. For other OSes please replace "open" in the subprocess line with the proper executable.
EDIT: I looked into it a bit more and Mark's comment above made me read the RFC (2368) for mailto URL scheme.
The special hname "body" indicates that the associated hvalue is the
body of the message. The "body" hname should contain the content for
the first text/plain body part of the message. The mailto URL is
primarily intended for generation of short text messages that are
actually the content of automatic processing (such as "subscribe"
messages for mailing lists), not general MIME bodies.
And a bit further down:
8-bit characters in mailto URLs are forbidden. MIME encoded words (as
defined in [RFC2047]) are permitted in header values, but not for any
part of a "body" hname."
So it looks like this is not possible as per RFC, although that makes me question why the JavaScript solution in the JSFiddle provided by naota works at all.
I leave my previous answer as is below, although it does not work.
I have run into same issues with Python 2.7.x quite a couple of times now and every time a different solution somehow worked.
So here are several suggestions that may or may not work, as I haven't tested them.
a) Force unicode strings:
webbrowser.open(u"mailto:%s?subject=%s&cc=%s&body=%s" % (mail_list, subject, cc_list, email_en_jp), new=1)
Notice the small u right after the opening ( and before the ".
b) Force the regex to use unicode:
email_jp = re.sub(ur'{{date}}', date_range, email_content_jp).encode("UTF-8")
# or maybe
email_jp = re.sub(ur'{{date}}', date_range, email_content_jp)
c) Another idea regarding the regex, try compiling it first with the re.UNICODE flag, before applying it.
pattern = re.compile(ur'{{date}}', re.UNICODE)
d) Not directly related, but I noticed you write the combined text via the normal open method. Try using the codecs.open here as well.
f = codecs.open(email_to_send, "w", "UTF-8")
Hope this helps.
I have the following code for managing file download through django.
def serve_file(request, id):
file = models.X.objects.get(id=id).file #FileField
file.open('rb')
wrapper = FileWrapper(file)
mt = mimetypes.guess_type(file.name)[0]
response = HttpResponse(wrapper, content_type=mt)
import unicodedata, os.path
filename = unicodedata.normalize('NFKD', os.path.basename(file.name)).encode("utf8",'ignore')
filename = filename.replace(' ', '-') #Avoid browser to ignore any char after the space
response['Content-Length'] = file.size
response['Content-Disposition'] = 'attachment; filename={0}'.format(filename)
#print response
return response
Unfortunately, my browser get an empty file when downloading.
The printed response seems correct:
Content-Length: 3906
Content-Type: text/plain
Content-Disposition: attachment; filename=toto.txt
blah blah ....
I have similar code running ok. I don't see what can be the problem. Any idea?
PS: I have tested the solution proposed here and get the same behavior
Update:
Replacing wrapper = FileWrapper(file) by wrapper = file.read() seems to fix the problem
Update: If I comment the print response, I get similar issue:. the file is empty. Only difference: FF detects a 20bytes size. (the file is bigger than this)
File object is an interable, and a generator. It can be read only once before being exausted. Then you have to make a new one, of use a method to start at the begining of the object again (e.g: seek()).
read() returns a string, which can be read multiple times without any problem, this is why it solves your issue.
So just make sure that if you use a file like object, you don't read it twice in a row. E.G: don't print it, then returns it.
From django documentation:
FieldFile.open(mode='rb') Behaves like the standard Python open()
method and opens the file associated with this instance in the mode
specified by mode.
If it works like pythons open then it should return a file-object, and should be used like this:
f = file.open('rb')
wrapper = FileWrapper(f)
Greetings all,
I have a python CGI script which using
print "Location: [nextfilename]"
print
After "forwarding" (the reason for the quotes there is apparent in a second), I see the HTML of the page to which it has forwarded fine, but all images, etc. are not showing. The address bar still shows the cgi script as the current location, not the HTML file itself. If I go to the HTML file directly, it displays fine.
Basically, the CGI script, which is stored in the cgi-bin, whereas the HTML files are not, is trying to render images with relational links that are broken.
How do I actually forward to the next page, not just render the next page through the cgi script?
I have gone through the script with a fine-toothed comb to make sure that i wasn't actually using a print htmlDoc command anywhere that would be interrupting and screwing this up.
Sections of Code that are Applicable:
def get_nextStepName():
"""Generates a random filename."""
nextStepBuilder = ["../htdocs/bcc/"]
fileLength = random.randrange(10)+5
for i in range(fileLength):
j = random.choice(varLists.ALPHANUM)
nextStepBuilder.append(j)
nextStepName = ""
for char in nextStepBuilder:
nextStepName += char
nextStepName += ".html"
return nextStepName
def make_step2(user, password, email, headerContent, mainContent, sideSetup, sideContent, footerContent):
"""Creates the next step of user registration and logs valid data to a pickle for later confirmation."""
nextStepName = get_nextStepName()
mainContent = "<h1>Step Two: The Nitty Gritty</h1>"
mainContent += "<p>User Name: %s </p>" % (user)
mainContent += """\
[HTML CODE GOES HERE]
"""
htmlDoc = htmlGlue.glue(headerContent, mainContent, sideSetup, sideContent, footerContent)
f = open(nextStepName, "w")
f.write(htmlDoc)
f.close()
nextStepName = nextStepName[9:] #truncates the ../htdocs part of the filename to fix a relational link issue when redirecting
gotoNext(nextStepName)
def gotoNext(filename):
nextLocation = "Location:"
nextLocation += filename
print(nextLocation)
print
Any thoughts? Thanks a ton. CGI is new to me.
You need to send a 30X Status header as well. See RFC 2616 for details.