Need some advice to use rebase and/or include.
For building a flexible concept with a variable menu system I need to insert a 'menuY.tpl' template into different 'mainX.tpl' pages.
Sounds easy but not only the pages need
thisTemplate = template('mainX', keys)
but also the menus need changing menukeys
thisMenu = template('menuY', menukeys)
How to define the different instructions?
python
#app.route('/doit')
def week():
...some code here to load keys etc ...
thisTemplate = template('mainX', keys)
return thisTemplate
mainX.tpl with
<body>
% insert ('menuY', rv)
<section class="container">
<p>{{param1}}</p>
some html code for the main page
</section>
</body>
menuY.tpl with just html code for the menu code like this
<div id="hambgMenu">
Home - {{titleY}}
{{titleZ}}
</div>
This will not work, at the mainX.tpl line with % insert python says:
NameError: name 'insert' is not defined
Also how are the variables (titleY,titleZ) passed to that 'menuY'? There is no reference for 'rv' with the coding above.
The solution was described here How to pass html directly to template ... very easy, just to add ! to the template reference.
I did some further steps just to have on Python:
#app.route('/doit')
def doit():
page = insertTPL('mainX', keys, 'menuY', menukeys, 'menuTag')
return page
.. with menuTag as declared as follows:
So mainX.tpl becomes
<body>
{{!menuTag}}
<section class="container">
<p>{{param1}}</p>
some html code for the main page
</section>
</body>
The mentioned insertTPL python function has:
def insertTPL(tpl, keys, menu, menukeys, menuTag):
subtpl = template(menu, menukeys)
rv = combineKeys(keys, {menuTag:subtpl}) # combine the menu code with other keys!
return template(tpl, rv)
def combineKeys(rv1, rv2):
try:
keys = {key: value for (key, value) in (rv1.items() + rv2.items())}
except:
keys = rv1
return keys
I am trying to design and implement a basic calculator in HTML and Python(using CGI). Below given is a static HTML web page and it is being redirected to a python script (calci.py) where, I am able to calculate the sum but unable to append the resultant to the 'output' textbox.
calculator.html
<html>
<head>
<title>Calculator</title>
</head>
<body>
<form action="python_scripts/calci.py" method="post">
Input 1 : <input type="text" name="input1"/><br>
Input 2 : <input type="text" name="input2"/><br>
<input type="submit" value="+" title="add" /><br>
output : <input type="text" name="output"/><br>
</form>
</body>
</html>
calci.py
import cgi
form = cgi.FieldStorage()
input1 = form.getvalue('input1')
input2 = form.getvalue('input2')
output = str(int(input1)+int(input2))
#how to return the response to the html
#and append it to the textbox
Thanks
This is not the way Web applications work - not hat simply, at least.
If you want to rely only on the browser, and plain HTML for your application, each request has to send the whole html page as a string. You have to use Python's string formatting capabilities to put the resulting number in the correct place in the HTML form.
This way of working is typical of "Web 1.0" applications (as opposed to the "Web 2.0" term used about ten years ago).
Modern web applications use logic that runs on the client side, in Javascript code, to make an HTTP request to retrieve only the needed data - and them, this client-side logic would place your result in the proper place in the page, without reloading the page. This is what isgenerally known as "ajax". It is not that complex, but the html + javascript side of the application become much more complex.
I think one should really understand the "Web 1.0" way before doing it the "Ajax way". in your case, let's suppose your HTML containing the calculator form is in a file called "calc.html". Inside it, where the result should lie, put a markup that can be understood by Python's built-in native string formatting methods, like {result} -
<html>
<body>
...
calculator body
...
Answer: <input type="text" readonly="true" value={result} />
</body>
</html>
And rewrite your code like:
import cgi
form = cgi.FieldStorage()
input1 = form.getvalue('input1')
input2 = form.getvalue('input2')
result = int(input1)+int(input2)
html = open("calc.html".read())
header = "Content-Type: text/html; charset=UTF-8\n\n"
output = header + html.format(result=result)
print (output)
The CGI way is outdated, but is nice for learning: it relies on your whole program being run, and whatever it prints to the standard output to be redirected to the HTTP request as a response. That includes the HTTP Headers, which are included, in a minimal form, above.
(I will leave the complete implementation of a way for the raw '{result}' string not to show up in the inital calculator form as an exercise from where you are 0- the path is to get the initial calculator html template through a CGI script as well, instead of statically, (maybe the same) as well - and just populate "result" with "0" or an empty string)
you can transfer response with the help of java script.
use under print("window.location=url")
I am trying to work with forms on python and I have 2 problems which I can't decide for a lot of time.
First if I leave the text field empty, it will give me an error. url like this:
http://localhost/cgi-bin/sayHi.py?userName=
I tried a lot of variants like try except, if username in global or local and ect but no result, equivalent in php if (isset(var)). I just want to give a message like 'Fill a form' to user if he left the input empty but pressed button Submit.
Second I want to leave what was printed on input field after submit(like search form). In php it's quite easy to do it but I can't get how to do it python
Here is my test file with it
#!/usr/bin/python
import cgi
print "Content-type: text/html \n\n"
print """
<!DOCTYPE html >
<body>
<form action = "sayHi.py" method = "get">
<p>Your name?</p>
<input type = "text" name = "userName" /> <br>
Red<input type="checkbox" name="color" value="red">
Green<input type="checkbox" name="color" value="green">
<input type = "submit" />
</form>
</body>
</html>
"""
form = cgi.FieldStorage()
userName = form["userName"].value
userName = form.getfirst('userName', 'empty')
userName = cgi.escape(userName)
colors = form.getlist('color')
print "<h1>Hi there, %s!</h1>" % userName
print 'The colors list:'
for color in colors:
print '<p>', cgi.escape(color), '</p>'
On the cgi documentation page are these words:
The FieldStorage instance can be indexed like a Python dictionary. It allows membership testing with the in operator
One way to get what you want is to use the in operator, like so:
form = cgi.FieldStorage()
if "userName" in form:
print "<h1>Hi there, %s!</h1>" % cgi.escape(form["userName"].value)
From the same page:
The value attribute of the instance yields the string value of the field. The getvalue() method returns this string value directly; it also accepts an optional second argument as a default to return if the requested key is not present.
A second solution for you might be:
print "<h1>Hi there, %s!</h1>" % cgi.escape(form.getvalue("userName","Nobody"))
How do I insert a variable into an HTML email I'm sending with python? The variable I'm trying to send is code. Below is what I have so far.
text = "We Says Thanks!"
html = """\
<html>
<head></head>
<body>
<p>Thank you for being a loyal customer.<br>
Here is your unique code to unlock exclusive content:<br>
<br><br><h1><% print code %></h1><br>
<img src="http://example.com/footer.jpg">
</p>
</body>
</html>
"""
Use "formatstring".format:
code = "We Says Thanks!"
html = """\
<html>
<head></head>
<body>
<p>Thank you for being a loyal customer.<br>
Here is your unique code to unlock exclusive content:<br>
<br><br><h1>{code}</h1><br>
<img src="http://example.com/footer.jpg">
</p>
</body>
</html>
""".format(code=code)
If you find yourself substituting a large number of variables, you can use
.format(**locals())
Another way is to use Templates:
>>> from string import Template
>>> html = '''\
<html>
<head></head>
<body>
<p>Thank you for being a loyal customer.<br>
Here is your unique code to unlock exclusive content:<br>
<br><br><h1>$code</h1><br>
<img src="http://example.com/footer.jpg">
</p>
</body>
</html>
'''
>>> s = Template(html).safe_substitute(code="We Says Thanks!")
>>> print(s)
<html>
<head></head>
<body>
<p>Thank you for being a loyal customer.<br>
Here is your unique code to unlock exclusive content:<br>
<br><br><h1>We Says Thanks!</h1><br>
<img src="http://example.com/footer.jpg">
</p>
</body>
</html>
Note, that I used safe_substitute, not substitute, as if there is a placeholder which is not in the dictionary provided, substitute will raise ValueError: Invalid placeholder in string. The same problem is with string formatting.
use pythons string manipulation:
http://docs.python.org/2/library/stdtypes.html#string-formatting
generally the % operator is used to put a variable into a string, %i for integers, %s for strings and %f for floats,
NB: there is also another formatting type (.format) which is also described in the above link, that allows you to pass in a dict or list slightly more elegant than what I show below, this may be what you should go for in the long run as the % operator gets confusing if you have 100 variables you want to put into a string, though the use of dicts (my last example) kinda negates this.
code_str = "super duper heading"
html = "<h1>%s</h1>" % code_str
# <h1>super duper heading</h1>
code_nr = 42
html = "<h1>%i</h1>" % code_nr
# <h1>42</h1>
html = "<h1>%s %i</h1>" % (code_str, code_nr)
# <h1>super duper heading 42</h1>
html = "%(my_str)s %(my_nr)d" % {"my_str": code_str, "my_nr": code_nr}
# <h1>super duper heading 42</h1>
this is very basic and only work with primitive types, if you want to be able to store dicts, lists and possible objects I suggest you use cobvert them to jsons http://docs.python.org/2/library/json.html and https://stackoverflow.com/questions/4759634/python-json-tutorial are good sources of inspiration
Hope this helps
I have been through many iterations of this problem so far, searched out many different examples, and have been all through the documentation.
I am trying to combine Plupload (http://www.plupload.com/) with the AWS S3 direct post method (http://aws.amazon.com/articles/1434). However, I believe there's something wrong with the way I am constructing my policy and signature for transmission. When I submit the form, I don't get a response from the server, but rather my connection to the server is reset.
I have attempted using the python code in the example:
import base64
import hmac, sha
policy = base64.b64encode(policy_document)
signature = base64.b64encode(
hmac.new(aws_secret_key, policy, sha).digest())
I have also tried to use the more up-to-date hashlib library in python. Whatever method I use to construct my policy and signature, I always get different values than those generated here:
http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html
I have read through this question:
How do I make Plupload upload directly to Amazon S3?
But I found the examples provided to be overly complicated and wasn't able to accurately implement them.
My most recent attempts have been to use portions of the boto library:
http://boto.cloudhackers.com/ref/s3.html#module-boto.s3.connection
But using the S3Commection.build_post_form_args method has not worked for me either.
If anyone could provide a proper example of how to create the post form using python, I would very much appreciate it. Even some simple insights on why the connection is always reset would be nice.
Some caveats:
I would like to use hashlib if possible.
I want to get an XML response from Amazon (presumably "success_action_status = '201'" does this)
I need to be able to upload largish type files, max size ~2GB.
One final note, when I run this in Chrome, it provides upload progress, and the upload usually fails around 37%.
Nathan's answer helped get me started. I've included two solutions that are currently working for me.
The first solution uses plain Python. The second uses boto.
I tried to get boto working first, but kept getting errors. So I went back to the Amazon ruby documentation and got S3 to accept files using python without boto. (Browser Uploads to S3 using HTML POST)
After understanding what was going on, I was able to fix my errors and use boto, which is a simpler solution.
I'm including solution 1 because it shows explicitly how to setup the policy document and signature using python.
My goal was to create the html upload page as a dynamic page, along with the "success" page the user sees after a successful upload. Solution 1 shows the dynamic creation of the form upload page, while solution 2 shows the creation of both the upload form page and the success page.
Solution 1:
import base64
import hmac, hashlib
###### EDIT ONLY THE FOLLOWING ITEMS ######
DEBUG = 1
AWS_SECRET_KEY = "MySecretKey"
AWS_ACCESS_KEY = "MyAccessKey"
HTML_NAME = "S3PostForm.html"
EXPIRE_DATE = "2015-01-01T00:00:00Z" # Jan 1, 2015 gmt
FILE_TO_UPLOAD = "${filename}"
BUCKET = "media.mysite.com"
KEY = ""
ACL = "public-read" # or "private"
SUCCESS = "http://media.mysite.com/success.html"
CONTENT_TYPE = ""
CONTENT_LENGTH = 1024**3 # One gigabyte
HTTP_OR_HTTPS = "http" # Or "https" for better security
PAGE_TITLE = "My Html Upload to S3 Form"
ACTION = "%s://%s.s3.amazonaws.com/" % (HTTP_OR_HTTPS, BUCKET)
###### DON'T EDIT FROM HERE ON DOWN ######
policy_document_data = {
"expire": EXPIRE_DATE,
"bucket_name": BUCKET,
"key_name": KEY,
"acl_name": ACL,
"success_redirect": SUCCESS,
"content_name": CONTENT_TYPE,
"content_length": CONTENT_LENGTH,
}
policy_document = """
{"expiration": "%(expire)s",
"conditions": [
{"bucket": "%(bucket_name)s"},
["starts-with", "$key", "%(key_name)s"],
{"acl": "%(acl_name)s"},
{"success_action_redirect": "%(success_redirect)s"},
["starts-with", "$Content-Type", "%(content_name)s"],
["content-length-range", 0, %(content_length)d]
]
}
""" % policy_document_data
policy = base64.b64encode(policy_document)
signature = base64.b64encode(hmac.new(AWS_SECRET_KEY, policy, hashlib.sha1).digest())
html_page_data = {
"page_title": PAGE_TITLE,
"action_name": ACTION,
"filename": FILE_TO_UPLOAD,
"access_name": AWS_ACCESS_KEY,
"acl_name": ACL,
"redirect_name": SUCCESS,
"policy_name": policy,
"sig_name": signature,
"content_name": CONTENT_TYPE,
}
html_page = """
<html>
<head>
<title>%(page_title)s</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body>
<form action="%(action_name)s" method="post" enctype="multipart/form-data">
<input type="hidden" name="key" value="%(filename)s">
<input type="hidden" name="AWSAccessKeyId" value="%(access_name)s">
<input type="hidden" name="acl" value="%(acl_name)s">
<input type="hidden" name="success_action_redirect" value="%(redirect_name)s">
<input type="hidden" name="policy" value="%(policy_name)s">
<input type="hidden" name="signature" value="%(sig_name)s">
<input type="hidden" name="Content-Type" value="%(content_name)s">
<!-- Include any additional input fields here -->
Browse to locate the file to upload:<br \> <br \>
<input name="file" type="file"><br> <br \>
<input type="submit" value="Upload File to S3">
</form>
</body>
</html>
""" % html_page_data
with open(HTML_NAME, "wb") as f:
f.write(html_page)
###### Dump output if testing ######
if DEBUG:
if 1: # Set true if not using the LEO editor
class G:
def es(self, data):print(data)
g = G()
items = [
"",
"",
"policy_document: %s" % policy_document,
"ploicy: %s" % policy,
"signature: %s" % signature,
"",
"",
]
for item in items:
g.es(item)
Solution 2:
from boto.s3 import connection
###### EDIT ONLY THE FOLLOWING ITEMS ######
DEBUG = 1
AWS_SECRET_KEY = "MySecretKey"
AWS_ACCESS_KEY = "MyAccessKey"
HTML_NAME = "S3PostForm.html"
SUCCESS_NAME = "success.html"
EXPIRES = 60*60*24*356 # seconds = 1 year
BUCKET = "media.mysite.com"
KEY = "${filename}" # will match file entered by user
ACL = "public-read" # or "private"
SUCCESS = "http://media.mysite.com/success.html"
CONTENT_TYPE = "" # seems to work this way
CONTENT_LENGTH = 1024**3 # One gigabyte
HTTP_OR_HTTPS = "http" # Or https for better security
PAGE_TITLE = "My Html Upload to S3 Form"
###### DON'T EDIT FROM HERE ON DOWN ######
conn = connection.S3Connection(AWS_ACCESS_KEY,AWS_SECRET_KEY)
args = conn.build_post_form_args(
BUCKET,
KEY,
expires_in=EXPIRES,
acl=ACL,
success_action_redirect=SUCCESS,
max_content_length=CONTENT_LENGTH,
http_method=HTTP_OR_HTTPS,
fields=None,
conditions=None,
storage_class='STANDARD',
server_side_encryption=None,
)
form_fields = ""
line = ' <input type="hidden" name="%s" value="%s" >\n'
for item in args['fields']:
new_line = line % (item["name"], item["value"])
form_fields += new_line
html_page_data = {
"page_title": PAGE_TITLE,
"action": args["action"],
"input_fields": form_fields,
}
html_page = """
<html>
<head>
<title>%(page_title)s</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body>
<form action="%(action)s" method="post" enctype="multipart/form-data" >
%(input_fields)s
<!-- Include any additional input fields here -->
Browse to locate the file to upload:<br \> <br \>
<input name="file" type="file"><br> <br \>
<input type="submit" value="Upload File to S3">
</form>
</body>
</html>
""" % html_page_data
with open(HTML_NAME, "wb") as f:
f.write(html_page)
success_page = """
<html>
<head>
<title>S3 POST Success Page</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<script src="jquery.js"></script>
<script src="purl.js"></script>
<!--
Amazon S3 passes three data items in the url of this page if
the upload was successful:
bucket = bucket name
key = file name upload to the bucket
etag = hash of file
The following script parses these values and puts them in
the page to be displayed.
-->
<script type="text/javascript">
var pname,url,val,params=["bucket","key","etag"];
$(document).ready(function()
{
url = $.url();
for (param in params)
{
pname = params[param];
val = url.param(pname);
if(typeof val != 'undefined')
document.getElementById(pname).value = val;
}
});
</script>
</head>
<body>
<div style="margin:0 auto;text-align:center;">
<p>Congratulations!</p>
<p>You have successfully uploaded the file.</p>
<form action="#" method="get"
>Location:
<br />
<input type="text" name="bucket" id="bucket" />
<br />File Name:
<br />
<input type="text" name="key" id="key" />
<br />Hash:
<br />
<input type="text" name="etag" id="etag" />
</form>
</div>
</body>
</html>
"""
with open(SUCCESS_NAME, "wb") as f:
f.write(success_page)
###### Dump output if testing ######
if DEBUG:
if 1: # Set true if not using the LEO editor
class G:
def es(self, data):print(data)
g = G()
g.es("conn = %s" % conn)
for key in args.keys():
if key is not "fields":
g.es("%s: %s" % (key, args[key]))
continue
for item in args['fields']:
g.es(item)
I tried using Boto but found it didn't let me put in all of the headers I wanted. Below you can see what I do to generate the policy, signature, and a dictionary of post form values.
Note that all of the x-amz-meta-* tags are custom header properties and you don't need them. Also notice that pretty much everything that is going to be in the form needs to be in the policy that gets encoded and signed.
def generate_post_form(bucket_name, key, post_key, file_id, file_name, content_type):
import hmac
from hashlib import sha1
from django.conf import settings
policy = """{"expiration": "%(expires)s","conditions": [{"bucket":"%(bucket)s"},["eq","$key","%(key)s"],{"acl":"private"},{"x-amz-meta-content_type":"%(content_type)s"},{"x-amz-meta-file_name":"%(file_name)s"},{"x-amz-meta-post_key":"%(post_key)s"},{"x-amz-meta-file_id":"%(file_id)s"},{"success_action_status":"200"}]}"""
policy = policy%{
"expires":(datetime.utcnow()+settings.TIMEOUT).strftime("%Y-%m-%dT%H:%M:%SZ"), # This has to be formatted this way
"bucket": bucket_name, # the name of your bucket
"key": key, # this is the S3 key where the posted file will be stored
"post_key": post_key, # custom properties begin here
"file_id":file_id,
"file_name": file_name,
"content_type": content_type,
}
encoded = policy.encode('utf-8').encode('base64').replace("\n","") # Here we base64 encode a UTF-8 version of our policy. Make sure there are no new lines, Amazon doesn't like them.
return ("%s://%s.s3.amazonaws.com/"%(settings.HTTP_CONNECTION_TYPE, self.bucket_name),
{"policy":encoded,
"signature":hmac.new(settings.AWS_SECRET_KEY,encoded,sha1).digest().encode("base64").replace("\n",""), # Generate the policy signature using our Amazon Secret Key
"key": key,
"AWSAccessKeyId": settings.AWS_ACCESS_KEY, # Obviously the Amazon Access Key
"acl":"private",
"x-amz-meta-post_key":post_key,
"x-amz-meta-file_id":file_id,
"x-amz-meta-file_name": file_name,
"x-amz-meta-content_type": content_type,
"success_action_status":"200",
})
The returned tuple can then be used to generate a form that posts to the generated S3 url with all of the key value pairs from the dictionary as hidden fields and your actual file input field, whose name/id should be "file".
Hope that helps as an example.
I've been struggling with this same exact problem, using almost the same exact code, for days. ( See Python Generated Signature for S3 Post ) Just tried to encode my policy in accordance with White Box Dev's code, but still not coming up with the same as AWS suggests I should have. I eventually gave up and used...
http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html
...and just inserted the values it returns into the HTML form. Works great.
#Mr. Oodles: if you're storing your aws_secret_key in a separate file, use bash command ls -al to check it's number of bytes prior to generating the signature. It should be 40 bytes long. As White Box Dev points out, AWS does not like \n, and it's possible you're bundling up this hidden character (or a carriage return or ^M) with your aws_secret_key string when saving it...hence making it 41 bytes long. You can try .replace("\n", "") or .rstrip() to get rid of it after reading it into your script, The .encode("utf-8") may work for you, too. None of those worked for me, however. Curious if you're running Python on Windows or Unix... You could also try using emacs to save the string without the \n being automatically inserted by your editor.
Try checking out https://github.com/burgalon/plupload-s3mixin
It combines PLUPLOAD with direct S3 upload