I'm using AppEngine to create a page that I would like to update from the program. Specifically, I am getting some market data and would like to have a table (or something else appropriate) that shows current prices. Let me be clear: I am new to this and think my problem is that I'm not asking the question well enough to find a good (best) answer. I'm not even sure AppEngine is necessarily the way to go. I'll also caveat that I've been learning via Udacity so if code looks familiar -- kudos to Steve Huffman.
I've created the page via jinja2 and I've managed to wrangle the appropriate libraries and sandbox parameters to get market updates. I've created an html table and passed in a dictionary with values for exchanges and bid/ask pairs. The table creates fine -- but when I render again, I get tables repeating down the page rather than one table with updating market prices.
Here is the html/jinja2 (I ditched all the styling to make it shorter):
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Table template</title>
</head>
<body>
<h1>Table template</h1>
<table>
{% for exch in mkt_data %}
<tr>
<td> <div>{{exch}}</div></td>
<td> <div>{{mkt_data[exch][0]}}</div></td>
<td><div>{{mkt_data[exch][1]}}</div></td>
</tr>
{% endfor %}
</table>
</body>
</html>
Here is the code:
import os
import jinja2
import webapp2
import ccxt
template_dir = os.path.join(os.path.dirname(__file__), 'templates')
jinja_env = jinja2.Environment(loader = jinja2.FileSystemLoader(template_dir),
autoescape=True)
class Handler(webapp2.RequestHandler):
def write(self, *a, **kw):
self.response.out.write(*a, **kw)
def render_str(self, template, **params):
t = jinja_env.get_template(template)
return t.render(params)
def render(self, template, **kw):
self.write(self.render_str(template, **kw))
class MainPage(Handler):
def get(self):
self.render("table.html", mkt_data=btc)
for x in range(3):
for exch in exchanges:
orderbook=exch.fetch_order_book('BTC/USD')
bid = orderbook['bids'][0][0] if len(orderbook['bids'])>0 else None
ask = orderbook['asks'][0][0] if len(orderbook['asks'])>0 else None
btc[exch.id]=[bid,ask]
self.render("table.html", mkt_data=btc)
gdax = ccxt.gdax()
gemini = ccxt.gemini()
exchanges = [gdax, gemini]
btc = {"gemini":[0,1], "gdax":[1,2]}
for exch in exchanges:
exch.load_markets()
app = webapp2.WSGIApplication([('/', MainPage)], debug=True)
I have 2 questions:
First, why am I getting the table repeating? I think I know why, but I want to hear a formal reason.
Second, what should I be doing? I originally started learning javascript/node but then it seemed very hard to wrap all the appropriate libraries (was looking into browserify but then thought appengine may be better so I could more easily host something for others to see). I tried integrating some javascript but that did not get me anywhere. Now I've run into Firebase but before I go learn yet another "thing" I wanted to ask how other people do this. I'm certain there are multiple ways but I'm new to web programming; I'm viewing a web page as a nice UI & delivery mechanism.
Some add'l notes: using Ubuntu, virtualenv, ccxt library (for cryptocurrency).
edit: I checked Dan's answer because it offered a solution. I'd love to hear about whether Firebase is "a" more correct solution rather than auto-refreshing.
The repeated table is the result of the multiple self.render() calls inside your MainPage.get() - both above and repetead ones inside the for loop(s).
Update your code to make a single such call, after the for loops building the template values (at the end of MainPage.get())
Related
I am trying to create a very simple one-page Flask application for a python script that I have. The script requires multiple user inputs in a for-loop with the number of loops being user input as well.
Here is the code in my script to make it more clear:
def shared_books():
import requests as re
from bs4 import BeautifulSoup
import time
num_lists = int(input('Enter the number of lists you would like to search:'))
urls = []
page_counts = []
for i in range(num_lists):
urls.append(input(f'Enter the url for list {i + 1}:'))
page_counts.append(int(input(f'Enter the number of pages for list {i + 1}:')))
I want a simple HTML that will ask the user for the number of lists, then the URL and page count for each list as is shown in my function. Then it will run the entire function.
The HTML code I have right now is super simple and I don't want much else outside of the input parts:
<html>
<head>
<title>Goodreads-App</title>
</head>
<body>
<h1>Welcome to my app!</h1>
<<p>This app will allow you to see books that are
shared between multiple lists on goodreads</p>
</body>
</html>
Please let me know how I can set up this application!
Firstly, I suggest you take a look at the Flask docs. You are doing it right in terms of having a view function, but the input() python keyword doesn't work like that in Flask. Instead, you should render an html template which you can then put your form input field into. Here is an example:
from flask import Flask, render_template
#flask initialising stuff, read docs for info
#app.route("/home")
def home():
return render_template("home.html")
Flask runs on your computer's local server "localhost", which is not publicly accessible. It conventionally runs on port 5000, which gives the name "localhost:5000".
When someone visits "localhost:5000/home", flask will look for a file called "home.html" in a pre-designated templates folder – the default is a directory called "templates" which you should put your html files into.
So if this is your "home.html" file:
<html>
<head>
<title>Goodreads-App</title>
</head>
<body>
<h1>Welcome to my app!</h1>
<p>This app will allow you to see books that are
shared between multiple lists on goodreads</p>
</body>
</html>
When you load the page associated with a specific function, it will return a template which is rendered as html. The above should look something like this:
And that is how to start.
Thank you for the answers! I haven't quite solved the previous issue but have approached it from a different angle which is working now! I will potentially post again if I don't solve it.
I am using flask forms to do what I was trying.
Suppose I want to render a page (not just JSON) using Flask with some specific data that I fetch from the database. For example
display_data.html includes:
<script src='display_data.js'></script>
...
<h1>Data display page!</h1>
<div id="chartContainer"></div>
display_data.js:
$(function() {
draw_chart($("#chartContainer"), json_data);
//draw_chart is defined elsewhere and json_data is what I want to pass in
});
Python:
#app.route('/<data_id>')
def get_display_data_page(data_id):
data = get_data_by_id(data_id)
return render_template('display_data.html', data = data)
I think that if I want to just "render template", I'd have to include elsewhere in display_data.html the following:
<script>window.json_data = {{ data | tojson | safe}}</script>
This pattern smells bad: I'm leaving an object on the global namespace (so that my JS file can access it), displaying the data as plain text, and rendering a string in that is parsed into JSON so the JS can use it. Looks bad but this does work.
Two other options:
Return the data with AJAX. Given the title of this post I'm specifically trying to avoid ajax. The reason for this is mainly that I'm building a mobile site and want to reduce the number of pings back to the server. I'm also thinking (perhaps more metaphysically) about encapsulating the page: once you have it, you have all of it.
Render my JS file via Flask and Jinja. This seems like a bummer because I'd have to then write a route down and render the JS based on the same logic that I have in the get_display_data_page: looking up the data by its id, etc. Code duplication and dynamic JS sound like big no-no's to me.
Is there a known pattern to doing this well?
There's no need to leave data in the global scope if you don't want to. In your template you can do something like this:
<script>
function registerTask(f, args) {
$(function() {
f.call(this, args);
});
}
{% for name, args in js_tasks %}
registerTask({{name}}, {{ args | tojson | safe }});
{% endfor %}
</script>
Then, in your JS file, redefine draw_chart to just take the data (or have a wrapper around it that you use as your task registry name):
function draw_chart_task(data) {
draw_chart($('#chartContainer'), data);
}
Finally, in your controller, simply provide the data and the task name as a tuple:
return render_template('display_data.html', js_tasks=[('draw_chart_task', data)])
This ensures that your JavaScript is not just plucking its dependencies out of the global scope, and you are not making extra network calls.
The data is visible in the raw text output of the page, but it is visible if you make an AJAX call too, you just have to look in a different panel of your browser's developer's tools to see it.
I am a behavorial scientist and usually collect data by letting participants do some tasks on a computer and record their responses (I write the programs using the pyglet wrapper PsychoPy). That is, the program runs locally and the data is stored locally.
Now I would like to know if there is a way to use Python to display a (local) website with html-forms to the user and collect the input (locally). The reason for this idea is that currently whenever I want to display checkboxes, radiobuttons, or input fields I use wxPython. This works quite well, but programming and layouting in wxPython is kind of cumbersome and I would prefer html with forms.
A requirement would be that it would need to rum without any borders, adress field, menu bar, ... The reason is that I need it in kind of fullscreen mode (I currently open a non-fullscreen pygflet window in the size of the screen to hide the desktop) so that participants can do nothing but work on the forms.
So I am looking for a way to (a) display html websites including html form above a pyglet window with no menu bar or whatsoever, (b) collect the input when clicking on the Ok button (i.e., the form is send), (c) control what is presented prior and after viewing this website, and (d) everything of this should happen locally!
My idea would be that the data is collected when participants hit the "Send away" button in the following example pic and the next page is displayed.
Update: I use windows (XP or 7).
This is a solution using Qt Webkit for rendering HTML. The default navigation request handler is wrapped by a function that checks for submitted form requests. The form uses the "get" method, so the data is included in the url of the request and can be retrieved that way. The original request is declined and you can change the content of the displayed web page as you wish.
from PyQt4 import QtGui, QtWebKit
app = QtGui.QApplication([])
view = QtWebKit.QWebView()
# intercept form submits
class MyWebPage(QtWebKit.QWebPage):
def acceptNavigationRequest(self, frame, req, nav_type):
if nav_type == QtWebKit.QWebPage.NavigationTypeFormSubmitted:
text = "<br/>\n".join(["%s: %s" % pair for pair in req.url().queryItems()])
view.setHtml(text)
return False
else:
return super(MyWebPage, self).acceptNavigationRequest(frame, req, nav_type)
view.setPage(MyWebPage())
# setup the html form
html = """
<form action="" method="get">
Like it?
<input type="radio" name="like" value="yes"/> Yes
<input type="radio" name="like" value="no" /> No
<br/><input type="text" name="text" value="Hello" />
<input type="submit" name="submit" value="Send"/>
</form>
"""
view.setHtml(html)
# run the application
view.show()
app.exec_()
As AdamKG mentioned, using a webframework would be a good choice. Since Django and similar might be an overkill here, using a micro webframework like 'flask' or 'bottle' would be a great choice.
This link demonstrates via step by step instruction how to make a simple form via a To-DO application. It assumes zero previous knowledge.
You can run it only locally also.
your want a simple solution, so just write a http server and run your simple page.
using python.BaseHTTPServer, coding a 15 line web server:
import BaseHTTPServer
class WebRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def do_GET(self):
if self.path == '/foo':
self.send_response(200)
self.do_something()
else:
self.send_error(404)
def do_something(self):
print 'hello world'
server = BaseHTTPServer.HTTPServer(('',80), WebRequestHandler)
server.serve_forever()
easy enough,but i suggest using some web frameworks. They are easy too.
for example, web.py. here is what u want in 50 line codes:
install web.py
make a dir with 2 files:
./
|-- app.py
`-- templates
`-- index.html
index.html
$def with (form, ret)
<html>
<head>
<title> another site </title>
</head>
<body>
<h1> hello, this is a web.py page </h1>
<form action="" method="post">
$:form.render()
</form>
<h2>$:ret</h2>
</body>
</html>
app.py logic file:
import web
### Url mappings
urls = (
'/', 'Index', )
### Templates
render = web.template.render('templates')
class Index:
form = web.form.Form(
web.form.Textbox('fav_name', web.form.notnull, description="Favorite Name:"),
web.form.Textbox('cur_name', web.form.notnull, description="Current Name:"),
web.form.Button('Send Away'),
)
def GET(self):
""" Show page """
form = self.form()
return render.index(form, "")
def POST(self):
""" handle button clicked """
form = self.form()
if not form.validates():
return render.index(form, "INPUT ERROR")
# save data by ur method, or do some task
#pyglet.save_data(form.d.fav_name, form.d.cur_name)
#pyglet.draw(some_pic)
#os.system(some_cmd)
form = self.form()
return render.index(form, "YOUR DATA SAVED")
app = web.application(urls, globals())
if __name__ == '__main__':
app.run()
run this server in your windows:
python app.py 9999
open browser: http://127.0.0.1:9999/
by the way, if ur data is only strings, u can save them in web.by by sqlite.
My suggestion would be:
Use some python server as, for example SimpleHTTPServer. It is needed because the submit button on forms sends the information to a server. There you should manage the received info some way;
Have your browser configured with one of those Kiosk extensions, which disallow even the use of Alt+F4. An example would be Open Kiosk extension for Firefox
Optionally, if you have affinity with scripts in general, you could create a script which, when executed, would at the same time run the python server AND open your html file in the browser. That would ease a lot your setup work for every subject in your group.
EDIT: I've read you need the pyglet over the browser window. That could be included in the script of step 3, using "always on top" option and absolute positioning of the pyglet (I can tell this would probably be simpler on Linux, which could be run from persistent LiveUSB - just a thought!)
EDIT (regarding the posted comment):
I think the most reliable option for output would be to disk (file or database) instead or RAM (running python object), then you read the info from file afterwards. Then, in case of a surprise (system hang, power failure), the already-entered data would be there.
The only (and most important) part I don't know HOW to do is to handle the content of the form's "submit" on the server-side. Probably some server-side script file (php, python) shoud be created and left on the server root, so the server would receive an http request containing the info, and send the info to the script, which then handles the processing and file/database storage activities.
This might be of your interest:
"The POST request method is used when the client needs to send data to the server as part of the request, such as when uploading a file or submitting a completed form." (from wikipedia on "POST(HTTP)" ENTRY)
In another link, some thoughts on using SimpleHTTPServer itself for handling POST requests:
http://islascruz.org/html/index.php/blog/show/Python%3A-Simple-HTTP-Server-on-python..html
Hope this helps.
The reason for this idea is that currently whenever I want to display
checkboxes, radiobuttons, or input fields I use wxPython. This works
quite well, but programming and layouting in wxPython is kind of
cumbersome and I would prefer html with forms.
You can combine the ease of HTML and still create native Windows applications using Flex with a Python backend.
If you are averse to Flex, a bit more - involved - but still native windows application generator is Camelot
Edit
Instead of typing it out again - I would suggest the django + flex + pyamf article on Adobe that explains it all with screenshots as well. You can replace django with flask or bottle as they are more lightweight, however the PyAMF library provides native support for django which is why it was used in the example.
PyAMF provides Action Message Format (a binary protocol to exchange object with the flash runtime) support for Python.
I would like to construct (constructing a table is easy, I do it inside templates) a table and choose a row from it. I am using pyramid as a framework and I think somehow I need to make the templates talk to the model, but I am not sure how. Can someone show me an example or direct me to link where I can see an explanation and an example (I couldn't find one). This is my HTML for the table:
<table border="1">
<tr>
<th>Course Number</th>
<th>Course Name</th>
</tr>
<tr>
<td>111</td>
<td>What ever the name is</td>
</tr>
</table>
Making some assumptions, what you describe would be handled via ajax, which is a technology based on javascript, and outside the scope of they Pyramid framework but easily added. Below is a very simple example of what the HTML might look like.
<html>
<head>
<script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
<script type="text/javascript">
$(document).ready(function() {
$('.row').bind('click',function(){
var row = $(this);
options = {
url:"/some_pyramid_url_to_a_view/",
type:"get",
data:{'row_id':$(this).attr('id')},
success:function(event){$(row).text(event.data.method_result)},
error:function(){alert('error')},
dataType:'json'
}
$.ajax(options);
});
});
</script>
</head>
<body>
<table>
<tr id="${row.id}" class="row">
<td>Row</td>
</tr>
</body>
</html>
The code between the tags is the javascript, here I am using the jQuery library to create the ajax call to the url '/some_pyramid_url_to_a_view/' which is tied to the view function 'some_pyramid_view'. I'm binding a click event to the row element in the table using the jQuery selector on the class 'row' "$('.row').bind('click',..." the event is then handled by the function block "function(){...}" immediately following. I'm setting up the call in the options object and passing the row id in the data "data:{'row_id':$(this).attr('id')}," where the code "$(this).attr('id')" is accessing the id set in the template '...' Finally I send of the request to the view using '$.ajax(options)'.
import json
#view_config(xhr=True)
def some_pyramid_view(request):
json_dict = dict()
session = DBSession()
if request.method == 'GET':
row_id = request.GET['row_id']
row = session.query(Row).get(row_id)
json_dict['method_result'] = row.some_method_call()
return Response(json.dumps(json_dict))
There is another piece here, JSON is being used to communicate back to the javascript from the Pyramid view. JSON stands for JavaScript Object Notation and is a data-interchange format. I create a Python dictionary and using the 'json' package convert it to JSON which I send back to the 'success' call back function in the javascript with the results of the method call. In the success call back function I update the text in the table row with the result of the method call in the view. '$(row).text(event.data.method_result)'
This might be more than you were expecting but it is well worth learning how to use ajax to add functionality to your websites.
In python, what is the most elegant way to generate HTML documents. I currently manually append all of the tags to a giant string, and write that to a file. Is there a more elegant way of doing this?
You can use yattag to do this in an elegant way. FYI I'm the author of the library.
from yattag import Doc
doc, tag, text = Doc().tagtext()
with tag('html'):
with tag('body'):
with tag('p', id = 'main'):
text('some text')
with tag('a', href='/my-url'):
text('some link')
result = doc.getvalue()
It reads like html, with the added benefit that you don't have to close tags.
I would suggest using one of the many template languages available for python, for example the one built into Django (you don't have to use the rest of Django to use its templating engine) - a google query should give you plenty of other alternative template implementations.
I find that learning a template library helps in so many ways - whenever you need to generate an e-mail, HTML page, text file or similar, you just write a template, load it with your template library, then let the template code create the finished product.
Here's some simple code to get you started:
#!/usr/bin/env python
from django.template import Template, Context
from django.conf import settings
settings.configure() # We have to do this to use django templates standalone - see
# http://stackoverflow.com/questions/98135/how-do-i-use-django-templates-without-the-rest-of-django
# Our template. Could just as easily be stored in a separate file
template = """
<html>
<head>
<title>Template {{ title }}</title>
</head>
<body>
Body with {{ mystring }}.
</body>
</html>
"""
t = Template(template)
c = Context({"title": "title from code",
"mystring":"string from code"})
print t.render(c)
It's even simpler if you have templates on disk - check out the render_to_string function for django 1.7 that can load templates from disk from a predefined list of search paths, fill with data from a dictory and render to a string - all in one function call. (removed from django 1.8 on, see Engine.from_string for comparable action)
If you're building HTML documents than I highly suggest using a template system (like jinja2) as others have suggested. If you're in need of some low level generation of html bits (perhaps as an input to one of your templates), then the xml.etree package is a standard python package and might fit the bill nicely.
import sys
from xml.etree import ElementTree as ET
html = ET.Element('html')
body = ET.Element('body')
html.append(body)
div = ET.Element('div', attrib={'class': 'foo'})
body.append(div)
span = ET.Element('span', attrib={'class': 'bar'})
div.append(span)
span.text = "Hello World"
if sys.version_info < (3, 0, 0):
# python 2
ET.ElementTree(html).write(sys.stdout, encoding='utf-8',
method='html')
else:
# python 3
ET.ElementTree(html).write(sys.stdout, encoding='unicode',
method='html')
Prints the following:
<html><body><div class="foo"><span class="bar">Hello World</span></div></body></html>
There is also a nice, modern alternative: airium: https://pypi.org/project/airium/
from airium import Airium
a = Airium()
a('<!DOCTYPE html>')
with a.html(lang="pl"):
with a.head():
a.meta(charset="utf-8")
a.title(_t="Airium example")
with a.body():
with a.h3(id="id23409231", klass='main_header'):
a("Hello World.")
html = str(a) # casting to string extracts the value
print(html)
Prints such a string:
<!DOCTYPE html>
<html lang="pl">
<head>
<meta charset="utf-8" />
<title>Airium example</title>
</head>
<body>
<h3 id="id23409231" class="main_header">
Hello World.
</h3>
</body>
</html>
The greatest advantage of airium is - it has also a reverse translator, that builds python code out of html string. If you wonder how to implement a given html snippet - the translator gives you the answer right away.
Its repository contains tests with example pages translated automatically with airium in: tests/documents. A good starting point (any existing tutorial) - is this one: tests/documents/w3_architects_example_original.html.py
I would recommend using xml.dom to do this.
http://docs.python.org/library/xml.dom.html
Read this manual page, it has methods for building up XML (and therefore XHTML). It makes all XML tasks far easier, including adding child nodes, document types, adding attributes, creating texts nodes. This should be able to assist you in the vast majority of things you will do to create HTML.
It is also very useful for analysing and processing existing xml documents.
Here is a tutorial that should help you with applying the syntax:
http://www.postneo.com/projects/pyxml/
I am using the code snippet known as throw_out_your_templates for some of my own projects:
https://github.com/tavisrudd/throw_out_your_templates
https://bitbucket.org/tavisrudd/throw-out-your-templates/src
Unfortunately, there is no pypi package for it and it's not part of any distribution as this is only meant as a proof-of-concept. I was also not able to find somebody who took the code and started maintaining it as an actual project. Nevertheless, I think it is worth a try even if it means that you have to ship your own copy of throw_out_your_templates.py with your code.
Similar to the suggestion to use yattag by John Smith Optional, this module does not require you to learn any templating language and also makes sure that you never forget to close tags or quote special characters. Everything stays written in Python. Here is an example of how to use it:
html(lang='en')[
head[title['An example'], meta(charset='UTF-8')],
body(onload='func_with_esc_args(1, "bar")')[
div['Escaped chars: ', '< ', u'>', '&'],
script(type='text/javascript')[
'var lt_not_escaped = (1 < 2);',
'\nvar escaped_cdata_close = "]]>";',
'\nvar unescaped_ampersand = "&";'
],
Comment('''
not escaped "< & >"
escaped: "-->"
'''),
div['some encoded bytes and the equivalent unicode:',
'你好', unicode('你好', 'utf-8')],
safe_unicode('<b>My surrounding b tags are not escaped</b>'),
]
]
I am attempting to make an easier solution called
PyperText
In Which you can do stuff like this:
from PyperText.html import Script
from PyperText.htmlButton import Button
#from PyperText.html{WIDGET} import WIDGET; ex from PyperText.htmlEntry import Entry; variations shared in file
myScript=Script("myfile.html")
myButton=Button()
myButton.setText("This is a button")
myScript.addWidget(myButton)
myScript.createAndWrite()
I wrote a simple wrapper for the lxml module (should work fine with xml as well) that makes tags for HTML/XML -esq documents.
Really, I liked the format of the answer by John Smith but I didn't want to install yet another module to accomplishing something that seemed so simple.
Example first, then the wrapper.
Example
from Tag import Tag
with Tag('html') as html:
with Tag('body'):
with Tag('div'):
with Tag('span', attrib={'id': 'foo'}) as span:
span.text = 'Hello, world!'
with Tag('span', attrib={'id': 'bar'}) as span:
span.text = 'This was an example!'
html.write('test_html.html')
Output:
<html><body><div><span id="foo">Hello, world!</span><span id="bar">This was an example!</span></div></body></html>
Output after some manual formatting:
<html>
<body>
<div>
<span id="foo">Hello, world!</span>
<span id="bar">This was an example!</span>
</div>
</body>
</html>
Wrapper
from dataclasses import dataclass, field
from lxml import etree
PARENT_TAG = None
#dataclass
class Tag:
tag: str
attrib: dict = field(default_factory=dict)
parent: object = None
_text: str = None
#property
def text(self):
return self._text
#text.setter
def text(self, value):
self._text = value
self.element.text = value
def __post_init__(self):
self._make_element()
self._append_to_parent()
def write(self, filename):
etree.ElementTree(self.element).write(filename)
def _make_element(self):
self.element = etree.Element(self.tag, attrib=self.attrib)
def _append_to_parent(self):
if self.parent is not None:
self.parent.element.append(self.element)
def __enter__(self):
global PARENT_TAG
if PARENT_TAG is not None:
self.parent = PARENT_TAG
self._append_to_parent()
PARENT_TAG = self
return self
def __exit__(self, typ, value, traceback):
global PARENT_TAG
if PARENT_TAG is self:
PARENT_TAG = self.parent