I'm trying to learn web programming in python, and have the following project in mind: mine the yahoo finance api for instrument data, and display it in real time, as well as plot charts based on instrument data.
I already did something similar using wxpython, and I'm interested in how I would accomplish this in a web application.
My first thought was to use django and matplotlib on the server, and have the client request updated chart images through jquery at a certain time interval, but after a bit of research I came upon libraries like twisted and tornado...and now I'm confused. Would they work better for this web app than django ?
After the above rambling, my question is: what library should I use for writing the web app i have in mind ? I'm also thinking that I should abandon matplotlib, and generate the chart on client side, but I'm not sure what javascript library would allow me to do that, if any.
Few tips:
1/ Do not plot your data at backend . Instead use the browsers to generate charts.I would recommend
using jqplot, or highcharts.
2/ Yes, you can use tornado or twisted instead of django, as they are asynchronous servers, and would provide faster handling of requests.
3/ You should create a REST interface of your application, with server side only sending JSON data, and do all the UI templating and charting on client side.
4/ Backbone.js (recommended, but you can use some other MVC framework), would also prove to be helpful if your app grows too complex.
Related
I got 6 real-time videos which are inference output from heavy deep learning calculation from python, and I've tried to display them to PyQt that lots of threading issues with ugly GUI!!. So, I want to use a framework only to display well and fancy GUI! There seem lots of frameworks based on my google search, and I have no idea
which one is the best for my current project among Node js, Django, and Flask!
I need to display 6 real-time videos with 10~15 FPS.
Communicating well with python.
Easy to build GUI, I have some features like logging, displaying real-time graph(optional)
For video streaming, you will require to setup a socket URL for listening to incoming packet. Definitely, you can dedicate a server just for this to handle high amount of traffic.
Now, as such it dependents on your application, if you are just going to use it for streaming then Flask will do, it is lightweight. On the other hand, Django and NodeJS provides ready made function for socket programming and they are very useful.
I would say it doesn't really matter much as Django would be good because of its pre built features... Plus data transfer is very reliable in django along with your expertise in python, django is preferred. NodeJs would be easier to implement tho. Really prefers upon you, but both django and nodeJs works pretty well.
Sorry, no idea on flask
My opinion - flask faster development, node - better efficiency, some thoughts why are explained here: https://hinty.io/ivictbor/flask-vs-node/
Could you give me an idea/concepts (not in code) on how could I link NodeJS and Python?
Let's say,
I have NodeJS up and running in PM2 (assuming I already know REST API) and I have a ton of data sets that I need to be ready to display to client side using socket.io (assumming I already know socket.io) as soon as possible.
I'm thinking to use Python. This is for me to implement the basics of machine learning.
In what concept should I start? I'd really love to hear your ideas.
Well you seem to be assuming way too many things, okay from your description I would suggest you to have a look at concept called microservice architecture.
This is how it will work let us assume you want to build an online shopping application where you have 2 main scenarios first is sell all items on your website and second you want to recommend products to your user(Your ML comes into play over here)
So as you said you already know REST API so what you would do is create a microservice (Consider it as a small nodejs application(Using either express or sails or any other framework) which has APIs exposed for all shopping related business logic) also you end up using fromtend technology viz. angularjs for your client side code. You'll show all this shopping stuff by calling your nodejs REST APIs from your angularjs client code. Node provides socket support via socket.io.
Similarly you write a small microservice in python(using Flask and Python-SocketIO) which takes your huge amount of data from datastore does all ML magic and returns recommended products for the particular user(which you received from your angularjs client application), and return it using Python-SocketIO to angularjs(or node application if you're maintaining your frontend logic there instead of angular).
You have provided very less detail so this is abstract view of what you can look into.
Since you're Python oriented for your ML code I'd suggest you to reduce the list of skills you need to learn and/or improve using Python for everything.
You could use Python-SocketIO and Flask, for example.
I want to implement a dynamic web based dash board with Python 3.x (Like Shiny for R)
Bokeh seems promising from what I have read and seen in YouTube.
What isn’t clear is when and where will I need a Bokeh server and a Flask/Django Server? Will they complement each other? Or I can work on either one? Will they both be serving the same thing?
The Bokeh server is based on Tornado which is itself a capable Python web framework and asynchronous networking library. Depending on your needs, it may be possible to simply write a Bokeh app, have it show everything you want, in the way you want. Bokeh exposes some capability to customize app appearance via Jinja2 templates. You can see an example of this in the Gapminder Demo on http://demo.bokeh.org.
However, you may have more sophisticated needs, especially around authentication and access, or have an existing site that you need to integrate into. In this case, you are probably looking at embedding a Bokeh app in another page, which might be served from Flask, or Django, or IIS, or whatever. There are two basic ways to do this:
using server_document to generate a <script> tag that you can template into your page, which will embed an app from a Bokeh server into the page
using <iframe> to embed a URL from a Bokeh server into the page
Either of these can work just fine. Depending on the sophistication of your deployment environment, there may be more "devops-y" type things to do to use a Bokeh server behind a proxy or with a load balancer, etc. The Running a Bokeh Server section of the User's Guide has a lot more information for anyone needing to dive into those details.
If you are embedding an app from a Bokeh server into another web page, the Bokeh server does need to be up and running to serve the app! How to accomplish this is a separate concern, there are a few ways you might do it:
start up as an external process, and manage with something like supervisord. You can see a full example deployment like this at https://github.com/bokeh/demo.bokeh.org
embed a Bokeh server "inside" your Flask/Django/whatever app by starting your own Tornado IOLoop. You can see one example of this technique in examples/howto/server_embed. Also, this should probably be considered fairy advanced usage.
Here's the setup: On a single-board computer with a very rudimentary linux I'm running a Django app. This app is, when a button is pressed or as a response to the data described below, supposed to call either a function from a library written in C, or a compiled C program, to write data to system memory at a specified address, poke/peek like. (Python doesn't seem to be able to do that natively).
The Django app should also display data, continuously, which is being read from the memory from the same library / program.
My question now is how to even begin with setting up the scenario described above. Is this even possible with a web app? Is a Django or more fundamentally any web framework even the right approach here? I'm at a bit of a loss here, since I've spent quite a few hours now trying to figure out how to do this while not getting the most basic starting point...
Disclaimer: I'm pretty new to the entire web framework thing, and more importantly web development in general, so sorry if this is a bad question as in, I could have easily found information on this topic online, but I couldn't really find a good starting point on this.
I wanted to add a comment but not enough space... anyway
You can write a native extension in C for Python that could do what you need, check this.
Now for the fact of displaying data continuously this is kind of vague, if this C library is switching this hypothetical address, very often and very fast you have to update a browser client as fast as possible.
I think websockets would do the trick but they are js related, so I think NodeJs would be a better candidate for the server side of your application instead of Django.
If you want to stick to Django you can also expose an URL with the generated address value and have a webpage continuously (with a little Interval) checking that URL using a simple ajax call, kind of ugly and inefficient but would work.
Anyway IMHO your best bet is for websockets because with them you have a fullduplex communication between client and server.
Good Luck with your project.
Info:
Websockets in Django with socket.io
Nodejs socket.io
I am looking for a pure Javascript/Python upload example, that uses server polling instead of client-side SWF to display upload progress (like the one on rapidshare.com for example)
Currently, website is running on the standalone wsgi server included with Werkzeug framework, but may be moved to mod_wsgi if the load increases.
I've tried the gp.fileupload middleware, but can't get it to work. Examples on their website wont work either :|
Website already uses Glow library for other misc client-side stuff, but there is no specific upload-related functionality in it.
If you don't have support on the web side to track the size of the temporary file (or in-memory buffer) of the uploading data as it arrives, I don't know how you'll do this. Some of the popular web servers have special support for this, mostly experimental, but it's not widely supported and what you're trying to do is pretty awkward in general. I've researched this recently and it's pretty poorly supported all around.