System requirement for an embedded Python web server - python

I'm working on an embedded device that runs Linux on ARM7 with 64MB RAM and 64MB storage (12MB free). The device should be configured via web therefore it needs to run an embedded web server. Currently it's using Lighttpd and LUA, but I'm thinking about replacing LUA (or maybe even Lighttpd) with Python. The server will occasionally be accessed by one or two users for making changes to internal settings of the C program that is running in Linux. So the server load isn't really a lot. I also need it to be Open Source Software. Web.py seems to be small enough but I still need to compile Python which I haven't done before. So I'm wondering what are the system requirements of Python? LUA seems to do quite well for small embedded systems but I don't like its syntax for C-binding.
However, I couldn't find updated information about system requirements for embedding Python in such settings. This page from Michael Lauer seems to be old.
Any ideas? Suggestions? hints? links?

I'm working on this device using OpenWRT + Python:
http://wiki.openwrt.org/oldwiki/OpenWrtDocs/Hardware/Meraki/Mini
The first python run is veeeeeery slow but it are metacompiling all .pyc files, next it work well.

Related

localhost server in compiled Python program

Background:
I am still new to Python so apologies if this is a naive question. I was originally looking to develop a tool for Windows with a GUI, however, reading around I get the impression that it may be simpler to have my compiled python code somehow 'wrapped' in simpleHTTPServer so users can interact via localhost (much like Jupyter notebook/labs).
Question:
Is there a standard/recommended way to have a compiled python program interface with the user via a localhost port in their web browser? I assume this is a fairly common approach and that a package may already even exist but I've not been able to find anything that gives guidance on how to tackle it.
Alternatively, is is feasible to somehow compile a Django development project to run on localhost?
Ideally, I want an executable file that users can run and their browser opens up at the localhost port required.
If you wish to quickly make a GUI based Python tool, you should consider using Django, a very straightforward framework for WebApps. It is very quick to learn and implement, and you shall very easily be able to integrate highly advanced functionalities too into your tool with ease.
When it comes to distribution of code, you can convert the entire Django App into a windows exe file for ease of distribution and use. There are many python packages that allow you to port python to windows executable files. I would recommend using PyInstaller. You may find this link useful: Recipe Executable From Django.
Alternatively, as someone suggested, you may use Docker, but that would be heavier in terms of space utilized, and will require the user to install the same.
Hope this helps!

Is there an easy way to distribute a Flask server as an executable?

I'm building some simple editors with Backbone.js, and I'm hoping to be able to distribute them as apps for users to edit content in a mostly client-side way (i.e., I don't want users to have to futz with setting up stuff like MySQL or Apache).
So I was imagining a scenario like:
User downloads a .zip file
In the resulting opened folder, the user clicks index.html
That opens in a browser
Backbone app starts, stores data in localStorage
The user can then export to CSV.
Believe it or not, that would solve my problem: I want to help users edit data in a browser and then get it back out in a familiar format (CSV can be loaded into Excel, for instance).
And I’d like to do this without forcing them to configure a server. It seems like this is almost possible in the HTML5 stack. However, in at least one browser (Chrome), this doesn't work, because I get errors like this one:
XMLHttpRequest cannot load file:///users/me/project/data/Appdata.json. Origin null is not allowed by Access-Control-Allow-Origin.
(Oddly enough, I don't get that error in Firefox, and the .js or .json files load fine.)
So at this point it seems to me that there's no way around having these users use something kind of local server to serve up the Backbone interface.
So, I'm trying to figure out how to build a distributable, cross-platform executable that will allow my users to start a Flask server. (I hope to build a REST backend to a Backbone.js app.)
Is this wishful thinking? I'm assuming I can get the people in question to install Python.
Is this doable? There seem to be many ways to package up Python programs, (pyinstaller? py2exe? ...) So I thought I would ask here in case someone might know of a solution for the stack I have in mind.
TIA!
You can use Anthony Gordon McMillan’s Pyinstaller or Tuininga’s cx_Freeze
Quoting the PyInstaller website:
Features
Packaging of Python programs into standard executables, that work on
computers without Python installed.
Multiplatform: works under
Windows (32-bit and 64-bit),
Linux (32-bit and 64-bit),
Mac OS X (32-bit only, 64-bit in git, see Features/MacOsCompatibility) and
experimentally Solaris and AIX (in git).
Multiversion: works under any version of Python from 2.2 up to 2.7.
My suggestion would be to create a thin service wrapper around your code. This will allow the server to run independently of your main codebase - also allowing the user to shut down the server directly (simply right clicking the service icon and selecting "Exit").
This SO answer should help you get started.
After reading your updated question, I think something like mongoose might be more suited to your task. It is an embeddable web server that is FLOSS and has python bindings. Flask might be overkill.
Not easily. On Windows, you'd have to include Python itself. Mac and Linux usually have Python installed, but you can't be sure of what version so it's often easier to bundle your specific Python for them as well. Then you'd have to include all the dependencies that you want to run with in your package or be able to install them with pip, easy_install, etc.
You can use py2app and py2exe. This won't be cross-platform as you'll still need to make a different version for each target OS. The only way to make it cross-platform is to bundle all versions and have some cross-platform code execute the appropriate version for that platform.
If you need databases like MySQL or even SQLite things get more complicated as you'll have to include those too.

Where can I find a browser based editor for python that works with IE7?

I'd like to get some code editing for a free software project done during some free time that I will have today at my university. There's just one small problem: We're almost entirely a windows environment (mostly thin clients at that) and the only source code editor installed is jGRASP.
So, I need a source code editor which is usable without installing anything (something browser, or Java Web Start based). I would use Notepad++, but I think downloading and running some unapproved exe may be in violation of our acceptable use policy, and it isn't that efficient of an editor for python anyway.
Any suggestions are appreciated.
The best webbased editor I know is Ace.
If you can't execute downloaded exes, this maybe mute, but SciTe is reasonably useable and does not require you to install anything - just download and run.
Putty and vim would be my solution (Requires a remote host running 'nix)
Putty is obviously a 3rd party binary, but a fairly common one.
I'm sure there are web based ssh clients to be had.

Hosting a non-trivial python program on the web?

I'm a complete novice in this area, so please excuse my ignorance.
I have three questions:
What's the best (fastest, easiest, headache-free) way of hosting a python program online?
I'm currently looking at Google App Engine and Web Frameworks for Python, but all the options are a bit overwhelming.
Which gui/viz libraries will transfer to a web app environment without problems?
I'm willing to sacrifice some performance for the sake of simplicity.
(Google App Engine can't do C libraries, so this is causing a dilemma.)
Where can I learn more about running a program locally vs. having a program continuously run on a server and taking requests from multiple users?
Currently I have a working Python program that only uses standard Python libraries. It currently uses around 2.7gb of ram, but as I increase my dataset, I'm predicting it will use closer to 6gb. I can run it on my personal machine, and everything is just peachy. I'd like to continue developing on the front end on my home machine and implement the web app later.
Here is a relevant, previous post of mine.
Depending on your knowledge with server administration, you should consider a dedicated server. I was doing running some custom Python modules with Numpy, Scipy, Pandas, etc. on some data on a shared server with Godaddy. One program I wrote took 120 seconds to complete. Recently we switched to a dedicated server and it now takes 2 seconds. The shared environment used CGI to run Python and I installed mod_python on the dedicated server.
Using a dedicated server allows COMPLETE control (including root access) to the server which allows the compilation and/or installation of anything. It is a bit pricy but if you're making money with your stuff it might be worth it.
Another option would be to use something like http://www.dyndns.com/ where you can host a domain on your own machine.
So with that said, perhaps some answers:
It depends on your requirements. ~4gb of RAM might require a dedicated server. What you are asking is not necessarily an easy task so don't be afraid to get your hands dirty.
Not sure what you mean here.
A server is just a computer that responds to requests. On the dedicated server (I keep mentioning) you are operating in a Unix (or Windows) environment just like you would locally. You use SOFTWARE (e.g. Apache web server) to serve client requests. My vote is mod_python.
It's a greater headache than a dedicated server, but it should be much closer to your needs to go with an Amazon EC2 instance.
http://aws.amazon.com/ec2/#instance
Their extra large instance should be more than large enough for what you need to do, and you only turn the instance on when you need it so you don't have the massive bill that you get with a dedicated server that's the same size.
There are some nice javascript based visualization toolkits out there, so you can model your application to return raw (json) data and render that on the client.
I can mention d3.js http://mbostock.github.com/d3/ and the JavaScript InfoVis Toolkit http://thejit.org/

What are some successful methods for deploying a Django application on the desktop?

I have a Django application that I would like to deploy to the desktop. I have read a little on this and see that one way is to use freeze. I have used this with varying success in the past for Python applications, but am not convinced it is the best approach for a Django application.
My questions are: what are some successful methods you have used for deploying Django applications? Is there a de facto standard method? Have you hit any dead ends? I need a cross platform solution.
I did this a couple years ago for a Django app running as a local daemon. It was launched by Twisted and wrapped by py2app for Mac and py2exe for Windows. There was both a browser as well as an Air front-end hitting it. It worked pretty well for the most part but I didn't get to deploy it out in the wild because the larger project got postponed. It's been a while and I'm a bit rusty on the details, but here are a few tips:
IIRC, the most problematic thing was Python loading C extensions. I had an Intel assembler module written with C "asm" commands that I needed to load to get low-level system data. That took a while to get working across both platforms. If you can, try to avoid C extensions.
You'll definitely need an installer. Most likely the app will end up running in the background, so you'll need to mark it as a Windows service, Unix daemon, or Mac launchd application.
In your installer you'll want to provide a way to locate a free local TCP port. You may have to write a little stub routine that the installer runs or use the installer's built-in scripting facility to find a port that hasn't been taken and save it to a config file. You then load the config file inside your settings.py and whatever front-end you're going to deploy. That's the shared port. Or you could just pick a random number and hope no other service on the desktop steps on your toes :-)
If your front-end and back-end are separate apps then you'll need to design an API for them to talk to each other. Make sure you provide a flag to return the data in both raw and human-readable form. It really helps in debugging.
If you want Django to be able to send notifications to the user, you'll want to integrate with something like Growl or get Python for Windows extensions so you can bring up toaster pop-up notifications.
You'll probably want to stick with SQLite for database in which case you'll want to make sure you use semaphores to tackle multiple requests vying for the database (or any other shared resource). If your app is accessed via a browser users can have multiple windows open and hit the app at the same time. If using a custom front-end (native, Air, etc...) then you can control how many instances are running at a given time so it won't be as much of an issue.
You'll also want some sort of access to local system logging facilities since the app will be running in the background and make sure you trap all your exceptions and route it into the syslog. A big hassle was debugging Windows service startup issues. It would have been impossible without system logging.
Be careful about hardcoded paths if you want to stay cross-platform. You may have to rely on the installer to write a config file entry with the actual installation path which you'll have to load up at startup.
Test actual deployment especially across a variety of firewalls. Some of the desktop firewalls get pretty aggressive about blocking access to network services that accept incoming requests.
That's all I can think of. Hope it helps.
If you want a good solution, you should give up on making it cross platform. Your code should all be portable, but your deployment - almost by definition - needs to be platform-specific.
I would recommend using py2exe on Windows, py2app on MacOS X, and building deb packages for Ubuntu with a .desktop file in the right place in the package for an entry to show up in the user's menu. Unfortunately for the last option there's no convenient 'py2deb' or 'py2xdg', but it's pretty easy to make the relevant text file by hand.
And of course, I'd recommend bundling in Twisted as your web server for making the application easily self-contained :).

Categories