We Keep Coding

Install-less python web app on headless CentOS 5.x-6.x

I have a Linux web application which installs a webserver, a database, digital certificates etc using a set of .sh scripts.
In order to simplify user interaction during installation such as entering passwords, certificate details and such, I want to create a GUI installer. After much deliberation, following are some decisions and related questions
The target systems may or may not have a Desktop or a monitor installed. So providing a web interface to the install process may be the way to go. User would copy the application to the target machine, start the webservice which would then expose a web interface to continue the install. Would Python be a good choice for this?
Since this is an installer itself, the requirements to run it must be practically nil. This requires
Use python's built in SimpleHTTPServer. This will be used the one time during installation and then be killed. Any caveats to using the default python web server?
Compile app into standalone binary using one of the Freezing utilities. We don't want to depend on the user having python on their system and have been asked to account for admins who've removed python due to whatever reason. Is this precaution necessary?
Any comments on the general approach or alternative options will be greatly appreciated.

Is there an easy way to distribute a Flask server as an executable?

I'm building some simple editors with Backbone.js, and I'm hoping to be able to distribute them as apps for users to edit content in a mostly client-side way (i.e., I don't want users to have to futz with setting up stuff like MySQL or Apache).
So I was imagining a scenario like:
User downloads a .zip file
In the resulting opened folder, the user clicks index.html
That opens in a browser
Backbone app starts, stores data in localStorage
The user can then export to CSV.
Believe it or not, that would solve my problem: I want to help users edit data in a browser and then get it back out in a familiar format (CSV can be loaded into Excel, for instance).
And I’d like to do this without forcing them to configure a server. It seems like this is almost possible in the HTML5 stack. However, in at least one browser (Chrome), this doesn't work, because I get errors like this one:
XMLHttpRequest cannot load file:///users/me/project/data/Appdata.json. Origin null is not allowed by Access-Control-Allow-Origin.
(Oddly enough, I don't get that error in Firefox, and the .js or .json files load fine.)
So at this point it seems to me that there's no way around having these users use something kind of local server to serve up the Backbone interface.
So, I'm trying to figure out how to build a distributable, cross-platform executable that will allow my users to start a Flask server. (I hope to build a REST backend to a Backbone.js app.)
Is this wishful thinking? I'm assuming I can get the people in question to install Python.
Is this doable? There seem to be many ways to package up Python programs, (pyinstaller? py2exe? ...) So I thought I would ask here in case someone might know of a solution for the stack I have in mind.
TIA!

You can use Anthony Gordon McMillan’s Pyinstaller or Tuininga’s cx_Freeze
Quoting the PyInstaller website:
Features
Packaging of Python programs into standard executables, that work on
computers without Python installed.
Multiplatform: works under
Windows (32-bit and 64-bit),
Linux (32-bit and 64-bit),
Mac OS X (32-bit only, 64-bit in git, see Features/MacOsCompatibility) and
experimentally Solaris and AIX (in git).
Multiversion: works under any version of Python from 2.2 up to 2.7.

My suggestion would be to create a thin service wrapper around your code. This will allow the server to run independently of your main codebase - also allowing the user to shut down the server directly (simply right clicking the service icon and selecting "Exit").
This SO answer should help you get started.
After reading your updated question, I think something like mongoose might be more suited to your task. It is an embeddable web server that is FLOSS and has python bindings. Flask might be overkill.

Not easily. On Windows, you'd have to include Python itself. Mac and Linux usually have Python installed, but you can't be sure of what version so it's often easier to bundle your specific Python for them as well. Then you'd have to include all the dependencies that you want to run with in your package or be able to install them with pip, easy_install, etc.
You can use py2app and py2exe. This won't be cross-platform as you'll still need to make a different version for each target OS. The only way to make it cross-platform is to bundle all versions and have some cross-platform code execute the appropriate version for that platform.
If you need databases like MySQL or even SQLite things get more complicated as you'll have to include those too.

How to remotely update Python applications

What is the best method to push changes to a program written in Python? I have a piece of software that is written in Python that will regularly be updated. What would be the best way to do this? All the machines will have Windows 7.
Also, excuse the ambiguity of my question. This will be my first time having to implement an updating procedure. Feel free to mention specifics you would like me ot add.

If you're not already packaging your program with InnoSetup, I strongly recommend you switch to it, because it has facilities to make this sort of thing easier. You can specify any special situations, such as files that should not be updated if they already exist (i.e. if you have any internal configuration files or things like that), in the InnoSetup script.
Next, to allow the client machine to find out about new versions of your app, keep a very small file on your public web server that has the version number of the current release and the URL to the latest version's installer exe. For this file to be useful, whenever you release a newer version of your program you must update this file, as well as the version number in the InnoSetup script, and also some kind of APP_VERSION constant in your program.
Then, you'll need to handle these parts of the updater yourself:
Detecting when a newer version is available by retrieving the current-version file from your web server over HTTP, and comparing the version number there to the app's own APP_VERSION. Make sure to do this query in a way that fails gracefully if the client machine doesn't have Internet access, and that doesn't block the GUI while it is doing the request (in case there's a network issue that forces the query to wait a long while for a timeout).
If a newer version is available, asking the user if they want to update, and if they say yes downloading an updated installer to the TEMP directory. Depending on what GUI toolkit you are using, there are various mechanisms for displaying a progress dialog during the download; this is a good idea since the installer is likely to be at least an MB.
Closing your app, running a special update script in the background, then starting up the app again.
The update script will wait for the original process to die completely (easiest way to do this is to pass in the original process's PID as a command line argument and have the update script send a query signal 0 to that process every second or so until it goes away.) It can then run the installer silently in the background, perhaps while displaying a "Please Wait..." dialog to the user. Once the installer is done and reports success in its return code, the updater can restart your program.
Depending on how big your app is, this is more wasteful of bandwidth than the method using git or another SCM. Every update with this approach would involve downloading the entire installer for the latest version of the app, whereas an SCM would only download the files that have changed. However, it has the advantage that it requires no special server facilities except a regular web server, and no special installation of the SCM client on the user's computer.
Plus, InnoSetup is just generally cool. :-)

I would suggest using a source control program such as git or subversion. Also, if you are okay with everyone seeing the code, you can post the code on github, where anyone can pull from it. You could make it private, but you would have to pay for it and all the users would also have create a github account and set it up with their git install.
If you use a source control program, the other people will have to pull the edits manually by running a command, but you could make a script pr batch file that does this and have it run at start up or at regular intervals.
Just to be clear, if you want to do this, you will have to put the code on a server with and SSH support and set up git. If you don't want to go through all of the server set up, I would reccomend github.
git- http://git-scm.com/ (For windows version, go to downloads and select msysGit)
github - https://github.com/

For those of you that would be looking into something a little less dated, I was just looking at how to create python applications that can be updated remotely (though not limited to Windows like OP).
It seems like esky as been a solution for a while. Though it's been deprecated since 2018.
The latest and most up to date solution seem to be a combination of pyinstaller and pyupdater. Note that I don't have personal experience with it, I'm looking for a friend.
It seems to support windows, linux and Mac though and both python 2 and 3 so definitely worth having a look.

The basic principles of application updates are described well by DSimon's answer.
However, update security is a different matter altogether: You don't want your clients to end up installing malicious files.
PyUpdater, as suggested in jlengrand's answer, does provide some secure update functionality, but, unfortunately, PyUpdater 4.0 is broken and there has not been a new release in over half a year (now Aug 2022).
There's also python-tuf, which is the reference implementation of The Update Framework (TUF).
TUF (python-tuf) does everything humanly possible to ensure your update files are distributed securely. However, it does not handle application-specific things like checking for new application versions and installation on the client side.

Python: tool to keep track of deployments

I'm looking for a tool to keep track of "what's running where". We have a bunch of servers, and on each of those a bunch of projects. These projects may be running on a specific version (hg tag/commit nr) and have their requirements at specific versions as well.
Fabric looks like a great start to do the actual deployments by automating the ssh part. However, once a deployment is done there is no overview of what was done.
Before reinventing the wheel I'd like to check here on SO as well (I did my best w/ Google but could be looking for the wrong keywords). Is there any such tool already?
(In practice I'm deploying Django projects, but I'm not sure that's relevant for the question; anything that keeps track of pip/virtualenv installs or server state in general should be fine)
many thanks,
Klaas
==========
EDIT FOR TEMP. SOLUTION
==========
For now, we've chosen to simply store this information in a simple key-value store (in our case: the filesystem) that we take great care to back up (in our case: using a DCVS). We keep track of this store with the same deployment tool that we use to do the actual deploys (in our case: fabric)
Passwords are stored inside a TrueCrypt volume that's stored inside our key-value store.
==========
I will still gladly accept any answer when some kind of Open Source solution to this problem pops up somewhere. I might share (part of) our solution somewhere myself in the near future.

pip freeze gives you a listing of all installed packages. Bonus: if you redirect the output to a file, you can use it as part of your deployment process to install all those packages (pip can programmatically install all packages from the file).
I see you're already using virtualenv. Good. You can run pip freeze -E myvirtualenv > myproject.reqs to generate a dependency file that doubles as a status report of the Python environment.

Perhaps you want something like Opscode Chef.
In their own words:
Chef works by allowing you to write
recipes that describe how you want a
part of your server (such as Apache,
MySQL, or Hadoop) to be configured.
These recipes describe a series of
resources that should be in a
particular state - for example,
packages that should be installed,
services that should be running, or
files that should be written. We then
make sure that each resource is
properly configured, only taking
corrective action when it's
neccessary. The result is a safe,
flexible mechanism for making sure
your servers are always running
exactly how you want them to be.
EDIT: Note Chef is not a Python tool, it is a general purpose tool, written in Ruby (it seems). But it is capable of supporting various "cookbooks", including one for installing/maintaining Python apps.

Practices while releasing the python/ruby/script based web applications on production

I am purely a windows programmer and spend all my time hacking VC++.
Recently I have been heading several web based applications and myself built applications with python (/pylons framework) and doing projects on rails. All the web projects are hosted on ubuntu linux.
The RELEASE procedures and check list we followed for building and releasing VC++ windows application are merely no more useful when it comes to script based language.
So we don't built any binaries now. I copied asp/php files into IIS folder through ftp server when using open source cms applications.
So FTP is the one of the way to host the files to the web server. Now we feel lazy or not so passionate to copy files via ftp instead we use the SVN checkout and we simply do svn update to get the latest copy.
Is SVN checkout and svn update are the right methods to update the latest build files into the server? Are there any downside in using svn update? Any better method to release the script/web based scripts into the production server?
PS: I have used ssh server at some extension on linux platform.

I would create a branch in SVN for every release of web application and when the release is ready there, I would check it out on the server and set to be run or move it into the place of the old version.

Is SVN checkout and svn update are the right methods to update the latest build files into the server?
Very, very good methods. You know what you got. You can go backwards at any time.
Are there any downside in using svn update? None.
Any better method to release the script/web based scripts into the production server?
What we do.
We do not run out of the SVN checkout directories. The SVN checkout directory is "raw" source sitting on the server.
We use Python's setup.py install to create the application in /opt/app/app-x.y directory tree. Each tagged SVN branch is also a branch in the final installation.
Ruby has gems and other installation tools that are probably similar to Python's.
Our web site's Apache and mod_wsgi configurations refer to a specific /opt/app/app-x.y version. We can then stage a version, do testing, do things like migrate data from production to the next release, and generally get ready.
Then we adjust our Apache and mod_wsgi configuration to use the next version.
Previous versions are all in place. And left in place. We'll delete them some day when they confuse us.

The one downside of doing an svn update on your web root is that the .svn directories can potentially be made public, so be careful about permissions on the web server.
That said, there are far better ways to deploy an app built with dynamic languages. In the Rails world Capistrano is a mature deployment tool, as well as Vlad the Deployer. Capistrano can easily be used for non-rails deployments
There are many deployment strategies based on version control. You could go through the tutorials and get some ideas.
I'd also like to add that even though we do not "build" (compile) the project in dynamic languages, we do "build" (test/integration) them. A serious project would use a continuous integration server to validate the integrated "build" of project on every commit or integration, well before it gets to production.

One downside of doing an svn update: though you can go back in time, to what revision do you go back to? You have to look it up. svn update pseudo-deployments work much cleaner if you use tags - in that case you'd be doing an svn switch to a different tag, not an svn update on the same branch or the trunk.
You want to tag your software with the version number something like 1.1.4 , and then have a simple script to zip it up application-1.1.4,zip, and deploy it - then you have automated repeatable releases and rollbacks as well as greater visibility into what is changing between releases.