I had a similar issue when running fast-cgi and I was told there is no way to fix it: Files being served are stale / cached ; Python as fcgi + web.py + nginx without doing custom work. I was told to use the python method, which invokes a local "web server" to host the python page.
Even doing that, the files served are stale / cached. If I make edits to the files, save and refresh, the python web server is still serving the stale / cached file.
The only way to get it to serve the modified file is to kill (ctrl+c) the script, and then restart...this takes about 5 seconds every-time and seriously impedes my development workflow.
Ideally any change to the script would be reflected next time the page is requested from the web server.
EDIT
#Jordan: Thanks for the suggestions. I've tried #2, which yields the following error:
app = web.application(urls, globals(), web.reloader)
AttributeError: 'module' object has no attribute 'reloader'
Per the documentation here: http://webpy.org/tutorial2.en
I then tried suggestion #4,
web.config.debug = True
Both still cause 'stale' files to get served.
Understandably you want a simple, set it up once and never worry about it again, solution. But you might be making this problem more difficult than it needs to be.
I generally write applications for an apache/modwsgi/nginx stack. If I have a caching problem, I just restart apache and voila, my python files are re-interpreted. I don't remember the commands to restart apache on all of my different boxes (mac's, ubuntu, centos, etc), and I shouldn't need to.
That is what command line aliases are for...
A python application is interpreted before it is run, and when run on a webserver, it is run once and should be considered stateless. This is unlike javascript running in a browser, which can be considered to have state since it is a continually running VM. You can edit javascript while it is running and that is probably fine for most applications of the language.
In python you generally write the code, run it, and if that doesn't work you start over. You don't edit the code in real time. That means you are knowingly saving the source and changing contexts to run it.
I am betting that you are editing your source from a Graphical IDE instead of a command-line editor like vi or emacs (I might be wrong, and I'm not saying there is anything 'wrong' with that). I only write iOS applications using an IDE, everything else I stick to ViM. Why? Because then I am always on the command line, and I am not distracted by anything (animations, mouse pointers, notifications). I finish writing my code, i quickly type ':wq' (write and quit), and then quickly type 'restartweb' (actually i usually type 're' then <\tab> to auto-complete) which is my alias to whatever the command to restart apache is. Voila my python is reinterpreted.
My point is that you should probably keep it simple and use something like an alias to solve your problem. It might not be the coolest thing you could do. But it is what Ninja coders have been doing for the last 20 years to get work done fast and simple.
Now obviously I only suggested a solution for apache, and I have never used web.py before. But the same possible solution still applies. Make a bash script that goes in your project directory, call it something like restart.bash. In it put something like:
rm -r *.pyc
Which will recursively remove all compiled pyc files, forcing your app to reload. Then make an alias in your ~/.bashrc that runs that file
Something like:
alias restartproject="bash /full/path/to/restart.bash"
Magical, now you have a solution that works everywhere, regardless of which type of web server you choose to run your application from.
Edit:
Now you have a solution that works everywhere but on a Windows IIS server. And if you are trying to run python from Windows, you should probably Stahp! hugz
We are using virtualenv right? :) We want to keep our python nice and system-agnostic so we can sell it to anyone right? :) And you should really check out ViM and emacs if you don't use them... you will bang your head against the wall for a week getting used to it, then never want to touch a mouse again after that.
Right, so Python is a compiled language when run on a web server. It's outputting a .pyc file that's the compiled version. Your goal is to tell the web server that the .pyc file is out of date and is no longer valid.
You have a few options:
Delete the relevant .pyc file
For web.py, use the reloader middleware
Send it a HUP signal (I'm lazy and usually do killall -SIGHUP python). You can do this automatically with a file watching tool like watchdog (thanks barracel).
web.config.debug = True should be the default in your application
None of those options are working for you?
Related
I have a website on my local server and I like to execute system commands on my local server with a button press in the html file that is displayed. Is there a way to either execute system commands like gpio write 0 1 or to run python scripts? And how can I get the output of a system command as a string, like /opt/vc/bin/vcgencmd measure_temp| egrep "[0-9.]{4,}" -o output e.g. 44.4?
Thanks
David
You will have to a webserver with some kind of server-side script. There's lots of ways you can do this. If you know PHP, that may be easiest. If you want to use python check out uwsgi.
Here is a pretty simple project I wrote with uwsgi that might help you get started if you go that route. I found a lot of the examples didn't help a lot, so you might have some luck with that code.
Edit: Actually, uwsgi on the pi is a pretty old version, and it does some weird things on ARM if you try to compile it.
I created a proof of concept for you here using gunicorn instead. Just follow the instructions under the Installing section.
I finished up some python tutorials and would like to go a bit further. I can open the IDLE and execute the code just fine by pressing f5 (save and run) on my desktop but that is the limit of my abilities. I would like to be able to execute the programs on a webpage
I tried simply uploading the file to my server, then browsing to it in chrome. I'm sure you know what happened: the url displayed text on the screen.
Since I am brand new to python, I am not sure where to start or even what questions to ask. Basically I would like to run the program in the browswer as if the browswer was the IDLE, or better yet, create an html/css button that runs the program when clicked.
I'd advise you to look into something like flask. It's a micro-framework that includes a basic web server. The documentation should get you most of the way that you want to go.
Running python on the web usually means that when you hit an URL, the server runs some kind of python script, and returns a string - the HTML content of the page you're requesting. You can't use python to 'script' the webpage as you'd use javascript.
You can run python in an interactive interpreter running within a webpage though - just check out try ipython.
Flask is good.
Cherrypy - http://www.cherrypy.org/ - is also a great choice for a really simple way to run python on the server.
Fundamentally, you just configure your web server to execute the file instead of display it. Typically you set this for *.py files but you could restrict it, say, to files in a particular directory. Apparently, your server already has such a setting for PHP files.
Wrapping Python with PHP (obviously) adds neither speed nor security or utility.
Down the line you might want to look at frameworks, mod_python, WSGI, etc, but for your immediate problem, those are severe overkill.
This is limited to static server-side code; JavaScript runs interactively in the visitor's browser, allowing for much richer user interaction. A server-side script runs when the browser attempts to load the page, and the page load finishes when the script is done. If you want something like IDLE in your browser, that's a JavaScript challenge rather than a Python task (and that particular wheel has already been invented, productized, marketed, and sold to the Americans: http://pythonfiddle.com/)
like this:
python -c "import urllib2; exec urllib2.urlopen("http://localhost:8000/test.py").read()"
I am attempting to build an educational coding site, similar to Codecademy, but I am frankly at a loss as to what steps should be taken. Could I be pointed in the right direction in including even a simple python interpreter in a webapp?
One option might be to use PyPy to create a sandboxed python. It would limit the external operations someone could do.
Once you have that set up, your website would take the code source, send it over ajax to your webserver, and the server would run the code in a subprocess of a sandboxed python instance. You would also be able to kill the process if it took longer than say 5 seconds. Then you return the output back as a response to the client.
See these links for help on a PyPy sandbox:
http://doc.pypy.org/en/latest/sandbox.html
http://readevalprint.com/blog/python-sandbox-with-pypy.html
To create a fully interactive REPL would be even more involved. You would need to keep an interpreter alive to each client on your server. Then accept ajax "lines" of input and run them through the interp by communicating with the running process, and return the output.
Overall, not trivial. You would need some strong dev skills to do this comfortably. You may find this task a bit daunting if you are just learning.
There's more to do here than you think.
The major problem is that you cannot let people run arbitrary Python code on your webserver. For example, what happens if they do
import os
os.system("rm -rf *.*")
So clearly you have to run this Python code securely. But then you have the problem of securing Python, which is basically impossible because of how dynamic it is. And so you'll probably have to run the Python shell in a virtual machine, which comes with its own headaches.
Have you seen e.g. http://code.google.com/p/google-app-engine-samples/downloads/detail?name=shell_20091112.tar.gz&can=2&q=?
One recent option for this is to use repl.
This option is awesome because the compilers are made using JavaScript so the compilation and execution is made in the user-side, meaning that the server is free of vulnerabilities.
They have compilers for: Python3, Python, Javascript, Java, Ruby, PHP...
I strongly recommend you to check their site at http://repl.it
Look into LXC Containers. They have a pretty cool api that you can use to create lightweight linux containers. You could run the subprocess commands inside that container that way the end user could not mess with your main server.
So I've been working on my first Django / Python project and I got my production server up and running. I was wondering if it's possible to make Python/FastCGI (not really sure which is responsible for the task) to recompile my code. As of right now, when I upload updated code, I need to restart the server for the changes to take place. I read that you can add some kind of mysite.fcgi file to lighttpd so it see's that you've updated the code, can you do the same for Nginx / FastCGI?
for anyone else that was interested in my question.. this is only a partial solution, but I ended up finding my answer here: How to gracefully restart django running fcgi behind nginx?
You can just run the script (I'm going to modify it a bit), everytime you edit your code and it will gracefully restart everything without dropping connections.
This is a general guide from the mod_wsgi project that outlines how you can monitor code changes from your app_wsgi.py and restart the current process if any of the modules have changed. You need to restart the Python process, because threads contending over modules could mean that a freshly reloaded module has outdated references from other modules that are still waiting to get discovered for reload.
If you want something that works nicely with nginx, Django and wsgi apps in general, take a peek at Spawning as your wsgi server. It's approach to code reloading is about as graceful as it gets.
It has great documentation, well documented request handling model and it just works, which makes it such a no-brainer to configure. You'd need less than five minutes from now to having your Django instance running on Spawning. Here's another topical blog to get your juices running.
Can I use os.system() or subprocess.call() to execute a Python program on a webserver?
I mean can I write these functions in a .py script and run it from a web browser and expect the program to be executed?
Thanks a lot.
EDIT:
Sorry for all the confusion, I am giving you more background to my problem.
The reason I am trying to do is this.
I have a Python program that accepts an XML file and returns me TTF file.
I run that program in terminal like this:
ttx somefile.xml
After which it does all the work and generates a ttf file.
Now when I deploy this script as a module on web server. I use a to allow user to browse and select the XML file.
Then I read the file data to temporary file and then pass the file to the module script to be executed like this:
ttx.main([temp_filename])
Is this right way to do it? Because at this point, I don't get any error in the log or in browser. I get blank screen.
When this didn't work, I was going to try os.system or subprocess.call
You do not use os.system or subprocess.call to execute something as a cgi process.
Maybe you should read the Python cgi tutorial here:
http://www.cs.virginia.edu/~lab2q/
If you want your cgi process to communicate with another process on your local machine, you might want to look at "REST frameworks" for Python.
So long as your server is configured to run CGI scripts (Apache's documentation for that is here, for example), yes, you can execute a python script from a webserver. Simply make sure the script is in the appropriate cgi-bin/ directory and that the file has executable permission on the server.
With regards to your comments:
You can, if you really want, explicitly allow other folders on the server to run executable code. I don't know what server you're using, but on Apache this is done by setting Option +ExecCGI for the folder you want. Again, see the docs I linked to.
You need to give an absolute path with respect to the server. As an example, a site I develop has the layout: /public_html/cgi-bin/ When I want to access .cgi or .py files, the url for the site is something like http://chess.narnia.homeunix.com/cgi-bin/index.cgi. You can also set up re-directs to certain files if you want.
One way to pass parameters through your browser is to append them to the URL like an HTTP POST method. Here's a good example of doing that.
Is that what you were looking for with your question, or did you want to actually invoke the python script with os.system()?
Yes, I do it all the time. Import as you would do normally, stick your .py in your cgi-bin folder and make sure the server is capable of handling python.
Another option would be to simply create an application on Google's App Engine. That gives you oodles of resources and APIs for Python execution.
http://code.google.com/appengine
I've done it quite a bit in classic ASP on IIS 5 and above. I would have the ASP engine execute python code (instead of, e.g., vbscript (hearkening back to the old days, here)). Behind those asp pages would be python modules written in straight python that could be imported and could execute pretty much arbitrary code. As others have mentioned, the effective user needs to have execute permission on the thing you're trying to execute.