Can I use os.system() or subprocess.call() to execute a Python program on a webserver?
I mean can I write these functions in a .py script and run it from a web browser and expect the program to be executed?
Thanks a lot.
EDIT:
Sorry for all the confusion, I am giving you more background to my problem.
The reason I am trying to do is this.
I have a Python program that accepts an XML file and returns me TTF file.
I run that program in terminal like this:
ttx somefile.xml
After which it does all the work and generates a ttf file.
Now when I deploy this script as a module on web server. I use a to allow user to browse and select the XML file.
Then I read the file data to temporary file and then pass the file to the module script to be executed like this:
ttx.main([temp_filename])
Is this right way to do it? Because at this point, I don't get any error in the log or in browser. I get blank screen.
When this didn't work, I was going to try os.system or subprocess.call
You do not use os.system or subprocess.call to execute something as a cgi process.
Maybe you should read the Python cgi tutorial here:
http://www.cs.virginia.edu/~lab2q/
If you want your cgi process to communicate with another process on your local machine, you might want to look at "REST frameworks" for Python.
So long as your server is configured to run CGI scripts (Apache's documentation for that is here, for example), yes, you can execute a python script from a webserver. Simply make sure the script is in the appropriate cgi-bin/ directory and that the file has executable permission on the server.
With regards to your comments:
You can, if you really want, explicitly allow other folders on the server to run executable code. I don't know what server you're using, but on Apache this is done by setting Option +ExecCGI for the folder you want. Again, see the docs I linked to.
You need to give an absolute path with respect to the server. As an example, a site I develop has the layout: /public_html/cgi-bin/ When I want to access .cgi or .py files, the url for the site is something like http://chess.narnia.homeunix.com/cgi-bin/index.cgi. You can also set up re-directs to certain files if you want.
One way to pass parameters through your browser is to append them to the URL like an HTTP POST method. Here's a good example of doing that.
Is that what you were looking for with your question, or did you want to actually invoke the python script with os.system()?
Yes, I do it all the time. Import as you would do normally, stick your .py in your cgi-bin folder and make sure the server is capable of handling python.
Another option would be to simply create an application on Google's App Engine. That gives you oodles of resources and APIs for Python execution.
http://code.google.com/appengine
I've done it quite a bit in classic ASP on IIS 5 and above. I would have the ASP engine execute python code (instead of, e.g., vbscript (hearkening back to the old days, here)). Behind those asp pages would be python modules written in straight python that could be imported and could execute pretty much arbitrary code. As others have mentioned, the effective user needs to have execute permission on the thing you're trying to execute.
Related
I have a server side Python script that imports a big package called nltk. It ran at a command prompt, but would not run in Apache server.
I tried the logging and put it as the first import and created a log file immediately after. But nothing is written to the file before the script crashes.
Is there a way to see the "ImportError: no module ..." when the script runs on Apache?
As far as I'm aware, there's no built-in way to tell Apache to simply dump error log messages directly to the browser. I'm betting the error_log output happens in an independent part of the Apache pipeline -- just a guess.
Most of the time when someone wants output errors directly to the browser, that person is a developer working in a web script such as PHP, Python, Perl, or other lang. In order to output error messages that occur within your lang interpretter, you have to format the output and pass it through apache as if it was normal output.
PHP
Some languages provide an easy way to do this, such as the well known error_reporting switch in PHP.
Python
Unfortunately some languages, make this painful or almost impossible and Python is one of them.
At one time this was apparently possible using https://docs.python.org/3/library/cgitb.html
However that library has been deprecated. Seems like the Python community doesn't have much love for developers who want to get immediate feedback about errors in the browser. :(
I have a python script which executes a string of code with the exec function. I need a way to restrict the read/write access of the script to the current directory. How can I achieve this?
Or, is there a way to restrict the python script's environment directly through the command line so that when I run the interpreter, it does not allow writes out of the directory? Can I do that using a virtualenv? How?
So basically, my app is a web portal where people can write and execute python apps and get a response - and I've hosted this on heroku. Now there might be multiple users with multiple folders and no user should have access to other's folders or even system files and folders. The permissions should be determined by the user on the nodejs app (a web app) and not a local user. How do I achieve that?
Execute the code as a user that only owns that specific directory and has no permissions anywhere else?
However- if you do not completely trust the source of code, you should simply not be using exec under any circumstances. Remember, say you came up with a python solution... the exec code could literally undo whatever restrictions you put on it before doing its nefarious deeds. If you tell us the problem you're trying to solve, we can probably come up with a better idea.
The question boils down to: How can I safely execute the code I don't trust.
You can't.
Either you know what the code does or you don't execute it.
You can have an isolated environment for your process, for example with docker. But the use cases are far away from executing unsafe code.
I'm doing a python class this semester, and I'd like to add graphical user interfaces to my programs in the form of web pages. Partly I can't be bothered to learn Tkinter, partly I'm just challenging myself, and partly I just like coding interfaces in HTML/JS.
I know the basics on creating HTTP servers with SimpleHTTPServer, but what I need is a way for my web pages to fire ajax commands to python scripts on the server, which will then be executes server side, and then receive the script output. Kind of like how Sage does things.
How can I create an extension for SimpleHTTPServer that I can then use to serve up the outputs of python scripts? I need a solution that's very general, so that given any python script I write, I can easily wrap it in some sort of interface, put it in the server's folder, and not have to do anything else but call it with AJAX, and my server will handle the rest.
My current idea is to have a ServerSideScript class that my scripts can extend, with a single function called output. Every script I have should contain a subclass of ServerSideScript named Script. When the server is asked to serve, for example, foo.py, it notices the extension and does something like:
if self.path[-3:]: == ".py":
return getScriptOutput(self.path)
...
def getScriptOutput(self, path):
from path import Script # obviously not going to work, since path is a string
return Script().output()
That can't work for the reason pointed out in the comment, I'm aware of things like the import_module function, but importing seems like an ugly way to do this to begin with.
What's recommended?
Python comes with batteries included; CGIHTTPServer will run Python scripts using the CGI standard:
The CGIHTTPServer module defines a request-handler class, interface compatible with BaseHTTPServer.BaseHTTPRequestHandler and inherits behavior from SimpleHTTPServer.SimpleHTTPRequestHandler but can also run CGI scripts.
It'll run Python scripts as long as they have a .py or .pyw extension, and are executable; other paths mapping to files are treated as regular files and have their content served instead.
You may be interested in the cgi and cgitb modules as well, to help ease CGI script development.
You might want to take a look at IPython notebooks - it essentially gives you a web-based interactive Python shell, which even integrates with matplotlib etc. You can save Python scripts as notebooks, add text inbetween statements etc.
You can also use the nbviewer if you want to share the output, but not the interactivity.
I finished up some python tutorials and would like to go a bit further. I can open the IDLE and execute the code just fine by pressing f5 (save and run) on my desktop but that is the limit of my abilities. I would like to be able to execute the programs on a webpage
I tried simply uploading the file to my server, then browsing to it in chrome. I'm sure you know what happened: the url displayed text on the screen.
Since I am brand new to python, I am not sure where to start or even what questions to ask. Basically I would like to run the program in the browswer as if the browswer was the IDLE, or better yet, create an html/css button that runs the program when clicked.
I'd advise you to look into something like flask. It's a micro-framework that includes a basic web server. The documentation should get you most of the way that you want to go.
Running python on the web usually means that when you hit an URL, the server runs some kind of python script, and returns a string - the HTML content of the page you're requesting. You can't use python to 'script' the webpage as you'd use javascript.
You can run python in an interactive interpreter running within a webpage though - just check out try ipython.
Flask is good.
Cherrypy - http://www.cherrypy.org/ - is also a great choice for a really simple way to run python on the server.
Fundamentally, you just configure your web server to execute the file instead of display it. Typically you set this for *.py files but you could restrict it, say, to files in a particular directory. Apparently, your server already has such a setting for PHP files.
Wrapping Python with PHP (obviously) adds neither speed nor security or utility.
Down the line you might want to look at frameworks, mod_python, WSGI, etc, but for your immediate problem, those are severe overkill.
This is limited to static server-side code; JavaScript runs interactively in the visitor's browser, allowing for much richer user interaction. A server-side script runs when the browser attempts to load the page, and the page load finishes when the script is done. If you want something like IDLE in your browser, that's a JavaScript challenge rather than a Python task (and that particular wheel has already been invented, productized, marketed, and sold to the Americans: http://pythonfiddle.com/)
like this:
python -c "import urllib2; exec urllib2.urlopen("http://localhost:8000/test.py").read()"
I had a similar issue when running fast-cgi and I was told there is no way to fix it: Files being served are stale / cached ; Python as fcgi + web.py + nginx without doing custom work. I was told to use the python method, which invokes a local "web server" to host the python page.
Even doing that, the files served are stale / cached. If I make edits to the files, save and refresh, the python web server is still serving the stale / cached file.
The only way to get it to serve the modified file is to kill (ctrl+c) the script, and then restart...this takes about 5 seconds every-time and seriously impedes my development workflow.
Ideally any change to the script would be reflected next time the page is requested from the web server.
EDIT
#Jordan: Thanks for the suggestions. I've tried #2, which yields the following error:
app = web.application(urls, globals(), web.reloader)
AttributeError: 'module' object has no attribute 'reloader'
Per the documentation here: http://webpy.org/tutorial2.en
I then tried suggestion #4,
web.config.debug = True
Both still cause 'stale' files to get served.
Understandably you want a simple, set it up once and never worry about it again, solution. But you might be making this problem more difficult than it needs to be.
I generally write applications for an apache/modwsgi/nginx stack. If I have a caching problem, I just restart apache and voila, my python files are re-interpreted. I don't remember the commands to restart apache on all of my different boxes (mac's, ubuntu, centos, etc), and I shouldn't need to.
That is what command line aliases are for...
A python application is interpreted before it is run, and when run on a webserver, it is run once and should be considered stateless. This is unlike javascript running in a browser, which can be considered to have state since it is a continually running VM. You can edit javascript while it is running and that is probably fine for most applications of the language.
In python you generally write the code, run it, and if that doesn't work you start over. You don't edit the code in real time. That means you are knowingly saving the source and changing contexts to run it.
I am betting that you are editing your source from a Graphical IDE instead of a command-line editor like vi or emacs (I might be wrong, and I'm not saying there is anything 'wrong' with that). I only write iOS applications using an IDE, everything else I stick to ViM. Why? Because then I am always on the command line, and I am not distracted by anything (animations, mouse pointers, notifications). I finish writing my code, i quickly type ':wq' (write and quit), and then quickly type 'restartweb' (actually i usually type 're' then <\tab> to auto-complete) which is my alias to whatever the command to restart apache is. Voila my python is reinterpreted.
My point is that you should probably keep it simple and use something like an alias to solve your problem. It might not be the coolest thing you could do. But it is what Ninja coders have been doing for the last 20 years to get work done fast and simple.
Now obviously I only suggested a solution for apache, and I have never used web.py before. But the same possible solution still applies. Make a bash script that goes in your project directory, call it something like restart.bash. In it put something like:
rm -r *.pyc
Which will recursively remove all compiled pyc files, forcing your app to reload. Then make an alias in your ~/.bashrc that runs that file
Something like:
alias restartproject="bash /full/path/to/restart.bash"
Magical, now you have a solution that works everywhere, regardless of which type of web server you choose to run your application from.
Edit:
Now you have a solution that works everywhere but on a Windows IIS server. And if you are trying to run python from Windows, you should probably Stahp! hugz
We are using virtualenv right? :) We want to keep our python nice and system-agnostic so we can sell it to anyone right? :) And you should really check out ViM and emacs if you don't use them... you will bang your head against the wall for a week getting used to it, then never want to touch a mouse again after that.
Right, so Python is a compiled language when run on a web server. It's outputting a .pyc file that's the compiled version. Your goal is to tell the web server that the .pyc file is out of date and is no longer valid.
You have a few options:
Delete the relevant .pyc file
For web.py, use the reloader middleware
Send it a HUP signal (I'm lazy and usually do killall -SIGHUP python). You can do this automatically with a file watching tool like watchdog (thanks barracel).
web.config.debug = True should be the default in your application
None of those options are working for you?