I'm running apache, Django and wsgi. I also use this other software called SAS to do statistical analysis. Just to give you some context. My end goal is when a client hits submit on a form written in the django, the appropriate sas script is called (via a python wsgi script) which performs calculations on the server, and then redirects the client to the output page.
I have a basic script called test5.py. It looks like this:
import os
import subprocess
def application(environ, start_response):
status = '200 OK'
output = 'Running External Program!'
f = open("C:\Documents and Settings\eric\Desktop\out.txt", 'a')
f.write('hi')
f.close()
#os.system(r'start "C:\Program Files\SAS92\SASFoundation\9.2\sas.exe"')
#subprocess.call([r'C:\Program Files\SAS92\SASFoundation\9.2\sas.exe'])
#os.startfile(r'C:\Program Files\SAS92\SASFoundation\9.2\sas.exe')
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
#start_response('301 Redirect', [('Location', 'http://myserver/reports'),])
start_response(status, response_headers)
return [output]
So what happens is that the out.txt file does get created and have hi written in the file. That's quite cool. The first 3 commented lines were 3 attempts to have this same script also call sas.exe which lives on the server. I'm just trying to get any .exe to work right now, so calling paint or wordpad would be fine. Those lines however do not seems to execute in the wsgi context. If I just load the Python command line, I can get the .exes to execute just fine. Also the last comment seems to be working properly in the redirecting. I'm not sure if I need to configure apache to add executables. Please forgive me if I'm using terms incorrectly. I am still quite new to all of this.
Thanks
Hi Paulo,
I was trying to look into your last comment. I am a bit confused as to exactly what i am looking for or how to look for it. Here is some information that I have gathered. By the way i am running on windows XP and using Apache 2.2.
My apache is installed for all users as in regedit the variable ServerRoot is under HKEY_LOCAL_MACHINE (http://httpd.apache.org/docs/2.2/platform/windows.html). Also I believe SAS is installed under all users. I tested this by having my coworker sign in using her login and I still had access. I’m not sure if that is a sufficient test though.
The log I get when I run the wsgi is the following. I’m not sure if it matters that the process is empty.
[Mon Aug 20 10:33:17 2012] [info] [client 10.60.8.71] mod_wsgi (pid=5980, process='', application='..com|/test5'): Reloading WSGI script 'C:/Sites/cprm/pyscripts/test5.wsgi'.
Also I tried the .bat trick from the link I posted in the comment i posted earlier to no avail. I made a simple batch file that just echoes 'hi' and placed it in the same directory where my wsgi scripts live. I feel like there should be no access problems there, but I may be mistaken. I also just tried calling a simple python script using subprocess just to test. Also nothing happened.
Also just to show you, my httpd.conf file looks like such:
AllowOverride None
Options None
Order allow,deny
Allow from all
WSGIScriptAlias /test1 "C:/sites/cprm/pyscripts/test1.wsgi"
WSGIScriptAlias /test2 "C:/sites/cprm/pyscripts/test2.py"
WSGIScriptAlias /test3 C:/sites/cprm/pyscripts/test3.py
WSGIScriptAlias /test4 "C:/sites/cprm/pyscripts/test4.py"
WSGIScriptAlias /test5 "C:/sites/cprm/pyscripts/test5.wsgi"
WSGIScriptAlias / "C:/sites/cprm/wsgi.py"
Is this information helpful or not really? Also, am i looking for a specific environ variable or something?
Thanks again
For web applications that perform background calculations or other tasks, IMHO it is best to queue the tasks for processing instead of calling an external process from a Django view and hang everything until the task completes. This leads to better:
user experience (request returns instantly - use ajax to signal task status and present the download link once task completes)
security (background process can run under safer credentials)
scalability (tasks can be distributed among servers)
resilience (by default many webservers will send an 'error 500' if your application fails to answer under 30 seconds or so)
For a background daemon processing all entries in the queue, there are several approaches depending on how big you want to scale:
a cron job
a daemon using supervisor (or your watchdog of choice)
an AMQP module like django-celery
[edit]
The process you start from a WSGI script will run under the same user that is running the webserver. In linux it is generally 'www-data' or 'nobody', in Windows/IIS it is 'IUSR_MachineName' (or authenticated user if using IIS authentication). Check if you can start the program using the same credentials your WSGI is running under.
Related
I have just found out this "feature" under my web hosting cPanel called "Setup Python App" and very much would like to play around with a nice flask app if I could however it does not seem to work so I come here looking for some help. Here is a brief description of the setup and it's symptoms:
Clicking on this takes me to a very simple page asking for a directory and URI however when "setting it up" not much information is reported back but a directory is generated with a passenger_wsgi.py file with what looks like a basic server setup however trying to land on it simply downloads the file.
passenger_wsgi.py
import os
import sys
sys.path.insert(0, os.path.dirname(__file__))
def application(environ, start_response):
start_response('200 OK', [('Content-Type', 'text/plain')])
message = 'It works!\n'
version = 'Python %s\n' % sys.version.split()[0]
response = '\n'.join([message, version])
return [response.encode()]
To solve this I figured I would setup an Apache Handler to include .py under the cgi-script handlers which then makes landing on this page a simple 500 Server Error.
I am running out of ideas on how to get this working and can't find any documentation about this "Setup Python App" feature. If anyone has had experience with it I would very much appreciate some assistance. I have already reached out to web hosting support and they also do not know what is going on. I do suspect that there is a phusion passenger server that gets setup in the process but not correctly maybe? Also there doesn't seem to be a way for me to know which port it is running under and no redirect is automatically generated (I say this because when generating a Rails app an internal redirect is automatically created pointing to the correct internal port, which I would expect to happen here too).
I've had the Python file set at 644 and added the following to the top of my .htaccess file to allow both root URL and non-root URL to work:
RewriteEngine on
RewriteRule ^http://%{HTTP_HOST}%{REQUEST_URI} [END,NE]
Got this info from: https://stackoverflow.com/a/63971427/10122266
Apologies that I couldn't add this as a comment as I do not have sufficient rights yet. But this solution worked for me and it would be a pitty if this little gem is hidden from those who may arrive here.
So would appreciate if this post is not deleted.
I am trying to get a grip on mod_wsgi by writing a simple WSGI server running under Apache on Linux. However I notice that requests are not always served in the order in which they arrive, even if they come from the same client/browser.
If a page e.g. contains images A and B respectively, A may be served as answer to the request for B, so it is shown at the wrong place. I'm convinced I must be making a very trivial mistake, but I am unable to find out what.
I am aware that the WSGI callable must be reentrant, and by logging requests and responses, I see that indeed it is sometimes entered a second time before the first result is served. But surely when the browser asks for B it should not get A as result from a previous GET. Or am I missing something very fundamental about HTTP?
Global Apache directives:
LoadModule wsgi_module /home/sterlicht/modWsgi/mod_wsgi.so
Virtual host directives:
WSGIScriptAlias / /home/sterlicht/debug/app.py
Found out what was wrong.
My callable was not reentrant after all.
The code in the call member of my application class used some instance data, which were overwritten by concurrent invocations.
Later note: the issues in the original posting below have been largely resolved.
Here's the background: For an introductory comp sci course, students develop html and server-side Python 2.7 scripts using a server provided by the instructors. That server is based on CGIHTTPRequestHandler, like the one at pointlessprogramming. When the students' html and scripts seem correct, they port those files to a remote, slow Apache server. Why support two servers? Well, the initial development using a local server has the benefit of reducing network issues and dependency on the remote, weak machine that is running Apache. Eventually porting to the Apache-running machine has the benefit of publishing their results for others to see.
For the local development to be most useful, the local server should closely resemble the Apache server. Currently there is an important difference: Apache requires that a script start its response with headers that include a content-type; if the script fails to provide such a header, Apache sends the client a 500 error ("Internal Server Error"), which too generic to help the students, who cannot use the server logs. CGIHTTPRequestHandler imposes no similar requirement. So it is common for a student to write header-free scripts that work with the local server, but get the baffling 500 error after copying files to the Apache server. It would be helpful to have a version of the local server that checks for a content-type header and gives a good error if there is none.
I seek advice about creating such a server. I am new to Python and to writing servers. Here are the issues that occur to me, but any helpful advice would be appreciated.
Is a content-type header required by the CGI standard? If so, other people might benefit from an answer to the main question here. Also, if so, I despair of finding a way to disable Apache's requirement. Maybe the relevant part of the CGI RFC is section 6.3.1 (CGI Response, Content-Type): "If an entity body is returned, the script MUST supply a Content-Type field in the response."
To make a local server that checks for the content-type header, perhaps I should sub-class CGIHTTPServer.CGIHTTPRequestHandler, to override run_cgi() with a version that issues an error for a missing header. I am looking at CGIHTTPServer.py __version__ = "0.4", which was installed with Python 2.7.3. But run_cgi() does a lot of processing, so it is a little unappealing to copy all its code, just to add a couple calls to a header-checking routine. Is there a better way?
If the answer to (2) is something like "No, overriding run_cgi() is recommended," I anticipate writing a version that invokes the desired script, then checks the script's output for headers before that output is sent to the client. There are apparently two places in the existing run_cgi() where the script is invoked:
3a. When run_cgi() is executed on a non-Unix system, the script is executed using Python's subprocess module. As a result, the standard output from the script will be available as an in-memory string, which I can presumably check for headers before the call to self.wfile.write. Does this sound right?
3b. But when run_cgi() is executed on a *nix system, the script is executed by a forked process. I think the child's stdout will write directly to self.wfile (I'm a little hazy on this), so I see no opportunity for the code in run_cgi() to check the output. Ugh. Any suggestions?
If analyzing the script's output is recommended, is email.parser the standard way to recognize whether there is a content-type header? Is another standard module recommended instead?
Is there a more appropriate forum for asking the main question ("How can a CGI server based on CGIHTTPRequestHandler require...")? It seems odd to ask if there is a better forum for asking programming questions than Stack Overflow, but I guess anything is possible.
Thanks for any help.
I'm maintaining an open-source document asset management application called NotreDAM, which is written in Django running on Apache an instance of TwistedWeb.
Whenever any user downloads a file, the application hangs for all users for the entire duration of the download. I've tracked down the download command to this point in the code, but I'm not enough versed with Python/Django to know why this may be happening.
response = HttpResponse(open(fullpath, 'rb').read(), mimetype=mimetype)
response["Last-Modified"] = http_date(statobj.st_mtime)
response["Content-Length"] = statobj.st_size
if encoding:
response["Content-Encoding"] = encoding
return response
Do you know how I could fix the application hanging while a file downloads?
The web server reads the whole file in the memory instead of streaming it. It is not well written code, but not a bug per se.
This blocks the Apache client (pre-forked) for the duration of whole file read. If IO is slow and the file is large it may take some time.
Usually you have several pre-forked Apache clients configured to satisfy this kind of requests, but on a badly configured web server you may exhibit this kind of problems and this is not a Django issue. Your web server is probably running only one pre-forked process, potentially in a debug mode.
notreDAM serves the asset files using the django.views.static.serve() command, which according to the Django docs "Using this method is inefficient and insecure. Do not use this in a production setting. Use this only for development." So there we go. I have to use another command.
I have a j2me client that would post some chunked encoded data to a webserver. I'd like to process the data in python. The script is being run as a CGI one, but apparently apache will refuse a chunked encoded post request to a CGI script. As far as I could see mod_python, WSGI and FastCGI are no go too.
I'd like to know if there is a way to have a python script process this kind of input. I'm open to any suggestion (e.g. a confoguration setting in apache2 that would assemble the chunks, a standalone python server that would do the same, etc.) I did quite a bit of googling and didn't find anything usable, which is quite strange.
I know that resorting to java on the server side would be a solution, but I just can't imagine that this can't be solved with apache + python.
I had the exact same problem a year ago with a J2ME client talking to a Python/Ruby backend. The only solution I found which doesn't require application or infrastructure level changes was to use a relatively unknown feature of mod_proxy.
Mod_proxy has the ability to buffer incoming (chunked) requests, and then rewrite them as a single request with a Content-Length header before passing them on to a proxy backend. The neat trick is that you can create a tiny proxy configuration which passes the request back to the same Apache server. i.e. Take an incoming chunked request on port 80, "dechunk" it, and then pass it on to your non-HTTP 1.1 compliant server on port 81.
I used this configuration in production for a little over a year with no problems. It looks a little something like this:
ProxyRequests Off
<Proxy http://example.com:81>
Order deny,allow
Allow from all
</Proxy>
<VirtualHost *:80>
SetEnv proxy-sendcl 1
ProxyPass / http://example.com:81/
ProxyPassReverse / http://example.com:81/
ProxyPreserveHost On
ProxyVia Full
<Directory proxy:*>
Order deny,allow
Allow from all
</Directory>
</VirtualHost>
Listen 81
<VirtualHost *:81>
ServerName example.com
# Your Python application configuration goes here
</VirtualHost>
I've also got a full writeup of the problem and my solution detailed on my blog.
I'd say use the twisted framework for building your http listener.
Twisted supports chunked encoding.
http://python.net/crew/mwh/apidocs/twisted.web.http._ChunkedTransferEncoding.html
Hope this helps.
Apache 2.2 mod_cgi works fine for me, Apache transparently unchunks the request as it is passed to the CGI application.
WSGI currently disallows chunked requests, and mod_wsgi does indeed block them with a 411 response. It's on the drawing board for WSGI 2.0. But congratulations on finding something that does chunk requests, I've never seen one before!
You can't do what you want with mod_python. You can do it with mod_wsgi if you are using version 3.0. You do however have to step outside of the WSGI 1.0 specification as WSGI effectively prohibits chunked request content.
Search for WSGIChunkedRequest in http://code.google.com/p/modwsgi/wiki/ChangesInVersion0300 for what is required.
Maybe it is a configuration issue? Django can be fronted with Apache by mod_python, WSGI and FastCGI and it can accept file uploads.