I was having issues with my Django app (running on Debian with Apache 2.2.22) serving occasional bursts of 500 errors with the mod_wsgi "Premature Script exit" errors in the logs. I dug around and it looked like the crashes might be a result of running out of memory, as a result I found a bad memory hog bug and fixed it. However at the same time I was reading about mod_wsgi configs and decided to make a couple other changes to my setup. I added...
stack-size=524288 maximum-requests=500 inactivity-timeout=5
to my Apache config, making it look like this...
WSGIScriptAlias / /home/myapp/src/myapp-api/myapp/wsgi.py
WSGIDaemonProcess myapp user=me group=team processes=4 threads=16 stack-size=524288 maximum-requests=500 inactivity-timeout=5 python-path=/home/myapp/src/myapp-api:/home/myapp/.virtualenvs/myapp/lib/python2.7/site-packages
WSGIProcessGroup myapp
WSGIPassAuthorization On
I tested it out on my dev servers and all seemed fine, but once I deployed this to servers under more load I saw a large number of sporadic 500 errors, again logging "Premature Script exit" but this time there was no memory issues and the system had plenty of free RAM.
I'm at a loss as to what caused this. The only other error that was new in the logs was
[Wed Jan 14 20:17:16 2015] [error] Exception AttributeError: "'NoneType' object has no attribute 'error'" in <bound method TCPTransport.__del__ of <amqp.transport.TCPTransport object at 0x7ff4b0e69d10>> ignored
which seems to relate to my usage of Celery and AMQP, so perhaps this is a problem with C module interop?
If the stack size issue was a problem I would expect to see errors in my django code, not in mod_wsgi, so what else could cause this?
Related
I am trying to load a Flask app in a subdirectory on my domain. Below is my virtual host configuration. the Flask app is kept at /var/www/FlaskApps/Recommender/
Apache's error log keeps saying "Target WSGI script not found or unable to stat: /var/www/FlaskApps/FlaskApp.wsgi" when i try to reach the endpoint /recommender
From online other people said it might be a permission issue, but I'm not sure if it's the case per snapshot below.
Thanks for your help
You have FlaskApps.wsgi in directory, but FlaskApp.wsgi in WSGIScriptAlias directive. So they don't match.
I'm trying to use mod_wsgi-express for a django project. This project will be served through the main instance of apache, with a proxy. My purpose is to serve a python 3 application with a mod_wsgi for python 2 installed on the "main" apache server. (I've many apps on the same server)
I've created a systemd service which should launch the mod_wsgi-express instance, using my own service file :
https://gist.github.com/frague59/87529fc28b098dd116f09be92cf66af0
and
https://gist.github.com/frague59/8de1d03800042db95c82452af280dffe
I've to mention that those scripts works on another server, same distro, same version (debian oldstable)
... but the mod_wsgi-express does not start : no error message, no log...
I've noticed that a dir is created in tmp:
/tmp/mod_wsgi-127.0.0.1:8081:993
I've tried to start the apachectl from here:
# apachectl configtest
And I have a weird message:
AH00526: Syntax error on line 241 of /tmp/mod_wsgi-127.0.0.1:8081:993/httpd.conf:
Invalid option to WSGI daemon process definition.
I posts the generated httpd.conf file for example:
https://gist.github.com/frague59/a6d8d26b704565b39f7352a7c16e07d3
Error seems to be around:
send-buffer-size=0 \
I am taking over a django project which another developer maintained. The service is run on an Ubuntu machine. ZEO is used for content caching. Ajax/Dajax is used for asynchronious content. Celery is used for task management and Django is used for the project itself.
The service is usually reached via a specific IP address which limits access to specific URLs. http://my_server_ip. Without knowingly changing anything, this started to not work. Instead of taking me to the splash page, entering the IP would hang, unsuccesfully connecting. I don't get a 404, 500 or some other error, it just sits and continually tries to load as if waiting to connect or to receive content.
I attempted to restart the service in the hopes that this would solve the problem, it did not. I performed a system reboot and followed the following commands, as per the prior developer's documentation, to reboot the server.
From within the django project:
runzeo -a localhost:8090 -f /path/to/operations_cache.fs
su project_owner
python manage.py celery worker -n multiprocessing_worker --loglevel=debug -Q multiprocessing_queue
python manage.py celery worker --concurrency=500 --pool=eventlet --loglevel=debug -Q celery -n eventlet_worker
The two celery commands had to be run as the owner of the project directory.
Finally I ran sudo service apache2 restart. Upon completion, I tried to navigate to the webpage but received the same response: hung on connecting. The trac pages do work at http://my_server_ip/trac.
The following is all I have found in the apache log files.
error.log
[Fri Feb 06 16:01:11 2015] [error] /usr/lib/python2.7/dist-packages/configobj.py:145: DeprecationWarning: The compiler package is deprecated and removed in Python 3.x.
[Fri Feb 06 16:01:11 2015] [error] import compiler
[Fri Feb 06 16:01:11 2015] [error]
access.log
my.ip - user [06/Feb/2015:15:55:40 -0500] "GET / HTTP/1.1" 500 632 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0 Gecko/20100101 Firefox/35.0"
I have tried looking into the django logs, but nothing appears there. Perhaps I am not finding the correct logs.
As a good start, where can I find out what the system is hanging on. How do I determine if it is an issue with django, apache, the interaction between the two, etc. This would help me zero in on specifically what is happening.
Edit
I was able to resolve my problem, though I cannot say for sure what resolved the issue. I suspect that the issue had to do with permissions on the static files folder. I serve my content through the www-user. After going through the steps described above I then ran yes yes | python manage.py collectstatic as the user www-data. The server was able restart and I was able to access the trac content as well as the django content.
I am hesitant to post this as an answer because I do not know with certainty whether my solution described here is the step which solved the problem.
Just installed and configured mod_python 3.2.8 on a CentOS 5 (Apache 2.2.3) server with Python 2.4.3. It is loaded fine by Apache.
I activated the mpinfo test page and it works. So I wrote a simple "Hello World" with the following code:
from mod_python import apache
def handler(req):
req.content_type = 'text/plain'
req.write("Hello World!")
req.flush()
return apache.OK
It outputs a blank page, with no text and no source. If I consciously create a syntax error I get the error output on the URL, for example (when I put a space before "def"):
Mod_python error: "PythonHandler mod_python.cgihandler"
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/mod_python/apache.py", line 299, in HandlerDispatch
result = object(req)
File "/usr/lib/python2.4/site-packages/mod_python/cgihandler.py", line 96, in handler
imp.load_module(module_name, fd, path, desc)
File "/var/www/vhosts/localhost/httpdocs/mptest.py", line 3
def handler(req):
^
SyntaxError: invalid syntax
I have spent about five hours browsing different tutorials, faqs and trouble shooting guides but can't find a description of this exakt issue.
What do you think could be the issue/cause?
EDIT: Here is the Apache configuration for the site...
<Directory />
Options FollowSymLinks
AllowOverride None
AddHandler mod_python .py
PythonHandler mptest
PythonDebug On
</Directory>
EDIT 2: Ah, another thing I forgot to mention is that I intend to use mod_python to write Apache extensions. The application itself is written in PHP but I need to make some security tweeks on the server :)
Don't use mod_python.
A common mistake is to take mod_python as "mod_php, but for python" and that is not true. mod_python is more suited to writing apache extensions, not web applications.
The standartized protocol to use between python web applications and web servers (not only apache) is WSGI. Using it ensures that you can publish your application to any wsgi-compliant webserver (almost all modern web servers are wsgi-compliant)
On apache, use mod_wsgi instead.
Your example rewritten using the wsgi standard and mod_wsgi on apache:
mywebapp.py:
def application(environ, start_response):
start_response('200 OK', [('content-type', 'text/plain')])
return ['Hello World']
Apache configuration:
WSGIScriptAlias /myapp /usr/local/www/wsgi-scripts/mywebapp.py
<Directory /usr/local/www/wsgi-scripts>
Order allow,deny
Allow from all
</Directory>
Now just go to http://localhost/myapp and the script will run. Additionally, any access under this root (i.e. http://localhost/myapp/stuff/here) will be handled by this script.
It's a good idea to choose a web framework. CherryPy. Pylons. Django. They make things even easier.
A good website to look at is wsgi.org
Your original problem is that mod_python.cgihandler is being called to handle the request. This means your Python script file is being interpreted as a CGI script. Thus, no wonder it doesn't return anything.
You likely have conflicting definition in your Apache configuration which is enabling the mod_python.cgihandler.
I make a complete new answer for clarity...
I decided to install mod_wsgi instead. So I've set it up and when I go to my testfile I just see the page source. I haven't been spending any time on finding the issue yet, so I'll get back to you when I either solve the problem or decide that I need more help :)
Thank you :)
Is there a way to have mod_wsgi reload all modules (maybe in a particular directory) on each load?
While working on the code, it's very annoying to restart apache every time something is changed. The only option I've found so far is to put modname = reload(modname) below every import.. but that's also really annoying since it means I'm going to have to go through and remove them all at a later date..
The link:
http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode
should be emphasised. It also should be emphaised that on UNIX systems daemon mode of mod_wsgi must be used and you must implement the code monitor described in the documentation. The whole process reloading option will not work for embedded mode of mod_wsgi on UNIX systems. Even though on Windows systems the only option is embedded mode, it is possible through a bit of trickery to do the same thing by triggering an internal restart of Apache from the code monitoring script. This is also described in the documentation.
The following solution is aimed at Linux users only, and has been tested to work under Ubuntu Server 12.04.1
To run WSGI under daemon mode, you need to specify WSGIProcessGroup and WSGIDaemonProcess directives in your Apache configuration file, for example
WSGIProcessGroup my_wsgi_process
WSGIDaemonProcess my_wsgi_process threads=15
More details are available in http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives
An added bonus is extra stability if you are running multiple WSGI sites under the same server, potentially with VirtualHost directives. Without using daemon processes, I found two Django sites conflicting with each other and turning up 500 Internal Server Error's alternatively.
At this point, your server is in fact already monitoring your WSGI site for changes, though it only watches the file you specified using WSGIScriptAlias, like
WSGIScriptAlias / /var/www/my_django_site/my_django_site/wsgi.py
This means that you can force the WSGI daemon process to reload by changing the WSGI script. Of course, you don't have to change its contents, but rather,
$ touch /var/www/my_django_site/my_django_site/wsgi.py
would do the trick.
By utilizing the method above, you can automatically reload a WSGI site in production environment without restarting/reloading the entire Apache server, or modifying your WSGI script to do production-unsafe code change monitoring.
This is particularly useful when you have automated deploy scripts, and don't want to restart the Apache server on deployment.
During development, you may use a filesystem changes watcher to invoke touch wsgi.py every time a module under your site changes, for example, pywatch
The mod_wsgi documentation on code reloading is your best bet for an answer.
I know it's an old thread but this might help someone. To kill your process when any file in a certain directory is written to, you can use something like this:
monitor.py
import os, sys, time, signal, threading, atexit
import inotify.adapters
def _monitor(path):
i = inotify.adapters.InotifyTree(path)
print "monitoring", path
while 1:
for event in i.event_gen():
if event is not None:
(header, type_names, watch_path, filename) = event
if 'IN_CLOSE_WRITE' in type_names:
prefix = 'monitor (pid=%d):' % os.getpid()
print "%s %s/%s changed," % (prefix, path, filename), 'restarting!'
os.kill(os.getpid(), signal.SIGKILL)
def start(path):
t = threading.Thread(target = _monitor, args = (path,))
t.setDaemon(True)
t.start()
print 'Started change monitor. (pid=%d)' % os.getpid()
In your server startup, call it like:
server.py
import monitor
monitor.start(<directory which contains your wsgi files>)
if your main server file is in the directory which contains all your files, you can go like:
monitor.start(os.path.dirname(__file__))
Adding other folders is left as an exercise...
You'll need to 'pip install inotify'
This was cribbed from the code here: https://code.google.com/archive/p/modwsgi/wikis/ReloadingSourceCode.wiki#Restarting_Daemon_Processes
This is an answer to my duplicate question here: WSGI process reload modules