So I was trying to run my flask app in a virtualenv. But when trying to import mariadb which uses a .so file for the module I got this error:
File "(pathToEnv)/FlaskEnv/lib/python3.10/site- packages/mariadb/__init__.py", line 10$
[Mon Mar 21 14:52:43.074678 2022] [wsgi:error] [pid 27865] [client 192.168.1.128:42536] from ._mariadb import (
[Mon Mar 21 14:52:43.074723 2022] [wsgi:error] [pid 27865] [client 192.168.1.128:42536] ModuleNotFoundError: No module named 'mariadb._mariadb'
The same error happens for xxhash which also uses a .so file (possible correlation?) Every other module works fine under flask and when using either mariadb or xxhash in a normal python script they import fine.
Also notable is that this only started after I put Flask into a virtualenv. Before I had it run of the system wide ptyhon3.7 and now virtalenv python 3.10. I'm really puzzled what could be the cause of this, because it works fine in normal python scripts, but not on flask under apache
I'm getting this error in my logs with django app. I thought such errors occur when using Python 2.x, but in my virtual env Python - 3.6.5. Django - 2.0.7. VESTA Control Panel with wsgi. I will be grateful for the help.
Error log:
[Thu Jul 26 19:02:48 2018] [error] [client 86.32.36.143] File "/home/project/private/django/project/env/lib/python3.6/site-packages/django/utils/version.py", line 61, in
[Thu Jul 26 19:02:48 2018] [error] [client 86.32.36.143] #functools.lru_cache()
[Thu Jul 26 19:02:48 2018] [error] [client 86.32.36.143] AttributeError: 'module' object has no attribute 'lru_cache'
django.wsgi
import os, sys
sys.path.insert(0, '/home/user/web/project/private/django/project/env/lib/python3.6/site-packages')
sys.path.insert(0, '/home/user/web/project/private/django/project/project/src/shared/')
sys.path.insert(0, '/home/user/web/project/private/django/project/project/src/')
os.environ['DJANGO_SETTINGS_MODULE'] = 'main.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
I had the same issue but then found out that my site was built with python3 and apache was configured for python2.7. The following link helped:
Target WSGI script cannot be loaded as Python module
I had mod_wsgi working fine on RHEL/Apache2.4/python3.4/Django1.11/virtualenv. Then, I used pip within the virtualenv to upgrade numpy from 1.14.0 to 1.14.2, install pandas, and install a couple other modules. Then I restarted Apache and started getting server errors indicating AttributeError: 'module' object has no attribute 'arange' when trying to call numpy.arange, or similar errors when trying to call matplotlib.use('Agg'), etc.
I replaced wsgi.py with the following and it works fine. As soon as I uncomment any of the numpy or matplotlib calls it returns a 500 error and in the apache error log it shows an error similar to AttributeError: 'module' object has no attribute ...
import datetime
import sys
import numpy as np
import matplotlib as mpl
import mod_wsgi
def application(env, start_response):
#mpl.use('Agg')
#a = np.arange(15).reshape(3, 5)
#a = np.array([2,3,4])
status = '200 OK'
output = 'Hello World! This is the wsgi app generated from python!'
output += '\n\ncurrent time is: '+str(datetime.datetime.now())
output += '\n\nsys.executable='+sys.executable
output += '\n\nsys.path='+str(sys.path)
output += '\n\nsys.version=' + str(sys.version)
output += '\n\nsys.prefix=' + str(sys.prefix)
output += '\n\nmod_wsgi.version='+str(mod_wsgi.version)
output += '\n\n\nEnvironment Variables:\n\n'+'\n'.join('%s: %s' % (k, v) for (k, v) in env.items())
output = bytes(output,'utf-8')
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)
return [output]
Relevant output from this wsgi.py:
sys.executable=/usr/bin/python3
sys.path=['/usr/local/virtualenvs/myapp/wsgiapp', '/usr/local/virtualenvs/myapp/lib64/python34.zip', '/usr/local/virtualenvs/myapp/lib64/python3.4', '/usr/local/virtualenvs/myapp/lib64/python3.4/plat-linux', '/usr/local/virtualenvs/myapp/lib64/python3.4/lib-dynload', '/usr/lib64/python3.4', '/usr/lib/python3.4', '/usr/local/virtualenvs/myapp/lib/python3.4/site-packages']
sys.version=3.4.8 (default, Mar 23 2018, 10:04:27)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
sys.prefix=/usr/local/virtualenvs/myapp
mod_wsgi.version=(4, 5, 24)
mod_wsgi.process_group: mywsgiapp
mod_wsgi.application_group:
The Apache httpd.conf didn't change. Relevant lines:
LoadModule wsgi_module "/usr/local/virtualenvs/myapp/lib64/python3.4/site-packages/mod_wsgi/server/mod_wsgi-py34.cpython-34m.so"
WSGIDaemonProcess mywsgiapp python-home=/usr/local/virtualenvs/myapp python-path=/usr/local/virtualenvs/myapp/wsgiapp
WSGIProcessGroup mywsgiapp
WSGIApplicationGroup %{GLOBAL}
WSGIScriptAlias /myapp /usr/local/virtualenvs/myapp/wsgiapp/wsgi.py process-group=mywsgiapp application-group=%{GLOBAL}
I'm not sure what caused this to break. I know this usually happens when mod_wsgi is compiled for the wrong version of python. But this was working fine before and everything still appears to be configured correctly: mod_wsgi is functioning, the output from the wsgi.py indicates it is using the correct version of python, and the site-packages directory of the virtualenv is in the python-path.
I have mod_wsgi installed in the main python3.4 location and tried pointing the Apache LoadModule at that instance but still get the same errors.
What else should I check?
Should sys.executable be pointing to the virtualenv instead of the main python install? Should I pip uninstall all the modules and start over?
Also, to install pandas, I had to install Cython, which in turn required that I install gcc-c++ (sudo yum install gcc-c++). Would that affect anything?
Edited to add error traces. I uncommented the line a = np.arange(15).reshape(3, 5) in the above wsgi.py file.
HTTP Error Page from Apache:
Internal Server Error
The server encountered an internal error or
misconfiguration and was unable to complete
your request.
Please contact the server administrator at
root#localhost to inform them of the time this error occurred,
and the actions you performed just before this error.
More information about this error may be available
in the server error log.
Apache error log (note I x'ed out the IP address before posting here):
[Thu Jul 12 23:01:15.664941 2018] [wsgi:error] [pid 21490] [remote 10.x.x.x:59961] mod_wsgi (pid=21490): Exception occurred processing WSGI script '/usr/local/virtualenvs/myapp/wsgiapp/wsgi.py'.
[Thu Jul 12 23:01:15.727450 2018] [wsgi:error] [pid 21490] [remote 10.x.x.x:59961] Traceback (most recent call last):
[Thu Jul 12 23:01:15.727814 2018] [wsgi:error] [pid 21490] [remote 10.x.x.x:59961] File "/usr/local/virtualenvs/myapp/wsgiapp/wsgi.py", line 14, in application
[Thu Jul 12 23:01:15.727831 2018] [wsgi:error] [pid 21490] [remote 10.x.x.x:59961] a = np.arange(15).reshape(3, 5)
[Thu Jul 12 23:01:15.727865 2018] [wsgi:error] [pid 21490] [remote 10.x.x.x:59961] AttributeError: 'module' object has no attribute 'arange'
Error when trying to show np.file:
[Thu Jul 12 23:25:59.196534 2018] [wsgi:error] [pid 22747] [remote 10.x.x.x:60353] mod_wsgi (pid=22747): Exception occurred processing WSGI script '/usr/local/virtualenvs/myapp/wsgiapp/wsgi.py'.
[Thu Jul 12 23:25:59.257214 2018] [wsgi:error] [pid 22747] [remote 10.x.x.x:60353] Traceback (most recent call last):
[Thu Jul 12 23:25:59.257500 2018] [wsgi:error] [pid 22747] [remote 10.x.x:60353] File "/usr/local/virtualenvs/myapp/wsgiapp/wsgi.py", line 23, in application
[Thu Jul 12 23:25:59.257535 2018] [wsgi:error] [pid 22747] [remote 10.x.x.x:60353] output += '\\n\\nnumpy file location='+str(np.__file__)
[Thu Jul 12 23:25:59.257569 2018] [wsgi:error] [pid 22747] [remote 10.x.x.x:60353] AttributeError: 'module' object has no attribute '__file__'
Also, just for reference, importing the modules works just fine if I activate the virtualenv and use the interpreter:
Python 3.4.8 (default, Mar 23 2018, 10:04:27)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.__file__
'/usr/local/virtualenvs/myapp/lib/python3.4/site-packages/numpy/__init__.py'
Printing dir(np) from the wsgi.py file results in:
dir(np)=['__doc__', '__loader__', '__name__', '__package__', '__path__', '__spec__']
So it doesn't error out, but it is not the full list of identifiers. Running dir(np) from the virtualenv interpreter results in:
['ALLOW_THREADS', 'AxisError', 'BUFSIZE', 'CLIP', 'ComplexWarning', 'DataSource', 'ERR_CALL', ..... clipped for brevity - there's a full screen of items in the list
Since it does list some identifiers, here are the results of printing each of them from within wsgi.py. It does seem to be picking up the package from the correct path.
np.__path__ =_NamespacePath(['/usr/local/virtualenvs/myapp/lib/python3.4/site-packages/numpy'])
np.doc=None
np.loader=<_frozen_importlib._NamespaceLoader object at 0x7f83b82e0c88>
np.package=numpy
np.name=numpy
np.Spec=ModuleSpec(name='numpy', loader=None, origin='namespace', submodule_search_locations=_NamespacePath(['/usr/local/virtualenvs/myapp/lib/python3.4/site-packages/numpy']))
Just to be sure wsgi/python is truly finding the numpy package, I changed the wsgi.py file to import a non-existent package import foo as np, and when I ran it again it errored out with ImportError: No module named 'foo'. So it does seem to be actually finding the site-packages... but just not fully loading them?
Fun fact: when pip installs a new module it sets the permissions such that the user is able to import and use the modules as expected but "other" permissions allow the modules to be seen but not execute. Thus, mod_wsgi running under Apache was able to import the modules but calling any methods resulted in an error.
Solution: double check permissions after installing or updating python modules! I set all directories to 755 and all files to 644 and that seemed to work. Or, as mentioned in the comment, ensure that your umask is set appropriately before running pip.
I'm learning about deploying flask apps using apache2.4 and mod_wsgi on an Ubuntu 16.04 server by building the simplest of "Hello, Word!" apps, but I encountered a weird syntax error related to my wsgi file and I'm stumped.
Apache seems to be running fine but when I hit the app's domain, I get apache's standard 500 error and the logs list the following:
[wsgi:error] [pid 28427:tid 140540431517440] mod_wsgi (pid=28427): Target WSGI script '/var/www/myapp.domain.com/myapp.wsgi' cannot be loaded as Python module.
[wsgi:error] [pid 28427:tid 140540431517440] mod_wsgi (pid=28427): Exception occurred processing WSGI script '/var/www/myapp.domain.com/myapp.wsgi'.
[wsgi:error] [pid 28427:tid 140540431517440] Traceback (most recent call last):
[wsgi:error] [pid 28427:tid 140540431517440] File "/var/www/myapp.domain.com/myapp.wsgi", line 5, in <module>
[wsgi:error] [pid 28427:tid 140540431517440] exec(file_.read(), dict(__file__=activate_this))
[wsgi:error] [pid 28427:tid 140540431517440] File "<string>", line 4
[wsgi:error] [pid 28427:tid 140540431517440] deactivate () {
[wsgi:error] [pid 28427:tid 140540431517440] ^
[wsgi:error] [pid 28427:tid 140540431517440] SyntaxError: invalid syntax
The syntax error referenced in the logs are pointing to code in my virtual environment’s activation script located at ./venv/bin/activate
I'm wondering if my use the venv module vs. virtualenv is causing this issue
Here is the content of my app's wsgi file (test is the name of the module were flask is instantiated):
#!/usr/bin/python3
activate_this = '/var/www/myapp.domain.com/venv/bin/activate'
with open(activate_this) as file_:
exec(file_.read(), dict(__file__=activate_this))
import sys
sys.path.insert(0, '/var/www/myapp.domain.com')
from test import app as application
Using the activate_this.py script is not recommended even if you had got the name correct. For how to use a Python virtual environment with mod_wsgi see the documentation at:
http://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html
Ensure you are also using daemon mode of mod_wsgi.
The details in that document are too much to replicate here and what method you use depends on how using mod_wsgi.
I have solved this problem by change the virtual environment which I built by 'python3 -m venv'. now I create the venv by virtualenv, and then re-install the necessary modules.
The reason is that the venv created by python3 hasn't the script of activate_this.py which is important in writing the myapp.wsgi file.
In the professional document, the reason is following:
When needing to activate the Python virtual environment from within the
WSGI script file as described, it is preferred that you be using the either
virtualenv or virtualenvwrapper to create the Python virtual environment.
This is because they both provide the activate_this.py script file which
does all the work of setting up sys.path. When you use either pyvenv or
python -m venv with Python 3, no such activation script is provided.
so good luck!
No matter what follows the curly brace (a set or dict display or dict comprehension) deactivate () { will not be legal Python code. I do not see anything odd or mysterious about getting a SyntaxError when you try to run it.
I was getting started with AWS' Elastic Beanstalk.
I am following this tutorial to deploy a Django/PostgreSQL app.
I did everything before the 'Configuring a Database' section. The deployment was also successful but I am getting an Internal Server Error.
Here's the traceback from the logs:
mod_wsgi (pid=30226): Target WSGI script '/opt/python/current/app/polly/wsgi.py' cannot be loaded as Python module.
[Tue Sep 15 12:06:43.472954 2015] [:error] [pid 30226] [remote 172.31.14.126:53947] mod_wsgi (pid=30226): Exception occurred processing WSGI script '/opt/python/current/app/polly/wsgi.py'.
[Tue Sep 15 12:06:43.474702 2015] [:error] [pid 30226] [remote 172.31.14.126:53947] Traceback (most recent call last):
[Tue Sep 15 12:06:43.474727 2015] [:error] [pid 30226] [remote 172.31.14.126:53947] File "/opt/python/current/app/polly/wsgi.py", line 12, in <module>
[Tue Sep 15 12:06:43.474777 2015] [:error] [pid 30226] [remote 172.31.14.126:53947] from django.core.wsgi import get_wsgi_application
[Tue Sep 15 12:06:43.474799 2015] [:error] [pid 30226] [remote 172.31.14.126:53947] ImportError: No module named django.core.wsgi
Any idea what's wrong?
Have you created a requirements.txt in the root of your application? Elastic Beanstalk will automatically install the packages from this file upon deployment. (Note it might need to be checked into source control to be deployed.)
pip freeze > requirements.txt
(You will probably want to do that from within a virtualenv so that you only pick up the packages your app actually needs to run. Doing that with your system Python will pick up every package you've ever installed system-wide.)
The answer (https://stackoverflow.com/a/47209268/6169225) by carl-g is correct. One thing that got me was that requirements.txt was in the wrong directory. Let's say you created a django project called mysite. This is the directory in which you run the eb command(s) --> make sure requirements.txt is in this directory.
If you forget the .ebextensions folder you will get the same error.
I was following along with a good simple (non Elastic Beanstalk) tutorial and missed step 3 & 4 of Elastic Beanstalk.
I was using Django 1.11 and Python 2.7