I am trying to deploy a Flask web app via mod_wsgi on Apache. I cannot use the default Python environment because it was compiled with UCS-2 Unicode instead of UCS-4, and I cannot recompile it for this one case. Thus, a virtual environment. Virtual environments would have been used anyway, but that error means that I can't get away with using the default Python install and just adding the virtual environment's modules to the PYTHONPATH, which otherwise would have let me avoid the current problem entirely by accident.
I found the documentation for mod_wsgi to change which Python executable to use. However, when attempting to do so, the server fails to work properly. /var/log/httpd/error_log rapidly floods with the line ImportError: No module named site.
I have checked every similar question I can find here and elsewhere, and not yet had success. Experimentation has shown that as far as I can tell, the problem occurs when changing PYTHONHOME without activating a virtual environment - and the way the automated deployment works (via Fabric), as far as I can tell I can't activate a virtual environment.
Apache config
My current httpd.conf for the app:
WSGIPythonPath /path/to/dir/containing/wsgi/file/and/app:/path/to/virtualenv/lib:/path/to/virtualenv/lib/site-packages
WSGIPythonHome /path/to/virtualenv
WSGISocketPrefix /var/run/wsgi
User user
Group group
<VirtualHost *>
ServerName servername.generic.com
DocumentRoot /path/to/dir/containing/wsgi/file/and/app/static_dev/
WSGIDaemonProcess appname user=user group=group threads=2
WSGIScriptAlias / /path/to/dir/containing/wsgi/file/and/app/app.wsgi
<Directory /path/to/dir/containing/wsgi/file/and/app>
WSGIProcessGroup appname
WSGIApplicationGroup %{GLOBAL}
Require all granted
</Directory>
</VirtualHost>
Data I've found from failed attempts
I know that the error is not in my app.wsgi, because when I added the line raise Exception('tried to open the file') at the very top to check that, the existing ImportError kept happening instead of that new Exception.
I have confirmed via ldd that my version of mod_wsgi is for Python 2.7.
I have tried setting WSGIPythonHome /path/to/virtualenv/bin/ and WSGIPythonHome /path/to/virtualenv/bin/python, with the same result as the current state.
I have tried omitting the WSGIPythonHome directive, in which case it loads the app.wsgi as it should, but breaks on a later import as described at the top (the reason I can't just do that).
I have tried omitting the WSGIPythonPath directive and leaving it up to app.wsgi to add things to the PYTHONPATH, with the same result as the current state.
I have tried putting the path-setting as an argument to WSGIDaemonProcess instead of as the WSGIPythonPath directive, with the same result as the current state.
I have confirmed that there is a site.py in /path/to/virtualenv/lib.
I have confirmed that no other non-app-specific Apache .conf files being used (default settings, automatic module loads, etc) contain the string "WSGI", so I don't think there's any conflicts here.
If I activate the virtual environment from the command line I can import site without an error, just for the sake of testing that it does in fact exist in the environment. However, this is insufficient because it needs to start smoothly from a single call to sudo systemctl start httpd.service due to the deployment tools in use, and that seems to not care about the venv of the current shell session.
If, from a default state, I export PYTHONHOME=/path/to/virtualenv, attempting to open the Python REPL exits immediately with ImportError: No module named site.
If I activate the virtual environment and then set PYTHONHOME, I get the same import error.
If I activate the virtual environment and don't touch PYTHONHOME, echo $PYTHONHOME outputs a blank line, and the Python REPL works fine. In the Python REPL while in the virtualenv:
(virtualenv)-bash-4.2$ python
Python 2.7.8 (default, Aug 14 2014, 13:26:38)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'/path/to/virtualenv'
>>> sys.exec_prefix
'/path/to/virtualenv'
Even though setting PYTHONHOME to the same value didn't work.
If I try export PYTHONHOME=:/path/to/virtualenv or export PYTHONHOME=/path/to/virtualenv:, explicitly setting only one of prefix and exec_prefix, it fails with the same import error in either case.
If I activate the virtual environment and set PYTHONHOME in one of those latter two ways, the unset one appears to default to / rather than to the usual default value, but the Python REPL runs fine:
# Setting only exec_prefix
(virtualenv)-bash-4.2$ export PYTHONHOME=:/path/to/virtualenv
(virtualenv)-bash-4.2$ python
Python 2.7.8 (default, Aug 14 2014, 13:26:38)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'/'
>>> sys.exec_prefix
'/path/to/virtualenv'
>>> quit()
# Setting only prefix
(.virtualenv)-bash-4.2$ export PYTHONHOME=/path/to/virtualenv:
(.virtualenv)-bash-4.2$ python
Python 2.7.8 (default, Aug 14 2014, 13:26:38)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'/path/to/virtualenv'
>>> sys.exec_prefix
'/'
Unfortunately, since the deployment script doesn't care what environment is activated, that doesn't solve it. Trying to set WSGIPythonHome in such a fashion makes no difference whatsoever.
I have noticed one further thing: The Python in the virtualenv is 2.7.8. The Python run outside the virtualenv (usr/bin/python) is 2.7.5. I do not know - would this affect the setting of PYTHONHOME somehow? I would hope not - since that seems to defeat the entire purpose of using WSGIPythonHome to run a virtualenv as compared to just setting sys.path inside the app.wsgi file, the ability to start from a different executable - but I cannot rule it out, clueless as I am.
The 2.7.8 Python in /path/to/virtualenv/bin/python has a sys.real_prefix of /network-mounted-drive/sw/python/python-2.7.8.
I changed the deployment to build from /network-mounted-drive/sw/python/python-2.7.5, then did more tests. Results as follows:
Attempting to start httpd gives the same import error as before.
Setting PYTHONHOME to the location of the virtual environment, then running python:
-bash-4.2$ echo $PYTHONHOME
/path/to/virtualenv
-bash-4.2$ python
ImportError: No module named site
Setting PYTHONHOME to the location of the virtual environment, then explicitly running the virtual environment's python binary (activating the virtual environment and then running python gives the same result):
# In the directory just above the virtualenv
-bash-4.2$ ./virtualenv/bin/python
Python 2.7.5 (default, Mar 14 2016, 14:13:09)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'/path/to/virtualenv'
>>> sys.exec_prefix
'/path/to/virtualenv'
>>> sys.real_prefix
'/network-mounted-drive/sw/python/python-2.7.5
Does anyone have any idea?
Resolution found: The issue seems to have been in trying to use a virtual environment built from something other than the local python install on the system.
Solved by pushing the problem of "local python install on the deployment VM doesn't have pip installed" up the chain to someone with the permissions required to install pip, since no attempted workarounds via networked python installs worked.
The issue of actually using a virtual environment chained from a Python install on a network drive for mod_wsgi may be insoluble, or at least I couldn't figure it out in a reasonable amount of time relative to the bureaucratic solution.
Related
Hi My python installation is in different directory and i am using a docker image which is mac based and it is referring shebang line as /user/local/bin/python3 from other folder in shell script .
my python installation path
Python 3.4.3 (default, Oct 14 2015, 20:28:29)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/home/myuser/project', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']
>>>
so is there a way without changing the shebang line i can redirect or link to my installation of python3 to get out of this error.
is it recommended to install python3 in given path. ?
please advice.
If you can't modify the shebang of the file, and you have access to the Dockerfile that creates your docker image, you can add a command directive to create a symbolic link: ln -s /usr/bin/python3 /usr/local/bin/.
If you don't have access to the Dockerfile. Then you can run the above command from within the running docker instance. That should solve your issue without having to modify the file.
https://docs.docker.com/engine/reference/builder/#cmd
You could set you shebang to "/usr/bin/env python" as usual, then set your path appropriately so that the correct version of python is on the path for your executable. In bash you can set the path on the command line using:
PATH=python/path:$PATH app
I will sometimes ignore the shebang and type python/path/python $(which app) in order to control which python interpreter is running.
I am using PythonAnyhwere for my django production environment. I have a script which should run on scheduled basis.
Since I installed django in a virtualenv, the script starts like
#!/usr/bin/env python
activate_this = '/home/myname/.virtualenvs/myenv/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))
The error I get for this is
/usr/bin/env python: no such Python interpreter
No problem. So I changed it to
#!/usr/bin/env python2.7
Then I got
/usr/bin/env python2.7: no such Python interpreter
or
/usr/bin/env python3.4: no such Python interpreter
I said ok, what if I don't have a shebang line at all?
Error in log:
line 1: activate_this: command not found
line 2: syntax error near unexpected token `activate_this,'
line 2: `execfile(activate_this, dict(__file__=activate_this))'
What is the way then?
You can know where your Python interpreter by typing
$ which python
Also you try something like this (or maybe without env):
$ env python
Python 3.5.0 (default, Sep 20 2015, 11:28:25)
[GCC 5.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path
<module 'posixpath' from '/usr/lib/python3.5/posixpath.py'>
And then change lib to bin and omit /posixpath.py part
While trying to figure out how to properly import the Google Identity Toolkit into my GAE project I observed that when running the project in PyCharm (v. 4.5.2 pro) the PYTHONPATH is NOT the same as in PyCharm's Python Console.
For example in the Console I properly find the oauth2client library from the GAE SDK (v. 1.9.23):
/usr/bin/python2.7 -u /home/usr_local/pycharm-4.5.2/helpers/pydev/pydevconsole.py 43085 43754
Python 2.7.8 (default, Sep 30 2014, 15:34:38) [GCC]
Type "copyright", "credits" or "license" for more information.
IPython 2.2.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
PyDev console: using IPython 2.2.0
import sys; print('Python %s on %s' % (sys.version, sys.platform))
sys.path.extend(['/home/user/src/myapp', '/usr/local/google_appengine', '/usr/local/google_appengine/lib/antlr3', '/usr/local/google_appengine/lib/apiclient', '/usr/local/google_appengine/lib/endpoints-1.0', '/usr/local/google_appengine/lib/yaml-3.10', '/usr/local/google_appengine/lib/markupsafe-0.15', '/usr/local/google_appengine/lib/django-1.3', '/usr/local/google_appengine/lib/requests', '/usr/local/google_appengine/lib/fancy_urllib', '/usr/local/google_appengine/lib/webob_0_9', '/usr/local/google_appengine/lib/graphy', '/usr/local/google_appengine/lib/distutils', '/usr/local/google_appengine/lib/concurrent', '/usr/local/google_appengine/lib/cacerts', '/usr/local/google_appengine/lib/six', '/usr/local/google_appengine/lib/pyasn1', '/usr/local/google_appengine/lib/setuptools-0.6c11', '/usr/local/google_appengine/lib/jinja2-2.6', '/usr/local/google_appengine/lib/portpicker', '/usr/local/google_appengine/lib/django-0.96', '/usr/local/google_appengine/lib/django-1.2', '/usr/local/google_appengine/lib/PyAMF-0.6.1', '/usr/local/google_appengine/lib/sqlcmd', '/usr/local/google_appengine/lib/oauth2client', '/usr/local/google_appengine/lib/uritemplate', '/usr/local/google_appengine/lib/httplib2', '/usr/local/google_appengine/lib/protorpc-1.0', '/usr/local/google_appengine/lib/python-gflags', '/usr/local/google_appengine/lib/rsa', '/usr/local/google_appengine/lib/grizzled', '/usr/local/google_appengine/lib/django-1.5', '/usr/local/google_appengine/lib/ipaddr', '/usr/local/google_appengine/lib/django-1.4', '/usr/local/google_appengine/lib/argparse', '/usr/local/google_appengine/lib/google-api-python-client', '/usr/local/google_appengine/lib/mox', '/usr/local/google_appengine/lib/cherrypy', '/usr/local/google_appengine/lib/prettytable', '/usr/local/google_appengine/lib/webapp2-2.3', '/usr/local/google_appengine/lib/oauth2', '/usr/local/google_appengine/lib/pyasn1_modules', '/usr/local/google_appengine/lib/webapp2-2.5.2', '/usr/local/google_appengine/lib/websocket', '/usr/local/google_appengine/lib/yaml', '/usr/local/google_appengine/lib/docker', '/usr/local/google_appengine/lib/simplejson', '/usr/local/google_appengine/lib/webapp2-2.5.1', '/usr/local/google_appengine/lib/deprecated_enum'])
Python 2.7.8 (default, Sep 30 2014, 15:34:38) [GCC] on linux2
In[2]: from oauth2client import crypt
In[3]: crypt
Out[3]: <module 'oauth2client.crypt' from '/usr/local/google_appengine/lib/oauth2client/oauth2client/crypt.py'>
In[4]:
But when running the project I get:
File "/home/user/src/myapp/main/identitytoolkit/gitkitclient.py", line 47, in <module>
from oauth2client import crypt
ImportStringError: import_string() failed for 'gitkit.GitKitHandler'. Possible reasons are:
- missing __init__.py in a package;
- package or module path not included in sys.path;
- duplicated package or module name taking precedence in sys.path;
- missing module, class, function or variable;
Original exception:
ImportError: No module named oauth2client
I'm also dumping the sys.path before the exception (real paths are displayed, /usr/local/google_appengine is a symlink to /home/usr_local/google_appengine_1.9.23):
ERROR 2015-06-30 18:59:46,199 gitkit.py:13] ['/home/user/src/myapp/main', '/home/usr_local/google_appengine_1.9.23', '/home/usr_local/google_appengine_1.9.23', '/usr/lib64/python2.7', '/usr/lib64/python2.7/lib-dynload', '/home/usr_local/google_appengine_1.9.23/lib/webapp2-2.5.2', '/home/usr_local/google_appengine_1.9.23/lib/jinja2-2.6', '/home/usr_local/google_appengine_1.9.23/lib/pycrypto-2.6', '/home/usr_local/google_appengine_1.9.23/lib/markupsafe-0.15', '/home/usr_local/google_appengine_1.9.23/lib/setuptools-0.6c11', '/home/usr_local/google_appengine_1.9.23/lib/protorpc-1.0', '/home/usr_local/google_appengine_1.9.23/lib/webob-1.1.1', '/home/usr_local/google_appengine_1.9.23/lib/yaml-3.10']
Is there a way to configure PyCharm to apply the same PYTHONPATH from the Console (which is IMHO correct) to the development server execution?
Alternatively - how does PyCharm determine/configure the development server execution path? (I don't mind even configuring the correct path manually if needed).
Donno if relevant, but I've upgraded the PyCharm and GAE SDK versions several times while working on this project.
It turned out that the GAE development server adjusts its own execution path depending on the SDK installation and the executed app code itself.
The failures in my case were caused by incorrect 3rd party lib vendoring in a multi-modules app. I documented my fix here: Is Google Identity Toolkit (v3) compatible with GAE/python sandbox?
I'm runnning a virtualenv in the django-web folder. The folder has a Scripts folder which contains the python file django-admin.py. I would like to run this python file as: python django-admin.py from C:\Users\name\django-web with the virtualenv activated. The python file isn't found unfortunatly:
python: can't open file 'django-admin.py': [Errno 2] No such file or directory
when I look for the pythonpath in my virtual env:
(django-web) C:\Users\name\django-web>python Python 3.4.1
(v3.4.1:c0e311e010fc, May 18 2014, 10:38:22) [MSC v.1600 32 bit (In
tel)] on win32 Type "help", "copyright", "credits" or "license" for
more information. import sys; print(sys.path) ['',
'C:\Users\name\django-web\lib\site-packages\django-1.6.5-py3.4.egg'
, 'C:\Python34\Lib', 'C:\Python34\DLLs',
'C:\Windows\system32\python34.zi p',
'C:\Users\name\django-web\DLLs',
'C:\Users\name\django-web\lib',
'C:\Users\name\django-web\Scripts', 'C:\Users\name\django-web', 'C:\
\Users\name\django-web\lib\site-packages']
The path 'C:\Users\name\django-web\Scripts' where the file is located is clearly in the python path.
Running the file as python Scripts\django-admin.py from C:\Users\name\django-web runs fine.
First of all, PYTHONPATH helps Python to find required modules while importing.
In your case it does not matter because you are trying to run the script, it is not an import
seems like you are executing python django-admin.py in wrong directory.
Also you have to run virtualenv's python binary instead of regular one (just need to activate virtualenv)
virtual env HOW TO
I tried to install cx_Oracle from pypi source since there is no available port for it in cygwin. I did make some changes as suggested in http://permalink.gmane.org/gmane.comp.python.db.cx-oracle/2492 and modified my setup.py. However, I still get the following error :-
$ python
Python 2.7.3 (default, Dec 18 2012, 13:50:09)
[GCC 4.5.3] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cx_Oracle
/usr/lib/python2.7/site-packages/cx_Oracle-5.1.3-py2.7-cygwin-1.7.24-i686.egg/cx_Oracle.py:3: UserWa
rning: Module cx_Oracle was already imported from /usr/lib/python2.7/site-packages/cx_Oracle-5.1.3-p
y2.7-cygwin-1.7.24-i686.egg/cx_Oracle.pyc, but /home/zerog/cx_Oracle-5.1.3 is being added to sys.pat
h
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.cygwin-1.7.24-i686/egg/cx_Oracle.py", line 7, in <module>
File "build/bdist.cygwin-1.7.24-i686/egg/cx_Oracle.py", line 6, in __bootstrap__
ImportError: Exec format error
>>>
If someone can please help me fix this ?
TIA.
Fixed this by specifying the path to instantclient as below :
$ export PATH=$PATH:/cygdrive/d/Tools/instantclient_11_2
(Other, possibly important stuff) :
$ echo $LD_LIBRARY_PATH
/cygdrive/d/Tools/instantclient_11_2
$ echo $ORACLE_HOME
/cygdrive/d/Tools/instantclient_11_2
Now, I get :-
$ python
Python 2.7.3 (default, Dec 18 2012, 13:50:09)
[GCC 4.5.3] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cx_Oracle
>>>
It's hard to pin down from the error message alone, but I am guessing that you have two different copies of cx_Oracle in your sys.path. The error message is complaining that a different version of the same module had already been import-ed.
Presumably the pristine upstream version is installed system-wide in /usr/lib/python2.7/site-packages/cx_Oracle-5.1.3-py2.7-cygwin-1.7.24-i686.egg, and your modified version in /home/zerog/cx_Oracle-5.1.3.
Does it work if you pare down sys.path so that only the original, or only your modified version, is included?
(You might want to use virtualenv if you need to switch back and forth between two versions frequently.)
I ran into this error "Exec format error."
For me, this was likely caused by a mismatch between cygwin being installed as 64 bit, but the instant client being installed as 32 bit. Double check that everything (oracle, cygwin) is either 32 bit or 64 bit.
What fixed my issue:
Since my cygwin is 64 bit (see uname -a, and look for x86_64), I downloaded the 64 bit instant client from oracle's website, and unzipped
I set the env vars in .profile, to point where it was unzipped:
export ORACLE_HOME=/cygdrive/c/oracle/instantclient_x64_11_2
export LD_LIBRARY_PATH=$ORACLE_HOME
export DYLD_LIBRARY_PATH=$ORACLE_HOME
export TNS_ADMIN='//optional/path/to/your/oracle/tns/files/'
source ~/.profile
To test, you should now be able to run this python command with no error:
import cx_Oracle
To verify the path is correct, if you run ls, you should see something like
ls $ORACLE_HOME
adrci.exe genezi.exe oci.sym ociw32.dll ojdbc6.jar
oraocci11.dll oraociei11.sym uidrvci.exe vc9
adrci.sym genezi.sym ocijdbc11.dll ociw32.sym orannzsbb11.dll
oraocci11.sym orasql11.dll uidrvci.sym xstreams.jar
BASIC_README oci.dll ocijdbc11.sym ojdbc5.jar orannzsbb11.sym
oraociei11.dll orasql11.sym vc8