Dockerized Django tests only look at first $PATH location - python

I'm running a Django test suite in a Docker container, and some of these tests use a program I've had to apt-get install (wkhtmltopdf). Now I can see that it's been installed correctly:
$ wkhtmltopdf --version
wkhtmltopdf 0.12.5
but for some reason the Django test can't use it. The installation location is definitely on my $PATH (third to last element):
$ echo $PATH
/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
$ find / -name wkhtmltopdf
/usr/bin/wkhtmltopdf
However when I run tests I get a stack trace ending in:
OSError: No wkhtmltopdf executable found: "/usr/local/bin/wkhtmltopdf"
If this file exists please check that this process can read it. Otherwise please install wkhtmltopdf - https://github.com/JazzCore/python-pdfkit/wiki/Installing-wkhtmltopdf
Now it's absolutely correct that there is no /usr/local/bin/wkhtmltopdf, because it got installed elsewhere (/usr/bin/) but both those locations are on $PATH.
I've tried moving /usr/bin/ to the start of $PATH, but I then get the error:
Traceback (most recent call last):
File "./manage.py", line 8, in <module>
from django.core.management import execute_from_command_line
ImportError: No module named django.core.management
presumably because it's now looking in /usr/bin/ for django when it's actually in usr/local/bin/, which is no longer the first location on $PATH.
I'm not sure if the problem is a Docker one, a Django one, a testing one, or just me misunderstanding one or more things going on here.

So it turns out that the project is using pdfkit as a wrapper for wkhtmltopdf, which includes setting the wkhtmltopdf path directly:
config = pdfkit.configuration(wkhtmltopdf=settings.WKHTMLTOPDF_BIN)
which after a quick look at the Django settings file was set to:
WKHTMLTOPDF_BIN = '/usr/local/bin/wkhtmltopdf'
Mystery solved! The fix in this case was to simply set it to the actual path desired (/usr/bin/wkhtmltopdf).

Related

How to fix ModuleNotFound error in python building?

I trying to build the ungoogled chrome source from github. I was following the instructions in the link below, but I really do not know how to continue.
I installed python 2.7 and 3.7, set them in the PATH.
Used the git clone command and jumped the replace comands and the git checkout too, because I didin't got them.
So, I tried the "py build.py" command and got this error.
C:\Users\aquasp\ungoogled-chromium-windows>py build.py
Traceback (most recent call last):
File "build.py", line 24, in <module>
import buildkit.config
ModuleNotFoundError: No module named 'buildkit'
Is there any suggestions?
This are the commands that I was folowing:
git clone --recurse-submodules https://github.com/ungoogled-software/ungoogled-chromium-windows.git
# Replace TAG_OR_BRANCH_HERE with a tag or branch name
git checkout --recurse-submodules TAG_OR_BRANCH_HERE
py build.py
py package.py
https://github.com/ungoogled-software/ungoogled-chromium-windows
I'm assuming you are on a windows machine. Try running the dos 'which' command with an argument of 'buildkit' as follows:
which buildkit
The output will be the search of directories in the path variable of the windows machine as follows:
which: no buildkit in (/c/WINDOWS/system32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/WINDOWS/System32/WindowsPowerShell/v1
etc...
'buildkit' is not found on path or the python.exe is not on the path.
Try 'which python' or 'which py' to test for python.exe or py in the machine path.
if python.exe is found output is as follows:
/c/Program Files/python/python
or if not found
which: no python found in (/c/WINDOWS/system32: etc...
Last but not least, add buildkit or python.exe or py to your machines path variable as follows:
set path=%path%;plus\new\path
C:\Users>echo %path%
C:\WINDOWS\system32;plus\new\path

git executable not found in python

I was trying to clone a git repo with access key, but when I am trying to run it, It throws an exception saying git executable not found.
But i have installed git and the in_it.py shows correct path "C:\Program Files\Git\bin" Also I have installed gitpython to use the library in python
here's my code...
import git
git.Git("D:/madhav/myrep/").clone("#github.com:myrepo/scripts")
========= and it throws the following exception =================
Traceback (most recent call last): File
"C:\Users\1096506\Desktop\gitclone.py", line 1, in <module>
from git import Repo File "C:\Users\1096506\AppData\Local\Programs\Python\Python36-32\lib\site-packages\git\__init__.py",
line 84, in <module>
refresh() File "C:\Users\1096506\AppData\Local\Programs\Python\Python36-32\lib\site-packages\git\__init__.py",
line 73, in refresh
if not Git.refresh(path=path): File "C:\Users\1096506\AppData\Local\Programs\Python\Python36-32\lib\site-packages\git\cmd.py",
line 293, in refresh
raise ImportError(err) ImportError: Bad git executable. The git executable must be specified in one of the following ways:
- be included in your $PATH
- be set via $GIT_PYTHON_GIT_EXECUTABLE
- explicitly set via git.refresh()
All git commands will error until this is rectified.
This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
- quiet|q|silence|s|none|n|0: for no warning or exception
- warn|w|warning|1: for a printed warning
- error|e|raise|r|2: for a raised exception
Example:
export GIT_PYTHON_REFRESH=quiet
I had the same issue. What I did is:
I went to: System Properties -> Environment Variables
On System Variables Section I clicked Path, Edit and Move Up.
Environment Variables
Edit and Moved Up two places from the bottom
Error occurs because git is not in the path. So its not able to import git module.
Couple of ways to resolve it.
As suggested above adding the git binary path to environment variable path.
If git is not being used directly in the module and its only a dependent module import thats throwing this exception before importing git we could add
os.environ["GIT_PYTHON_REFRESH"] = "quiet"
and import git after this line, this would suppress the error caused due to git import
Had the same problem got it to work thanks to Muthukumaran. Just make Muthukumaran answer more clear.
Follow these steps:
import os
os.environ["GIT_PYTHON_REFRESH"] = "quiet"
import git
See if you have installed Git in the OS.
If not install git first this will solve your error.
Centos
sudo yum -y install git
Ubuntu/Debian
sudo apt-get install git
Mac Os
sudo brew install git
This solved my problem.
Make sure you're not in an inaccessible directory on *nix, such as when you've just been root and then done a su username
you may still be in root's home folder and that will trigger this error (assuming you have the correct environment variables set, and have sourced the .profile or .bashrc etc with source ~/.bashrc )
which git
/usr/bin/git
I was getting this error even after setting the environment:
in ~/.bashrc
# for bench
PATH=$PATH:/usr/bin/git
export PATH
GIT_PYTHON_GIT_EXECUTABLE=/usr/bin/git
export GIT_PYTHON_GIT_EXECUTABLE
cd
and it's working
$ bench --version
WARN: Command not being executed in bench directory
5.3.0
I came across similar problem recently and installing git followed by restarting Windows Powershell CommandLine solved the problem. May it helps.
For those who are using a Lambda layer with it. It worked adding as the comment above says just adding GIT_PYTHON_REFRESH=quiet as an environment variable.
Execute GIT_PYTHON_REFRESH=quiet in your terminal and then try to run the code.

Python Script not Running - Has to be something simple

OS: Fedora 21
Python: 2.7.6
I run a python script as root or using sudo it runs fine. If I run it as just the user I get the following:
Traceback (most recent call last):
File "/home/user/dev_ad_list.py", line 12, in
import ldap
ImportError: No module named ldap
selinux=disabled -- What other security is preventing a user from running a python script that imports ldap
If it works fine under sudo, it simply sounds like a file access issue.
A quick fix for this would be to run something along the lines of:
sudo chmod -R a+rX /usr/lib/python2.7
But you may wish to be more specific with the directory (or even file) that you actually apply this to.
Path to python was different than other user. User was pointing to canopy.

Why is PYTHONPATH being ignored?

I am setting PYTHONPATH to have a directory that includes a few .py files.
When I go into python and type "import file", the file cannot be find (it says "No module named wsj10").
If, however, I cd to the directory, and repeat the same process, then the file is found.
I am just not sure why PYTHONPATH is being ignored. I followed exact instructions from installation instructions of some software, so I know I am doing the right thing.
Any circumstances under which PYTHONPATH will be ignored, or import won't work?
Thanks.
Following a comment below, here is a transcript:
untar file1.tgz to file1/. file1.tgz contains a library/file called file1.py.
type in the shell:
export PYTHONPATH=`pwd`/file1/:./
echo $PYTHONPATH shows the variable was set.
run python and type "import file1"
I get the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named file1
If I do first "cd file1" and then to import file1 it identifies the file.
Any circumstances under which PYTHONPATH will be ignored, or import won't work?
Yes. I've set PYTHONPATH in my /home/me/.bashrc and all worked ok from terminal, but when Apache w/ mod_wsgi starts my python scripts, it acts under sysem or dedicated user, which knows nothing of my .bashrc.
For this particular situation, I just used apache config to set python path for apache (WSGIPythonPath option).

How can I use Django with MySQL in MAMP stack?

I have difficulty especially in installing MySQLdb module (MySQL-python-1.2.3c1), to connect to the MySQL in MAMP stack.
I've done a number of things such as copying the mysql include directory and library (including plugin) from a fresh installation of mysql (version 5.1.47) to the one inside MAMP (version 5.1.37).
Now, the MySQLdb module build and install doesnt give me error.
The error happens when I'm calling 'import MySQLdb' from python shell (version 2.6).
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.macosx-10.6-universal/egg/MySQLdb/__init__.py", line 19, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 7, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so, 2): Symbol not found: _mysql_affected_rows
Referenced from: /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
Expected in: flat namespace
in /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
Any idea, what else do I need to do to make it works?
Thanks a bunch,
Robert
=========
Add the system response after using virtualenv as suggested by Hank Gay below...
(MyDjangoProject)MyMacPro:MyDjangoProject rhenru$ which python
/Users/rhenru/Workspace/django/MyDjangoProject/bin/python
After I run python in virtualenv, importing MySQLdb:
>>> import MySQLdb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.macosx-10.6-universal/egg/MySQLdb/__init__.py", line 19, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 7, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so, 2): Symbol not found: _mysql_affected_rows
Referenced from: /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
Expected in: flat namespace
in /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
import sys and sys.path
>>> import sys
>>> print sys.path
['', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/site-packages/distribute-0.6.10-py2.6.egg', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/site-packages/pip-0.7.1-py2.6.egg', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python26.zip', '/Library/Python/2.6/site-packages/PyXML-0.8.4-py2.6-macosx-10.6-universal.egg', '/Library/Python/2.6/site-packages/pydot-1.0.2-py2.6.egg', '/Library/Python/2.6/site-packages/pyparsing-1.5.2-py2.6.egg', '/Library/Python/2.6/site-packages/vobject-0.8.1c-py2.6.egg', '/Library/Python/2.6/site-packages/pytz-2010h-py2.6.egg', '/Library/Python/2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg', '/Library/Python/2.6/site-packages/distribute-0.6.12-py2.6.egg', '/Library/Python/2.6/site-packages/pip-0.7.1-py2.6.egg', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/plat-darwin', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/plat-mac', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/plat-mac/lib-scriptpackages', '/Users/rhenru/Workspace/django/MyDjangoProject/Extras/lib/python', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/lib-tk', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/lib-old', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/lib-dynload', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/site-packages', '/Library/Python/2.6/site-packages', '/Library/Python/2.6/site-packages/PIL', '/Library/Python/2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode']
How are you installing MySQL-Python? I just tested in a fresh virtualenv and pip install mysql-python seems to have done the trick.
UPDATE:
pip is sort of like a package manager for Python packages.
By default, pip installs to your current site-packages directory, which is on your $PYTHONPATH. This lets other libraries/applications (like Django) access it. pip also works well with virtualenv (it should; Ian Bicking wrote them both), which is a nifty library that lets you sandbox an application. This is nice because it means you can try out new things without polluting (or even needing write access to) the global site-packages directory.
It probably seems like yak-shaving right now, but I'd say it's worth the effort to get up to speed on pip and virtualenv (you may also want to look into virtualenvwrapper, but we'll skip that for now; it's just sugar for virtualenv). It will lead to a slightly more complicated deployment scenario than putting everything in the global site-packages, but for development it's really no harder, and there are lots of good guides to deploying using a virtualenv.
I'd recommend something like the following:
curl -0 http://python-distribute.org/distribute_setup.py
python distribute_setup.py
easy_install pip
pip install virtualenv
virtualenv --distribute MyDjangoProject --no-site-packages
cd MyDjangoProject
source bin/activate (this activates the sandbox that virtualenv created)
pip install django mysql-python
At this point, you should have a totally functional Django+MySQL install (if I missed any steps, just comment and I'll try to add it in). You can start your Django project like this: django-admin.py startproject MyDjangoProject. cd into your project's directory, edit your settings.py file to point to your MySQL database, and run the dev server to test it out like so: ./manage.py runserver (you may need to chmod u+x your manage.py file). Voila! You should be able to access your site on localhost:8000. When you're done working on the project, you can just use deactivate to exit the virtualenv sandbox.
Try not to hold all this against Django: a lot of it is just best practices stuff for working with Python libraries. You could get by with a lot less, but this way it's more reproducible and you're less likely to accidentally mess up one of this project's dependencies when working on a different project.
I had this problem and it turned out to be due to an errant configuration:
export VERSIONER_PYTHON_PREFER_32_BIT=yes
I can't recall what I had this enabled for (some package that required 32-bit), probably related to Google AppEngine. But Setting it to 'no' solved by issues.
Otherwise I just installed everything using homebrew and pip.

Categories