We Keep Coding

Cython "ModuleNotFoundError" even though it was just installed via requirements.txt while building docker image

Installing packages when building a docker image via the requirements.txt. In the Dockerfile it reads
COPY requirements.txt /app/requirements.txt
RUN pip3 install -r /app/requirements.txt
In the requirements.txt I put down deploy token for installing a private git repo, such as this example one:
git+https://gitlab+deploy-token-123456:a1b2c3d4e5f6g7h8j9#gitlab.com/example/lib.git#a1b2c3d4e5f6g7h8j9k1l2m3n4o5p6q7r8s9t
It throws an error, ModuleNotFoundError: No module named 'Cython'.
However, even though I include Cython in the requirements and it gets collected correctly, I get the same error.
Here is the complete message:
docker build -t docker.example/example:dev .
Sending build context to Docker daemon 8.704kB
Step 1/7 : FROM python:3.6.6-alpine
---> a78e257617d1
Step 2/7 : RUN apk update
---> Using cache
---> 35e6b7308e61
Step 3/7 : RUN apk add gcc build-base musl-dev libc-dev util-linux-dev linux-headers python3-dev git ca-certificates libffi-dev bash postgresql-libs postgresql-dev
---> Using cache
---> 5ca54839bb7c
Step 4/7 : RUN python3 -m ensurepip
---> Using cache
---> 03d95f700108
Step 5/7 : RUN pip install --upgrade pip
---> Using cache
---> e462d855f07f
Step 6/7 : COPY requirements.txt /app/requirements.txt
---> f69aa355732a
Step 7/7 : RUN pip3 install -r /app/requirements.txt
---> Running in d75911af46c7
Collecting git+https://gitlab%2Bdeploy-token-171703:****#gitlab.com/example/lib.git#f75285be6ee56360d6f0cfbf64f09a86819def5c (from -r /app/requirements.txt (line 2))
Cloning https://gitlab%2Bdeploy-token-171703:****#gitlab.com/example/lib.git (to revision f75285be6ee56360d6f0cfbf64f09a86819def5c) to /tmp/pip-req-build-yt5mk7q5
Running command git clone -q 'https://gitlab%2Bdeploy-token-171703:****#gitlab.com/example/lib.git' /tmp/pip-req-build-yt5mk7q5
Collecting Cython==0.29.17
Downloading Cython-0.29.17-py2.py3-none-any.whl (971 kB)
Collecting psychrolib==2.5.0
Downloading PsychroLib-2.5.0.zip (9.5 kB)
Collecting influxdb==5.2.2
Downloading influxdb-5.2.2-py2.py3-none-any.whl (70 kB)
Collecting configparser==3.5.0
Downloading configparser-3.5.0.tar.gz (39 kB)
Collecting reportlab
Downloading reportlab-3.5.42.tar.gz (2.9 MB)
Collecting PyPDF2
Downloading PyPDF2-1.26.0.tar.gz (77 kB)
Collecting matplotlib>=2.2.2
Downloading matplotlib-3.2.1.tar.gz (40.3 MB)
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mqz7vwwc/matplotlib/setup.py'"'"'; __file__='"'"'/tmp/pip-install-mqz7vwwc/matplotlib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-mqz7vwwc/matplotlib/pip-egg-info
cwd: /tmp/pip-install-mqz7vwwc/matplotlib/
Complete output (112 lines):
Processing numpy/random/_bounded_integers.pxd.in
Processing numpy/random/_bounded_integers.pyx.in
Traceback (most recent call last):
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 61, in process_pyx
from Cython.Compiler.Version import version as cython_version
ModuleNotFoundError: No module named 'Cython'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 238, in <module>
main()
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 234, in main
find_process_files(root_dir)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 225, in find_process_files
process(root_dir, fromfile, tofile, function, hash_db)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 191, in process
processor_function(fromfile, tofile)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 94, in process_tempita_pyx
process_pyx(pyxfile, tofile)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 66, in process_pyx
raise OSError('Cython needs to be installed in Python as a module')
OSError: Cython needs to be installed in Python as a module
Running from numpy source directory.
/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py:461: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
run_build = parse_setuppy_commands()
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
_execfile(setup_script, ns)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 488, in <module>
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 469, in setup_package
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 275, in generate_cython
)
RuntimeError: Running cythonize failed!
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-mqz7vwwc/matplotlib/setup.py", line 274, in <module>
cmdclass=cmdclass,
File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 139, in setup
_install_setup_requires(attrs)
File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 134, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/usr/local/lib/python3.6/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 777, in resolve
replace_conflicting=replace_conflicting
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1060, in best_match
return self.obtain(req, installer)
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1072, in obtain
return installer(requirement)
File "/usr/local/lib/python3.6/site-packages/setuptools/dist.py", line 581, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 676, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 702, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 887, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1155, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1141, in run_setup
run_setup(setup_script, args)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 253, in run_setup
raise
File "/usr/local/lib/python3.6/contextlib.py", line 99, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python3.6/contextlib.py", line 99, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 166, in save_modules
saved_exc.resume()
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 141, in resume
six.reraise(type, exc, self._tb)
File "/usr/local/lib/python3.6/site-packages/setuptools/_vendor/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
_execfile(setup_script, ns)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 488, in <module>
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 469, in setup_package
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 275, in generate_cython
)
RuntimeError: Running cythonize failed!
Edit setup.cfg to change the build options; suppress output with --quiet.
BUILDING MATPLOTLIB
matplotlib: yes [3.2.1]
python: yes [3.6.6 (default, Sep 12 2018, 02:15:29) [GCC 6.4.0]]
platform: yes [linux]
sample_data: yes [installing]
tests: no [skipping due to configuration]
agg: yes [installing]
tkagg: yes [installing; run-time loading from Python Tcl/Tk]
macosx: no [Mac OS-X only]
Cythonizing sources
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 20.0.2; however, version 20.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
The command '/bin/sh -c pip3 install -r /app/requirements.txt' returned a non-zero code: 1
Makefile:8: recipe for target 'build' failed
make: *** [build] Error 1

I was able to resolve the problem by including
RUN pip3 install Cython
in the Dockerfile, above the requirements installation.

So the problem is you installed two versions of Python, I'm guessing:
The Docker image has Python included.
You then installed the Alpine system Python with apk.
You should remove the python-dev package from apk install line.
Also—you only need to compile matplotlib, numpy, etc. because you're using Alpine. If you switch to a base image that isn't Alpine based (e.g. python:3.6-slim-buster) you'll be able to use pre-compiled wheels and your build process will be much faster. See https://pythonspeed.com/articles/alpine-docker-python/ for longer discussion, but basically Alpine uses libc that's different than standard Linux glibc and so wheels from PyPI don't work.

Django: No module named 'psycopg2' when launching migrations to heroku

I am new to django and I created a website and now I am in the deployment phase, all is well run until launching migrating from the database to the server. I'll show you everything I've done since the beginning
structure of my project
OC_project8/
pur_beurre/
catalog/
dumps/
catalog.json
... # other classic file
media/
pur_beurre/
static/
.gitignore
... # other classic file
static/
templates/
users/
manage.py
Procfile
venv/
requirements.txt/
certifi==2019.9.11
chardet==3.0.4
coverage==4.5.4
dj-database-url==0.5.0
Django==2.2.6
django-debug-toolbar==2.0
gunicorn==19.9.0
idna==2.8
mysqlclient==1.4.4
Pillow==6.2.0
pytz==2019.3
requests==2.22.0
sqlparse==0.3.0
urllib3==1.25.6
whitenoise==4.1.4
.gitignore/
All command was run in a virtual environement, for exemple:
(venv) ~/OC_project8/pur_beurre$ sudo apt-get install python-psycopg2
I did all the necessary steps before runing the command git push heroku master
(git commit, heroku create my-app ...)
if you need more detail on these steps tell me.
After runing the command git push heroku master I had this error:
....
remote: Collecting pkg-resources==0.0.0 (from -r /tmp/build_cac68f54540915062647a425c52dc61b/requirements.txt (line 11))
remote: Could not find a version that satisfies the requirement pkg-resources==0.0.0 (from -r/tmp/build_cac68f54540915062647a425c52dc61b/requirements.txt (line 11)) (from versions: )
remote: No matching distribution found for pkg-resources==0.0.0 (from -r /tmp/build_cac68f54540915062647a425c52dc61b/requirements.txt (line 11))
remote: ! Push rejected, failed to compile Python app.
remote:
remote: ! Push failed
remote: Verifying deploy...
remote:
remote: ! Push rejected to pur-beurre-oc8.
remote:
To https://git.heroku.com/pur-beurre-oc8.git
! [remote rejected] master -> master (pre-receive hook declined)
error: impossible de pousser des références vers 'https://git.heroku.com/pur-beurre-oc8.git'
I saw in stackoverflow that I have to do that:
pip uninstall pkg-resources==0.0.0
After I made a commit:
git add requirements.txt
git commit -m "uninstalling pkg-resources==0.0.0"
And now git push heroku master work fine, but when I access my website I had the error: Apllication error. I don't know if I'm getting wet but I think because I have not done the migration of the database yet.
So I run the command heroku run python pur_beurre/manage.py migrate
And now I had this error:
Running python pur_beurre/manage.py migrate on ⬢ pur-beurre-oc8... up, run.4138 (Free)
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 20, in <module>
import psycopg2 as Database
ModuleNotFoundError: No module named 'psycopg2'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "pur_beurre/manage.py", line 21, in <module>
main()
File "pur_beurre/manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/app/.heroku/python/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/app/.heroku/python/lib/python3.6/site-packages/django/core/management/__init__.py", line 357, in execute
django.setup()
File "/app/.heroku/python/lib/python3.6/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/app/.heroku/python/lib/python3.6/site-packages/django/apps/registry.py", line 114, in populate
app_config.import_models()
File "/app/.heroku/python/lib/python3.6/site-packages/django/apps/config.py", line 211, in import_models
self.models_module = import_module(models_module_name)
File "/app/.heroku/python/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/auth/models.py", line 2, in <module>
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/auth/base_user.py", line 47, in <module>
class AbstractBaseUser(models.Model):
File "/app/.heroku/python/lib/python3.6/site-packages/django/db/models/base.py", line 117, in __new__
new_class.add_to_class('_meta', Options(meta, app_label))
File "/app/.heroku/python/lib/python3.6/site-packages/django/db/models/base.py", line 321, in add_to_class
value.contribute_to_class(cls, name)
File "/app/.heroku/python/lib/python3.6/site-packages/django/db/models/options.py", line 204, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
File "/app/.heroku/python/lib/python3.6/site-packages/django/db/__init__.py", line 28, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/app/.heroku/python/lib/python3.6/site-packages/django/db/utils.py", line 201, in __getitem__
backend = load_backend(db['ENGINE'])
File "/app/.heroku/python/lib/python3.6/site-packages/django/db/utils.py", line 110, in load_backend
return import_module('%s.base' % backend_name)
File "/app/.heroku/python/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/app/.heroku/python/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 24, in <module>
raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named 'psycopg2'
I don't know why I have this error since psycopg2 is a library required to use PostgreSQL and I am using MySQL, here is my settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'openfoodfacts',
'USER': 'root',
'PASSWORD': 'my-password',
'HOST': '',
'PORT': '5432',
}
}
In any case after having this error I tried to install psycopg2:
pip install psycopg2
But I had another error:
Collecting psycopg2
Downloading https://files.pythonhosted.org/ packages/84/d7/6a93c99b5ba4d4d22daa3928b983cec66df4 536ca50b22ce5dcac65e4e71/psycopg2-2.8.4.tar.gz (377kB)
100% |████████████████████████████████| 378kB 1.8MB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/rouizi/OC_project8/venv/lib/python3.6/site-packages/setuptools/__init__.py", line 12, in <module>
import setuptools.version
File "/home/rouizi/OC_project8/venv/lib/python3.6/site-packages/setuptools/version.py", line 1, in <module>
import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-gf4fxs4k/psycopg2/
Maybe I got this error because I uninstalled
in the beginning pkg-resources ! But I tried to reinstall pkg-resources:
sudo apt-get install --reinstall python-pkg-resources
And I rerun the command pip install psycopg2, but still have the same error
By searching on google I found here No module named pkg_resources that it was necessary to run this command:
wget https://bootstrap.pypa.io/ez_setup.py -O - | python
And I rerun the command pip install psycopg2, but I have another error:
Collecting psycopg2
Using cached https://files.pythonhosted.org/ packages/84/d7/6a93c99b5ba4d4d22daa3928b983cec66df 4536ca50b22ce5dcac65e4e71/psycopg2-2.8.4.tar.gz
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
Error: pg_config executable not found.
pg_config is required to build psycopg2 from source. Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.
For further information please check the 'doc/src/install.rst' file (also at
<http://initd.org/psycopg/docs/install.html>).
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-sglmdb2o/psycopg2/
And if I run the command which pg_config it return nothing, but after some research I fond here pg_config executable not found that it was necessary to run this command:
sudo apt-get install libpq-dev
and now which pg_config return this:
/usr/bin/pg_config
So I rerun pip install psycopg2 and another error occured:
Collecting psycopg2
Using cached https://files.pythonhosted.org/ packages/84/d7/6a93c99b5ba4d4d22 daa3928b983cec66df4536ca50b22ce5dcac65e4e71/psycopg2-2.8.4.tar.gz
Building wheels for collected packages: psycopg2
Running setup.py bdist_wheel for psycopg2 ... error
Complete output from command /home/rouizi/OC_project8/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-lu4ihs81/psycopg2/setup.py';f=getattr(tokenize,
'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmphgjf714ppip-wheel- --python-tag cp36:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
Failed building wheel for psycopg2
Running setup.py clean for psycopg2
Failed to build psycopg2
Installing collected packages: psycopg2
Running setup.py install for psycopg2 ... error
#I cut here because it's very long
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-lu4ihs81/psycopg2/setup.py", line 611, in <module>
'Documentation': 'http://initd.org/psycopg/docs/',
File "/home/rouizi/OC_project8/venv/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/home/rouizi/OC_project8/venv/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/usr/lib/python3.6/distutils/command/install.py", line 601, in run
self.run_command(cmd_name)
File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/home/rouizi/OC_project8/venv/lib/python3.6/site-packages/setuptools/command/install_scripts.py", line 17, in run
import setuptools.command.easy_install as ei
File "/home/rouizi/OC_project8/venv/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 47, in <module>
from setuptools.sandbox import run_setup
File "/home/rouizi/OC_project8/venv/lib/python3.6/site-packages/setuptools/sandbox.py", line 15, in <module>
import pkg_resources.py31compat
ModuleNotFoundError: No module named 'pkg_resources.py31compat'
----------------------------------------
Command "/home/rouizi/OC_project8/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-lu4ihs81/psycopg2/setup.py';f=getattr(tokenize, 'open', open)(__file__)
;code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-eya3xv3b-record/install-record.txt --single-version-externally-managed
--compile --install-headers /home/rouizi/OC_project8/venv/include/site/python3.6/psycopg2" failed with error code 1 in /tmp/pip-build-lu4ihs81/psycopg2/
I tried this command:
sudo apt-get install --reinstall python-setuptools
And now when I try to run pip install psycopg2 he return to me:
Requirement already satisfied: psycopg2 in /home/rouizi/OC_project8/venv/lib/python3.6/site-packages
But if I try to run the command heroku run python pur_beurre/manage.py migrate I still have the error:
django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named 'psycopg2'
But if I run sudo apt-get install python-psycopg2 It worked but I still have the error when runing heroku run python pur_beurre/manage.py migrate
Since I still had the same error, so I tried uninstall psycopg2:
sudo pip uninstall psycopg2
But it return to me that:
The directory '/home/rouizi/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Not uninstalling psycopg2 at /usr/lib/python2.7/dist-packages, outside environment /usr
And there I am stuck for a day without finding a solution, I hope you can help
sorry for the long post and for my misspelling

I solved my problem !
I ran the command pip install psycopg2, it return to me the error:
ModuleNotFoundError: No module named 'pkg_resources.py31compat'
But if I try to rerun the command pip install psycopg2, it says:
Requirement already satisfied: psycopg2 in /home/rouizi/OC_project8/venv/lib/python3.6/site-packages
So I tryied to commit my change:
pip freeze > requirements.txt
git add requirements.txt
git commit -m "installing psycopg2"
git push heroku master
And now heroku run python pur_beurre/manage.py migrate work fine.
I don't know why since I had an error, if someone has an explanation will be well

Python Briefcase django: setup.py returned non-zero exit status 1

Ubuntu 18.04 LTS
Python 3.6
Node 8.11.4 (LTS)
NPM 5.6.0
Trying to install the tutorial django project by following the instructions in the briefcase tutorial. It installs the Linux file just fine, but attempting to install it the same way but with django python3 setup.py django -s gives me the following failure:
running django
* Updating user code...
* Installing requirements...
No requirements.
* Installing plaform requirements...
Collecting toga-django==0.3.0.dev9
Using cached https://files.pythonhosted.org/packages/9e/88/49d9ab4c128e61a47bb365201fdfa3ebf1fd8905c0c5baa06dc248db9f0a/toga_django-0.3.0.dev9-py3-none-any.whl
Collecting toga-core==0.3.0.dev9 (from toga-django==0.3.0.dev9)
Using cached https://files.pythonhosted.org/packages/50/a1/ebc3b57145b03b644cc53c2693838eb3afafdb686eec2dffd90cf3548d09/toga_core-0.3.0.dev9-py3-none-any.whl
Collecting django-environ==0.4.1 (from toga-django==0.3.0.dev9)
Using cached https://files.pythonhosted.org/packages/f4/06/8dd165534eae19e72ca78bd6e845566cd1e0aacc9c20cc43984675748345/django_environ-0.4.1-py2.py3-none-any.whl
Collecting django==1.10.6 (from toga-django==0.3.0.dev9)
Using cached https://files.pythonhosted.org/packages/b9/bb/723f78e6f6aea78590331eba4e42b8a09c33ce154204a942525a91101d0b/Django-1.10.6-py2.py3-none-any.whl
Collecting travertino>=0.1.0 (from toga-core==0.3.0.dev9->toga-django==0.3.0.dev9)
Using cached https://files.pythonhosted.org/packages/56/3f/f03efe63c2bdc72763ac39777c9ebbab98605ced5c74aefe48b4fa0abb4d/travertino-0.1.2-py3-none-any.whl
Collecting six (from django-environ==0.4.1->toga-django==0.3.0.dev9)
Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Installing collected packages: travertino, toga-core, django, six, django-environ, toga-django
Found existing installation: travertino 0.1.2
Uninstalling travertino-0.1.2:
Successfully uninstalled travertino-0.1.2
Found existing installation: toga-core 0.3.0.dev9
Uninstalling toga-core-0.3.0.dev9:
Successfully uninstalled toga-core-0.3.0.dev9
Found existing installation: Django 1.10.6
Uninstalling Django-1.10.6:
Successfully uninstalled Django-1.10.6
Found existing installation: six 1.11.0
Uninstalling six-1.11.0:
Successfully uninstalled six-1.11.0
Found existing installation: django-environ 0.4.1
Uninstalling django-environ-0.4.1:
Successfully uninstalled django-environ-0.4.1
Found existing installation: toga-django 0.3.0.dev9
Uninstalling toga-django-0.3.0.dev9:
Successfully uninstalled toga-django-0.3.0.dev9
Successfully installed django-1.10.6 django-environ-0.4.1 six-1.11.0 toga-core-0.3.0.dev9 toga-django-0.3.0.dev9 travertino-0.1.2
* Installing project code...
Traceback (most recent call last):
File "setup.py", line 79, in <module>
'toga-django==0.3.0.dev9',
File "/home/hv/Projects/tutorial/venv/lib/python3.6/site-packages/setuptools/__init__.py", line 140, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/home/hv/Projects/tutorial/venv/lib/python3.6/site-packages/briefcase/app.py", line 478, in run
self.install_code()
File "/home/hv/Projects/tutorial/venv/lib/python3.6/site-packages/briefcase/app.py", line 314, in install_code
stderr=subprocess.STDOUT,
File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/usr/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['pip', 'install', '--upgrade', '--force-reinstall', '--no-dependencies', '--target=/home/hv/Projects/tutorial/helloworld/django', '.']' returned non-zero exit status 1.
I have attempted searching around, and even running the pip command directly with pip install --upgrade --force-reinstall --no-dependencies --target=/home/hv/Projects/tutorial/helloworld/django . which outputs:
Processing /home/hv/Projects/tutorial/helloworld
Installing collected packages: helloworld
Running setup.py install for helloworld ... done
Successfully installed helloworld-0.0.1
I'm quite new to this, and looking for some advice. This is within a virtualenv and with versions:
pip 18.0 from /home/hv/Projects/tutorial/venv/lib/python3.6/site-packages/pip (python 3.6)
UPDATE:
Removed the specific requirement for toga-django==0.3.0.dev9 in the setup.py file, and it appears to have successfully installed. However, it cannot actually launch the Django server:
python setup.py django -s -v
running django
* Updating user code...
* Installing requirements...
No requirements.
* Installing plaform requirements...
No platform requirements.
* Installing project code...
Processing /home/hv/Projects/tutorial/helloworld
Installing collected packages: helloworld
Running setup.py install for helloworld: started
Running setup.py install for helloworld: finished with status 'done'
Successfully installed helloworld-0.0.1
* No icons defined - using default...
* No splash screen defined...
* Installing NPM requirements...
/usr/bin/env: ‘node’: Not a directory
Installation complete.
* Building Webpack assets...
/usr/bin/env: ‘node’: Not a directory
* Starting Django server on localhost:8042
* Opening browser...
Traceback (most recent call last):
File "./manage.py", line 22, in <module>
execute_from_command_line(sys.argv)
File "/home/hv/Projects/tutorial/venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
utility.execute()
File "/home/hv/Projects/tutorial/venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 341, in execute
django.setup()
File "/home/hv/Projects/tutorial/venv/lib/python3.6/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/hv/Projects/tutorial/venv/lib/python3.6/site-packages/django/apps/registry.py", line 85, in populate
app_config = AppConfig.create(entry)
File "/home/hv/Projects/tutorial/venv/lib/python3.6/site-packages/django/apps/config.py", line 116, in create
mod = import_module(mod_path)
File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'toga.django'
App started.

Unable to install pyzmail - "Command "python setup.py egg_info" failed with error code 1"

I am trying to install pyzmail for Python 3.4. I am using Visual Studio Community Edition (Windows) but have also tried to install using the command line and get the following dump:
----- Installing 'pyzmail' -----
Collecting pyzmail
Using cached pyzmail-1.0.3.tar.gz
Collecting distribute (from pyzmail)
Using cached distribute-0.7.3.zip
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info\distribute.egg-info
writing requirements to pip-egg-info\distribute.egg-info\requires.txt
writing top-level names to pip-egg-info\distribute.egg-info\top_level.txt
writing dependency_links to pip-egg-info\distribute.egg-info\dependency_links.txt
writing pip-egg-info\distribute.egg-info\PKG-INFO
Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Temp\pip-build-v5zvr2p3\distribute\pkg_resources.py", line 2680, in _dep_map
return self.__dep_map
File "C:\Users\user\AppData\Local\Temp\pip-build-v5zvr2p3\distribute\pkg_resources.py", line 2525, in __getattr__
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\user\AppData\Local\Temp\pip-build-v5zvr2p3\distribute\setup.py", line 58, in <module>
setuptools.setup(**setup_params)
File "C:\Python34\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Python34\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Users\user\AppData\Local\Temp\pip-build-v5zvr2p3\distribute\setuptools\command\egg_info.py", line 177, in run
writer = ep.load(installer=installer)
File "C:\Users\user\AppData\Local\Temp\pip-build-v5zvr2p3\distribute\pkg_resources.py", line 2241, in load
if require: self.require(env, installer)
File "C:\Users\user\AppData\Local\Temp\pip-build-v5zvr2p3\distribute\pkg_resources.py", line 2254, in require
working_set.resolve(self.dist.requires(self.extras),env,installer)))
File "C:\Users\user\AppData\Local\Temp\pip-build-v5zvr2p3\distribute\pkg_resources.py", line 2471, in requires
dm = self._dep_map
File "C:\Users\user\AppData\Local\Temp\pip-build-v5zvr2p3\distribute\pkg_resources.py", line 2682, in _dep_map
self.__dep_map = self._compute_dependencies()
File "C:\Users\user\AppData\Local\Temp\pip-build-v5zvr2p3\distribute\pkg_resources.py", line 2699, in _compute_dependencies
from _markerlib import compile as compile_marker
ImportError: No module named '_markerlib'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-build-v5zvr2p3\distribute\
----- Failed to install 'pyzmail' -----
After Googling, I checked I had the ez_setup and setuptools modules installed, which I do. Any help would be much appreciated!

I've been able to reproduce your problem, and you're right, using pip install pyzmail wont' be good (pyzmail's installer is buggy) so i've tried this instead:
easy_install pyzmail
And it succeed, then I could just import pyzmail without any problem.

You can also try installing the pyzmail36 fork, which installs the same modules for Python 3.6 and later: pip install pyzmail36 on Windows and pip3 install pyzmail36 on Linux and macOS.

You can try easy_install pyzmail
That worked for me in 2020.

xcode-select error during pyobjc-core installation

I am trying to install pyobjc-core which is a requirement for pyautogui. I am getting the below error while installing. I have already updated setuptools using
pip3 install --upgrade setuptools
and also using
sudo python3 ez_setup.py
I am on OSX Yosemite and using Python 3.5.
I see that the code below has a requirement for xcode. Could that be the reason and why would xcode be needed for this installation?
pip3 install pyobjc-core
Collecting pyobjc-core
Using cached pyobjc-core-3.1.1.tar.gz
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/pyobjc_core.egg-info
writing include/pyobjc-compat.h to pip-egg-info/pyobjc_core.egg-info/include/pyobjc-compat.h
writing include/pyobjc-api.h to pip-egg-info/pyobjc_core.egg-info/include/pyobjc-api.h
writing namespace_packages to pip-egg-info/pyobjc_core.egg-info/namespace_packages.txt
writing top-level names to pip-egg-info/pyobjc_core.egg-info/top_level.txt
writing dependency_links to pip-egg-info/pyobjc_core.egg-info/dependency_links.txt
writing pip-egg-info/pyobjc_core.egg-info/PKG-INFO
writing manifest file 'pip-egg-info/pyobjc_core.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance
Traceback (most recent call last):
File "", line 1, in
File "/private/var/folders/zw/573csqzn0pnbjkb44gw2s7800000gn/T/pip-build-72bxsxfe/pyobjc-core/setup.py", line 696, in
**parse_package_metadata()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/core.py", line 148, in setup
dist.run_commands()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/private/var/folders/zw/573csqzn0pnbjkb44gw2s7800000gn/T/pip-build-72bxsxfe/pyobjc-core/setup.py", line 371, in run
egg_info.egg_info.run(self)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-23.0.0-py3.5.egg/setuptools/command/egg_info.py", line 193, in run
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-23.0.0-py3.5.egg/setuptools/command/egg_info.py", line 216, in find_sources
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-23.0.0-py3.5.egg/setuptools/command/egg_info.py", line 300, in run
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-23.0.0-py3.5.egg/setuptools/command/egg_info.py", line 329, in add_defaults
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-23.0.0-py3.5.egg/setuptools/command/sdist.py", line 132, in add_defaults
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/cmd.py", line 299, in get_finalized_command
cmd_obj.ensure_finalized()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/private/var/folders/zw/573csqzn0pnbjkb44gw2s7800000gn/T/pip-build-72bxsxfe/pyobjc-core/setup.py", line 525, in finalize_options
universal_newlines=True).strip()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py", line 629, in check_output
**kwargs).stdout
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py", line 711, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/bin/xcodebuild', '-version', '-sdk', 'macosx', 'Path']' returned non-zero exit status 1
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/zw/573csqzn0pnbjkb44gw2s7800000gn/T/pip-build-72bxsxfe/pyobjc-core/

The error is clear: you need xcode...
It's silly but unfortunately there are these developer tools that are tied to it, and since many packages require dev tools (in your specific case xcodebuild needs it) you have to install xcode (anyway it's free)