Cython embed binary (docker): Failed to import the site module - python

I'm building Hello World using cython embed. Using docker python-3.5.3 image (Debian Jessie, with libpython3.5-dev additionaly installed)
hello.py:
print('hello world')
building:
cython --embed -o hello.c hello.py
gcc -I /usr/include/python3.5 -o hello hello.c -lpython3.5m
After trying to run i get ImportError:
Failed to import the site module
Traceback (most recent call last):
File "/usr/lib/python3.5/site.py", line 580, in <module>
main()
File "/usr/lib/python3.5/site.py", line 566, in main
known_paths = addusersitepackages(known_paths)
File "/usr/lib/python3.5/site.py", line 287, in addusersitepackages
user_site = getusersitepackages()
File "/usr/lib/python3.5/site.py", line 263, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/usr/lib/python3.5/site.py", line 253, in getuserbase
USER_BASE = get_config_var('userbase')
File "/usr/lib/python3.5/sysconfig.py", line 595, in get_config_var
return get_config_vars().get(name)
File "/usr/lib/python3.5/sysconfig.py", line 538, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/usr/lib/python3.5/sysconfig.py", line 410, in _init_posix
from _sysconfigdata import build_time_vars
File "/usr/lib/python3.5/_sysconfigdata.py", line 6, in <module>
from _sysconfigdata_m import *
ImportError: No module named '_sysconfigdata_m'
I suggest that something is wrong with PATH or PYTHONPATH, but launching binary file with directly set environment fails with same error.

I used
find / -name '_sysconfigdata_m.py'
/usr/lib/python3.5/plat-x86_64-linux-gnu/_sysconfigdata_m.py
And then
cp /usr/lib/python3.5/plat-x86_64-linux-gnu/_sysconfigdata_m.py /usr/lib/python3.5 Everything worked fine.
But I still wonder - why _sysconfigdata_m is not placed in /usr/lib/python3.5 by default.

Related

cannot uninstall enum34 (Python 3.6, MacOS)

I'm trying to open my jupyter notebook with Conda Python3.6. When I try to open it, I'm getting this error
VikMacBook-Pro:~ vik$ /anaconda3/bin/jupyter_mac.command ; exit;
Failed to import the site module
Traceback (most recent call last):
File "/anaconda3/lib/python3.6/site.py", line 541, in <module>
main()
File "/anaconda3/lib/python3.6/site.py", line 522, in main
known_paths = addusersitepackages(known_paths)
File "/anaconda3/lib/python3.6/site.py", line 282, in
addusersitepackages
user_site = getusersitepackages()
File "/anaconda3/lib/python3.6/site.py", line 258, in
getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/anaconda3/lib/python3.6/site.py", line 248, in getuserbase
USER_BASE = get_config_var('userbase')
File "/anaconda3/lib/python3.6/sysconfig.py", line 608, in
get_config_var
return get_config_vars().get(name)
File "/anaconda3/lib/python3.6/sysconfig.py", line 587, in
get_config_vars
import _osx_support
File "/anaconda3/lib/python3.6/_osx_support.py", line 4, in <module>
import re
File "/anaconda3/lib/python3.6/re.py", line 142, in <module>
class RegexFlag(enum.IntFlag):
AttributeError: module 'enum' has no attribute 'IntFlag'
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
And I have attempted to solve by following this endorsed answer, using command
pip uninstall enum34
However, I'm still getting the following error after running this command
Vik-MacBook-Pro:~ vik$ pip uninstall enum34
Skipping enum34 as it is not installed.
I do not have my own code file named enum.py
try using "pip3 uninstall enum34" it worked for me

_sysconfigdata error when trying to Freeze a Python 3.6.5 application

I'm trying to Freeze a Python 3.6.5 app into a standalone executable on Linux (Fedora 27 64 bit), using the default Python freeze.py utility. I have a checkout of Python 3.6.5 compiled, and I'm trying to freeze my application with the following (sanitized) commands:
LOCAL_PYTHON=(path to my compiled local python directory)
$LOCAL_PYTHON/python $LOCAL_PYTHON/Tools/freeze/freeze.py -p $LOCAL_PYTHON
-P $LOCAL_PYTHON -o frozen myapp.py
This freeze command seems to work fine, and generates the myapp executable as I would expect, but running the executable throws the following exception:
Failed to import the site module
Traceback (most recent call last):
File "$LOCAL_PYTHON/Lib/importlib/_bootstrap.py", line 971, in _find_and_load
return _find_and_load_unlocked(name, import_)
File "$LOCAL_PYTHON/Lib/importlib/_bootstrap.py", line 955, in _find_and_load_unlocked
module = _load_unlocked(spec)
File "$LOCAL_PYTHON/Lib/importlib/_bootstrap.py", line 665, in _load_unlocked
spec.loader.exec_module(module)
File "$LOCAL_PYTHON/Lib/importlib/_bootstrap.py", line 807, in exec_module
exec(code, module.__dict__)
File "$LOCAL_PYTHON/Lib/site.py", line 544, in <module>
main()
File "$LOCAL_PYTHON/Lib/site.py", line 530, in main
known_paths = addusersitepackages(known_paths)
File "$LOCAL_PYTHON/Lib/site.py", line 282, in addusersitepackages
user_site = getusersitepackages()
File "$LOCAL_PYTHON/Lib/site.py", line 258, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "$LOCAL_PYTHON/Lib/site.py", line 248, in getuserbase
USER_BASE = get_config_var('userbase')
File "$LOCAL_PYTHON/Lib/sysconfig.py", line 601, in get_config_var
return get_config_vars().get(name)
File "$LOCAL_PYTHON/Lib/sysconfig.py", line 550, in get_config_vars
_init_posix(_CONFIG_VARS)
File "$LOCAL_PYTHON/Lib/sysconfig.py", line 421, in _init_posix
_temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
File "$LOCAL_PYTHON/Lib/importlib/_bootstrap.py", line 971, in _find_and_load
return _find_and_load_unlocked(name, import_)
File "$LOCAL_PYTHON/Lib/importlib/_bootstrap.py", line 953, in _find_and_load_unlocked
raise ModuleNotFoundError(_ERR_MSG.format(name), name=name)
ModuleNotFoundError: No module named '_sysconfigdata_m_linux_x86_64-linux-gnu'
My application works fine if I run it without freezing (e.g. $LOCAL_PYTHON/python myapp.py), and the same freeze command and app generate a working executable on Windows. Are there some platform specific Freeze parameters or environment variables that I'm missing?
So I was able to reproduce the issue by using docker. I ran a docker container using below
docker run -it fedora:27 bash
And then ran below commands to setup python from source
yum install -y git gcc unzip zip wget curl make cmake zlib-devel libsq3-devel readline-devel openssl-devel which vim findutils
mkdir py
cd py/
wget "https://github.com/python/cpython/archive/v3.6.5.zip"
unzip v3.6.5.zip
cd cpython-3.6.5/
./configure --with-debug
make -j8
After that I created a simple app.py
print('this is from app')
And then I ran
export LANG=en_US.UTF-8
LOCAL_PYTHON=/py/cpython-3.6.5/
$LOCAL_PYTHON/python $LOCAL_PYTHON/Tools/freeze/freeze.py -p $LOCAL_PYTHON -P $LOCAL_PYTHON -o frozen app.py
cd frozen/
make
./app
And I got below exception
[root#54ba4757d177 frozen]# ./app
Failed to import the site module
Traceback (most recent call last):
File "/py/cpython-3.6.5/Lib/importlib/_bootstrap.py", line 971, in _find_and_load
return _find_and_load_unlocked(name, import_)
File "/py/cpython-3.6.5/Lib/importlib/_bootstrap.py", line 955, in _find_and_load_unlocked
module = _load_unlocked(spec)
File "/py/cpython-3.6.5/Lib/importlib/_bootstrap.py", line 665, in _load_unlocked
spec.loader.exec_module(module)
File "/py/cpython-3.6.5/Lib/importlib/_bootstrap.py", line 807, in exec_module
exec(code, module.__dict__)
File "/py/cpython-3.6.5/Lib/site.py", line 544, in <module>
main()
File "/py/cpython-3.6.5/Lib/site.py", line 530, in main
known_paths = addusersitepackages(known_paths)
File "/py/cpython-3.6.5/Lib/site.py", line 282, in addusersitepackages
user_site = getusersitepackages()
File "/py/cpython-3.6.5/Lib/site.py", line 258, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/py/cpython-3.6.5/Lib/site.py", line 248, in getuserbase
USER_BASE = get_config_var('userbase')
File "/py/cpython-3.6.5/Lib/sysconfig.py", line 601, in get_config_var
return get_config_vars().get(name)
File "/py/cpython-3.6.5/Lib/sysconfig.py", line 550, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/py/cpython-3.6.5/Lib/sysconfig.py", line 421, in _init_posix
_temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
File "/py/cpython-3.6.5/Lib/importlib/_bootstrap.py", line 971, in _find_and_load
return _find_and_load_unlocked(name, import_)
File "/py/cpython-3.6.5/Lib/importlib/_bootstrap.py", line 953, in _find_and_load_unlocked
raise ModuleNotFoundError(_ERR_MSG.format(name), name=name)
ModuleNotFoundError: No module named '_sysconfigdata_m_linux_x86_64-linux-gnu'
Same like you got. Then I ran make install to install python and tried again
make install -C /py/cpython-3.6.5/
And then the app again
[root#54ba4757d177 frozen]# ./app
this is from app
And it worked, so I understood that the issue is that the generated app assumes python is installed on machine. And then I found Gold in the form of
Python freeze.py generated bin doesn't run
I updated the /py/cpython-3.6.5/Tools/freeze/makefreeze.py file and added Py_NoSiteFlag = 1; before PyImport_FrozenModules = _PyImport_FrozenModules;.
Of course I created the container again to make sure no python was installed on system. And then recompiled the app again and bingo!
[root#06284cc1ae0c frozen]# ./app
this is from app
To go one step further, I deleted all python source, build files and ran app again and it worked great

ImportError: No module named sysconfig

I get this error many times whenever I try to run a python module.. for example i tried running pyspark from spark bin folder and getting the error shown below..
soundarya#soundarya:~/spark/spark-1.5.1-bin-hadoop2.6/bin$ ./pyspark
Traceback (most recent call last):
File "/usr/lib/python2.7/site.py", line 563, in <module>
main()
File "/usr/lib/python2.7/site.py", line 545, in main
known_paths = addusersitepackages(known_paths)
File "/usr/lib/python2.7/site.py", line 272, in addusersitepackages
user_site = getusersitepackages()
File "/usr/lib/python2.7/site.py", line 247, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/usr/lib/python2.7/site.py", line 236, in getuserbase
from sysconfig import get_config_var
ImportError: No module named sysconfig
soundarya#soundarya:~$ whereis python2.7
python2: /usr/bin/python2.6 /usr/bin/python2.7 /usr/bin/python2.6-config /usr/bin/python2 /usr/bin/python2.7-config /usr/lib/python2.6 /usr/lib/python2.7 /etc/python2.6 /etc/python2.7 /usr/local/lib/python2.6 /usr/local/lib/python2.7 /usr/include/python2.6 /usr/include/python2.7 /opt/splunk.old/bin/python2.7 /opt/splunk.old/bin/python2 /opt/splunk/bin/python2.7 /opt/splunk/bin/python2 /usr/share/man/man1/python2.1.gz
can anyone help to fix this issue? thank you so much in advance.

Linux - Weird Python Output

When ever i mistype or do a error into the console the following message come up:
Traceback (most recent call last):
File "/usr/lib/python3.3/site.py", line 629, in <module>
main()
File "/usr/lib/python3.3/site.py", line 614, in main
known_paths = addusersitepackages(known_paths)
File "/usr/lib/python3.3/site.py", line 284, in addusersitepackages
user_site = getusersitepackages()
File "/usr/lib/python3.3/site.py", line 260, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/usr/lib/python3.3/site.py", line 250, in getuserbase
USER_BASE = get_config_var('userbase')
File "/usr/lib/python3.3/sysconfig.py", line 610, in get_config_var
return get_config_vars().get(name)
File "/usr/lib/python3.3/sysconfig.py", line 560, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/usr/lib/python3.3/sysconfig.py", line 432, in _init_posix
from _sysconfigdata import build_time_vars
File "/usr/lib/python3.3/_sysconfigdata.py", line 6, in <module>
from _sysconfigdata_m import *
ImportError: No module named '_sysconfigdata_m'
I have both Python 2.7 and 3.3 install with Anaconda. I wonder if this is normal or it was a conflict between python 2.7 and 3.3
Assuming you are using ubuntu, here is the relevant bug report https://bugs.launchpad.net/ubuntu/+source/python3.3/+bug/1192890
You need to patch your /etc/bash.bashrc. See comment #6 for details
Actually, it's possible you have another python3 in your PATH before /usr/bin (most probably in /usr/local/bin) in your system, one which lacks the ubuntu-provided libraries (used by /usr/bin/python3). That happens with hashbangs using #!env python3 and the PATH environment variable.
If this is the case, then temporarily make unavailable (eg renaming them) all such non-/usr/bin/python3* executables, do your system administration stuff and then make them again available.

Setting python path for WinPython to use f2py

I installed the Winpython distribution on my copy of Windows 7. Launching iPython consoles and other items from the distribution from within the folder it copied to works fine.
I'd like to use the f2py module from numpy to be able to call Fortran subroutines from Python. My understanding is that f2py must be called from the command line, but the system does not seem to find f2py, returning ImportError: no module named site when I call it either with or without flags. This same error is returned when I try to run python itself from the command line.
When I manually navigate to the Winpython directory (e.g. C:\Users\AGK\WinPython-32bit-2.7.5.3\python-2.7.5) and call f2py -c --help-fcompiler to see if f2py is found there, I receive the following error
Traceback (most recent call last):
File ".\lib\site.py", line 538, in main
main()
File ".\lib\site.py", line 530, in main
known_paths = addusersitepackages(known_paths)
File ".\lib\site.py", line 266, in addusersitepackages
user_site = getusersitepackages()
File ".\lib\site.py", line 241, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File ".\lib\site.py", line 231, in getuserbase
USER_BASE = get_config_var('userbase')
File "C:\Users\AGK\WinPython-32bit-2.7.5.3\python-2.7.5\lib\sysconfig.py",
line 516, in get_config_var
return get_config_vars().get(name)
File "C:\Users\AGK\WinPython-32bit-2.7.5.3\python-2.7.5\lib\sysconfig.py",
line 449, in get_config_vars
import re
File "C:\Users\AGK\WinPython-32bit-2.7.5.3\python-2.7.5\lib\re.py", line 1
05, in <module>
import sre_compile
File "C:\Users\AGK\WinPython-32bit-2.7.5.3\python-2.7.5\lib\sre_parse.py"
", line 14, in <module>
import sre_parse
File "C:\Users\AGK\WinPython-32bit-2.7.5.3\python-2.7.5\lib\sre_constants.py",
line 17, in <module>
from sre_constants import *
File "C:\Users\konings\WinPython-32bit-2.7.5.3\pyt
py", line 18, in <module>
from _sre import MAXREPEAT
ImportError: cannot import name MAXREPEAT
Could somebody explain to me what this error means and/or how to use f2py with Winpython? Thanks!
You can set your environment variable PYTHONPATH pointing to the folder where these modules are. This should solve the ImportErrors.

Categories