Python tarfile fails when installing sqlite3 via nodejs/npm on embedded linux - python

I'm trying to install an node-based webserver on a cortax a7 embedded system. The repos is pulled via git and I need to run npm install to install the node modules.
The server uses sqlite3, but the package fails when installing, specifically at the build stage, because python cannot find the tarfile module.
node -v // 4.3.1
npm -v // 2.14.12
root#imx6ul-var-dart:~/gateway-server# npm install
> sqlite3#3.1.1 install /home/root/gateway-server/node_modules/sqlite3
> node-pre-gyp install --fallback-to-build
Traceback (most recent call last):
File "./extract.py", line 2, in <module>
import tarfile
ImportError: No module named tarfile
deps/action_before_build.target.mk:13: recipe for target 'Release/obj/gen/sqlite-autoconf-3090100/sqlite3.c' failed
make: *** [Release/obj/gen/sqlite-autoconf-3090100/sqlite3.c] Error 1
make: Leaving directory '/home/root/gateway-server/node_modules/sqlite3/build'
I'm aware that many users experience issues with gyp/node-gyp, but here it seems its actually the python file extract.py that fails, when trying import tarfile ... but this seems to be a core module.
I've not been able to find references to this in my searches and bluntly I'm not a python guy. Ideas?

For anyone finding this after-the-fact, it seems you have to install python-modules package (in my case, with opkg) to get ALL the python standard libraries .... dunno why ... but this brings over tarfile and subsequent dependent zlib

Related

Import ldap in python gives gives "DLL load failed" error

I'm using the "import ldap" in a python code. This is on a windows 10 machine.
I installed the python-ldap module
pip3 install python-ldap
Installed the dependencies based on the instructions at Python Can't install packages
Also resolved all the pip deployment issues based on Installing python-ldap in a virtualenv on Windows
I'm now getting the following error when executing the import ldap statement. am I missing something here? Any ideas to resolve it?
thon39\site-packages\ldap\__init__.py", line 34, in <module>
import _ldap
ImportError: DLL load failed while importing _ldap: The specified module could not be found.
Visit the unofficial Python binaries page:
https://www.lfd.uci.edu/~gohlke/pythonlibs/#python-ldap
Download the appropriate WHL package for your system.
For example, if you're using Python 3.8 on an x64 system, download python_ldap‑3.3.1‑cp38‑cp38‑win_amd64.whl
(hint: do NOT download the +sasl version unless you have the Cyrus SASL code running on your system...)
Start the VirtualEnv for your project, if you're using one (C:\Users\youruser\.virtualenv\YourVirtualEnv\Scripts\activate.bat) -- if you're not, skip this step.
Then run pip3 install C:\Path\To\python_ldap_x.x.x-cpXX-cpXX-winXX.whl and this should install the Python DLL (pyd) file for you.

How to import python module after cloning and modifying the module's github repo

I'm trying to make an open source contribution to a python module that is hosted on Github (pypika).
I cloned the repo from github and ran pip in editable install mode such that any future imports would point to my version of the code.
But when I try running a test file within the repo, I get an error when trying to import the module. What am I doing wrong? How can I make it so that the import will use the modified module that I'm working on?
$ cd Dev
$ git clone https://github.com/kayak/pypika.git
$ pip install -e /Users/me/Dev/pypika
Obtaining file:///Users/me/Dev/pypika
Installing collected packages: PyPika
Running setup.py develop for PyPika
Successfully installed PyPika
$ python3 ./pypika/pypika/tests/test_functions.py
Traceback (most recent call last):
File "./pypika/pypika/tests/test_functions.py", line 3, in <module>
from pypika import (
ImportError: No module named 'pypika'
PyPika maintainer here. In order to contribute it's best to make a fork on Github and put your contributions in a separate branch which you can then pull request.
In order to run the tests you simply need to execute python -m unittest in the project folder of PyPika. (Or use the Python test runner functionality of your favourite IDE.)
pip install -e /Users/me/Dev/pypika will work if executed in the environment of the project in which you want to use PyPika.

Unresolved symbols after building HDF5, netCDF-C and netCDF4-python from source

After using the netCDF4-python package without any trouble, I needed to enable parallel file access. As I could not find suitable ready-made combinations of HDF5 and netCDF C libraries built against the same MPI library, I decided to build each of the packages from source with OpenMPI. However, importing the netCDF4 package fails due to unresolved symbols:
--------------------------------------------------------------------------- ImportError Traceback (most recent call
last) in
----> 1 import netCDF4
~/anaconda3/lib/python3.6/site-packages/netCDF4-1.5.1.2-py3.6-linux-x86_64.egg/netCDF4/init.py
in
1 # init for netCDF4. package
2 # Docstring comes from extension module _netCDF4.
----> 3 from ._netCDF4 import *
4 # Need explicit imports for names beginning with underscores
5 from ._netCDF4 import doc, pdoc
ImportError: /usr/local/lib/libnetcdf.so.15: undefined symbol:
H5Pset_dxpl_mpio
I tried installing the netcdf4-python package via pip install, and it imports works fine for serial file access, but still fails to load a file with parallel=True, stating that it requires parallel-enabled netcdf-c.
I am installing HDF5 with
export NCPROCS=4
export CC=mpicc
./configure --prefix=/usr/local/ --enable-parallel --enable-hl
make check
sudo make install
and netCDF-C with
export NCPROCS=4
export CC=mpicc
./configure --prefix=/usr/local/ --enable-parallel-tests
make check
sudo make install
and netCDF4-python using nc-config with
export CC=mpicc
python setup.py install
Each step recognizes the parallel functionalities. Am I missing a linking step in the build process somewhere, or why can't libnetcdf.so.15 find the symbols from the HDF5 library?

How do I solve the issue "No module name Botan"

I am using windows 8 and python 3.6.1 I've done the following command in my cmd:
pip install cryptoshop
However, when I run the following python code:
from cryptoshop import encryptfile
from cryptoshop import decryptfile
result1 = encryptfile(filename="test", passphrase="mypassphrase", algo="srp")
print(result1)
result2 = decryptfile(filename="test.cryptoshop", passphrase="mypassphrase")
print(result2)
I get the following error:
Traceback (most recent call last):
File "C:/Users/Owner/Desktop/test.py", line 1, in
from cryptoshop import encryptfile
File "C:\Users\Owner\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cryptoshop__init__.py", line 26, in
from cryptoshop.cryptoshop import encryptfile
File "C:\Users\Owner\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cryptoshop\cryptoshop.py", line 56, in
from ._cascade_engine import encry_decry_cascade
File "C:\Users\Owner\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cryptoshop_cascade_engine.py", line 27, in
from ._nonce_engine import generate_nonce_timestamp
File "C:\Users\Owner\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cryptoshop_nonce_engine.py", line 39, in
import botan
ModuleNotFoundError: No module named 'botan'
Now, I obviously know that you must install botan into python in order to use it. However, this is where I am running into an issue. I've downloaded Botan from this link as instructed:
https://github.com/randombit/botan
And then I've followed these instructions in an attempt to install Botan:
./configure.py [--prefix=/some/directory]
make
make install
However, when I type make into the command line I get an error saying there is no such command. And then when I go to run my above Python code I still get the no module Botan error. So obviously I am doing something run. How can I properly install Botan into my Python 3.6 directories so that I can use cryptoshop.
I've also attempted to do pip install Botan, as that is how I've installed so many other python libraries but that has been unsuccessful as well.
make is a linux command
According to the botan website you can use nmake as a replacement on windows ( http://wiki.c2.com/?UsingNmake ) :
On Windows
You need to have a copy of Python installed, and have both Python and
your chosen compiler in your path. Open a command shell (or the SDK
shell), and run:
$ python configure.py --cc=msvc (or --cc=gcc for MinGW) [--cpu=CPU]
$ nmake
$ botan-test.exe
$ nmake install
Botan supports the nmake replacement Jom which enables you to run
multiple build jobs in parallel.
source : https://botan.randombit.net/manual/building.html
For completeness, here's how I made it work on a Mac
Assuming you have brew installed.
brew install botan
You may need to install other functionality first:
brew install gmp
brew install mpfr
brew install mpc
Find out where botan got installed with brew info botan.
My location is /usr/local/Cellar/botan/2.6.0
In that folder, you'll find lib/python2.7/site-packages, copy the contents of this folder into your Python's installation site-packages folder.
Note 1: At the time of this writing, only python 2.7 seems to be supported, but I'm using python 3.6 and everything seems to be working.
Note 2: If the file is called botan2.py, you may need to rename it to botan.py in your python's site-packages folder.

Cannot import sqlite3 in Python3

I am unable to import the sqlite3 module in Python, version 3.5.0. Here's what I get:
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/sqlite3/__init__.py", line 23, in <module>
from sqlite3.dbapi2 import *
File "/usr/local/lib/python3.5/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: No module named '_sqlite3'
I know, I know, there are PLENTY of StackOverflow posts and support forums across the web where people complain about this problem, but none of the posted solutions have worked for me so far. Here's where I've been:
I also have Python 2.6.6 installed on this server, which is running CentOS 6.8 x86_64. I can open up the Python REPL and import sqlite3 just fine when using Python 2.6.6. I can also use sqlite3 from straight from bash and nothing seems awry.
This helpful question looked promising. I tried to re-configure and re-compile Python3.5 with the --enable-loadable-sqlite-extensions option, as user jammyWolf suggested. Nope, same error still occurs.
I've been using virtual environments like a good boy, but I have root access to this server. So, I was a bad boy and ran python3 as root without any virtualenvs activated. Still no luck. So I don't think it has anything to do with permissions.
I noticed that in the error message, it says No module named '_sqlite3'. This thread suggests that the underscore before the module name means that the module is an implementation detail, and isn't exposed in the API. ... I'm not sure what to make of this information, but there may be a hint somewhere in there.
Any ideas?
Falsetru is correct, I am going to go into a bit more detail for those not familiar (linux instructions). If you are getting this error, chances are you are using a version of python that was compiled without the correct headers. Here's a step by step guide to get it sorted. (Python 3.X.X instructions)
Install the required sqlite libraries
sudo apt-get install libsqlite3-dev
Uninstall python (I'm using python 3.6.5 as an example in this guide)
sudo apt-get remove python3.6
Download python from source
cd /tmp && wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz
Unpack the archive
tar -xvf Python-3.6.5.tgz
Configure
cd Python-3.6.5 && ./configure
Make and install (and go make coffee while you're at it)
make && sudo make install
If you did everything correctly running "python3.6 -V" should give you your python version. Note you will have to rebuild any virtual environments you have as well.
One final caveat you may encounter.
zipimport.ZipImportError: can't decompress data; zlib not available
This happens if you don't have the following zlib library installed:
sudo apt-get install zlib1g-dev
Install sqlite-devel package which includes header, library that is required to build sqlite3 extension.
yum install sqlite-devel
NOTE: Python does not include sqlite3 library itself, but an extension module (wrapper).
I copied sqlite3.dll to this folder:
C:\Users\*****\Anaconda3\DLLs
It worked after two weeks trying everything else.

Categories