ModuleNotFoundError after pip install . (custom module) - python

I am starting a custom package called nate-givens-toolkit.
I want to use a module from that package in a Jupyter notebook in a different folder.
Here is my structure
- nate-givens-toolkit/
- setup.py
- nate_givens_toolkit/
- cloud_io.py
- __init__.py
- project/
- project_nb.ipynb
I am going into nate_givens_toolkit (in the console) and running: pip3 install .
The output I get looks like everything is just fine.
Processing /home/ec2-user/nate-givens-toolkit
Building wheels for collected packages: nate-givens-tooklit
Building wheel for nate-givens-tooklit (setup.py) ... done
Created wheel for nate-givens-tooklit: filename=nate_givens_tooklit-0.1-py3-none-any.whl size=2009 sha256=7de8c9d2930d531603c973c7d8079b66f3d4326fb274e63087128fb7d25d9e1b
Stored in directory: /home/ec2-user/.cache/pip/wheels/17/96/2f/0073c92cfdadbb032d855f24df4725bf190d39cd1c5bb1d233
Successfully built nate-givens-tooklit
Installing collected packages: nate-givens-tooklit
Attempting uninstall: nate-givens-tooklit
Found existing installation: nate-givens-tooklit 0.1
Uninstalling nate-givens-tooklit-0.1:
Successfully uninstalled nate-givens-tooklit-0.1
Successfully installed nate-givens-tooklit-0.1
But if I go into project_nb.ipynb and write:
from nate_givens_toolkit import cloud_io as cloud
I get: ModuleNotFoundError: No module named 'nate_givens_toolkit'
The same thing happens if I run python from the console. As long as I'm in the nate-givens-toolkit directory I can import and run it fine. But if I navigate up a level and try the import I get ModuleNotFoundError.
I don't understand why pip install seems to work, but I still get this ModuleNotFoundError. I thought the whole point of the pip install would be that I could then import from files in other directory without having to worry about relative paths and such. (I've done this exact same thing in a totally different context for work and it was fine.)
I've tried the solutions for similar issues like these:
after pip successful installed: ModuleNotFoundError
ModuleNotFoundError: No module named 'requests' after pip install
So far, haven't found anything that works.
Everything I'm doing is inside the same conda venv.
Help?

I found the answer to my question here: Import py file in another directory in Jupyter notebook. (Note: it wasn't the accepted answer on that post, it was the second answer after the accepted answer.)
The trick is that I needed to do use the -e flag when using pip install.
So instead of
pip install .
I had to run
pip install -e .
Once I ran that, it worked in Python from the console and also in Jupyter.

Related

python package is installed but not found in other directory

I have created a conda environment and activated it already.
Then inside the use_cases/ directory I execute: pip install -e use_case_b (https://github.com/geoHeil/dagster-demo/tree/master/use_cases):
...
...
Installing collected packages: use-case-b
Attempting uninstall: use-case-b
Found existing installation: use-case-b 0.0.0
Uninstalling use-case-b-0.0.0:
Successfully uninstalled use-case-b-0.0.0
Running setup.py develop for use-case-b
Successfully installed use-case-b
Now when inside the use_cases/ directory:
python
import use_case_b
works fine.
When switching to a different directory like: / (= the root of the repository I get an error message of:
ModuleNotFoundError: No module named 'use_case_b'
Why is it working once and failing in the second place? Could it be that it is not even working in the first place and only importing the sub_directory due to the __init__.py file?
How can I get the python package properly install into the virtual environment?
FYI: here you can find the full project https://github.com/geoHeil/dagster-demo
When using the cookiecutter to set up the package it works just fine.
https://github.com/audreyfeldroy/cookiecutter-pypackage
This is more like a workaround than an solution to the SF question - but works nicely and as an additional benefit has some best practices with regards to documentation and testing already built in.
I normally face that problem when im using pip to install a package. Im not sure if this gonna work for you.
In the file you are importing the package copy the following code and run it.
from pip._internal import main
main(["install", "use_case_b"])
Just ignore the warnings'
Good Luck, Let me know if it works.

Module gets imported under a syntactically false name space?

I am following along with the O'Riley Head First Python (2nd Edition) Course.
At one point you will create a webapp and deploy it to pythonanywhere (chapter5).
The webapp uses two functions, imported from a module, created earlier.
The module is called vsearch.py. I also created a readme.txt and a setup.py and used setuptools to create a source distribution file using :
python3 setup.py sdist
The code of the setup.py read as follows:
from setuptools import setup
setup(
name = "vsearch",
version = "1.0",
description = "The Head First Python Seach Tools",
author = "HF Python 2e",
author_email = "hfpy2e#gmail.com",
url = "headfirstlabs.com",
py_modules = ["vsearch"],
)
The source distribution file gets created without errors and creates a file called vsearch-1.0.tar.gz
The file then gets uploaded to pythonanywhere and installed via console using:
python3 -m pip install vsearch-1.0.tar.gz --user
Console outputs:
15:36 ~/mysite $ python3 -m pip install vsearch-1.0.tar.gz --user
Looking in links: /usr/share/pip-wheels
Processing ./vsearch-1.0.tar.gz
Building wheels for collected packages: vsearch
Running setup.py bdist_wheel for vsearch ... done
Stored in directory: /home/Mohr/.cache/pip/wheels/85/fd/4e/5302d6f3b92e4057d341443ed5ef0402eb04994663282c12f7
Successfully built vsearch
Installing collected packages: vsearch
Found existing installation: vsearch 1.0
Uninstalling vsearch-1.0:
Successfully uninstalled vsearch-1.0
Successfully installed vsearch-1.0
Now when I try to run my webapp I get the following error:
2020-03-24 16:18:14,592: Error running WSGI application
2020-03-24 16:18:14,592: ModuleNotFoundError: No module named 'vsearch'
2020-03-24 16:18:14,593: File "/var/www/mohr_eu_pythonanywhere_com_wsgi.py", line 16, in <module>
2020-03-24 16:18:14,593: from vsearch4web import app as application # noqa
2020-03-24 16:18:14,593:
2020-03-24 16:18:14,593: File "/home/Mohr/mysite/vsearch4web.py", line 3, in <module>
2020-03-24 16:18:14,593: from vsearch import search4letters
Judging from this error I assume that "vsearch" can not be found because it was installed as "vsearch-1.0". However when I try to change this line to:
from vsearch-1.0 import search4letters
I rightfully get a synthax error since I can not adress modules this way. So what can I do about this? When creating the module in the beginning I added a version number to the setup.py file because according to the lecture it is good practice. Setuptools then automatically creates the source distribution file with the "-1.0" at the end. Also when importing it using the command shown above i automatically gets importet as "vsearch-1.0" which in turn I am unable to reference in my python code because of bad synthax.
Am I doing something wrong? Is there a way to import this under another namespace? Is there a way to reference "vsearch-1.0" in my python code without getting a synthax error?
There are different python3 versions installed on PythonAnywhere. When you install something using python3 -m pip or pip3 you use default python3 that is probably not matching python version setting of your web app. Use python3.7 and pip3.7 or python3.6 and pip3.6 etc. for --user installations to be sure.
pip install --user (with emphasized --user) installed the package into your user directory: /home/Mohr/.local/lib/pythonX.Y/site-packages/.
To run your WSGI application you probably use a virtual environment in which the user-installed modules are not available. To use modules in the venv you have to install everything in the venv. So activate the venv in a terminal and install the module with the venv's pip:
pip install vsearch-1.0.tar.gz

Python: changing directory of Python package for development

Suppose I have a python package in following directory:
c:\Python27\site-package\monster
I want to change it to local branch:
c:\ddey\dev
I went to the directory c:\ddey\dev where I have \monster developer package
and tried: pip install -e .
But it isn't changing the path. How do I know?
in python I tried following:
>>>import monster
>>>print(monster.__file__)
c:\Python27\site-package\monster\__init__.pyc
expected:
>>>import monster
>>>print(monster.__file__)
c:\ddey\dev\monster\__init__.pyc
solution:
previously my pip install was getting completed but I never getting the following message at the end of it as shown below. There was an issue with a package during the operation which I didn't notice as pip install wasn't giving a clear error.Once I fixed the issue with the package, the installation worked fine and got expected result!
Successfully installed monster
Try using pip install --target=d:\path\not_default package_name

I cannot import a python submodule after successful installation of the module

I am trying to install the python submodule ANCA in my mac, for which I have run the following:
pip install git+https://github.com/acadev/anca.git
getting the following:
.
.
.
Installing collected packages: anca
Running setup.py install for anca ... done
Successfully installed anca-0.1.5
I have miniconda installed in my computer, and I have checked that the module and its files and dependencies are correctly located in /usr/local/miniconda2/lib/python2.7/site-packages/anca
When I import anca in my notebook it does not give me any error, but if I try to import a submodule it gives me the following error:
from anca import IterativeMeansAlign
ImportError: cannot import name IterativeMeansAlign
It is the same with other submodules, but all of them are apparently well installed with all their files.
I have also tried to install the package using Git:
git clone https://github.com/acadev/anca.git
sudo python setup.py install
with the same result.
Note: 'locate' is not able to locate the module anca unlike the rest of the python modules although I can see all the files in the right location. It is the first time this happens to me.
Thanks a lot for your help,
any suggestion will be very appreciated.

Installing GDAL Python binding in Ubuntu to use as standalone module

I am trying to export a GeoTiff with Blender using the Blender Python API (based on Python 3), so I've decided to install GDAL on Ubuntu (14.04). What I would like is to get the module as a standalone folder that I could put in the modules directory of Blender (/home/user/.config/blender/2.73/scripts/modules).
The thing is I've run through several different problems trying to install GDAL. I've tried to install from source (for GDAL 2.0.0) here : Official PyPi Gdal
I ran sudo apt-get install libgdal-dev gdal-bin (I list it here because it may be important)
When I am in the extracted GDAL folder, using python setup.py build & python setup.py install, the library installs to /usr/local/lib/python2.7/dist-packages/osgeo. However, when I run python from command line, running from osgeo import osr returns ImportError: No module named _gdal
Following GDAL via pip , I used pip (pip install GDAL) to install the library, and the folder it went to was /usr/lib/python3/dist-packages/osgeo (using pip show ...). Again, running python3 and trying to import results in the same error. Of course, when I copy-paste each folder in the blender module directory, I get the same error in the Blender Python console.
So I decided to compile the sources using ./configure --with-python & make & make install in the source folder. I then copied the folder GDAL-x.x.x/build/lib.linux-x86_64-3.4/osgeo to the blender modules directory and got this time the error when importing : ImportError: /home/yvesu/.config/blender/2.73/scripts/modules/osgeo/_gdal.so: undefined symbol: _Py_ZeroStruct.
Trying to compile with python3 using python3 setup.py build returns the error error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
EDIT 1:
I think I've found the solution : I went to the directory swig/python (not found in a GDAL-1.11.0 folder but gdal-1.11.0 fodler, can't remember where I downloaded it from), ran python3 setup.py build & python3 setup.py install and could finally find the folder in /usr/local/lib/python3.4/dist-packages/GDAL-1.11.0-py3.4-linux-x86_64.egg/osgeo. When I put this osgeo folder oni the Blender modules directory, I was able to import osgeo in Blender. I will report if anything went wrong.
I think I've listed all my attempts at installing GDAL on Ubuntu. Can anyone point me in the right direction? Do you think it is even possible to install it as a standalone module, or do I need linked libraries through LD_LIBRARY_PATH?
Here is the solution I've found :
Download Gdal sources (v2.0.0 is the current stable release) from ftp://ftp.remotesensing.org/gdal/2.0.0/ or http://download.osgeo.org/gdal/2.0.0/ and untar
Go to the directory gdal2.0.0/swig/python
Run python3 setup.py build & python3 setup.py install
Finally find the module folder in, on Ubuntu : /usr/local/lib/python3.4/dist-packages/GDAL-2.0.0-py3.4-linux-x86_64.egg/osgeo
I can now use it in Blender (copying in the modules directory)

Categories