Question
I installed a local package called credentials using
pip install -e c:\users\worker\src\clockwork\lib\credentials
But when I try to import the package from a sibling directory, it fails with an ImporError:
cd c:\users\worker\src\clockwork\bank
python -c "import credentials"
...
ImportError: No module named 'credentials'
Confusingly, the package credentials is listed as successfully installed as shown when I run pip list:
...
credentials (1.0.0, c:\users\worker\src\clockwork\lib\credentials)
...
How can I install my local package so that it can be imported?
Background
I am using Python 3.4 (32-bit). The package contains two files:
credentials\__init__.py
credentials\setup.py
The __init__.py file defines a single function. The setup.py file is short:
from distutils.core import setup
setup(name='credentials', version='1.0.0')
Workaround
I currently add the directory containing the package (c:\users\worker\src\clockwork\lib) to my PATH variable as a workaround. But my question is how to install the package properly so that I do not need to modify the PATH.
Uninstall the python package then install it using:
python -m pip install -e c:\users\worker\src\clockwork\lib\credentials
What is probably happening is that you have multiple python installs and pip is run from one install while you are trying to use the package from another. See also:
What's the difference between pip install and python -m pip install?
https://docs.python.org/3/installing/#basic-usage
The problem centers on setup.py. It needs to declare a package:
from distutils.core import setup
setup(name='credentials', version='1.0.0', packages=['credentials'])
But this setup.py must be in the parent directory of the credentials package, so in the end, the directory structure is:
...\credentials\setup.py
...\credentials\credentials\__init__.py
With this change, the module is found after reinstalling the package.
This could also be caused by two Python installs (but wasn't in my case), and #Mr_and_Mrs_D gives an answer for that case.
Related
i've installed a python project, and it imports modules(Like almost every project). The problem is when i want to install them(because i haven't got the modules), for example: In the project is imported a module called "a" but when i go and install "a" with pip install a, it says ERROR: Could not find a version that satisfies the requirement a (from versions: none) ERROR: No matching distribution found for a. How could i know the name of the module that is imported in that python project?
Edit:
btw i just found out the module that the project uses comes in the zip where the python project is. How could i install it so it works?
All pip packages are listed here. If you want to import a module called a inside a python script, the command to install it could be sometimes pip install b. Because the name of the stored package can varied from the python import name. To find how to install it the best is to get the pypi url of your package. You can googling the python error ModuleNotFoundError: No module named 'dgvd', it always show you the pypi url in top links.
The good practice in a project is to have a txt file called requirement.txt that you create in bash using this command:
pip freeze > requirement.txt
Then install all packages in once using:
pip install -r requirement.txt
For installing from zip simply use:
pip install *.zip
or specify the path directly:
pip install <path to .zip>
pip install ./my-archive.zip
Same applies for a tarball or any other format. It can be even a folder. However, it has to include a proper setup.py or other mechanism for pip to install it and pip has to support the packaging format (be it archive, networking protocol, version control system (git prefix), etc).
pip install ./my-folder
pip install ./
pip install .
pip install ..
etc
If, however, there is no setup.py present, you'll need to simply copy-paste the files somewhere where your project/module resides (or set PYTHONPATH or sys.path to that folder) to be able to import them. See this or this question for more.
I want to create a python package which will be cloned from its git repo when a build runs, so I will have the source inside the build agent. I would then like to run the python package as a command line tool, the package is called environment_manager.
Initially I thought I would follow a tutorial for creating a simple setup.py although this has proved to be a lot more difficult than I thought it would be and whenever I run python setup.py install --force I am not able to use my installed package, generally either module not found or the command is not recognised when I type it.
I have found that if I simply install with pip install . then I am actually able to use the tool from the command line and it works. I don't understand what the difference is, or why this only works when doing the pip install method.
Below is the setup.py file, I cannot see what is wrong with it:
from setuptools import setup, find_packages, find_namespace_packages
import pathlib
here = pathlib.Path(__file__).parent.resolve()
# Get the long description from the README file
long_description = (here / 'README.MD').read_text(encoding='utf-8')
setup(
name='environment_manager',
version='1.0.0',
package_dir={'': 'src'},
packages=find_namespace_packages(where='src', include='environment_manager.*'),
python_requires='>=3.8, <4',
install_requires=['boto3', 'botocore', 'pyyaml'],
extras_require={
'dev': ['pre-commit', 'black', 'pylint'],
'test': ['pytest', 'pytest-mock', 'coverage'],
},
entry_points={
'console_scripts': [
'environment-manager=environment_manager.environment_controller:main',
],
}
)
My project structure looks like:
environment_manager
/src
conf/
environment_manager/
environment_controller.py
config_parser.py
command.py
test/
unit_tests.py
I thought the correct way to install and run the tool from the command line was to use setup.py and setuptools but it seems like it is a lot easier and actually works if I just install it with pip.
Is installing it with pip over setup.py correct (as both ways the package appears when I type pip list) and are there any issues with my setup.py script? The script was taken from the pypa sample project and I removed most of what I didnt need.
setup.py is a python file, which usually tells you that the module/package you are about to install has been packaged and distributed with Distutils, which is the standard for distributing Python Modules. This allows you to easily install Python packages. Often it's enough to write: $ pip install .
In other words setup.py is a packaging file while pip is a package manager, therefore you should have setup.py file to be able to install with pip.
pip is a package manager which helps install, manage, and uninstall Python packages. It searches for them on PyPI, downloads them, and then runs their setup.py script.
Since you mentioned that you can run your binary executable after a pip install, but not a setup.py install, it is likely that each of them is installing the binary to separate locations.
One thing I would check is that you are using python and pip from the same version of Python, e.g:
% python --version
Python 3.8.6
% pip --version
pip 20.1.1 from /usr/lib/python3.8/site-packages/pip (python 3.8)
If these have different Python versions listed, they are likely installing to two separate directories - one in your PATH environment variable, and one which is not.
Next, I would check pip list -v after each install method, as this should list a Location header telling you where the package has been installed.
I've installed a module named rauth through terminal with pip3 install rauth command but when I import the module and run the code on Visual Studio Code with python3 interpreter, it gives ModuleNotFoundError: No module named 'rauth' error. But it is indeed installed and I can use it in Anaconda. The package file is stored here.
/Users/puffedricecracker/anaconda3/lib/python3.7/site-packages/rauth
And it seems like all my pip installed packages are stored in that path, but those are imported outside Anaconda with no problem. Tried several other commands as google search suggested.
• pip install instead of pip3 install
• python -m pip install
• python3 -m pip install
Let me know if there is any other information needed to be specified.
this is due to the module is installed into site-packages in Anaconda but not Visual Studio. Can you check if the module exists in Visual Studio folder? Another way to test it is to open Python IDLE and run the import, it should also return an error.
I don't know if this could be an universal solution but there was a way to set where to install a package using pip.
In python shell, find where your python modules usually go. It seemed like most of pip installed packages were installed in the second path so I chose that.
>>> import re
>>> print(re.__file__)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/re.py
>>> import sqlalchemy
>>> print(sqlalchemy.__file__)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/sqlalchemy/__init__.py
Uninstall the package using pip uninstall packagename
Reinstall with a path name.
pip install packagename -t /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages
(In fact, just copy pasting package files (in my case, they were rauth, rauth-0.7.3.dist-info) from anaconda package path in the post worked.)
I had to manually build a package and copy it to the site-packages directory. When I type pip list into a console it isn't listed, though I can use it in python scripts. How can I make pip aware of the package?
Installing it via pip is not an option.
You say "Installing it via pip is not an option.", but I'm assuming installing it via pip using a local copy still is. If so, the way to do that is to clone your library into a directory (say /my/lib/dir), where the root of the source for the root package appears below /my/lib/dir (ex: if the package you want to install is imported as import foo, then you should have /my/lib/dir/foo). If there is no file named setup.py in your copy of the code, then you need to create a simple one. Something like
# in a file called setup.py above the `foo` directory
from distutils.core import setup
setup(name='foo',
version='1.0',
packages=['foo'],
)
Finally, run pip install . from /my/lib/dir.
It's definitely a hack, but making pip aware of a package without installing it via pip is asking for a hack :-)
What I have:
local Python3 files that I want to turn into a module test_module
test_module folder containing an empty __init__.py, a setup.py file (see below) and subdirectories with several source
files
What I want:
continuously work on and improve test_module locally
have an easy way to install test_module and all its dependencies locally in my own virtual environment (created using python3 -m venv my_environment)
run files that make use of the module via python myexample.py, without having to take care of adapting my local PYTHONPATH variable each time i enter or exit the my_environment
share my python code with others via git, and allow them to install their code locally on their machines using the same procedure (as simple as possible)
learn best practices on how to create my own module
How I'm doing it at the moment:
pip freeze > requirements.txt and pip install -r requirements.txt for installing dependencies
adding export PYTHONPATH="${PYTHONPATH}:." to my_environment/bin/activate, to have my own module in the search path
(as found here: How do you set your pythonpath in an already-created virtualenv?)
I'd like to know if there are "cleaner" solutions based on setup.py, possibly involving something like pip install ./test_module or similar that takes care of 2.-3. automagically.
My current setup.py file looks as follows
from setuptools import setup
setup(
name='test_module',
version='0.1',
description='Some really good stuff, that I am still working on',
author='Bud Spencer',
author_email='bud.spencer#stackoverflow.com',
packages=['test_module'], # same as name
install_requires=['numpy', 'scipy', 'sklearn', 'argparse'], # external packages as dependencies
)
It sounds like you want to run pip install -e <path/url> from within your virtual env, which will install a package (with a setup.py file as you have) from either a local path or a Git repo. See https://pip.pypa.io/en/stable/reference/pip_install/#vcs-support for an explanation on the syntax of the latter.
Example:
pip install -e git+https://github.com/me/test_module/#egg=test-module
If you have already installed and want to pull the latest code from the repo, add an --upgrade switch to the above.