I'm trying to setup my environment for a project but python isn't able to find the modules I've installed with pip.
I did the following:
mkdir helloTwitter
cd helloTwitter
virtualenv myenv
Installing setuptools, pip, wheel...done.
source myenv/bin/activate
pip install tweepy
Collecting tweepy
Using cached tweepy-3.5.0-py2.py3-none-any.whl
Collecting six>=1.7.3 (from tweepy)
Using cached six-1.10.0-py2.py3-none-any.whl
Collecting requests>=2.4.3 (from tweepy)
Using cached requests-2.11.1-py2.py3-none-any.whl
Collecting requests-oauthlib>=0.4.1 (from tweepy)
Using cached requests_oauthlib-0.6.2-py2.py3-none-any.whl
Collecting oauthlib>=0.6.2 (from requests-oauthlib>=0.4.1->tweepy)
Installing collected packages: six, requests, oauthlib, requests-oauthlib, tweepy
Successfully installed oauthlib-2.0.0 requests-2.11.1 requests-oauthlib-0.6.2 six-1.10.0 tweepy-3.5.0
When I try to import the module it says it cannot be found.
The first entry in $PATH is helloTwitter/myenv/bin
All the packages are showing up in the environments site-packages directory.
I seem to be using the right python and pip.
Which python outputs helloTwitter/myenv/bin/python
Which pip outputs helloTwitter/myenv/bin/pip
Any advice on where I'm going wrong?
It looks like you're manually setting your $PATH to point to your virtual environment. The whole point of the myenv/bin/activate script is to take care of this for you.
Once you have activated your virtual environment, any package you install using pip will be placed in the relevant venv site-packages directory (in your case, myenv/lib/python2.7/site-packages). Things like pip --user are unnecessary when you are working in a virtual environment (assuming default behaviour). It's all automatic.
After running activate, you can check the python binary you are using with find -iname tweepy.
Aliases can cause issues too. which is an external command, and won't always pick these up. A type -a python will flush these out.
A quick test can be done by running helloTwitter/myenv/bin/python -c 'import tweepy' directly. If this behaves differently to however you are currently running python (i.e. doesn't throw an import exception), then this is your problem.
Hope that helps.
Ok, I think I found a solution, if not an answer.
I uninstalled the package and made sure it was not in system or user.
I re-created the virtual environment.
I checked that the environments python and pip were being used.
This time when installing my package I added the --no-cache-dir option. The packages install successfully. Now Python can find the package.
derptop:environmentScience Marcus$ python
>>> from tweepy import StreamListener
>>> StreamListener
<class tweepy.streaming.StreamListener'>
I checked the sys.path and it now includes the site-packages directory from the virtual environment, where previously it was absent.
Output of sys.path:
['', ....'/Users/Marcus/CodeProjects/environmentScience/myenv/lib/python2.7/site-packages']
As far as I can tell the sys.path was referencing the wrong site packages directory. Though I'm not sure why. I'm wondering if pips use of the cache was causing the site-packages reference to reset to system.
On Windows, if you are experiencing this issue, check that after activating your environment, the subsequent python commands are being run using the binary in the .venv\Scripts\Python executable.
I've seen this strange behaviour where even after the environment is activated using the activate.bat file in the virtual environment, AND the path to the Scripts\ directory is added to the system path, python commands still use the binary in the central Python installation in the system.
This leads to any packages being installed being installed NOT in the virtual environment, but in the global system location.
To work around this, when running pip or python commands after activating your environment, specify the correct path to the executable within your virtual environment, i.e.,
.venv\Scripts\python -m pip install setup.py
This should install the packages into the virtual environment and solve the issue.
I solved this problem by change project directory. Move your project directory from desktop to another place. It helped me on Windows. I moved it from /Desktop to C:/some_folder
Related
So I have a virtual environment set up in VS Code. I recently learned that it's better this way because it doesn't interfere with my global python data. The problem is that I'm still getting issues. When I try to install a package in that environment using pip, it gives a message like this and it doesn't install:
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: {PACKAGE NAME} in c:\users\lavaa\appdata\roaming\python\python311\site-packages
I activated the environment and selected it as my interpreter before this.
This confuses me because why should it care about that python directory when I'm in a virtual environment? It's not already installed in the virtual environment, so there should be no problem, right? I didn't want to mess anything up, so here I am.
One last thing: When I went to the appdata directory to find the issue, I saw that the package was installed there. I don't know why it didn't get placed in the virtual environment folder.
If you use pip command to install, the package will be installed in pyenv.
If you are using conda environment you can use the following command to install:
conda install packageName
You can read document about environment for more information.
I am trying to build a python library for embedded device without wifi access and without pip installation usage
First I start off with downloading the package from PyPi website. After opening the gz file, I found and read the setup.py and then download its dependencies accordingly.
For my environment, I start to trial out the installation on a VM as trial(yah it is far from being the perfect environment) and try to setup the packages in a virtual environment.
After running python3 setup.py install for all of them, I found that almost all the installation is installed onto /usr/local/lib/python3.8/dist-packages instead of in the virtual environment /helper/lib/python3.8/site-packages. Subsequently I copied the content in the /usr/local/lib/python3.8/dist-packages to the '/usr/local/lib/python3.8/dist-packages` and try to run python from the activated environment.
However the program give an error "zipp" module not found. Thinking that I have missed out a module, I proceed to downnload the package and tried to setup manually. However there is no setup.py in the folder. There seem to be no installation available
So my question is this:
How do i install to a virtual environment folder? Do I copied the contents /usr/local/lib/python3.8/dist-packages to /helper/lib/python3.8/site-packages like what I did? Or is there other files I will have to copy over?
How do I install packages like zipp which do not have any setup.py and there seem to be no installation instruction whatsoever. Do I copy zipp.egg-info over to the site folder
Thanks
My package Foo builds an extension libFoo.so that depends on another package Bar's extension libBar.so, and I don't know a priori where the dependency libBar.so is already installed. I need to know the dependency's directory in order to set libFoo.so's rpath so that it can find libBar.so at runtime.
I thought a good way to do this might be to import the dependency package Bar in my setup.py and inspect the module to get the directory, since I know where libBar.so will be relative to the Bar package's directory. That way if they installed from a virtual environment, I would locate the appropriate path.
However, when I try to do this, i.e., when I activate a virtual environment and install my package from PyPI, I find that import Bar in the setup.py ends up importing a version from a DIFFERENT virtual environment (I can tell by inspecting the built libFoo.so's rpath after installation).
On the other hand, if I install the Foo package locally by navigating to the source directory and doing pip install -e . with my virtual environment activated, the rpath is set correctly...
How can I get the right behavior when installing from PyPI? In case it helps, here is the relevant code from setup.py:
import Bar
barDir = os.path.dirname(inspect.getfile(Bar))
When I activate my environment and install from PyPI, barDir is
/Users/me/anaconda/envs/WRONGENVIRONMENT/lib/python2.7/site-packages/bar
When I activate my environment and install locally, barDir is
/Users/me/anaconda/envs/RIGHTENVIRONMENT/lib/python2.7/site-packages/bar
I'm running OS X 10.10.5 and using conda environments
Try using pkg_resources from setuptools: https://pythonhosted.org/setuptools/pkg_resources.html.
Something like:
from pkg_resources import resource_string
libbar = resource_string('Bar', 'libBar.so')
It turned out to be a stupid caching problem (pip install was not getting the latest package from PyPI, but using a cached copy of an older version I had been testing with). I had to do
pip install --no-cache-dir Foo
to ensure I was getting a fresh version
I just re-installed python & virtualenv after accidentally getting everything messy. I noticed that my requirements.txt in my newly created environment automatically includes wheel==0.24.0. Is this normal, and what is it? I understand that virtualenv should create new environments without any system-installed packages.
I came across this issue when setting up virtualenv in windows.
To avoid wheel from being installed you can use the --no-wheel option.
virtualenv venv --no-wheel
To my understanding wheel is used handle a compressed format of a python package (.whl) that can be used by pip to speed up the install time. https://wheel.readthedocs.org/en/latest/
Probably one of the itens listed in your requirements.txt needs wheel package installed to work.
Although it's a common (and good) pratice list the fundamental python packages of a project inside requirements.txt, not all of the packages the eventually will be installed are there.
Your pip log may helps you to point what package required wheel to be installed.
For work-related reasons, I'm trying to install Python 2.7.8 directly onto my machine (Mac OSX 10.9).
I am current running Python 2.7.6 in Enthought Canopy, and I really don't want to touch the existing libraries there.
My problem is that I'd like to use pip to install packages for the new instantiation of Python, but currently pip is bundled up with Enthought Canopy, so it only installs packages in the site-packages path for Enthought Canopy.
I first tried the following:
pip install --install-option="--prefix=$PREFIX_PATH/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages" scikit-learn
But got the following error:
Requirement already satisfied (use --upgrade to upgrade): scikit-learn in ./Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages
Then, I tried to add the existing Enthought folder to the path for the newly installed Python 2.7.8, By entering the following line at the end of the .bash_profile:
PYTHONPATH=$PYTHONPATH:Users/***/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
This led to errors when trying to import some of the packages, probably for this reason:
Cannot import Scikit-Learn
I would really prefer just to install a new version of scikit-learn in a separate folder. Anyone have any suggestions?
You can use virtualenv to create a self-contained python environment that can be configured and used separately from your regular python installation.
Create the virtualenv (for oldish versions of virtualenv you'd want to include --no-site-packages right after virtualenv):
$ virtualenv limitedenv
Using base prefix '/usr/local/Cellar/python3/3.3.3/Frameworks/Python.framework/Versions/3.3'
New python executable in limitedenv/bin/python3
Also creating executable in limitedenv/bin/python
Installing setuptools, pip...done.
Move into the virtualenv and activate it:
$ cd limitedenv/
$ source bin/activate
(limitedenv)$
Install the packages you're after with pip as you'd do globally:
(limitedenv)$ pip install scikit-learn
Downloading/unpacking scikit-learn
Downloading scikit-learn-0.15.0.tar.gz (7.0MB): ...
scikit-learn will now be installed just inside limitedenv, and as long as you have that environment active, invoking python or pip will be like this is your very own, secluded Python installation.
You can exit from the virtual environment by calling deactivate:
(limitedenv)$ deactivate
$
This allows you to have different versions of python by themselves, different versions of libraries pr. project and different configurations based on what your project requires. virtualenv is a very useful tool!