IPython sys.path different from python sys.path - python

I generally use IPython and only recently noticed that the the search path for imports is wrong in the regular python shell. From what I understand, sys.path inherits from PYTHONPATH (although I don't know where PYTHONPATH lives), is this different in IPython? I'm worried that this effecting installations. For instance I just tried
pip install --upgrade gensim
which failed because it couldn't resolve the scipy dependency, which I already have installed.
So I dove a little bit deeper and found in Ipython
import gensim
gensim.__version__
returns .9.1
while in python
import gensim
gensim.__version__
returns .8.9
Here is what the Ipython version of sys.path looks like:
['',
'/Users/change/anaconda/bin',
'/Users/change/anaconda/lib/python2.7/site-packages/pybing-0.12-py2.7.egg',
'/Users/change/anaconda/lib/python2.7/site-packages/httplib2-0.8-py2.7.egg',
'/Users/change/anaconda/python.app/Contents/lib/python27.zip',
'/Users/change/anaconda/python.app/Contents/lib/python2.7',
'/Users/change/anaconda/python.app/Contents/lib/python2.7/plat-darwin',
'/Users/change/anaconda/python.app/Contents/lib/python2.7/plat-mac',
'/Users/change/anaconda/python.app/Contents/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/change/anaconda/python.app/Contents/lib/python2.7/lib-tk',
'/Users/change/anaconda/python.app/Contents/lib/python2.7/lib-old',
'/Users/change/anaconda/python.app/Contents/lib/python2.7/lib-dynload',
'/Users/change/anaconda/lib/python2.7/site-packages',
'/Users/change/anaconda/lib/python2.7/site-packages/PIL',
'/Users/change/anaconda/lib/python2.7/site-packages/pygoogle',
'/Users/change/anaconda/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info',
'/Users/change/anaconda/lib/python2.7/site-packages/IPython/extensions']
and the regular python script sys.path:
['',
'/Users/change/anaconda/lib/python2.7/site-packages/pybing-0.12-py2.7.egg',
'/Users/change/anaconda/lib/python2.7/site-packages/httplib2-0.8-py2.7.egg',
'/Users/change/anaconda/lib/python27.zip',
'/Users/change/anaconda/lib/python2.7',
'/Users/change/anaconda/lib/python2.7/plat-darwin',
'/Users/change/anaconda/lib/python2.7/plat-mac',
'/Users/change/anaconda/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/change/anaconda/lib/python2.7/lib-tk',
'/Users/change/anaconda/lib/python2.7/lib-old',
'/Users/change/anaconda/lib/python2.7/lib-dynload',
'/Users/change/anaconda/lib/python2.7/site-packages',
'/Users/change/anaconda/lib/python2.7/site-packages/PIL',
'/Users/change/anaconda/lib/python2.7/site-packages/pygoogle',
'/Users/change/anaconda/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info']

Probably too late for you now, but the answer here may help you too: ipython reads wrong python version . Basically the ipython script can directly reference a specific python binary rather than the one that you'd get if you just ran python directly.

I had a brew installed version of Python on my mac. For some reason that couldn't use the system libraries. After brew uninstall python it worked again, because the default python was switched back to /usr/bin/python.
Also playing around with which python, which ipython and opening up /usr/bin/python, /usr/local/bin/python, /usr/bin/ipython and /usr/local/bin/ipython and doing the imports there might help you find out where it is exactly going wrong.

You may have your virtual environment activated which is ignoring the system path.
It can be fixed by running this in terminal (After closing notebook etc):
virtualenv --system-site-packages ~

Related

No module named 'stix2' [duplicate]

After installing mechanize, I don't seem to be able to import it.
I have tried installing from pip, easy_install, and via python setup.py install from this repo: https://github.com/abielr/mechanize. All of this to no avail, as each time I enter my Python interactive I get:
Python 2.7.3 (default, Aug 1 2012, 05:14:39)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mechanize
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named mechanize
>>>
The installations I ran previously reported that they had completed successfully, so I expect the import to work. What could be causing this error?
In my case, it is permission problem. The package was somehow installed with root rw permission only, other user just cannot rw to it!
I had the same problem: script with import colorama was throwing an ImportError, but sudo pip install colorama was telling me "package already installed".
My fix: run pip without sudo: pip install colorama. Then pip agreed it needed to be installed, installed it, and my script ran. Or even better, use python -m pip install <package>. The benefit of this is, since you are executing the specific version of python that you want the package in, pip will unequivocally install the package into the "right" python. Again, don't use sudo in this case... then you get the package in the right place, but possibly with (unwanted) root permissions.
My environment is Ubuntu 14.04 32-bit; I think I saw this before and after I activated my virtualenv.
I was able to correct this issue with a combined approach. First, I followed Chris' advice, opened a command line and typed 'pip show packagename'
This provided the location of the installed package.
Next, I opened python and typed 'import sys', then 'sys.path' to show where my python searches for any packages I import. Alas, the location shown in the first step was NOT in the list.
Final step, I typed 'sys.path.append('package_location_seen_in_step_1'). You optionally can repeat step two to see the location is now in the list.
Test step, try to import the package again... it works.
The downside? It is temporary, and you need to add it to the list each time.
It's the python path problem.
In my case, I have python installed in:
/Library/Frameworks/Python.framework/Versions/2.6/bin/python,
and there is no site-packages directory within the python2.6.
The package(SOAPpy) I installed by pip is located
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/
And site-package is not in the python path, all I did is add site-packages to PYTHONPATH permanently.
Open up Terminal
Type open .bash_profile
In the text file that pops up, add this line at the end:
export PYTHONPATH=$PYTHONPATH:/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/
Save the file, restart the Terminal, and you're done
The Python import mechanism works, really, so, either:
Your PYTHONPATH is wrong,
Your library is not installed where you think it is
You have another library with the same name masking this one
I have been banging my head against my monitor on this until a young-hip intern told me the secret is to "python setup.py install" inside the module directory.
For some reason, running the setup from there makes it just work.
To be clear, if your module's name is "foo":
[burnc7 (2016-06-21 15:28:49) git]# ls -l
total 1
drwxr-xr-x 7 root root 118 Jun 21 15:22 foo
[burnc7 (2016-06-21 15:28:51) git]# cd foo
[burnc7 (2016-06-21 15:28:53) foo]# ls -l
total 2
drwxr-xr-x 2 root root 93 Jun 21 15:23 foo
-rw-r--r-- 1 root root 416 May 31 12:26 setup.py
[burnc7 (2016-06-21 15:28:54) foo]# python setup.py install
<--snip-->
If you try to run setup.py from any other directory by calling out its path, you end up with a borked install.
DOES NOT WORK:
python /root/foo/setup.py install
DOES WORK:
cd /root/foo
python setup.py install
I encountered this while trying to use keyring which I installed via sudo pip install keyring. As mentioned in the other answers, it's a permissions issue in my case.
What worked for me:
Uninstalled keyring:
sudo pip uninstall keyring
I used sudo's -H option and reinstalled keyring:
sudo -H pip install keyring
In PyCharm, I fixed this issue by changing the project interpreter path.
File -> Settings -> Project -> Project Interpreter
File -> Invalidate Caches… may be required afterwards.
I couldn't get my PYTHONPATH to work properly. I realized adding export fixed the issue:
(did work)
export PYTHONPATH=$PYTHONPATH:~/test/site-packages
vs.
(did not work)
PYTHONPATH=$PYTHONPATH:~/test/site-packages
This problem can also occur with a relocated virtual environment (venv).
I had a project with a venv set up inside the root directory. Later I created a new user and decided to move the project to this user. Instead of moving only the source files and installing the dependencies freshly, I moved the entire project along with the venv folder to the new user.
After that, the dependencies that I installed were getting added to the global site-packages folder instead of the one inside the venv, so the code running inside this env was not able to access those dependencies.
To solve this problem, just remove the venv folder and recreate it again, like so:
$ deactivate
$ rm -rf venv
$ python3 -m venv venv
$ source venv/bin/activate
$ pip install -r requirements.txt
Something that worked for me was:
python -m pip install -user {package name}
The command does not require sudo. This was tested on OSX Mojave.
In my case I had run pip install Django==1.11 and it would not import from the python interpreter.
Browsing through pip's commands I found pip show which looked like this:
> pip show Django
Name: Django
Version: 1.11
...
Location: /usr/lib/python3.4/site-packages
...
Notice the location says '3.4'. I found that the python-command was linked to python2.7
/usr/bin> ls -l python
lrwxrwxrwx 1 root root 9 Mar 14 15:48 python -> python2.7
Right next to that I found a link called python3 so I used that. You could also change the link to python3.4. That would fix it, too.
In my case it was a problem with a missing init.py file in the module, that I wanted to import in a Python 2.7 environment.
Python 3.3+ has Implicit Namespace Packages that allow it to create a packages without an init.py file.
Had this problem too.. the package was installed on Python 3.8.0 but VS Code was running my script using an older version (3.4)
fix in terminal:
py .py
Make sure you're installing the package on the right Python Version
I had colorama installed via pip and I was getting "ImportError: No module named colorama"
So I searched with "find", found the absolute path and added it in the script like this:
import sys
sys.path.append("/usr/local/lib/python3.8/dist-packages/")
import colorama
And it worked.
I had just the same problem, and updating setuptools helped:
python3 -m pip install --upgrade pip setuptools wheel
After that, reinstall the package, and it should work fine :)
The thing is, the package is built incorrectly if setuptools is old.
If the other answers mentioned do not work for you, try deleting your pip cache and reinstalling the package. My machine runs Ubuntu14.04 and it was located under ~/.cache/pip. Deleting this folder did the trick for me.
Also, make sure that you do not confuse pip3 with pip. What I found was that package installed with pip was not working with python3 and vice-versa.
I had similar problem (on Windows) and the root cause in my case was ANTIVIRUS software! It has "Auto-Containment" feature, that wraps running process with some kind of a virtual machine.
Symptoms are: pip install somemodule works fine in one cmd-line window and import somemodule fails when executed from another process with the error
ModuleNotFoundError: No module named 'somemodule'
In my case (an Ubuntu 20.04 VM on WIN10 Host), I have a disordered situation with many version of Python installed and variuos point of Shared Library (installed with pip in many points of the File System). I'm referring to 3.8.10 Python version.
After many tests, I've found a suggestion searching with google (but' I'm sorry, I haven't the link). This is what I've done to resolve the problem :
From shell session on Ubuntu 20.04 VM, (inside the Home, in my case /home/hduser), I've started a Jupyter Notebook session with the command "jupyter notebook".
Then, when jupyter was running I've opened a .ipynb file to give commands.
First : pip list --> give me the list of packages installed, and, sympy
wasn't present (although I had installed it with "sudo pip install sympy"
command.
Last with the command !pip3 install sympy (inside jupyter notebook
session) I've solved the problem, here the screen-shot :
Now, with !pip list the package "sympy" is present, and working :
In my case, I assumed a package was installed because it showed up in the output of pip freeze. However, just the site-packages/*.dist-info folder is enough for pip to list it as installed despite missing the actual package contents (perhaps from an accidental deletion). This happens even when all the path settings are correct, and if you try pip install <pkg> it will say "requirement already satisfied".
The solution is to manually remove the dist-info folder so that pip realizes the package contents are missing. Then, doing a fresh install should re-populate anything that was accidentally removed
When you install via easy_install or pip, is it completing successfully? What is the full output? Which python installation are you using? You may need to use sudo before your installation command, if you are installing modules to a system directory (if you are using the system python installation, perhaps). There's not a lot of useful information in your question to go off of, but some tools that will probably help include:
echo $PYTHONPATH and/or echo $PATH: when importing modules, Python searches one of these environment variables (lists of directories, : delimited) for the module you want. Importing problems are often due to the right directory being absent from these lists
which python, which pip, or which easy_install: these will tell you the location of each executable. It may help to know.
Use virtualenv, like #JesseBriggs suggests. It works very well with pip to help you isolate and manage the modules and environment for separate Python projects.
I had this exact problem, but none of the answers above worked. It drove me crazy until I noticed that sys.path was different after I had imported from the parent project. It turned out that I had used importlib to write a little function in order to import a file not in the project hierarchy. Bad idea: I forgot that I had done this. Even worse, the import process mucked with the sys.path--and left it that way. Very bad idea.
The solution was to stop that, and simply put the file I needed to import into the project. Another approach would have been to put the file into its own project, as it needs to be rebuilt from time to time, and the rebuild may or may not coincide with the rebuild of the main project.
I had this problem with 2.7 and 3.5 installed on my system trying to test a telegram bot with Python-Telegram-Bot.
I couldn't get it to work after installing with pip and pip3, with sudo or without. I always got:
Traceback (most recent call last):
File "telegram.py", line 2, in <module>
from telegram.ext import Updater
File "$USER/telegram.py", line 2, in <module>
from telegram.ext import Updater
ImportError: No module named 'telegram.ext'; 'telegram' is not a package
Reading the error message correctly tells me that python is looking in the current directory for a telegram.py. And right, I had a script lying there called telegram.py and this was loaded by python when I called import.
Conclusion, make sure you don't have any package.py in your current working dir when trying to import. (And read error message thoroughly).
I had a similar problem using Django. In my case, I could import the module from the Django shell, but not from a .py which imported the module.
The problem was that I was running the Django server (therefore, executing the .py) from a different virtualenv from which the module had been installed.
Instead, the shell instance was being run in the correct virtualenv. Hence, why it worked.
This Works!!!
This often happens when module is installed to an older version of python or another directory, no worries as solution is simple.
- import module from directory in which module is installed.
You can do this by first importing the python sys module then importing from the path in which the module is installed
import sys
sys.path.append("directory in which module is installed")
import <module_name>
Most of the possible cases have been already covered in solutions, just sharing my case, it happened to me that I installed a package in one environment (e.g. X) and I was importing the package in another environment (e.g. Y). So, always make sure that you're importing the package from the environment in which you installed the package.
For me it was ensuring the version of the module aligned with the version of Python I was using.. I built the image on a box with Python 3.6 and then injected into a Docker image that happened to have 3.7 installed, and then banging my head when Python was telling me the module wasn't installed...
36m for Python 3.6
bsonnumpy.cpython-36m-x86_64-linux-gnu.so
37m for Python 3.7 bsonnumpy.cpython-37m-x86_64-linux-gnu.so
I know this is a super old post but for me, I had an issue with a 32 bit python and 64 bit python installed. Once I uninstalled the 32 bit python, everything worked as it should.
I have solved my issue that same libraries were working fine in one project(A) but importing those same libraries in another project(B) caused error. I am using Pycharm as IDE at Windows OS.
So, after trying many potential solutions and failing to solve the issue, I did these two things (deleted "Venv" folder, and reconfigured interpreter):
1-In project(B), there was a folder named("venv"), located in External Libraries/. I deleted that folder.
2-Step 1 (deleting "venv" folder) causes error in Python Interpreter Configuration, and
there is a message shown at top of screen saying "Invalid python interpreter selected
for the project" and "configure python interpreter", select that link and it opens a
new window. There in "Project Interpreter" drop-down list, there is a Red colored line
showing previous invalid interpreter. Now, Open this list and select the Python
Interpreter(in my case, it is Python 3.7). Press "Apply" and "OK" at the bottom and you
are good to go.
Note: It was potentially the issue where Virtual Environment of my Project(B) was not recognizing the already installed and working libraries.

No module named 'pyodbc' in Spyder

I'm trying to set up a script with SQL connection, and am running into the following error:
ModuleNotFoundError: No module named 'pyodbc'
from the line:
import pyodbc
I'm using Python 3.9.2, Spyder version 4.2.3. Updated to latest Spyder version 5.0.1, no change.
Weirdly I have no trouble importing the module from the command line, it's only when I run it through Spyder that I get this issue.
Normally, from what I've read this would be a case of multiple environments, but I have yet to set up any environments so I only have one python installation.
I also have the site-packages folder properly added to path, heck I've moved the pyodbc files into their own directory and added THAT to the path.
I've copy and pasted the files into Spyder's packages directory too, but still didn't fix the issue.
The only thing I can think of is this:
which indicates that Spyder is using Python 3.7.9 as its interpreter rather than the version I actually have installed?
Any idea what the issue could be?
Thank you Ewran for your help.
When I changed the interpreter it gave me an error message to update my spyder-kernels which seems to have fixed the issue.
To summarize for others:
1). Make sure you have a path to C:\Users<User>\AppData\Local\Programs\Python\Python39\Lib\site-packages
2). Make sure you're installing the module to the correct environment
3). Make sure Spyder is using the correct interpreter, point it to the desired python.exe and update Spyder's kernels if needed.
If you are fine with the Python 3.7.9 used by Spyder, you can type
!pip install pyodbc
directly in Spyder's console to install pyodbc.
Then, the import should work.
I was getting exactly the same issue with installing using pip, but after a while of being stuck I remembered that running CMD as administrator does the trick many times -- and it did

Importing python modules works in command line but not in the python 3.8 shell

I'm having a problem with importing modules in python.
When I run my program in the command line it works perfectly fine.
However, when I try to run the same program in the python shell I am prompted with the following error:
ModuleNotFoundError: No module named 'matplotlib'
I already successfully installed matplotlib using 'python -m pip install matplotlib'.
I've read this can happen when you have two different versions of python installed; however, I don't.
I've uninstalled and reinstalled python and I still am having the same issue. I've also uninstalled and reinstalled matplotlib using pip.
I believe my problem is the module paths that python uses to search for imported modules are different between the two.
When I use the 'print(sys.path)' command in the python shell and the command line I get two different outputs.
Any help would be greatly appreciated!!!
The file different system paths between the python shell and the command line
You have two versions of python. I would recommend you to remove all pythons you have and go for anaconda https://www.anaconda.com/distribution/. It will fix your path problems and allow you to create environments with different versions of python. This is the least painful way also for future :) good luck.
I suppose, you have both of the Python versions installed on the same computer.
If that is so, then my answer would be to go inside both Python script folders and install matplotlib on both of them.
I have also faced that issue. My path includes pip of Python 3.7.1 and whenevwer I try to import modules on Python 3.4. It throws an error!
Maybe, you could add both of the Pythons to the path.
I encountered this same problem – python -c "import sklearn" would work just fine, but import sklearn inside a Python program failed. Both my one-liner and program was using the same Python version (version 3.8.10).
I eventually got the program to work by replacing the shebang line (originally #!/usr/bin/python) with #!/bin/env python.
I don't know why this worked exactly (sorry). Presumably some path got reset, and the module loaded from a different location, but it might help someone so I'm posting it here nontheless. (If you know more, feel free to edit this answer.)

Why does importing Pytorch in Sublime Text cause a ModuleNotFoundError?

The command 'import torch' causes the build error: 'ModuleNotFoundError: No module named 'torch''.
But when I run the command in the terminal, pytorch works just fine (Python 3.7.X).
Are you using a venvin the project you are right now? If that is so, then you need to install torch in that venv, as it has been created just for the project, so it has not got any extra modules like that one.
Take a look at this. And make sure that the module is in the environment your project is!
In Pycharm for example, whenever you create a new project with a new venv(virtual environment), you need to add to your project settings->interpreter settings the module manually, so it can be imported. Because it may be installed in your system, but not in that venvspecifically, which will look in there!
try this
link
it might help.
Also Faced similar issue in Windows 10 with Anaconda installed. Installed pytorch package using conda install command which I found here: http://pytorch.org/
The usual import from command-line worked for my environment(torch):

Python packages are imported on terminal but not on IDLE

So I am new in Python. I downloaded Anaconda (and Homebrew) and using Terminal on my Mac I confirmed that, say, Numpy is installed:
pip install numpy
to get as a result
Requirement already satisfied: numpy in ./anaconda/lib/python3.6/site-packages
Then opening Python within Terminal I can indeed import the package. The story is not the same once I open IDLE. So when I open it and try to import, say, Numpy I get the following message:
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
import numpy
ModuleNotFoundError: No module named 'numpy'
I guess there is something I do not know, I assume it is a common rookie problem. Can you help me with this?
Since your IDLE is using Python 3 you should use
pip3 install numpy
to get it installed for the correct Python. If you want to use your default installed Python (2.x) instead, IDLE is located at /usr/bin/. From your terminal you can open it with
/usr/bin/idle
this should have the numpy you installed with pip install numpy
See when you are working with Anaconda it creates its own virtual environment.
Now, If you are new in python on MAC then you might not familiar with Virtual Environment.
I suggest you to download python from the official website www.python.org
or Click here to directly download Python 3.6.2 on your MAC.
Then, Download Pycharm(The Best IDE for Python)
Download the community version for beginner and it is also free.
Pycharm Community Version -> Click Here
And do whatever you want and also must read about Virtualenv
According to me, Pycharm is better then Anaconda's Jupyter Notebook
Wish you good luck and show your creativity in python ! !
Erm... well it appears as though - since you have 2 versions on your computer - that it might be because it imported it into the other file path directory. Now, I've never worked with Mac, but I think if you just specify which pip you want to download from, it might work.
For example, on Windows:
C:\Python34\Scripts\pip.exe install numpy
or if you were doing it for 3.6, you would follow the path to the folder, find pip and install. So, I suggest to install the normal IDLE before any other platform built upon it just because it is easier to import modules and is not as bad to break like yours has.
There can be two issues -
You are using python 2 as a kernel for IDLE, since numpy is installed for python3.6. This will raise an error.
The issue is with anaconda's configuration with IDLE. Anaconda installs numpy in ./anaconda/lib/python3.6/site-packages. If IDLE uses systems default python instead of anaconda's. You will not be able to import numpy.
You can try running this snippet -
import sys
print('\n'.join(sys.path))
to track the location of python exactly.
One of the workaround that I can think of is -
create a virtual environment using anaconda. Something like
conda create -n py352 python=3.5.2 anaconda
and then fire IDLE from your terminal. Though I am not entirely sure, if this works for mac. If it doesn't, let me know the output of -
import sys
print('\n'.join(sys.path))

Categories