I can't load a python module in IPython that works fine in the normal interpreter. I have analyzed the problem and somehow IPython does not find a module, whereas the standard console does:
This works in the normal interpreter:
>>> import sys
>>> sys.path.append(r'c:\development\...\ns.package-10.1.0.3-py2.7.egg')
>>> from ns import package
>>>
But on IPython it does not:
In [2]: import sys
In [3]: sys.path.append(r'c:\development\...\ns.package-10.1.0.3-py2.7.egg')
In [4]: from ns import package
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-4-c019e2988e33> in <module>()
----> 1 from ns import package
ImportError: cannot import name package
I find this pretty confusing. I am new to IPython and I don't know where to start. Thanks in advance.
The key thing you have to recall here is that usually there's not just one python interpreter on your machine. Many systems nowadays come with both python2.7 and python3.x, maybe there are more. Every interpreter maintains its own set of installed packages and has its own set of installed scripts, such as ipython or pip.
When you type pip in your shell it's often not obvious which pip you are actually calling. Is it python3's or python2's pip?
And here is where you can get into trouble:
The ipython und python executables in your PATH do not necessary belong to the same interpreter: Imagine that python and pip belong to a python2 installation but you then decide to install ipython into your python3 interpreter.
Now ipython sees the the packages of your python3 interpreter whereas python sees all your python2 packages.
If you compare the output of which ipython and which python in this case, you will notice that you get paths that belong to different interpreters.
So how can you call the script for your favorite interpreter? If python points to your favorite interpreter some packages give you a nice way of calling via -m parameter: Instead of pip install ipython you can write python -m pip install ipython and be sure that you called the pip version of your favorite python interpreter.
Similar you can start ipython notebook via python -m IPython notebook.
All the above is true, thanks #cel. I break my installs every month or so b/c I'm an admin installing and removing and such. I broke my stuff last week and had the same problem as above, except for python3 specifically. All I did was pip3 uninstall ipython; pip3 install ipython and I got my pandas and other libraries linked back usable.
Related
This is my first time asking on this site, so sorry if my question is not layed out correctly
y#DESKTOP-MQJ3NCT:~/Real-Time-Voice-Cloning$ python demo_toolbox.py
Traceback (most recent call last):
File "demo_toolbox.py", line 1, in <module>
from pathlib import Path
ImportError: No module named pathlib
I have tried:
pip3 install pathlib
and:
sudo -H pip3 install pathlib
but continue to get the same error
I am using the windows store version of ubuntu 18 LTS and python 3.7
When it comes to python, it's quite easy to make the mistake of just running "python ...". When you install python on windows "python" defaults to the python 2.7 installation ( probably changed now that 2.7 is no longer supported) if it is installed.
Ubuntu has the links "python2" and "python3" which makes so much more sense but can still lead to confusion.
If you have a local python Virtual environment, the "python" command defaults to the global install on windows (to further confuse people).
I find it generally best to create my own links to the global python "python27" and "python36" to avoid these confusions.
Same goes for pip. It's best to call
python3 -m pip install ...
Also. PyCharm is the most amazing Python IDE in the world and it helps with so much.
Sorry, turns out it was a simple mistake!
Instead of:
python demo_toolbox.py
I should have used
python3 demo_toolbox.py
As Linux defaults to Python 2 or something incompatible with pathlib
I've been trying for a couple of hours already. It seems IDLE can't find any third-party module. I am a Python beginner.
Here is some info about my system:
OSX version: 10.11.5
python version: Python 2.7, Python 3.4, Python 3.5
The initial installation using pip (among other methods) seems to work fine. When I repeat the installation, terminal responds with:
Requirement already satisfied (use --upgrade to upgrade): pyperclip in
./anaconda/lib/python3.4/site-packages
However, when I go to IDLE (Python 3.4) and try to import the module, IDLE responds with:
Traceback (most recent call last): File "", line 1, in
import pyperclip ImportError: No module named 'pyperclip'
I have read that it may have something to do with my PATH or some virtual environment. I’ll be frank, I’m not sure what to make of these as they seem beyond my current ability.
This inability to import modules is becoming an almost insurmountable roadblock to advancing with Python. If you can offer any ideas on what I can do or can ELI5 the solution, I am forever in your debt?
It seems you are using conda, but you are trying to install the pyperclip module with pip. Have you tried running conda install pyperclip?
As stated here:
Because Conda introduces a new packaging format, you cannot use pip and Conda interchangeably; pip cannot install the Conda package format. You can use the two tools side by side but they do not interoperate either.
I wrote myself a handy bash script, which solves the task of creating a virtualenv with its own compiled virtualenv and python. It aims at creating a mostly self contained virtualenv, with maybe only native libraries installed in system level if necessary, but installing all python packages and virtualenv and pip and such things inside the virtualenv.
The script can be found here.
I invoke the script as follows:
self_contained_venv.sh \
-n udacity_model_building_and_validation \
-p https://www.python.org/ftp/python/3.4.4/Python-3.4.4.tar.xz \
-v https://pypi.python.org/packages/c8/82/7c1eb879dea5725fae239070b48187de74a8eb06b63d9087cd0a60436353/virtualenv-15.0.1.tar.gz#md5=28d76a0d9cbd5dc42046dd14e76a6ecc \
-d pandas scikit-learn seaborn
Given the required packages for compiling python and virtualenv are installed on the system, the script creates a nice virtualenv. However, when I try to access any installed modules/packages from within the virtualenv, python is not able to find them. To demonstrate this, I'll put some output of commands and code here:
First of all of course I have to activate the virtualenv:
. bin/activate
output: None, works without problem.
Then I print the pythonpath python is aware of:
import sys
for i in sys.path:
print(i)
output:
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python34.zip
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/plat-linux
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/lib-dynload
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/plat-linux
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/site-packages
So far so good. Then I try to import a module / package I installed during usage of my bash script: pandas:
python
(IDLE is running)
import pandas as pd
output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'pandas'
Another try:
import numpy as np
output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'numpy'
Huh? So none of the packages is available? Lets check pip again:
which pip
output:
alias pip='localpython/bin/pip3.4'
./localpython/bin/pip3.4
Ok, so it's using my local pip.
Check packages:
pip list
output:
numpy (1.11.0)
pandas (0.18.1)
pip (8.1.2)
psutil (4.1.0)
Python-contrib-nbextensions (alpha)
python-dateutil (2.5.3)
pytz (2016.4)
PyYAML (3.11)
setuptools (18.2)
six (1.10.0)
virtualenv (15.0.1)
Hm the packages are there, so why can't python find them? Let's see where those packages are located, simply by trying to remove one:
pip uninstall pandas
output (shortened, because it fills many pages):
Uninstalling pandas-0.18.1:
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas-0.18.1-py3.4.egg-info
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__init__.py
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/__init__.cpython-34.pyc
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/_version.cpython-34.pyc
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/info.cpython-34.pyc
Aha, so the packages are in the path.
Another attempt on finding out if python looks in the right places:
>>> from distutils.sysconfig import get_python_lib
>>> print(get_python_lib())
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/site-packages
So this one indicates it does not look in the right place, if I understand correctly.
Note:
The script does the following things, which might not be obvious:
compile python with a prefix (local python)
compile virtualenv for the local python
it aliases pip to the local pip of the version of the installed python
it aliases virtualenv to the locally installed one
it installs packages from pypi if specified
it updates the local pip if there is a newer version available
I am a beginner still at writing bash scripts, so I think the structure and logic of the script is fairly easy to understand. It also prints information about success of its operations in the terminal.
Further Notes:
I did not use su or sudo to run the script.
My OS is a Fedora 22, although I think in this case any major distro would work the same way.
OS has been updated recently.
Question: So why can't Python find them / its own packages? (What do I need to change?)
can you try to compare path to pyhton and pip in your script and manually in your system:
which python and which pip?
After that check pip freeze | grep pandas it should return you package ...if no you need to add this package to site-packages folder of you python.
Getting Python right on Mac seems a constant challenge. I'm working with a Homebrew implementation, and now have difficulty running Jupyter except with sudo:
$ which python
/usr/local/bin/python
$ which pip
/Users/username/bin/pip
$ which jupyter
/usr/local/bin/jupyter
$ jupyter notebook
Traceback (most recent call last):
File "/usr/local/bin/jupyter", line 7, in <module>
from jupyter_core.command import main
ImportError: No module named jupyter_core.command
$ sudo jupyter notebook
The Jupyter Notebook is running at: http://localhost:8888/
Once running, Jupyter fails to import pandas (installed via pip install pandas):
import pandas
ImportError Traceback (most recent call last)
<ipython-input-18-d6ac987968b6> in <module>()
----> 1 import pandas
ImportError: No module named pandas
.. even though pandas is available (python -s 'import pandas' works fine). Guessing its path isn't available to root.
I'm wondering if Jupyter is particularly problematic or if this is just my setup..?
Grateful for assistance as this is becoming very tiresome. Any guidance on wiping python and reinstalling 'properly' from scratch would be handy.
Edit:
$ which virtualenv
/usr/local/bin/virtualenv
21:16 $ virtualenv
Traceback (most recent call last):
File "/usr/local/bin/virtualenv", line 7, in <module>
from virtualenv import main
ImportError: No module named virtualenv
What python is in the shebang (#!) line on pip and jupyter? You may have installed jupyter with a pip that was using /usr/bin/python. Which means it's not there in the libraries installed for Homebrew's /usr/local/bin/python.
You should be able to resolve this with something like this:
/usr/local/bin/python $(type -p pip) install jupyter
But in general I second #Ray's suggestion of using pyenv to manage a personal set of Python installations that you can switch between, independently of what's on the system. (I likewise use rbenv for Ruby and perlbrew for Perl.)
Stop using python via native install on OSX or via homebrew right now!!!
First learn to use pyenv to control your version of python being used. Pyenv allows you to safely switch between any version of python you want at any time without messing up the natively installed python (which may be needed by the system): https://amaral.northwestern.edu/resources/guides/pyenv-tutorial
Second learn how to use virtual environments (pyvenv) to sandbox your projects. Virtual environments will allow you to have different sets of libraries installed by pip and such available as needed: https://robinwinslow.uk/2013/12/26/python-3-4-virtual-environment/
With these two technologies in your toolbox you will never have the issue you are experiencing again.
Please note pyvenv and pyenv are two seperate technologies used to enable similar goals (to control/sandbox environments) with unfortunately VERY similar names. One thing to note is you'll see older references to virtualenv if you google around a bit. This is the same thing as pyvenv.
Someone put together a good overview of using these with Jupyter: http://www.alfredo.motta.name/create-isolated-jupyter-ipython-kernels-with-pyenv-and-virtualenv/.
The other option is to do everything in virtual machines (with say virtualbox) or better still Docker containers, but this a whole other ball of fun best left to after you master virtual environments (you'll want them when you build your docker images anyhow).
I am used to the R functionality of installing packages and I am trying to do the same thing with ipython. Sometimes the following method works but then again sometimes it doesn't and I would like to finally find out why it only works half the time.
Normally to install a module (like the requests module for example) I would type the following after opening a fresh terminal:
$ sudo pip install requests
Password: *******
This would then be followed by a message indicating that the install was successful or that it has already been installed.
Requirement already satisfied (use --upgrade to upgrade):
requests in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
Cleaning up...
Which suggests that the code can be accessed. And indeed if I run python now from the terminal it shows a good response without any errors whatsoever.
$ python
ActivePython 2.7.2.5 (ActiveState Software Inc.) based on
Python 2.7.2 (default, Jun 24 2011, 12:20:15)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>>
I now open PyLab through Alfred and it gives me an error.
Welcome to pylab, a matplotlib-based Python environment [backend: WXAgg].
For more information, type 'help(pylab)'.
In [1]: import requests
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
/Users/vincentwarmerdam/<ipython-input-1-686486c241c8> in <module>()
----> 1 import requests
ImportError: No module named requests
I've read some help from another question on StackOverflow (here) which suggests that I install the module from ipython shell. This gives an even more baffling response:
In [2]: !pip install requests
Requirement already satisfied (use --upgrade to upgrade): requests in
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
Cleaning up...
In [3]: import requests
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
/Users/vincentwarmerdam/<ipython-input-3-686486c241c8> in <module>()
----> 1 import requests
ImportError: No module named requests
This seems very strange to me. Are there multiple versions of python installed on the system? How could I check this? Do I need to point ipython to the location of the installed code?
actually there is a much much much more elegant solution. when pip is installed then within python you can do things like this as well:
import pip
def install(package):
pip.main(['install', package])
install('requests')
which is easier. once logged into a virtualenv you can just make sure that you have what you need in the session you are in. easy.
edit
Another alternative would be to use the %%bash magic.
%%bash
pip install requests
edit2
If you want the standard output, one could even use the exclamation bang.
! pip install requests
edit3
From within ipython this is the safest installation method.
%pip install requests
This ensures that everything is installed in the virtualenv that your ipython is installed in.
Here's what I did that made it work; open up iypthon through the command line and type
import sys
sys.path
This shows a list of folders where other python modules are located. For me this was:
['',
'/Library/Frameworks/Python.framework/Versions/7.3/bin',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/pandas-0.10.0-py2.7-macosx-10.5-i386.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/googlemaps-1.0.2-py2.7.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/oauth-1.0.1-py2.7.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/oauth2-1.5.211-py2.7.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/httplib2-0.7.7-py2.7.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/selenium-2.28.0-py2.7.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/jellyfish-0.2.0-py2.7-macosx-10.5-i386.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/python_yelp-0.1.1-py2.7.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/pymongo-2.4.2_-py2.7-macosx-10.5-i386.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/lucene_querybuilder-0.1.6-py2.7.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/mechanize-0.2.5-py2.7.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/html2text-3.200.3-py2.7.egg',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python27.zip',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/plat-darwin',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/plat-mac',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/plat-mac/lib-scriptpackages',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/lib-tk',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/lib-old',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/lib-dynload',
'/Users/vincentwarmerdam/Library/Python/2.7/lib/python/site-packages',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/PIL',
'/Library/Python/2.7/site-packages',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/IPython/extensions]
With this information, I now knew where ipython looks for the modules that one can import. So I downloaded the requests library manually, added it to the same root directory such that the following directory exists:
/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/requests
This folder contains the python modules that belong to requests. The only thing I now had to do was to make sure that ipython knows that this folder exists. Which was done by updating the sys.path.
req_link = '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/requests'
sys.path.append(req_link)
After this I no longer got the error.
import requests
Just works.
Also after restarting ipython, I found that ipython automatically updates the new path into the sys.path list.
If the new packages installed are imported on Terminal but not imported to ipython notebook then it is very likely that you have two versions of python installed on your library. Due to this there are separate site-packages directory from where packages are being imported on terminal and ipython notebook. To check if this is the case.
On terminal use:
import sys
sys.path
This will show where the python modules are located for python you using on terminal.
Now, in ipython notebook use:
import sys
sys.path
This will show where the python modules are located for python you using on terminal.
Now, if the two path are different you know you are using two different installations of python. To solve this problem, copy installed packages from site-packages directory of terminal python to site-packages directory of ipython.
I had this same problem when trying to install patool, but it turned out it was due to more than one interpreter installed, and when I ran it from ipython it worked.
This sorted me out running Spyder on Windows.
As the answer above, to find the directory:
import sys
sys.path
This site :
www.geeksforgeeks.org/how-to-install-pip-on-windows/
Directed me to this: https://bootstrap.pypa.io/get-pip.py
Use cmd to run that script from the directory of the version of Python you want to add modules to. Then just use pip to add the modules, again from the command line.
I didn't even bother adding any of it to my PATH, but you could do that.