I need to use netcdf but do not have install permission for python modules. I have downloaded netcdf-0.1.2.tar.gz from here: https://pypi.python.org/simple/netcdf/ and unzipped the tar ball. I have been following this stack overflow post in an attempt to use the module but have had no luck so far:
(Python) Use a library locally instead of installing it
here is what I have tried:
Installing virtualenv:
I do not have permission to do this
python setup.py install -- user:
again, I don't have permission
running my script with netcdf as my current working directory:
I tried this as well, here are the issues I have run into:
first I went into netcdf-0.1.2 and made a new file called asdf.py
which contains the following:
import netcdf
print("testing")
running python asdf.py gives the following error:
Traceback (most recent call last):
File "asdf.py", line 1, in <module>
import netcdf
File "/.../Downloads/netcdf-0.1.2/netcdf/__init__.py", line 1, in <module>
from netcdf import *
File "/.../Downloads/netcdf-0.1.2/netcdf/netcdf.py", line 1, in <module>
from netCDF4 import Dataset, numpy
ImportError: No module named netCDF4
I'm not sure how to fix this error, any help would be greatly appreciated
in case this is somehow relevant, the version of Linux I am using is 3.2.0-23-generic
also I have numpy installed already
Easest would be to install Anaconda or Miniconda with your user rights.
Anaconda already as netCDF4installed. In case of Miniconda install with:
conda install netcdf4
If you have Python 3 installed, then you will have the venv package in the standard library, so you do not need "virtualenv" to be installed for you separately (as would be the case with Python 2). Instead use python3 -mvenv , in a similar way to how you would use virtualenv, for example:
python3 -mvenv /path/to/my_venv
or to include any non-standard packages already installed on the system:
python3 -mvenv --system-site-packages /path/to/my_venv
After that, you should be able to activate the environment and pip install packages, e.g.
source /path/to/my_venv/bin/activate # for csh use activate.csh instead
pip install netCDF4
Remember to source the activate script at run time as well as installation time:
source /path/to/my_venv/bin/activate
python
and you should then find that in your python session you have the netCDF4 package available, e.g.
import netCDF4
my_dataset = netCDF4.Dataset('myfile.nc')
Of course, substitute the actual path in place of /path/to/my_venv above.
None of this requires any root privileges.
(And as someone else has suggested, another option for you is to use conda.)
I'd also like to highlight that the package is imported using capitals
import netCDF4 as nc
This might not matter on a mac, but for Windows it is key.
Related
I have installed the requests module
C:\Python34\Scripts\easy_install.exe requests
i got to the folder location
C:\Python34\Lib\site-packages\requests-2.13.0-py3.4.egg\requests
I have a path variable in system
C:\Python34\Lib\site-packages
yet when i run my script
C:\Users\beast\Desktop>update.py
I get the error No module named 'requests'
Traceback (most recent call last):
File "C:\Users\beast\Desktop\plex_playlist_update.py", line 17, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
I tried installing using pip just in case
python -m pip install requests
Requirement already satisfied: requests in c:\python34\lib\site-packages\requests-2.13.0-py3.4.egg
I am new to python and I cant find an answer anywhere.
UPDATE:
I found a command to check my python search location.
C:\Users\beast\Desktop>python -c "import site; print(site.getsitepackages())"
['C:\\Python34', 'C:\\Python34\\lib\\site-packages']
I think it has to do with C:\Python34 vs C:\python34? How do i check or fix this?
The problem here is maybe because of the different versions of python installed.
If you are a Windows user, you can go to Path in Environment variables and remove the paths to unnecessary versions of python (if any). Modules installed for one version of python won't work in another version.
So I ended up going with python environment. Per python documentation it is the way to go anyway. The below command are run on the root folder of the python app.
py -m venv env
then
./env/Scripts/activate
I then ran my pip upgrade and everything is working. Obviously this did not fix it computer wide. just for my python app. But from documentation this is better because of version control of the whole enviroment.
This creates a "env" folder at the root which will have all the packages installed. Need to install all requirements again or use the requirements.txt file.
I wrote myself a handy bash script, which solves the task of creating a virtualenv with its own compiled virtualenv and python. It aims at creating a mostly self contained virtualenv, with maybe only native libraries installed in system level if necessary, but installing all python packages and virtualenv and pip and such things inside the virtualenv.
The script can be found here.
I invoke the script as follows:
self_contained_venv.sh \
-n udacity_model_building_and_validation \
-p https://www.python.org/ftp/python/3.4.4/Python-3.4.4.tar.xz \
-v https://pypi.python.org/packages/c8/82/7c1eb879dea5725fae239070b48187de74a8eb06b63d9087cd0a60436353/virtualenv-15.0.1.tar.gz#md5=28d76a0d9cbd5dc42046dd14e76a6ecc \
-d pandas scikit-learn seaborn
Given the required packages for compiling python and virtualenv are installed on the system, the script creates a nice virtualenv. However, when I try to access any installed modules/packages from within the virtualenv, python is not able to find them. To demonstrate this, I'll put some output of commands and code here:
First of all of course I have to activate the virtualenv:
. bin/activate
output: None, works without problem.
Then I print the pythonpath python is aware of:
import sys
for i in sys.path:
print(i)
output:
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python34.zip
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/plat-linux
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/lib-dynload
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/plat-linux
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/site-packages
So far so good. Then I try to import a module / package I installed during usage of my bash script: pandas:
python
(IDLE is running)
import pandas as pd
output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'pandas'
Another try:
import numpy as np
output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'numpy'
Huh? So none of the packages is available? Lets check pip again:
which pip
output:
alias pip='localpython/bin/pip3.4'
./localpython/bin/pip3.4
Ok, so it's using my local pip.
Check packages:
pip list
output:
numpy (1.11.0)
pandas (0.18.1)
pip (8.1.2)
psutil (4.1.0)
Python-contrib-nbextensions (alpha)
python-dateutil (2.5.3)
pytz (2016.4)
PyYAML (3.11)
setuptools (18.2)
six (1.10.0)
virtualenv (15.0.1)
Hm the packages are there, so why can't python find them? Let's see where those packages are located, simply by trying to remove one:
pip uninstall pandas
output (shortened, because it fills many pages):
Uninstalling pandas-0.18.1:
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas-0.18.1-py3.4.egg-info
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__init__.py
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/__init__.cpython-34.pyc
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/_version.cpython-34.pyc
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/info.cpython-34.pyc
Aha, so the packages are in the path.
Another attempt on finding out if python looks in the right places:
>>> from distutils.sysconfig import get_python_lib
>>> print(get_python_lib())
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/site-packages
So this one indicates it does not look in the right place, if I understand correctly.
Note:
The script does the following things, which might not be obvious:
compile python with a prefix (local python)
compile virtualenv for the local python
it aliases pip to the local pip of the version of the installed python
it aliases virtualenv to the locally installed one
it installs packages from pypi if specified
it updates the local pip if there is a newer version available
I am a beginner still at writing bash scripts, so I think the structure and logic of the script is fairly easy to understand. It also prints information about success of its operations in the terminal.
Further Notes:
I did not use su or sudo to run the script.
My OS is a Fedora 22, although I think in this case any major distro would work the same way.
OS has been updated recently.
Question: So why can't Python find them / its own packages? (What do I need to change?)
can you try to compare path to pyhton and pip in your script and manually in your system:
which python and which pip?
After that check pip freeze | grep pandas it should return you package ...if no you need to add this package to site-packages folder of you python.
I am running Windows 7 and using Python 2.7.
I have installed openpyxl using easy_install. It looks like the installation was successful. I changed the directory and fired up Python.
>>> import openpyxl
>>>
So, this should mean that Python is able to find openpyxl. However, when I execute a simple test program excell_tutorial1.py and run it, I get the following:
Traceback (most recent call last):
File "C:/Python27/playground/excell_tutorial1.py", line 7, in <module>
from openpyxl import Workbook
ImportError: No module named openpyxl
Very confusing! It could find it in prompt line but not in the program!
import os, sys
the_module ="C:\\Python27\\Lib\\site-packages\\openpyxl-2.3.3-py2.7.egg\\openpyxl"
if the_module not in sys.path:
sys.path.append(the_module)
if the_module in sys.path:
print sys.path.index(the_module)
print sys.path[18]
so, this gives me:
18
C:\Python27\Lib\site-packages\openpyxl-2.3.3-py2.7.egg\openpyxl
Anyone can think of what the problem might be?
Much appreciated
I had the same problem solved using instead of pip or easy install one of the following commands :
sudo apt-get install python-openpyxl
sudo apt-get install python3-openpyxl
The sudo command also works better for other packages.
While not quite what you ran into here (since you state that you are using python 2.7), for those who run into this issue and are using python 3, you may be unintentionally installing to python 2 instead. To force the install to python 3 (instead of 2) use pip3 instead.
See this thread for more info:
No module named 'openpyxl' - Python 3.4 - Ubuntu
Try deleting all openpyxl material from C:\Python27\Lib\site-packages\
Once you do that try reinstalling it using pip. (This what worked for me)
At times this can be a simple permission issue. As it was in my case. I installed it in my local directory with my login.
python ./setup.py install
but some of the other user were not able to import the module.
They were getting this error:
ImportError: No module named openpyxl
Hence I simply gave exe permission to 'others'
chmod -R 755
That solves the problem at least in my case.
Go to the directory where pip is installed, for eg.C:\Python27\Scripts and open cmd (simply type cmd in address bar ). Now run the command "pip install openpyxl". It will install the package itself. Hope this will solve your problem.
Try this:
!pip install openpyxl
I had the same issue on 3.8.2
I found out that python was installed in two locations on my machine (probably py and python, just a guess)
Here:
C:\Users<userAccount>\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8\LocalCache\local-packages\Python38
and Here:
C:\Python38
I deleted the one in my C drive and everything is working well now. I would double check to see where your packages are getting installed first, before deleting. Which ever one is being used, keep that one.
For this case, check to see where this package got installed:
C:\Users\<userAccount>\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8\LocalCache\local-packages\Python38\site-packages\openpyxl
keep that directory.
What worked for me was to open the terminal as an administrator, cd to the 'scripts' file of where python (different for each version) is stored, and then install using pip:
cd C:\Users\Salfa\AppData\Local\Programs\Python\Python39\Scripts
pip install openpyxl
This resolved the problem for me.
Please only respond to this post if you use Python on Windows, rather than Mac or Linux.
Error message:
>>> import numpy
Traceback (most recent call last):
File "stdin", line 1, in <module>
ImportError: No module named 'numpy'
Questions:
Regarding windows 7, python 3.4.3 and numpy-1.9.2, are there any
conflicts that would prevent these from working together?
Into which directory (please write out the complete directory path
starting with "C:") should I extract the contents of the numpy zip
file?
What is the exact command that I need to type into python command
prompt in order to install numpy?
Thanks in advance for your assistance
I was also facing this issue where in I tried using
import numpy
But it has given me error "ImportError: No module named 'numpy'"
I installed numpy using "C:\user>python -mpip install numpy" and it was successfully installed.
However I was again getting the same error
Then I checked that the path where in numpy was intalled was not listed in
import sys
print(sys.path)
Then I appended my path wherein numpy was installed using ">>> sys.path.append(r"C:\Users\xxxx\AppData\Roaming\Python\Python38\site-packages").
The above command worked fine, still problem not resolved then I restarted my python session again.
Finally it worked..!!!!(As after every append we have to restart our python session).
1) No there isn't.
2) You can use pip. pip install numpy
If you don't have pip installed, install it, its the most often used way of installing python packages. Yes it is possible to do under windows.
3) Once you have the python command prompt, you already have python installed.
1) no
2) It doesn't actually matter as pip or the installer will figure that out for you. However, just so you know, pip or whatever will install numpy into your site-packages folder.
3) I recommend downloading the NumPy installer from SourceForge:
http://sourceforge.net/projects/numpy/files/NumPy/1.9.2/
Then you don't have to worry about having the right compiler installed. However, if you want to use pip, then you'll need to install the appropriate compiler. This is documented at the following locations:
Microsoft Visual C++ Compiler for Python 3.4
https://matthew-brett.github.io/pydagogue/python_msvc.html
I'm not even close to an expert on Python. That said, I like the notion of "keep it simple." I chose to install a Python distribution that already includes numpy. Specifially I installed Python(x,y). Everything seems to be working just fine right after install. I appreciate all those who commented on my question. Thanks
I'm not sure if the default python installation is the one that I've been installing modules to, and if that may be the cause of a conflicting Unicode byte size compatibility error. In short, I've installed Numpy 1.7 using Python 2.7.3 and when I try to install this other program that uses Python and Numpy as dependencies, I get this error:
Traceback (most recent call last):
File "setup.py", line 20, in <module>
from weblogolib import __version__
File "/home/chris/Documents/IS/Bioinformatics-Software/weblogo-3.3/weblogolib/__init__.py", line 108, in <module>
from numpy import array, asarray, float64, ones, zeros, int32,all,any, shape
File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 137, in <module>
import add_newdocs
File "/usr/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 9, in <module>
from numpy.lib import add_newdoc
File "/usr/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 4, in <module>
from type_check import *
File "/usr/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 8, in <module>
import numpy.core.numeric as _nx
File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 5, in <module>
import multiarray
ImportError: /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so: undefined symbol: PyUnicodeUCS4_AsUnicodeEscapeString
So I guess I have a conflicting unicode byte size (2-byte vs. 4-byte). I went to check to see if I had conflicting versions of Python that could be messing this up.
python --version
Python 2.7.3
But this seems at odds with
which python
/usr/local/bin/python
When I go to /usr/local/bin I find these files (relevant to python):
python
python2
python2.7
python-config
python2-config
python2.7-config
Now I've installed numpy into the dist-packages directory of /usr/lib/python2.7/dist-packages which corresponds to what I get for python --version. But the fact that when I try which python and get a directory for python and not python2.7 concerns me that this might be conflicting when I try to install the program that uses python and numpy as dependencies.
So I guess to clarify my question(s): Are these normal files to find for a python installation or have I somehow installed three different versions? Could they be causing my error with the unrecognized symbol? Is there a way to uninstall if they are indeed extraneous versions?
Thanks for any help you can provide!
Oh and here is a link to a previous question I had, where I edited the PYTHONPATH while trying to fix an ImportError I was getting, if that might be affecting things....ImportError: No module named numpy
Here are the results of trying virtualenv:
chris#ubuntu:~/Documents/IS/Bioinformatics-Software$ virtualenv weblogo-3.3
New python executable in weblogo-3.3/bin/python
Installing setuptools.............done.
Installing pip...............done.
chris#ubuntu:~/Documents/IS/Bioinformatics-Software$ cd weblogo-3.3
chris#ubuntu:~/Documents/IS/Bioinformatics-Software/weblogo-3.3$ source bin/activate
(weblogo-3.3)chris#ubuntu:~/Documents/IS/Bioinformatics-Software/weblogo-3.3$ pip install numpy
Requirement already satisfied (use --upgrade to upgrade): numpy in /usr/lib/python2.7/dist-packages
Cleaning up...
The problem indeed seems to be a mismatch of Python and Numpy compile settings.
/usr/local/bin is where custom Python is installed, you should try to run using /usr/bin/python instead.
Another solution is to use a virtualenv. Try this:
virtualenv myproject
cd myproject
source bin/activate
pip install numpy
Basically virtualenv sets up a different Python installation with its own packages in the "myproject" directory. Running the "activate" command tells the system that you want to use this installation instead of the default system. This lets you have a different Python environment for different projects. Using virtualenv, each project can have its own versions of Python packages even if they're incompatible with other projects or system packages.
Note you will have to repeat the "source" command each time you open a new shell and want to use that virtual environment. Also you might have to install the virtualenv command by using your OS package manager. If this isn't possible (e.g. you don't have root access) or your OS version is too old for some reason, you can also download it manually from https://pypi.python.org/packages/source/v/virtualenv/
If you do ls -l /usr/local/bin/python* you should see that python and python2 are actually symlinks to python2.7, and likewise python-config and python2-config are symlinks to python2.7-config.
What OS are you on? This is more a question for superuser, but try something like this. Ditch easy_install and use pip if you haven't already.
On Ubuntu:
sudo apt-get install python-setuptools
sudo easy_install pip
pip install --user numpy