My project structure looks like the following:
my-project:
dialog
utils
path.py
After install by 'pip install .', in PyCharm, I can import successfully either of the two:
from dialog.utils import path
import dialog.utils
Then I can refer to it as below:
path.module_path() // OR
dialog.utils.module_path()
It doesn't run into any problem. I can also do this in iPython's terminal. However, when I run a python through terminal:
python3 run_dialog
run_dialog.py import 'path' as shown above, it always reports this problem:
Traceback (most recent call last):
File "run_dialog.py", line 15, in <module>
import dialog.utils
ModuleNotFoundError: No module named 'dialog.utils'
So basically, I can run this code in both PyCharm and iPython terminal, but not in the project with 'python3 run_dialog'.
Why is that?
Try this
import sys
sys.path.append('my/path/to/myModule/dir')
import myModule
The Python interpreter needs to know to path to your imported module.
However, a better approach would be setting PYTHONPATH to your project directory like this
set PYTHONPATH=my/path/to/project
In my humble opinion, the Pythonic solution would be to install your package in the activated virtual environment using an "editable" install.
“Editable” installs are fundamentally “setuptools develop mode” installs.
use the pip install -e <path> option to do an editable install
(venv) $ pip install -e path/to/myproject # wherever myproject/setup.py is located
you can use pip list to confirm your package is installed into the virtualenv as an editable package, ie the path to the package is listed
$ pip list
myproject 0.1 path/to/myproject
pip 19.1.1
setuptools 41.0.1
changes saved to your package take effect each time you restart the interpreter and reload/import the package, letting you "edit" or "develop" your package
Related
I have just uploaded a Python package into pip called hqc:
https://pypi.org/project/hqc/0.0.1/
then successfully pip installed the package using:
pip install hqc
but I can't seem to import the package in Python Interpreter. When I do this, I get the following error message:
>>> import hqc
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import hqc
ModuleNotFoundError: No module named 'hqc'
What am I missing? Do I need to set the environment variable PATH or is it something else?
In the folder where setup.py is, make a folder called hqc and put an empty file called __init__.py (thats two underscores on each side). Right now, there's no file to import so python fails when importing it.
hqc-0.0.1:
hqc:
You can test the package by going to the directory where setup.py is and running
pip install .
Before trying again, be sure to uninstall
pip uninstall hqc
I had a quick look at your github repository and found that the package name, i.e. your folder name is skltemplate.
So, your import will be: (verified it to be working)
import skltemplate
And found that skltemplate package exposes the following modeules:
TemplateClassifier, TemplateTransformer and TemplateEstimator.
You can import the above by doing:
from skltemplate import TemplateClassifier
Now, if you want your package to be named as hqc, you need to rename skltemplate directory to hqc AND run the setup.py again. (You can refer this stackoverflow answer for more details).
Ensure that you increment the package version inside setup.py before running it.
For some reason when I install a package through Anaconda, it is not available in Spyder. When I execute the following command in anaconda and in spyder I get different files.
Anaconda:
import sys; sys.executable
'C:\\Users\\onp1ldy\\AppData\\Local\\conda\\conda\\envs\\deeplearning\\python.exe'
Spyder:
import sys; sys.executable
'C:\\Users\\onp1ldy\\AppData\\Local\\conda\\conda\\envs\\deeplearning\\pythonw.exe'
Can anyone help me with this? I am not sure what to do...
By Default: You can control which one of the executables will run your script. Such as, when opened from Explorer by choosing the right file_name like:
1. python.exe is a terminal-based (console) application to run and lunch CLI-Type Python-scripts.
*.py files are by default associated (invoked) with python.exe
2. pythonw.exe is a GUI-based app for lunching Graphical User interface-(No_UI_at_all_Scripts)
*.pyw files are by default associated (invoked) with pythonw.exe
TO SUMMARIZE AND COMPLEMENT MY OPINION:
First of all, Python-Binary that you're trying to run doesn't have Package installed. It does have a directory path named as package_name like torch at the search path of modules, and it is being treated as a Package namespace like for me:
torch.Tensor(5, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'torch' has no attribute 'Tensor'
For your current python binary, you need to install your package correctly. Visit Reference: Home-Page
python3.7 -m pip install http://download.pytorch.org/whl/cu80/torch-0.2.0.post3-cp35-cp35m-manylinux1_x86_64.whl
python3.7 -m pip install torchvision
More importantly, replace the pip or pip3 as the Home-Page instructions use with python3.7 -m pip;. In the end, don't forget to include python3.7 to be the full path to your Python binary.
Run this in spyder and see if you can access the package.
import subprocess
subprocess.call('pip install numpy', shell=True)
import numpy
based on your description the problem may be connected to the Python interpreter that you are using in Spyder. There is a similar issue on Stack Overflow at this url:
Issue on Anaconda and Spyder
You can try the solution proposed by Bremsstrahlung.
Hope this can help you.
Every time that I need to run my python program with:
python my_program.py
I get some error saying that some import was not found.
Some error like this:
Traceback (most recent call last):
File "graphic.py", line 1, in <module>
import matplotlib.pyplot as plt
ImportError: No module named 'matplotlib'
Than I run:
sudo python my_program.py
And every thing works just fine. How do I remove the sudo command to run my python codes?
ImportError: No module named 'matplotlib' happens when your Python doesn't find the module. sudo changes the enviornment variables; That's why.
To fix this, locate where matplotlib is installed in your computer, and verify the folder is part of your sys.path.
import sys
sys.path
['C:\\Python27\\tests', ..., ...]
Then you've two options: insert that path from your script, i.e adding a line such import sys; sys.path.append(<folder>) or configure PYTHONPATH env variable under your user appending the folder to the path.
PYTHONPATH env variable gets loaded to sys.path on startup.
Best solution to me is a general workflow for all projects: use virtualenviroment]1.
sudo pip3 install virtualenv
virtualenv myenv
source mynenv/bin/activate
Then you should install your libraries again with pip and they will be installed in your virtualenviroment, isolated from everything else.
I wrote myself a handy bash script, which solves the task of creating a virtualenv with its own compiled virtualenv and python. It aims at creating a mostly self contained virtualenv, with maybe only native libraries installed in system level if necessary, but installing all python packages and virtualenv and pip and such things inside the virtualenv.
The script can be found here.
I invoke the script as follows:
self_contained_venv.sh \
-n udacity_model_building_and_validation \
-p https://www.python.org/ftp/python/3.4.4/Python-3.4.4.tar.xz \
-v https://pypi.python.org/packages/c8/82/7c1eb879dea5725fae239070b48187de74a8eb06b63d9087cd0a60436353/virtualenv-15.0.1.tar.gz#md5=28d76a0d9cbd5dc42046dd14e76a6ecc \
-d pandas scikit-learn seaborn
Given the required packages for compiling python and virtualenv are installed on the system, the script creates a nice virtualenv. However, when I try to access any installed modules/packages from within the virtualenv, python is not able to find them. To demonstrate this, I'll put some output of commands and code here:
First of all of course I have to activate the virtualenv:
. bin/activate
output: None, works without problem.
Then I print the pythonpath python is aware of:
import sys
for i in sys.path:
print(i)
output:
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python34.zip
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/plat-linux
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/lib-dynload
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/plat-linux
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/site-packages
So far so good. Then I try to import a module / package I installed during usage of my bash script: pandas:
python
(IDLE is running)
import pandas as pd
output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'pandas'
Another try:
import numpy as np
output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'numpy'
Huh? So none of the packages is available? Lets check pip again:
which pip
output:
alias pip='localpython/bin/pip3.4'
./localpython/bin/pip3.4
Ok, so it's using my local pip.
Check packages:
pip list
output:
numpy (1.11.0)
pandas (0.18.1)
pip (8.1.2)
psutil (4.1.0)
Python-contrib-nbextensions (alpha)
python-dateutil (2.5.3)
pytz (2016.4)
PyYAML (3.11)
setuptools (18.2)
six (1.10.0)
virtualenv (15.0.1)
Hm the packages are there, so why can't python find them? Let's see where those packages are located, simply by trying to remove one:
pip uninstall pandas
output (shortened, because it fills many pages):
Uninstalling pandas-0.18.1:
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas-0.18.1-py3.4.egg-info
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__init__.py
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/__init__.cpython-34.pyc
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/_version.cpython-34.pyc
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/info.cpython-34.pyc
Aha, so the packages are in the path.
Another attempt on finding out if python looks in the right places:
>>> from distutils.sysconfig import get_python_lib
>>> print(get_python_lib())
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/site-packages
So this one indicates it does not look in the right place, if I understand correctly.
Note:
The script does the following things, which might not be obvious:
compile python with a prefix (local python)
compile virtualenv for the local python
it aliases pip to the local pip of the version of the installed python
it aliases virtualenv to the locally installed one
it installs packages from pypi if specified
it updates the local pip if there is a newer version available
I am a beginner still at writing bash scripts, so I think the structure and logic of the script is fairly easy to understand. It also prints information about success of its operations in the terminal.
Further Notes:
I did not use su or sudo to run the script.
My OS is a Fedora 22, although I think in this case any major distro would work the same way.
OS has been updated recently.
Question: So why can't Python find them / its own packages? (What do I need to change?)
can you try to compare path to pyhton and pip in your script and manually in your system:
which python and which pip?
After that check pip freeze | grep pandas it should return you package ...if no you need to add this package to site-packages folder of you python.
I have an assignment to read excel data in Python. I have Python 2.7 installed. I tried installing xlrd0.8.0 with the following commands in Windows.
C:\Python27\xlrd-0.8.0>python setup.py build
running build
running build_py
creating build
creating build\lib
creating build\lib\xlrd
copying xlrd\biffh.py -> build\lib\xlrd
....
C:\Python27\xlrd-0.8.0>python setup.py install
running install
running build
running build_py
running build_scripts
running install_lib
running install_scripts
running install_egg_info
Writing C:\Python27\Lib\site-packages\xlrd-0.8.0-py2.7.egg-info
I don't get any error message while installing. I also see xlrd-0.8.0 folder in site-packages in /lib folder...
But when I try to import it, Python is not able to recognize it...
>>> import xlrd
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
import xlrd
ImportError: No module named xlrd
Can you suggest how to find the issue?
How to reproduce and fix this error:
Open the python interpreter, try to import xlrt, you get an error:
python
>>> import xlrt
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named xlrt
1. Install, or make sure pip is installed:
What is the official "preferred" way to install pip and virtualenv systemwide?
2. Install xlrd
pip install xlrd
Protip: If you feel you have to use sudo pip install .... to get this to work then you need to stop and learn why this is dangerous. See: What are the risks of running 'sudo pip'?
Workarounds are to install pip using a certain user: pip install --user myuser ... use your own best judgment here. Make sure the directory your pip is operating in is owned by the user trying to install packages there. Use: chown -R $USER /Library/Python/2.7/site-packages.
3. Test it on the python interpreter:
python
>>> import xlrd
>>> type(xlrd)
<type 'module'>
>>>
Now it is imported it without a problem, xlrd is a python module.
Troubleshooting:
If your PYTHONPATH is not defined, you should define it:
PYTHONPATH=:/home/el/wherever/where_to_find_modules
Resolving issue with xlrd import in Python 2.7
open this link https://bootstrap.pypa.io/get-pip.py and save as get-pip.py and copy this file into C:\Python2.7\
C:\Python2.7\python.exe get-pip.py
After this pip is installed in your system now installing xlrd
C:\Python2.7\python.exe -m pip install xlrd
Open python and import xlrd
import xlrd
it will work.
For me, running python in spyder on a mac, it didn't work even after I installed xlrd using pip, because it was installed to a different location than the one spyder was using. To fix this, I first found where xlrd was installed to:
$pip install xlrd
Requirement already satisfied: xlrd in /usr/local/lib/python2.7/site-packages
Then copied the xlrd folder from there into where Spyder could access it:
$cd /Applications/Spyder.app/Contents/Resources/lib/python2.7/
$cp -r /usr/local/lib/python2.7/site-packages/xlrd.
Then updated the module within spyder, but I'm not sure whether this was necessary. Restarting Spyder might have also worked after making those changes.
If you're using conda,
conda install xlrd
Please add "C:\Python27\Lib\site-packages\" to your PYTHONPATH in your system variables.
If there is no such SYSTEM VARIABLE, please create it:
Right-click the My Computer icon on your desktop and select Properties.
Click the Advanced tab, then click the Environment Variables button.
Under System Variables, click New.
Enter the variable name as PYTHONPATH.
Enter the variable value as C:\Python27\Lib\site-packages\
Click OK.
Click Apply Changes.
I had the same problem, seems like the is better if you export your xlsx to a csv file and then run the following on python
df = pd.read_csv('FileName.csv')
It should work like that. If you're using iPython or even better Jupyter then run df.head() to check if pandas reads your table properly.
Note, I am using Ubuntu
python -m pip install xlrd
For my case I have both python2 and python3 installed, pip install xlrd default install xlrd into the Python3 library (/usr/local/lib/python3.6/site-packages/xlrd/. By specifying python2 for pip install solved my problem.