I have a package that I would like to automatically install and use from within my own Python script.
Right now I have this:
>>> # ... code for downloading and un-targzing
>>> from subprocess import call
>>> call(['python', 'setup.py', 'install'])
>>> from <package> import <name>
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named <package>
Then I can continue like this:
>>> exit()
$ python
>>> from <package> import <name>
And it works just fine. For some reason, Python is able to pick up the package just fine if I restart after running the setup.py file, but not if I don't. How can I make it work without having the restart step in the middle?
(Also, is there a superior alternative to using subprocess.call() to run setup.py within a python script? Seems silly to spawn a whole new Python interpreter from within one, but I don't know how else to pass that install argument.)
Depending on your Python version, you want to look into imp or importlib.
e.g. for Python 3, you can do:
from importlib.machinery import SourceFileLoader
directory_name = # os.path to module
# where __init__.py is the module entry point
s = SourceFileloader(directory_name, __init__.py).load_module()
or, if you're feeling brave that your Python path knows about the directory:
map(__import__, 'new_package_name')
Hope this helps,
I downloaded from seaborn from GitHub.
Through command prompt, cd to downloads\seaborn folder
python install setup.py
Then using spyder from anaconda, checked if it was installed by running the following in a console
import pip
sorted(["%s==%s" % (i.key, i.version)
for i in pip.get_installed_distributions()])
Seeing that it was not there, go to tools and select "Update module names list"
Again trying the previous code in a python console, the lib was still not showing.
Restarting Spyder and trying import seaborn worked.
Hope this helps.
Related
When installing gcloud for mac I get this error when I run the install.sh command according to docs here:
Traceback (most recent call last):
File "/path_to_unzipped_file/google-cloud-sdk/bin/bootstrapping/install.py", line 8, in <module>
from __future__ import absolute_import
I poked through and echoed out some stuff in the install shell script. It is setting the environment variables correctly (pointing to my default python installation, pointing to the correct location of the gcloud SDK).
If I just enter the python interpreter (using the same default python that the install script points to when running install.py) I can import the module just fine:
>>> from __future__ import absolute_import
>>>
Only other information worth noting is my default python setup is a virtual environment that I create from python 2.7.15 installed through brew. The virtual environment python bin is first in my PATH so python and python2 and python2.7 all invoke the correct binary. I've had no other issues installing packages on this setup so far.
If I echo the final line of the install.sh script that calls the install.py script it shows /path_to_virtualenv/bin/python -S /path_to_unzipped_file/google-cloud-sdk/bin/bootstrapping/install.py which is the correct python. Or am I missing something?
The script uses the -S command-line switch, which disables loading the site module on start-up.
However, it is a custom dedicated site module installed in a virtualenv that makes a virtualenv work. As such, the -S switch and virtualenvs are incompatible, with -S set fundamental imports such as from __future__ break down entirely.
You can either remove the -S switch from the install.bat command or use a wrapper script to strip it from the command line as you call your real virtualenv Python.
I had the error below when trying to run gcloud commands.
File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/lib/gcloud.py", line 20, in <module>
from __future__ import absolute_import
ImportError: No module named __future__
If you have your virtualenv sourced automatically you can specify the environment variable CLOUDSDK_PYTHON i.e. set -x CLOUDSDK_PYTHON /usr/bin/python to not use the virtualenv python.
In google-cloud-sdk/install.sh go to last line, remove variable $CLOUDSDK_PYTHON_ARGS as below.
"$CLOUDSDK_PYTHON" $CLOUDSDK_PYTHON_ARGS "${CLOUDSDK_ROOT_DIR}/bin/bootstrapping/install.py" "$#"
"$CLOUDSDK_PYTHON" "${CLOUDSDK_ROOT_DIR}/bin/bootstrapping/install.py" "$#"
i try to train.py in object_detection in under git url
https://github.com/tensorflow/models/tree/master/research/object_detection
However, the following error occurs.
ModuleNotFoundError: No module named 'object_detection'
So I tried to solve the problem by writing the following code.
import sys
sys.path.append('/home/user/Documents/imgmlreport/inception/models/research/object_detection')
from object_detection.builders import dataset_builder
This problem has not been solved yet.
The directory structure is shown below.
~/object_detection/train.py
~/object_detection/builders/dataset_bulider.py
and here is full error massage
/home/user/anaconda3/lib/python3.6/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated.
In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "train.py", line 52, in
import trainer
File"/home/user/Documents/imgmlreport/inception/models/research/object_detection/trainer.py", line 26, in
from object_detection.builders import optimizer_builder
ModuleNotFoundError: No module named 'object_detection'
how can i import modules?
Try install Tensorflow Object Detection Library Packaged
pip install tensorflow-object-detection-api
Cause of this error is installing object_detection library, So one of the solution which can work is running the below command inside models/research
sudo python setup.py install
If such solution does not work, please execute the below command one by one in the directory models/research
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
sudo python setup.py install
I hope this will work. I also faced the same problem while creating model from export_inference_graph.py. It worked for me.
You need to export the environmental variables every time you open a new terminal in that environment.
Please note that there are are back quotes on each of the pwd in the command as this might not be showing in the command below. Back quote is the same as the tilde key without pressing the shift key (US keyboard).
From tensorflow/models/research/
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
try this:
python setup.py build
python setup.py install
There are a number of modules in the object_detection folder, and I have created setup.py in the parent directory(research folder) to import all of them.
from setuptools import find_packages
from setuptools import setup
REQUIRED_PACKAGES = ['Pillow>=1.0', 'Matplotlib>=2.1', 'Cython>=0.28.1']
setup(
name='object_detection',
version='0.1',
install_requires=REQUIRED_PACKAGES,
include_package_data=True,
packages=[p for p in find_packages() if p.startswith('object_detection')],
description='Tensorflow Object Detection Library',
)
You did have "sys.path.append()" before you imported the object detection, so I am surprised that you are facing this error!
Please check that the path you have used in sys.path.append() is right.
Well, the only and obvious answer for the error is that the path of the module is not added properly.
Besides the various ways mentioned here, here is a way in which you can add the "object_detection" path permanently to the PYTHONPATH variable.
If you are using a Linux system, here is how you would go about it:
Go to the Home directory. Press Ctrl + H to show hidden files. You will see a file called ".bashrc". Open this file using a code editor (I used Visual Studio).
In the last line of .bashrc file, add the line:
export PYTHONPATH=/your/module/path:/your/other/module/path:your/someother/module/path
Then press "save" in the code editor. Since ".bashrc" is a "Read-only" file the editor will throw a pop-up saying the same. Also in the pop-up there will be an option that says: "Try with sudo". Hit this button and now you are good to go.
All your modules are now permanently added to the PYTHONPATH. This means that you need not run sys.path.append every time you open your terminal and start a session!
Below is the screenshot with no error when I followed the said steps:
Try this. I hope it helps.
And finally, If you've followed all the steps here and are at your wit's end...make sure the file that you're running (the one with your source code in it ya know), isn't named object_detection.py - that would preclude it being searched for as a module.
Certainly I've never done anything like this that led me to add an embarrassing answer on Stack Overflow...
I had to do:
sudo pip3 install -e . (ref)
sudo python3 setup.py install
System:
OS: Ubuntu 16.04, Anaconda (I guess this is why I need to use pip3 and python3 even I made virtual environment with Pyehon 3.8)
I've created the following package tree:
/main_package
/child_package
version.py
where version.py contains a single string variable (VERSION)
Inside my script in child package I'm importing version.py by the following line:
from main_package.version import VERSION
While I'm running the code from PyCharm everything works great, however when I'm running the code via the command line I'm getting the following error message:
C:\Users\usr\PycharmProjects\project\main_package\child_package>python script.py
Traceback (most recent call last):
File "script.py", line 2, in <module>
from main_package.version import VERSION
ModuleNotFoundError: No module named 'main_package'
I've found in the internet that I might need to add my package to the python path, however it doesn't seems to work for me
PyCharm sets the Python Path at the root of the project (by default). To mimic this in a quick'n'dirty fashion, you just need to do this once in your shell session before invoking python whatever:
set PYTHONPATH=C:\Users\usr\PycharmProjects\project
The pythonic way is to have a setup.py file to install your project in the system (check python Minimal Structure):
from setuptools import setup
setup(name='main_package',
version='0.1',
description='main package',
license='MIT',
packages=['main_package'],
zip_safe=False)
Then you install it as follow:
python setup.py install for global installation
OR
python setup.py develop for local installation in editable mode
when I run mitmproxy command in command line, I get the following error.
% mitmproxy
Traceback (most recent call last):
File "/usr/local/bin/mitmproxy", line 7, in <module>
from libmproxy.main import mitmproxy
File "/usr/local/lib/python3.5/site-packages/libmproxy/main.py", line 5, in <module>
import thread
ImportError: No module named 'thread'
I googled this error and found this stackoverflow Q&A page.
pydev importerror: no module named thread, debugging no longer works after pydev upgrade
according to the page above, the error occurs because module "thread" is renamed to "_thread" in python3.
So, I know what's causing this error, but then what?
I don't know what to do now in order to get rid of this error.
I'm new to python. I've just installed Python and pip into my mac OSX as shown below because I want to use mitmproxy.
% which pip
/usr/local/bin/pip
% pip --version
pip 8.1.1 from /usr/local/lib/python3.5/site-packages (python 3.5)
% which python
/usr/bin/python
% which python3
/usr/local/bin/python3
% python --version
Python 2.7.10
% python3 --version
Python 3.5.1
could anyone please tell me what to do now?
Additional Info
As #linusg answered, I created "thread.py" file in "site-packages" directory and pasted the code below in "thread.py"
from _thread import *
__all__ = ("error", "LockType", "start_new_thread", "interrupt_main", "exit", "allocate_lock", "get_ident", "stack_size", "acquire", "release", "locked")
After I did this, "ImportError: No module named 'thread'" disappeared, but now I have another ImportError, which is "import Cookie ImportError: No module named 'Cookie'".
It seems that in Python 3, Cookie module is renamed to http.cookies (stackoverflow.com/questions/3522029/django-mod-python-error).
Now what am I supposed to do?
What I have in "site-packages" directory
% ls /usr/local/lib/python3.5/site-packages (git)-[master]
ConfigArgParse-0.10.0.dist-info/ mitmproxy-0.15.dist-info/
OpenSSL/ netlib/
PIL/ netlib-0.15.1.dist-info/
Pillow-3.0.0.dist-info/ passlib/
PyYAML-3.11.dist-info/ passlib-1.6.5.dist-info/
__pycache__/ pathtools/
_cffi_backend.cpython-35m-darwin.so* pathtools-0.1.2.dist-info/
_markerlib/ pip/
_watchdog_fsevents.cpython-35m-darwin.so* pip-8.1.1.dist-info/
argh/ pkg_resources/
argh-0.26.1.dist-info/ pyOpenSSL-0.15.1.dist-info/
backports/ pyasn1/
backports.ssl_match_hostname-3.5.0.1.dist-info/ pyasn1-0.1.9.dist-info/
blinker/ pycparser/
blinker-1.4.dist-info/ pycparser-2.14.dist-info/
certifi/ pyparsing-2.0.7.dist-info/
certifi-2016.2.28.dist-info/ pyparsing.py
cffi/ pyperclip/
cffi-1.6.0.dist-info/ pyperclip-1.5.27.dist-info/
click/ setuptools/
click-6.2.dist-info/ setuptools-19.4-py3.5.egg-info/
configargparse.py sitecustomize.py
construct/ six-1.10.0.dist-info/
construct-2.5.2.dist-info/ six.py
cryptography/ test/
cryptography-1.1.2.dist-info/ thread.py
easy_install.py tornado/
hpack/ tornado-4.3.dist-info/
hpack-2.0.1.dist-info/ urwid/
html2text/ urwid-1.3.1.dist-info/
html2text-2015.11.4.dist-info/ watchdog/
idna/ watchdog-0.8.3.dist-info/
idna-2.1.dist-info/ wheel/
libmproxy/ wheel-0.26.0-py3.5.egg-info/
lxml/ yaml/
lxml-3.4.4.dist-info/
In Python 3 instead of:
import thread
Do:
import _thread
You are trying to run Python 2 code on Python 3, which will not work.
As of April 2016, mitmproxy only supports Python 2.7. We're actively working to fix that in the next months, but for now you need to use Python 2 or the binaries provided at http://mitmproxy.org.
As of August 2016, the development version of mitmproxy now supports Python 3.5+. The next release (0.18) will be the first one including support for Python 3.5+.
As of January 2017, mitmproxy only supports Python 3.5+.
Go to you site-packages folder, create a file called thread.py and paste this code in it:
from _thread import *
__all__ = ("error", "LockType", "start_new_thread", "interrupt_main", "exit", "allocate_lock", "get_ident", "stack_size", "acquire", "release", "locked")
This creates an 'alias' for the module _thread called thread. While the _thread module is very small, you can use dir() for bigger modules:
# Examle for the Cookies module which was renamed to http.cookies:
# Cookies.py in site-packages
import http.cookies
__all__ = tuple(dir(http.cookies))
Hope this helps!
Easiest solution is to create a virtualenv with python2 and run mitmproxy on this virtualenv
virtualenv -p `which python2` .env
source .env/bin/activate
pip install mitmproxy
.env/bin/mitmproxy
The name of the file saved could be threading, this would give an error as threading is a predefined class in Python. Try changing the name of your file. It would help....
I have a code in python that I have been working on and it builds and runs very well on my pc (Windows). I had to run the same code on my other machine which runs ubuntu,so I had to install all the packages on prior to runing the code. The problem is I ran into this error which I couldn't figure out. The error is triggered by one of the installed packages.
from qalsadi import analex
File "/usr/local/lib/python2.7/dist-packages/qalsadi/analex.py", line 14, in <module>
import pyarabic.araby as araby # basic arabic text functions
File "/usr/local/lib/python2.7/dist-packages/pyarabic/araby.py", line 28, in <module>
from stack import *
ImportError: No module named stack
I used the following command, "sudo pip install pyarabic", to install it. However, still the file stack.py doesn't exist among it's files. I searched in the folder /usr/local/lib/python2.7/dist-packages/pyarabic. The folder contains the following: araby.py and init.py and the coresponding pyc files only. I'v insalled and uninstalled it a number of times using "pip" but still the file is not there.
Check your pyarabic folder. Usually it's in Python27\Lib\site-packages\pyarabic.
There, there should be stack.py. If it doesn't exists, re-download pyarabic and then reinstall it.
After installation of pyarabic import STACK in this manner:
from pyarabic.stack import Stack
for window users
open cmd prompt and type the following to install the stack variable to python 3.x-
pip install pyarabic
To install and run with this code-
from pyarabic.stack import Stack
It seems like stack is not part of the Python Package Index so most probably it is a script you installed manually. The problem can be that the folder containing stack.py is not on your PYTHONPATH.
Open a terminal (Ctrl+ Alt + t) and edit the .bashrc file:
sudo gedit ~/.bashrc
Add the following line:
export PYTHONPATH=$PYTHONPATH:/path/to/the/folder/of/your/module
where you should substitute the part after the : to the full path to the directory
where stack.py can be found.
I hope this helps.