I'm trying to use virtualenv in my new mainly Python project. The code files are located at ~/Documents/Project, and I installed a virtual environment in there, located at ~/Documents/Project/env. I have all my packages and libraries I wanted in the env/bin folder.
The question is, how do I actually run my Python scripts, using this virtual environment? I activate it in Terminal, then open idle as a test, and try
"import django"
but it doesn't work. Basically, how can I use the libraries install in the virtual environment with my project when I run it, instead of it using the standard directories for installed Python libraries?
Check out the example below, and also the virtualenv documentation. It's actually fairly straightforward if you follow the steps:
virtualenv Project # creates a new Project dir
cd Project/bin # could just call Project/bin
. activate # should now have (Project) in the prompt name
pip install django # without this, won't be able to import django
deactivate # switch of virtual mode
I tried the above out in my Mac and worked fine. Here's a transcript for reference.
Transcript of operations
[MacMini]<Documents> :virtualenv Project
[MacMini]<Project> :cd bin/
[MacMini]<bin> :python2.7
Python 2.7.9 (v2.7.9:648dcafa7e5f, Dec 10 2014, 10:10:46)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named django
>>> quit()
[MacMini]<bin> :. activate
(Project)[MacMini]<bin> :pip install django
You are using pip version 6.0.6, however version 8.1.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting django
Downloading Django-1.9.4-py2.py3-none-any.whl (6.6MB)
100% |################################| 6.6MB 1.2MB/s
Installing collected packages: django
Successfully installed django-1.9.4
(Project)[MacMini]<bin> :python
Python 2.7.10 (default, Oct 23 2015, 18:05:06)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> quit()
(Project)[MacMini]<bin> :deactivate
It's also good practice to make a requires.txt file for all your dependencies. If for example your project requires Flask and pymongo, create a file with:
Flask==<version number you want here>
pymongo==<version number you want here>
Then you can install all the necessary libraries by doing:
pip install -r requires.txt
Great if you want to share your project or don't want to remember every library you need in your virtualenv.
Related
This is the first time I've tried developing a package rather than just scripts and a lot of it is very confusing to me. I am using Windows 10 with Python 3.7 installed and managed via Conda 4.7.11.
TLDR: Conda packages aren't properly isolated from each other when using pip install -e.
I have a basic package structured like this:
----my_package
| LICENSE
| README.md
| setup.py
|
|---my_package
my_module.py
__init__.py
In my_module.py is:
def cool_thing():
print("This is a cool thing!")
In __init__.py is:
from .my_module import cool_thing
So when I run Python from the outer my_package directory, I can do this:
$ python
Python 3.7.4 (default, Aug 9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)] :: Ana
conda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import my_package
>>> my_package.cool_thing()
This is a cool thing!
So far, so good. Now I want to install the package locally to a virtual Conda environment. My setup.py file has this inside:
from setuptools import setup
setup(name = "my_package",
version = "0.1",
description = "Do something cool!",
author = "Me",
license = "MIT",
packages = ["my_package"],
zip_safe = False)
In the terminal I do this:
conda create -n env-my-package
conda activate env-my-package
conda install pip
I read in a lot of places that this is how you're supposed to install local development packages:
pip install -e .
It tells me:
Obtaining file:///C:/long/path/to/my_package
Installing collected packages: my-package
Running setup.py develop for my-package
Successfully installed my-package
pip list tells me it's installed:
$ pip list
Package Version Location
------------ --------- --------------------------
certifi 2019.6.16
my-package 0.1 c:\long\path\to\my_package
pip 19.2.2
setuptools 41.0.1
wheel 0.33.4
wincertstore 0.2
And if I navigate to a different directory I can still import it in Python:
cd ..
python
$ python
Python 3.7.4 (default, Aug 9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import my_package
>>> my_package.cool_thing()
This is a cool thing!
So this seems fine. But then look at what happens when I switch to a different Conda environment:
conda deactivate
conda activate env-something-else
python
Python 3.7.4 (default, Aug 9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import my_package
The package somehow gets imported anyway, without an error message, despite the fact that we are in a totally separate Conda environment without my_package installed. On the other hand, actually trying to use the package results in an error:
>>> my_package.cool_thing()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'my_package' has no attribute 'cool_thing'
Investigating further, dir(my_package) shows it has no non-dunder attributes:
>>> dir(my_package)
['__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']
my_package.__file__ outputs None, while my_package.__path__ gives the path to the directory where the original files are: C:\long\path\to\my_package.
So it seems like the Conda environments aren't properly isolated. It can still see the package when it's not supposed to, even if it can't use functions from it. Before you ask, yes, I did make sure to run pip install -e with the original env-my-package environment activated.
This has made me completely distrust that what I'm doing in my Conda environment stays in my Conda environment. If it "leaks" in this way, what else might leak? I don't want to accidentally pollute any of my other projects while working on this package. Thankfully I'm able to uninstall it using pip and return to a clean state, but still, I can't go any further until I figure this out. My ultimate question:
What is the proper way to develop a package when using Conda?
I hope you'll forgive me for writing this all out to the point of tediousness, but I wanted to make sure I covered all the details just in case any are important.
I had python version 2.7.3 and i wanted to learn django so i installed django version 1.8.2 on my ubuntu 12.0.4 .
invivtus#invictus:~/bin$ python
Python 2.7.3 (default, Sep 26 2013, 20:08:41)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> django.VERSION
(1, 8, 2, 'final', 0)
Then i read that best way yo work with django is to work on python version 3.3 so i installed python version 3.3.6 on my system where py is symbolic link pointing to /opt/python3.3/bin/python3.3
invictus#invictus:~/bin$ py
Python 3.3.6 (default, Jun 21 2015, 16:13:35)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
when I try and import django here i get error
>>> import django
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'django
I see django got installed my python 2.7 directory.
>>> import django
>>> django
<module 'django' from '/usr/local/lib/python2.7/dist-packages/django/__init__.pyc'>
How can i use this django with my 3.3.6 version. My default python version is 2.7.3
What are possible workout here?
Yes, each Python version has its own folder with installed packages. You'll have to install Django separately for Python 3.3. The same is true for every package that is not available by default.
(If you're using Python 3, why not go for the latest and greatest, 3.4?)
As some of the comments said, you should be using a virtualenv to isolate your environments. You would do it like this:
1) Ensure you have virtualenv installed. On Ubuntu for instance, that would be package virtualenv.
2) Create a new, empty environment. You choose which python version it will be like this:
virtualenv -p /usr/bin/python3.4 env
3) That created an env folder. Activate the newly created environment:
. env/bin/activate
This updates your paths so now, when you run python or pip from this shell, they will execute in the context of your virtualenv.
4) Update the virtualenv (optional)
pip install -U pip
5) Install whatever packages you need. The recommended way is to have a requirements.txt file at the root of your project. You would pull them this way:
pip install -r myproject/requirements.txt
That's it. Use the pip command as usual. As long as you're working with the virtualenv active, your python command will only see the modules you explicitly install in it.
6) Don't forget to re-run . env/bin/activate in every new shell. If you think you'll probably forget, you can add this to your manage.py:
import sys
if __name__ == "__main__":
if not hasattr(sys, 'real_prefix'):
sys.stderr.write('Running outside of any virtualenv - did you forget to activate one?\n')
What are the benefits?
You have an isolated environment for every project (no conflicts).
You may use different versions of the same module in different projects.
System updates will not break your project.
You are not polluting your system with unmanaged files.
You never run stuff as root, which means both added isolation, and the possibility of running your project without having root access to the system.
As long as you keep your requirements.txt up to date (using pip freeze), you can rebuild the virtualenv on another system and it will work.
[edit: using requirements.txt]
That's just a file that has pip install specifications, one by line. It allows to rebuild the virtualenv from scratch easily. You can generate it from your current virtualenv using:
pip freeze > requirements.txt
So the idea is just to remember to re-run this command everytime you change your environment (installing, removing or upgrading some package).
I am trying to use twisted on OS X Mavericks, but I get this error message when I try to import it.
christohersmbp2:~ christopherspears$ python
Python 2.7.6 (default, Mar 8 2014, 09:29:01)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import twisted
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named twisted
My guess is that I am receiving this error because I am not using the default Python. I am using a Python installed with brew. Ideally, I would want to install twisted into a virtual environment to play with, but the docs are seriously lacking in details. Apparently, a dmg exists for Mac OS X 10.5, which is not helpful for me. I can install from the tarball into the virtual environment, but I am not sure how to do this. Any hints?
If you're using virtualenv, it doesn't matter whether you are using the system python or not.
Simply pip install twisted in your virtualenv, like:
$ workon MyTwistedEnv
$ pip install twisted
Currently, due to a bug in Xcode that affects all projects which contain extension modules, you may need to spell this
$ CFLAGS= pip install twisted
instead; hopefully this will be fixed soon. It may not affect brew installed Pythons, however.
I thought I'd move from using Tkinter to wxPython, but I'm having some troubles. All I get is this:
>>> import wx
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named wx
I have installed wxPython. When I run the uninstaller I clearly see that wxPython IS installed:
1. wxPython3.0-osx-cocoa-py2.7 3.0.0.0
Enter the number of the install to examine or 'Q' to quit:
When I start Python I see that my version should match the version of wxPython:
Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
I'm on OS X 10.7.5 32-bit.
Any help would be awesome! :)
Macs can have multiple versions of Python installed. Are you sure that you installed wxPython for the same python you invoke with the interpreter?
Try, which python, and make sure that this version of python has a wxredirect.pth file in site-packages pointing to the wxPython installation. (If it doesn't search for wxredirect.pth.)
Here's one version on my system...
> which python2.6
/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6
> more /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/wxredirect.pth
import site; site.addsitedir('/usr/local/lib/wxPython-3.0.0.0/lib/python2.6')
#import site; site.addsitedir('/usr/local/lib/wxPython-2.9.1.1/lib/python2.6')
#import site; site.addsitedir('/usr/local/lib/wxPython-unicode-2.8.12.1/lib/python2.6')
Here, btw, I can comment out lines in the wxredirect.pth to choose the version of wxPython I want to use.
I just find the same problem.
I used brew install wxpython to install it on mac.
I try the method above but no use.
I happen to find the solution when I install another package, it showen below:
brew install tbb
Downloading https://homebrew.bintray.com/bottles/tbb-2017_U7.sierra.bottle.t
############################################################ 100.0%
==> Pouring tbb-2017_U7.sierra.bottle.tar.gz
==> Caveats
Python modules have been installed and Homebrew's site-packages is not
in your Python sys.path, so you will not be able to import the modules
this formula installed. If you plan to develop with these modules,
please run:
mkdir -p /Users/ningrongye/.local/lib/python2.7/site-packages
echo 'import site; site.addsitedir("/usr/local/lib/python2.7/site-
packages")' >> /Users/ningrongye/.local/lib/python2.7/site-
packages/homebrew.pth`
this is what homebrew said and I just try those, and it works.
ningrong
I have Python 2.7 and 3.7 .
In /usr/local/bin/ there are symbolic links for 2.7 and 3.7 Python versions and also symbolic links for pip.
I've installed wxPython with pip3
pip3 install -U wxPython
Then i checked the installation for Python3
myname$ python3
Python 3.7.2 (v3.7.2:9a3ffc0492, Dec 24 2018, 02:44:43)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
>>> wx.version()
'4.0.4 osx-cocoa (phoenix) wxWidgets 3.0.5'
Antonio
In my case it worked by resetting the brew environment to 2.7:
brew link --overwrite python#2
I have build my virtual env with this command:
virtualenv env --distribute --no-site-packages
And then I have installed several modules (django etc) into env with pip, the problem is that when I wanted to run the code on the second machine it would not work, here is what I have done:
visgean#rewitaqia:~/scripty/project_name$ source ./env/bin/activate
(env)visgean#rewitaqia:~/scripty/project_name$ python
Python 2.7.1+ (r271:86832, Sep 27 2012, 21:12:17)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> django.__file__
'/home/visgean/scripty/project_name/env/lib/python2.7/site-packages/django/__init__.pyc'
but when I want to run it on the second machine:
(env)user#debian:~/project_name$ python
Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named django
>>>
I wild error appears! The first machine is ubuntu, the second one is ubuntu. There seems to be some broken links in /home/user/project_name/env/lib/python2.7 , is that the problem? and if so, how should I prevent it/repair it?
If you are just copying the virtualenv to the second machine you may encounter some issues. From the virtualenv site:
Normally environments are tied to a specific path. That means that you
cannot move an environment around or copy it to another computer. You
can fix up an environment to make it relocatable with the command:
$ virtualenv --relocatable ENV
This will make some of the files
created by setuptools or distribute use relative paths, and will
change all the scripts to use activate_this.py instead of using the
location of the Python interpreter to select the environment.
Note: you must run this after you've installed any packages into the
environment. If you make an environment relocatable, then install a
new package, you must run virtualenv --relocatable again.
I have just noticed that I did have a wrong version of python on the second machine - debian does not have python2.7, installing 2.7 and pip for is the solution