I'm getting familiar with nix by using it instead of brew on my Mac. I like it pretty much so far, but there is an issue with python environment. I can't figure out how to configure it properly.
I'm using powerline status, which is a python package itself, but also available as derivation (nixpkgs.python37Packages.powerline). So, I installed it together with python and some other packages:
# file: python.nix
with import <nixpkgs> {};
python3.withPackages (ps: with ps; [ pip pipenv powerline ipython ])
This is working as expected.
However, I'm also using some additional segments for powerline (ex. powerline-gitstatus), which are not available as derivations. So, I just installed them via pip install --user .... And then set PYTHONPATH=$HOME/.local/lib/python3.7/site-packages.
This is also working.
Apart from that I need to have awscli, which is a python package and available as standalone derivation (not nixpkgs.python37Packages.awscli), so I can't install it as a part of my custom python derivation, as far as I understand.
I tried to get it with nix-env -i awscli, but it didn't work together with set PYTHONPATH.
I tried to get it with pip install --user awscli, but it didn't work also.
In both cases it crashed because of lack of dependencies or version incompatibilities.
So, my question is: how to setup global python environment properly with nix?
I guess following well-documented very convenient approach could help in your case also.
How to consume python modules using pip in a virtualenv like I am used to on other Operating Systems ?
10.09.2020 Update:
Developing Python with Poetry & Poetry2nix: Reproducible flexible Python environments describes an other working with python on NixOS approach.
Related
I am trying to check what packages are installed in my virtual environments, I have different packages from different virtual environments depends on what project I am working on. Whatever virtual environment I switch I always get this error when using help("modules")
aiohttp dependency is not installed: No module named 'aiohttp'. Please re-install black with the '[d]' extra install to obtain aiohttp_cors: `pip install black[d]`
My question is how do I install this package correctly, and by that I mean should I install it in my main Python PATH? I'm not sure if this is a mandatory package that I need it installed in my main environment if that makes sense.
My Python version is 3.9.6 by the way.
EDIT: I was able to fetch all of my packages.
I had to do a . Scripts/activate first, but it was weird because before I did not have to do the activate command before, since VS Code displays on the lower left portion of the screen what environment you are currently using, I'm not sure if I had any settings change prior to this bug.
Maybe you should try to open new terminal
Being a "normal" user (i.e. not overly familiar with code), I find myself in a bit of a python conundrum and would very much like to try to muster some kind of assistance with the following:
I use svtplay-dl and youtube-dl via homebrew in Mac OSX 10.14.6, which also requires python (among many other extensions such as ffmpeg et al). However, I now seem to have three different python versions installed, and since python3.5 is no longer supported, I thought I would delete this and make sure that homebrew/svtplay-dl/youtube-dl uses a path to python3.9.
When I type "which" followed by the following in Terminal:
python
python3.5
python3.9
...I get the following respective results:
/usr/local/bin/python
/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5
/usr/local/bin/python3.9
Based upon this short background info, can someone advise how I best uninstall/remove python3.5 and get homebrew et al to use python3.9 every time python is required?
Any help with this would be very much appreciated!
/Mark
The normal answer is to use something like virtualenv (https://virtualenv.pypa.io/en/latest/) to isolate your python environment and dependencies locally.
You need to install virtualenv on your system, then you tell it what python version to install. Once activated, any dependencies are installed and isolated in that environment. Also see: https://gist.github.com/pandafulmanda/730a9355e088a9970b18275cb9eadef3
If you want to go a step further you can use docker (https://docs.docker.com/language/python/build-images/) to isolate everything, including the operating system, in that case you don't need virtualenv, but to start with I'd play around with virtualenv.
I am experiencing difficulty with installing Python correctly on my Mac (Fresh Install, I wiped my PC clean). I installed firstly
Home Brew
Xcode through App Store
Command line tools using command: "xcode-select --install"
On typing the following command:
which python3
Output is as follows:
gaurangsmacbookpro#Gaurangs-New-MacBook-Pro ~ % which python
/usr/bin/python
gaurangsmacbookpro#Gaurangs-New-MacBook-Pro ~ % which python3
/usr/bin/python3
upon running the following command which I was following off a blog from this site:
https://www.pyimagesearch.com/2016/12/05/macos-install-opencv-3-and-python-3-5/
brew install eigen tbb
Home brew installed python 3.9 dependencies so no when I type the following commands:
brew list python3
I get the following output:
gaurangsmacbookpro#Gaurangs-New-MacBook-Pro ~ % brew list python3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/2to3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/2to3-3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/easy_install-3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/idle3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/idle3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/pip3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/pip3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/pydoc3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/pydoc3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/python3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/python3-config
/usr/local/Cellar/python#3.9/3.9.1_8/bin/python3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/python3.9-config
/usr/local/Cellar/python#3.9/3.9.1_8/bin/wheel3
/usr/local/Cellar/python#3.9/3.9.1_8/Frameworks/Python.framework/ (3019 files)
/usr/local/Cellar/python#3.9/3.9.1_8/IDLE 3.app/Contents/ (8 files)
/usr/local/Cellar/python#3.9/3.9.1_8/lib/pkgconfig/ (4 files)
/usr/local/Cellar/python#3.9/3.9.1_8/libexec/bin/ (7 files)
/usr/local/Cellar/python#3.9/3.9.1_8/libexec/pip/ (480 files)
/usr/local/Cellar/python#3.9/3.9.1_8/libexec/setuptools/ (334 files)
/usr/local/Cellar/python#3.9/3.9.1_8/libexec/wheel/ (44 files)
/usr/local/Cellar/python#3.9/3.9.1_8/Python Launcher 3.app/Contents/ (16 files)
/usr/local/Cellar/python#3.9/3.9.1_8/share/man/ (2 files)
The contents of my ~./bash_profile file are as follows:
# Add Homebrew's executable directory to the front of the PATH
export PATH=/usr/local/bin:$PATH
export PATH="/usr/local/sbin:$PATH"
export PATH="/usr/local/opt/python#3.8/bin:$PATH"
What am I doing wrong? I just want a fresh clean install of Python 3.8 not 3.9 as there is no support for TF Lite on 3.9 yet. Can someone help me fix this mess? I will be very grateful!
I'm a bit late at the game, but I hope this will be useful to somebody. The best solution here is to use a virtual environment, but for the sake of argument and because it is a little bit more involved, I won't discuss that as an option here and will assume you just want a specific version installed on your machine where you can just add all the dependencies you need.
From looking at the page OP was using, the easiest solution here is probably to use a distribution like Anaconda if you are particularly interested in OpenCV or that type of libraries. Then it's just a case of setting up your path like export PATH=$HOME/anaconda/bin:$PATH or whatever path you used to install Anaconda with your shell of choice.
But let's imagine that Anaconda doesn't help you and you still need a specific version of Python, and you figure 3.8.2 is good enough for your needs and a newer version wouldn't work. Using the Python version that comes with the Xcode command line tools is probably a bad idea, just like using the system installed Python is. That version is really meant to support the Apple toolchain and while I've never run into any issues with it, I would not recommend it.
Next option is is to use homebrew. Great option since most people will already be using it, and you might already have python installed as part of a dependency but maybe like OP it's not the version you were hoping for. You can still make it work by installing an older version like so brew install python#3.8 however, it comes with a few caveats. Whenever one of the python dependencies is updated, you might need to unlink and relink it so that you can continue to use the correct version. Believe me, it's no way to live.
What I've found works (in combination with virtual environments is to use pyenv. Install it through homebrew, set your shell as described in the install doc and use it to install the python version you want (very easy) and then set the global interpreter to be whatever version you want and your system will stick to it until you set it to something else. It also works great in combination with tools like pipenv which allow you to manage virtual environments.
If I use a module such as tkinter, would somebody need to have that module installed as well in order for my code to run on their machine?
Definitely. You can use virtual environments or containers to deliver required packages or have a requrements.txt or similar to install the dependencies.
python comes with a number of standard modules pre-installed, if the other person is running python (the same version of you) then he/she won't need to install anything, it will just work, that's the case of tkinter. But if you use external packages that you installed to run your code, for example celery, then he/she will need to do the same thing.
If you gave your code to someone to run, they would need to download the same modules, unless you also sent the environment too. The only way I know around this is to freeze your code where you would create an executable. I've used cx_Freeze and pyInstaller and haven't had any issues but it also depends on your needs. You can find some more information through here:
https://docs.python-guide.org/shipping/freezing/
Hope this helps!
In your running environment do a, this file you add to your repo
pip freeze > requirements.txt
When people clone your repo, they only have to do a:
pip install -r requirements.txt
and they will install exactly the same pypi modules you have.
With virtualenv you can isolate a python environment to each project, with pyenv you can use different pythonversions withing the various environment also.
How to package Python itself into virtualenv? Is this even possible?
I'm trying to run python on a machine which it is not installed on, and I thought virtualenv made this possible. It activates, but can't run any Python.
When setting up the virtualenv (this can also be done if it already set up) simply do:
python -m virtualenv -p python env
And Python will be added to the virtualenv, and will become the default python of it.
The version of Python can also be passed, as python uses the first version found in the PATH.
virtualenv makes it convenient to use multiple python versions in different projects on the same machine, and isolate the pip install libraries installed by each project. It doesn’t install or manage the overall python environment. Python must be installed on the machine before you can install or configure the virtualenv tool itself or switch into a virtual environment.
Side note, consider using virtualenvwrapper — great helper for virtualenv.
You haven't specified the Operating System you are using.
In case you're using Windows, you don't use virtualenv for this. Instead you:
Download the Python embeddable package
Unpack it
Uncomment import site in the python37._pth file (only if you want to add additional packages)
Manually copy your additional packages (the ones you usually install with pip) to Lib\site-packages (you need to create that directory first, of course)
Such a python installation is configured in such a way that it can be moved and run from any location.
You only have to ensure the Microsoft C Runtime is installed on the system (but it almost always already is). See the documentation note:
Note The embedded distribution does not include the Microsoft C Runtime and it is the responsibility of the application installer to provide this. The runtime may have already been installed on a user’s system previously or automatically via Windows Update, and can be detected by finding ucrtbase.dll in the system directory.
You might need to install python in some location you have the permissions to do so.