I've been trying to switch my coding to Linux.
I have ironed out most of my issues but one last thing I am not able to find any explanation of is the virtualization of make and bash commands.
I have installed PyCharm which virtualizes everything from what I have seen.
However, when I am cloning repositories from Github, the instructions require building some code using make and then installing them and later on using bash to build dependencies.
I am running the commands in PyCharm terminal but instead of installing into the venv, it's installing the data in /usr/xxx instead.
How do I tell PyCharm to use bash and make in a similar way to pip to virtualize the setup ?
Edit:
One of the projects in question is gym-gazebo which requires:
git clone https://github.com/erlerobot/gym-gazebo/blob/master/INSTALL.md
Then make and make install which installs it in the root
Later on there is also
bash setup_kinetic.bash
Which also uses root folders and not venv
I was able to install it but it is not virtualized the way it should when compared with coding on Windows
Basically the answer based on Evert and CharlesDuffy, what I'm looking for is containerization, since the libraries I'm looking for are C based with a python wrapper (something like that)
Docker, Singularity and Conda are on of the solutions.
Related
I have an open source project called Djengu. To install it, the user must clone the repo and run make to initiate the setup script. The setup script creates a Python virtual environment using virtualenv. The command goes like
virtualenv -p python3.8 .python3.8_env
I'd like to pin the Python version to avoid anything breaking. I also cannot assume that any given user will have a python3.8 binary installed on their machine. And I cannot assume that they have pyenv installed either.
I imagine I will have to make a trade off somewhere. How can I pin Python without making assumptions on what the user has installed? Is there a standard way to do something like this?
Since your Djengu project is a development environment, I think it's completely fine to require that your users first install pyenv before calling make. Just tell them to do so in the Readme. You can then use their pyenv in your Makefile to install the Python version you need.
I've been going around but was not able to find a definitive answer...
So here's my question..
I come from javascript background. I'm trying to pickup python now.
In javascript, the basic practice would be to npm install (or use yarn)
This would install some required module in a specific project.
Now, for python, I've figured out that pip install is the module manager.
I can't seem to figure out how to install this specific to a project (like how javascript does it)
Instead, it's all global.. I've found --user flag, but that's not really I'm looking for.
I've come to conclusion that this is just a complete different schema and I shouldn't try to approach as I have when using javascript.
However, I can't really find a good document why this method was favored.
It may be just my problem but I just can't not think about how I'm consistently bloating my pip global folder with modules that I'm only ever gonna use once for some single project.
Thanks.
A.) Anaconda (the simplest) Just download “Anaconda” that contains a lots of python modules pre installed just use them and it also has code editors. You can creat multiple module collections with the GUI.
B.) Venv = virtual environments (if you need something light and specific that contains specific packages for every project
macOS terminal commands:
Install venv
pip install virtualenv
Setup Venve (INSIDE BASE Project folder)
python3 -m venv thenameofyourvirtualenvironment
Start Venve
source thenameofyourvirtualenvironment/bin/activate
Stop Venve
deactivate
while it is activated you can install specific packages ex.:
pip -q install bcrypt
C.) Use “Docker” it is great if you want to go in depth and have a solide experience, but it can get complicated.
Pip is a program used to manage Python distribution. You usually have one system distribution which is by default managed by Pip. When you do pip install scipy, you install package scipy to your system Python. Everytime you try to import scipy after it will work because your system Python has it.
Project specific distributions are acomplished by using virtual environments. python -m venv env or venv env creates a copy of system Python interpreter, pip, setuptools and a couple of other essential tools. In other words, virtual environment created this way is empty.
To use created virtual environement one should use source env/bin/activate. After that, everytime you invoke python command it will use activated Python interpreter. When you install packages using pip, it will install them in the virtual environment rather than to your system python. To use system Python again use deactivate.
Such usage is actually prefered for projects because some user applications could rely on system Python and some packages, and installing, updating etc. could be potentionally dangerous.
Further reading: venv documentation
I have a python library that I am wanting to help out with and fix some issues. I just don't know how to test my changes given the complexity of how python/pip installs libraries.
I have the library installed with pip and I can run python code connecting to the library by doing an "from import *". But now that I want to make changes to it I pulled the code with git and plan to branch to work on my changes. That's fine. I will then do a pull request to merge any changes given tests pass.
But after I make a change, how do I integrate my changes into python to test out the changes I made with the library? Can pip install my custom/modified version of the library?
I have looked around and haven't successfully found an answer to this but perhaps I'm not looking in the right spot.
Can pip install my custom/modified version of the library?
Yes.
There are various ways of approaching this question. A common solution is the use of Python virtual environments. This allows you to create an isolated Python environment that does not share the same packages as your system Python install. You can then install things into it (such as your modified Python library) to test it out.
To get started, you need the virtualenv tool. This is probably available as a package for your distribution, but you can also install it using pip. Once you have it, you can run in the same directory as your code:
virtualenv .venv
This creates a virtuelenv named .venv. You can call it anything you want, but naming it .venv (or anything starting with a .) means it won't clutter up the output of ls in your workspace.
Next, you need to activate the virtualenv:
. .venv/bin/activate.sh
This modifies your $PATH to place the virtualenv at the front of the list of directories. Now when you type python or pip, you'll be using the virtualenv version.
If your code has a setup.py file, you can install it like this:
pip install -e .
The -e means you want to perform an "editable" install, which means python will use the code "in place", and any changes you make will be immediately visible to the code you use for testing.
When you're done, you can run:
deactivate
This will remove the changes that activate made to your environment.
For more information:
Pipenv & Virtual Environments discusses a higher level tool for managing virtual environments.
Virtualenvwrapper is another take on a higher level management tool.
I am working on several projects on the same PyCharm. Like I "attached" them all together. But I recently noticed some weird behaviors. Like when I import a library I haven't installed yet to my script. It shows me a little error as expected. But when I try to install that using python -m pip install my_library, it tells me that it has already installed. I recently noticed that this is because it's using and other pip from another project. I doesn't use the one in the venv folder in the project. Also to run the scripts sometimes it uses python.exe from pythons original directory. It's a whole mess and I have no idea how I can solve it. Sometimes my projects requires different versions of the same library and you can imagine what happens when I change the version.
I make sure each project is using their own interpreter. Don't know what else to do other than this. I am using Python3.6.4 PyCharm2018.3.2 running on Windows10
it sounds like all your projects are configured to use the system's interpreter instead of the virtual environment you set up for each of them.
Follow this instruction to fix it https://www.jetbrains.com/help/pycharm-edu/creating-virtual-environment.html
In terms of using different version of the python library, you can address that by specifying it in requirements.txt file, which you can put in your venv folder for each project. then you can just do pip install -r requirements.txt after you set up your venv. (you need to ensure that the venv is activated - you don't need to worry about this if you have configured the project in PyCharm to use the venv's python interpreter.) You can check this by going to Terminal in your PyCharm and you should see (venv_name) hostusername#host:~/project_folder$
Let me first outline my desired solution and then elaborate on a specific question how to achieve this state.
I'm soon starting two coding projects in python. I've used python before but never on such big projects. My ideal scenario would be to have a setup where I can run virtual environments and different python version for various project. Some research pointed me to virtualenv / virtualenvwrapper and pyenv. It seems using pyenv-virtualenv or pyenv-virtualenvwrapper there is a nice way to specify the virtualenvironment and python version for a specific project.
Question: Once I've setup a virtualenvironment and python version for a specific project, how easily could I switch in a later stage to a newer python version? Let's say I've started project A with python 3.4 and in one year in the future I would like to move everything to python 3.6. Is this possible in a neat way?
Sure:
$ rm -r my-python-3.4-env
$ virtualenv -p python3.6 my-python-3.6-env
$ source my-python-3.6-env/bin/activate
In other words, each virtual environment is just a folder with the necessary files in it. You "activate" an environment with the source .../activate command (in case of virtualenv) and you leave it just as easily. To switch to a different environment you simply create a new one with a specific Python executable and activate it.
What you want to be careful about is to keep your installation repeatable, meaning if you depend on external modules (which modern projects typically do), you don't want to install each dependency by hand and instead automate that. For instance, you create a setuptools setup.py file which lists your dependencies, and then have it install them into your new environment automatically:
$ source my-python-3.6-env/bin/activate
(my-python-3.6-env) $ python setup.py develop