Dependencies in Python project - python

I am currently working on a Python project, and I want the user to automatically have access to the dependencies that I used. Do they have to download (e.g. pip install) them manually? If so, is there an easy way to make them download the necessary packages easily?

virtualenv is What you need. You install packages your project needed when developing it. After coding, you can run pip freeze > requirements.txt to save all packages to requirements.txt. pip install -r requirements.txt will install all packages automatically.
further, Docker is more better for releasing projects to PC.

You need to create a virtual environment, see the link on how to, Then could use pip freeze > requirements.txt, to store your env dependencies to a file and then your user can simply use pip install -r requirements.txt to install them in one go
See documentation for more details

Related

Pip install a package in a way that it gets not listed in requirements.txt

Is it possible to install a pip package in a way so that it gets not listed when doing pip freeze > requirements.txt?
I am thinkging of an equivalent to: poetry add --dev which adds (installs) a package as a development dependency, but it does not appear in dependency list.
Is there a way in pip to do something similar?
What you want is pipenv.
There are ways of making RStudio work with pipenv (link to an article).
This allows both complete package control, python version specification for a project as well as virtualenv, all in one.
Otherwise, you'd have to maintain your requirements.txt file manually, and further down the line use a constraints.txt file, also.
Think of pipenv files as what yarn.lock files (JS) vs package.json file + some extra sweet features.
You can use pipenv to generate a requirements.txt file by doing:
pipenv lock -r > requirements.txt
While you can add/install packages in development mode by:
pipenv install --dev <mypackage>

Do we need to upload virtual env on github too?

This is my GitHub repo
https://github.com/imsaiful/backmyitem
I push from my local machine and pull the changes in Amazon EC2.
Earlier I have not added the virtual env file in my repo but now I have changed some file in admin directory which is containing in the virtual env. So should I go for to add the virtual env too on my GitHub or instead I change the same thing on my remote server manually?
As was mentioned in a comment it is standard to do this through a requirements.txt file instead of including the virtualenv itself.
You can easily generate this file with the following:
pip freeze > requirements.txt
You can then install the virtualenv packages on the target machine with:
pip install -r requirements.txt
It is important to note that including the virtualenv will often not work at all as it may contain full paths for your local system. It is much better to use a requirements.txt file.
No - although the environment is 100% there, if someone else where to pull it down the path environment hasn't been exported not to mention Python version discrepancies will likely crop up.
The best thing to do is to create what is known as a requirements.txt file.
When you have created your environment, you can pip install this and pip install that. You'll start to built a number of project specific dependencies.
Once you start to build up a number of project dependencies I would then freeze your local python environment (analogoues to a package.json for node.js package dependency management). I would recommend doing the following in your terminal:
(local_python_environment) $ pip install django && pip freeze > requirements.txt
(local_python_environment) $ pip install requests && pip freeze > requirements.txt
That is to say, freeze your environment to a requirements.txt file every time a new dependency is installed.
Once a collaborator pulls down your project - they can then install a fresh python environment:
$ python3 -m venv local_python_environment
(* Please use Python 3 and not Python 2!)
And then activate that environment and install from your requirements.txt which you have included in your version control:
$ source local_python_environment/bin/activate
(local_python_environment) $ pip install -r requirements.txt
Excluding your virtual environment is probably analogous to ignoring node_modules! :)
No Its not necessary to upload virtualenv file on github. and even some time when you push your code to github then it ignore python file only if add into ignore.
Virtual Environment
Basically virtual environment is nothing but itis a tool that helps to keep dependencies required by different projects separate by creating isolated python virtual environments for them. This is one of the most important tools that most of the Python developers use. Apart from that you can add requirement.txt file into your project.
Requirement.txt
It is file that tells us to which library and application are need to run this application. you can add requirement.txt file with this simple command.
pip freeze > requirements.txt
After run this command all application and library add in this file. and if you make your project without activate any virtualenv then python automatically use system environment variable it will also add all the file that not necessary for your project.
You should add the virtualenv in your gitignore. Infact github has a recommended format for python, which files should be added and which shouldn't
Github recommendation for gitignore

Python equivalent of npm or rubygems?

I've been looking around for a package manager that can be used with python. I want to list project dependencies in a file.
For example ruby uses Gemfile where you can use bundle install.
How can I achieve this in Python?
The pip tool is becoming the standard in equivalent of Ruby's gems.
Like distribute, pip uses the PyPI package repository (by default) for resolving and downloading dependencies.
pip can install dependencies from a file listing project dependencies (called requirements.txt by convention):
pip install -r requirements.txt
You can "freeze" the current packages on the Python path using pip as well:
pip freeze > requirements.txt
When used in combination with the virtualenv package, you can reliably create project Python environments with a project's required dependencies.
Pipenv
(I know it's an old question, and it already has an answer but for anyone coming here looking for a different answer like me.)
I've found a very good equivalent for npm, It's called pipenv. It handles both virtualenv and pip requirements at the same time so it's more like npm.
Simple Use Case
pip install pipenv
then you can make a new virtualenv with third version of python, as well as making a pipfile that will be filled with your projects requirement and other stuff:
pipenv install --three
using your created virtualenv:
pipenv shell
installing a new python package:
pipenv install requests
running your .py file is like:
pipenv run python somefile.py
you can find it's doc here.
Python uses pip for a package manager. The pip install command has a -r <file> option to install packages from the specified requirements file.
Install command:
pip install -r requirements.txt
Example requirements.txt contents:
Foo >= 1.2
PickyThing <1.6,>1.9,!=1.9.6,<2.0a0,==2.4c1
SomethingWhoseVersionIDontCareAbout
See the Requirements Parsing section of the docs for a full description of the format: https://pip.pypa.io/en/stable/user_guide/#requirements-files
This is how I restrict pip's scope to the current project. It feels like the opposite if you're coming from NodeJS's npm or PHP's composer where you explicitly specify global installations with -g or --global.
If you don't already have virtualenv installed, then install it globally with:
pip install virtualenv
Each Python project should have its own virtualenv installation. It's easy to set one up, just cd to your project's root and:
python3 -m virtualenv env # creates env folder with everything you need
Activate virtualenv:
source env/bin/activate
Now, any interaction with pip is contained within your project.
Run pip install package_name==version for each of your dependencies. They are installed in ./env/lib/python3.x/site-packages/
When you want to save your project's dependencies to a file, run:
pip freeze > requirements.txt
You actually don't need -l or --local if you're in an activated project-specific virtualenv (which you should be).
Now, when you want to install your dependencies from requirements.txt, set up your virtualenv, and run:
pip install -r requirements.txt
That's all.
This is an old question but things are constantly evolving.
Further to the other answer about pipenv. There is also a python package manger called poetry.
There is a detailed comparison between pipenv and poerty here: Feature comparison between npm, pip, pipenv and poetry package managers. It also links the features to common npm features.
Here is a comparison of pipenv vs poetry vs pdm: https://dev.to/frostming/a-review-pipenv-vs-poetry-vs-pdm-39b4
The conclusion is that pdm is the winner.
But in my experience, poetry is easier than pdm to integrate with IDEs.

Does Django have an equivalent of Rails's "bundle install"?

One thing I like about Rails projects is that when deploying to a remote server, if everything is set up correctly you can just do:
$: bundle install
And the system will install the various dependencies (ruby gems) needed to run the project.
Is there something similar for Python/Django?
You can freeze requirements. This generates a list of all the Python modules that your project needs. I believe bundle is similar in concept.
For example:
virtualenv --no-site-packages myproject_env # create a blank Python virtual environment
source myproject_env/bin/activate # activate it
(myproject_env)$ pip install django # install django into the virtual environment
(myproject_env)$ pip install other_package # etc.
...
(myproject_env)$ pip freeze > requirements.txt
The last line generates a text file will all the packages that were installed in your custom environment. You can use that file to install the same requirements on other servers:
pip install -r requirements.txt
Of course you don't need to use pip, you can create the requirements file by hand; it doesn't have any special syntax requirements. Just a package and (possibly) version identifier on each line. Here is a sample of a typical django project with some extra packages:
Django==1.4
South==0.7.4
Werkzeug==0.8.3
amqplib==1.0.2
anyjson==0.3.1
celery==2.5.1
django-celery==2.5.1
django-debug-toolbar==0.9.4
django-extensions==0.8
django-guardian==1.0.4
django-picklefield==0.2.0
kombu==2.1.4
psycopg2==2.4.5
python-dateutil==2.1
six==1.1.0
wsgiref==0.1.2
xlwt==0.7.3
The closest is probably virtualenv, pip and a requirements file.
With those 3 ingredients is quite easy to write a simple bootstrap scripts.
More demanding and complex is buildout. But I would only go for it if virtualenv and pip are not sufficient.
And if you extend this approach with fabric and optional cuisine, you already have your project deployment automated. Check out these links for more information:
http://www.caktusgroup.com/blog/2010/04/22/basic-django-deployment-with-virtualenv-fabric-pip-and-rsync/
http://www.saltycrane.com/blog/2009/05/notes-using-pip-and-virtualenv-django/
http://www.saltycrane.com/blog/2009/05/notes-using-pip-and-virtualenv-django/

Any productive way to install a bunch of packages

I had one machine with my commonly used python package installed.
and i would like to install the same package on another machine or same machine with different python version. I would like to know whether pip or easy-install or some other method can let me install those packages in a batch. When i use perl, it has something like a bundle package, how to do that in python?
Pip has some great features for this.
It lets you save all requirements from an environment in a file using pip freeze > reqs.txt
You can then later do : pip install -r reqs.txt and you'll get the same exact environnement.
You can also bundle several libraries into a .pybundle file with the command pip bundle MyApp.pybundle -r reqs.txt, and later install it with pip install MyApp.pybundle.
I guess that's what you're looking for :)
I keep a requirements.txt file in one of my repositories that has all my basic python requirements and use PIP to install them on any new machine.
Each of my projects also has it's own requirements.txt file that contains all of it's dependencies for use w/virtualenv.

Categories