This is not so much of a problem but I have been adding requirements.txt everytime I start a new project and deploy it on heroku. I don't understand the purpose of requirements.txt though and why it is so important.
Consider the scenario where u have installed 2 or 3 packages during the development cycle of 1/2 months.
During deployment, you forget to install one of them explicitly. At this time requirements.txt will come very handy.
So whenever you install a new package make sure add that package name in requirements.txt
And during deployment just running a single command like
pip install -r requirements.txt
Will install all dependencies of your project.
And also useful when you clone the repo in new location.
Related
I've got a simple project that I want to be able to be pip installed - and where the 'development time' dependencies are the same as the 'use time' dependencies.
To avoid duplication between setup.py and requirements.txt - I'm using the 'trick' to have my requirements.txt simply contain . (to pick up dependencies from setup.py).
When working on the project, we can then clone it and pip install -r requirements.txt.
The problem I'm having with this is that this also seems to end up installing the actual project itself (presumably due to going through setup.py?)
Is there an option I can use with pip install to prevent this? Or some other way to work around it without ending up installing the project itself?
For context, I don't know much about Python, let alone idiomatic Python. I'm working on a brownfield project. Everything I say about docker may be irrelevant to the question I'm asking, but I can't tell: Our code runs in a Docker container. Instead of using virtual environments, we hardcode the Python version and run this in the Dockerfile:
ADD requirements.txt /
RUN pip install -r /requirements.txt \
&& rm -rf /requirements.txt
At the moment, we have two ways of adding requirements to the requirements.txt:
By running this command (using twilio as an example):
docker-compose run --rm django bash -c "pip install twilio && pip freeze > requirements.txt"
By going to pypi.org, finding a dependency's name and current version, and manually adding that line to the hosts requirements.txt.
Both seem to work, but my gut tells me there are latent downsides to one/both of these. What are the pros and cons of each choice and which one is considered idiomatic? If neither of these are considered idiomatic, what's the right way to add to the requirements.txt?
I've been googling, but a lot of the results are questionable because they are really old. e.g., pip 20.3.2020 added resolver functionality, and I don't know what ripples that had on best practices.
The requirements.txt file indicates all the dependencies that must be installed for your application to run correctly.
Running pip freeze will dump all the actually installed libraries (development, other projects, deprecated, etc) into a freshly created requirements.txt file.
Adding manually the dependencies is a more controlled manner to list your dependencies.
I recommend you adding manually the dependencies to the file while building the project. If you found some are not necessary, remove them. With the pip freeze, maybe other secondary libraries will remain.
I am trying to upgrade a Pyramid project to Python 3. I am also exploring various on how to improve the build system so that is more modern.
Currently we are using Python's buildout to setup an instance. The approach is simple:
All the required eggs (including app - which is my package) are stored in a buildout configuration file with their exact versions specified.
Run buildout to get the exact eggs (including third party stuff) from a local package server.
Run the instance using ./bin/pserve config.ini.
For my new app source code that has Python 3 changes, I am trying to get rid of everything and just use pip instead. This is what I have done now (in a docker container):
git clone git#github.com/org/app.git
cd project
# Our internal components are fetched using the `git` directive inside `requirements.txt`.
pip install -r requirements.txt # Mostly from PyPi.
pip install .
It works, but is this the correct way to deploy an application for deployment?
Will I be able to convert the entire installation to a simple: pip install app and run it using pserve config.ini if I do the following:
Upload the latest app egg to my package server.
Sync setup.py and requirements.txt so that Python to do pip install -r requirements.txt (or its equivalent) internally?
pip install app.
Copy config.ini to the machine where I am going to install.
Run pserver config.ini
I wanted to know if the above approach can be made to work before proceeding with the egg creation, mocking a simple package server etc. I am not sure if I can really do pip install for a web application; and I think requirements.txt has some significance in this case.
I haven't explored wheels yet, but if the above works, I will try that as well.
Since I am really new to packaging, would appreciate if I can some suggestions to modernize by build using the latest tools.
After reading some of the links like requirements.txt vs setup.py, I think requirements.txt is needed for Web Apps especially if you want a consistent behaviour for deployment purposes. A project or an application seems to be different than a library where pip install suffices.
If that is the case, I think the ideal way is to do pip install -r requirements.txt and then pip install app from a local package server without git cloning?
Resources: install_requires vs requirements files
In Python, the requirements.txt file declares a set of concrete dependencies of a python application.
Often, during development, dependencies change. When the set of new dependencies overlaps with the set of old dependencies, all is good, as a venv/bin/pip install -r requirements.txt updates them accordingly.
However, when the set of new dependencies is smaller than the set of old dependencies, old dependencies become "dangling" as they are no longer used, but still populate the pip freeze.
Is there any command to force pip to ensure that pip freeze after a pip install -r requirements.txt is the same as declared in the requirements.txt?
My current solution is to delete the venv and create a new one, which is mildly time-consuming.
Note that the goal is not to generate a requirements.txt file, is to make venv map to a given requirements file.
It sounds like pipreqs could solve your problem, pipreqs will create a requirments.txt file, but only include dependencies that are currently being used in your project (based on the imports in your code)
for more info check their github repo here
I'm writing a Python app to deploy on Heroku. Per Heroku's guide, I need to list package requirements in a Pip requirements.txt file. The guide instructs me to install the packages locally, then run pip freeze > requirements.txt to write the frozen requirements file.
However, one of the packages I want to use in deployment on Heroku can't be installed locally. It's incompatible with my operating system.
So how do I write a requirements.txt including this package suitable for Heroku?
The only way I can think of is to write it by hand - but this would be tedious, because the package in question has many dependencies of its own. Besides, this defeats the point of the package manager.
When deploying Ruby apps to Heroku, Bundler makes this easy. In my Gemfile I write
gem "pg", :group => :production
gem "sqlite3", :group => :development
The command bundle install then writes a frozen version list Gemfile.lock (analogous to requirements.txt). It doesn't install the packages listed under the 'production' group, but it still freezes a consistent list of versioned packages.
Example: Gemfile and Gemfile.lock
You can have more than one file, and call them different things, but Heroku does expect a requirements.txt. For instance, for dev, you could maintain a dev_requirements.txt
Locally you can run:
$ pip freeze > dev_requirements.txt
etc, and
$ pip install -r dev_requirements.txt
and Heroku will run:
$ pip install -r requirements.txt
It's not possible. Issue reported to pip https://github.com/pypa/pip/issues/747