Is there a way to install Airflow without pip?
I am trying to install Airflow on a offline computer that does not have pip. I have downloaded the packages from the internet but I am not sure how to run the installation without pip.
Does anybody knows how to run an installation with 'setup.py'?
# /usr/bin/bash
python setup.py build
python setup.py install
If you get problems like the lack of setuptools you can install it (depending on your system, it usually a package. If you want to be sure to have everything download and later install the python-devel (sometimes also python-dev) package with your OS package manager) or you could use airflow via docker, you will build the image and then export it to the offline unit.
To install it without pip,
download airflow zip from git-repo.
unzip the contents and follow the instructions in the INSTALL file. (Steps inside create a virtualenv and then start the airflow installation)
These are the steps you will find in INSTALL file:
python -m my_env
source my_env/bin/activate
# [required] by default one of Apache Airflow's dependencies pulls in a GPL
# library. Airflow will not install (and upgrade) without an explicit choice.
#
# To make sure not to install the GPL dependency:
# export SLUGIFY_USES_TEXT_UNIDECODE=yes
# In case you do not mind:
# export AIRFLOW_GPL_UNIDECODE=yes
# [required] building and installing
# by pip (preferred)
pip install .
# or directly
python setup.py install
Alternative, If you are facing any conflicts while installing through pip,
create a virtualenv
source-activate virtualenv
Follow steps from airflow to install and configure it.
One way or the other you will have to use pip as there might be a number of modules you may have to download using pip. Creating a virtualenv here will help.
Related
I am familiar with using pip to install Python packages but there is no way to install it in the environment I am working in. We have to call the directory with python.exe to run any Python code. Therefore, it is impossible to use pip install because, since there is no python, there is no pip. How could we install packages without using pip or installing pip via the python.exe file? Here is an image of the error:
Packages like pip can be executed from the python executable using python.exe -m pip install openpyxl. If you don't have sufficient firewall permissions (as you mentioned high security) you may not be able to connect to the package servers, which you would need to discuss with admin.
I'm on windows and want to use python and a few packages, however, due to our policies it is not possible to have admin rights and any fetch requests done by pip (or any other package manager I'd assume) are being blocked.
So far I've downloaded Python (3.9.6) and "installed" it by manually unzipping and setting the PATH env variable.
I know that packages can be manually downloaded and installed as well using setup.py, however I also do not have setuptools, so is there any way to manually install that?
pip doesn't require admin rights just use it with --user.If you want to avoid using pip here the steps
if the folder has setup.py in it
Download the packages and uncompress if needed open folder
holding shift click open with cmd or powershell
run python setup.py install --user or python3 setup.py install --user
if not present setup.py check the install documentation provided by module.
I am currently using conda for this purpose.
After generating local environment.yml, I run $ conda create -n environment.yml on the remote server.
But this doesn't include global packages that my code references.
I can add a requirements.txt using pipreqs and then run pip install -r requirements.txt remotely but this doesn't take into account dependencies like dlib or boost that a package may need for installation.
Is there any solution for this?
You have two diff type of dependencies. 1) Need to install via apt-get like boost, opencv 2) Need to install via pip.
You need to install apt-get library manually on the server and can define pip related libraries in the requirements.txt file. Because apt-get libraries are environment independent.
I am going through this post Numpy, Scipy, and Pandas - Oh My!, installing some python packages, but got stuck at the line for installing Pandas:
pip install -e git+https://github.com/pydata/pandas#egg=pandas
I changed 'wesm' to 'pydata' for the latest version, and the only other difference to the post is that I'm using pythonbrew.
I found this post, related to the error, but where is the Makefile for bz2 mentioned in the answer? Is there another way to resolve this problem?
Any help would be much appreciated. Thanks.
You need to build python with BZIP2 support.
Install the following package before building python:
Red Hat/Fedora/CentOS: yum install bzip2-devel
Debian/Ubuntu: sudo apt-get install libbz2-dev
Extract python tarball. Then
configure;
make;
make install
Install pip using the new python.
Alternative:
Install a binary python distribution using yum or apt, that was build with BZIP2 support.
See also: ImportError: No module named bz2 for Python 2.7.2
I spent a lot of time on the internet and got a partial answer everywhere. Here is what you need to do to make it work. Follow every step.
sudo apt-get install libbz2-dev Thanks to Freek Wiekmeijer for this.
Now you also need to build python with bz2. Previously installed python won't work. For that do following:
Download stable python version from https://www.python.org/downloads/source/ then extract that Gzipped source tarball file. You can use wget https://python-tar-file-link.tgz to download and tar -xvzf python-tar-file.tgz to extract it in current directory
Go inside extracted folder then run following commands one at a time
./configure
make
make install
This will build a python file with bz2 that you previously installed
Since this python doesn't have pip installed, idea was to create a virtual environment with above-built python then install pandas using previously installed pip
You will see python file in the same directory. Just create a virtual environment.
./python -m env myenv (create myenv in the same directory or outside it's your choice)
source myenv/bin/activate (activate virtual environment)
pip install pandas (install pandas in the current environment)
That's it. Now with this environment, you should be able to use pandas without error.
pyenv
I noticed that installing Python using source takes a long time (I am doing it on i7 :/ ); especially the make and make test...
A simpler and shorter solution was to install another version of Python (I did Python 3.7.8) using pyenv, install it using these steps.
It not only saved the problem of using multiple Python instances on the same system but also maintain my virtual environments without virtualenvwrapper (which turned buggy on my newly setup ubuntu-20.04).
As part of a deployment procedure, I upload Python source packages (generated with setup.py sdist) to a remote server and install them in a virtualenv using pip install mypackage-1.0.tar.bz2.
This has worked for long time both for new installs and upgrades (specifically, upgrades without a change in the package's version number). For some reason I cannot figure out, since yesterday, it fails to upgrade the packages. No error is reported, the files are just not changed. Now I'm sure I'm doing something differently but I can't explain the change in behaviour.
I can upgrade the package with the -U --no-deps flags, but this technique forces the deployment script to differentiate between first install and upgrades (--no-deps is required as otherwise dependencies would be downloaded each time from pypi).
Any ideas how I can get a single pip command to do installs and upgrades?
pip install package will only be executed with you don't have this package already.
With you want to upgrade the package you'll have to use: pip install -U package