I have a Pyramid server on an Openshift Python 3 cartridge that does not like the module Pandas, but I cannot figure how to figure out why it is so.
In setup.py the method setuptools.setup is called with the argument install_requires with a few modules (such as pyramid,biopython,numpy), which run fine. But when I add pandas it fails. This is what it says during pushing (✱ = censoring added for security):
remote: Installing collected packages: six, python-dateutil, pytz, pandas
remote: Found existing installation: six 1.3.0
remote: Not uninstalling six at /opt/rh/python33/root/usr/lib/python3.3/site-packages, outside environment /var/lib/openshift/✱✱✱✱✱/python/virtenv/venv
remote: Running setup.py install for pandas: started
remote: Running setup.py install for pandas: still running...
remote: Running setup.py install for pandas: still running...
remote: Running setup.py install for pandas: still running...
remote: Running setup.py install for pandas: still running...
remote: Running setup.py install for pandas: still running...
remote: Running setup.py install for pandas: still running...
Connection to pedel2-matteoferla.rhcloud.com closed by remote host.
fatal: The remote end hung up unexpectedly
error: error in sideband demultiplexer
To ssh://pedel2-matteoferla.rhcloud.com/~/git/pedel2.git/
✱✱✱ master -> master
error: failed to push some refs to 'ssh://✱✱✱✱#pedel2-matteoferla.rhcloud.com/~/git/pedel2.git/'
Adding a requirements.txt file does not affect it due to permission issues seen also with when I ssh into the server and pip install it manually. Namely /var/lib/openshift isn't my virtual environment.
remote: Activating virtenv
remote: Checking for pip dependency listed in requirements.txt file..
remote: The directory '/var/lib/openshift/✱✱✱✱/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Pip3 within the virtual python does the same.
cd /python/virtenv/venv/bin/
./pip3 install pandas
Easy_install does it right but has no permission, so this approach is wrong anyway.
pedel2-matteoferla.rhcloud.com bin]\> ./easy_install pandas
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
[Errno 122] Disk quota exceeded: '/var/lib/openshift/✱✱✱✱/python/virtenv/venv/lib/python3.3/site-packages/test-easy-install-247598.write-test'
I assume I ought to solve the problem with setuptools module. Which means I need to figure out why from there. But I am stuck.
EDIT. It is not my disk quota.
Matteos-Air-3:pedel2 matteo$ rhc show-app pedel2 --gears quota
Gear Cartridges Used Limit
------------------------ ---------- ------ -----
✱✱✱✱ python-3.3 268 MB 1 GB
I think you have a few issues.
size issue - it's difficult to reproduce it. But maybe we don't need it (see details below)
Modern Pandas versions are NOT compatible with Python 3.3 - see this
I'd recommend you to use "ready to go" distributives like Anaconda with already preinstalled numpy, pandas and lot more.
Beside that - don't overwrite systme Python - it might cause another problems.
Just install Anaconda into a separate directory and set Anaconda environment everytime you want to use it.
PS alternatively you can have Anaconda as Docker
Related
I was installing firebase-admin using pip on my raspberry pi 3b+ using the following command:
pip3 install firebase-admin
However it always ends with an error saying "Error building wheel for cryptography"
Here is the full error message:
/tmp/pip-build-env-k7qo7p54/overlay/lib/python3.7/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning: Installing 'cryptography.hazmat.bindings._rust' as data is deprecated, please list it in packages.
!!
############################
# Package would be ignored #
############################
Python recognizes 'cryptography.hazmat.bindings._rust' as an importable package,
but it is not listed in the `packages` configuration of setuptools.
'cryptography.hazmat.bindings._rust' has been automatically added to the distribution only
because it may contain data files, but this behavior is likely to change
in future versions of setuptools (and therefore is considered deprecated).
Please make sure that 'cryptography.hazmat.bindings._rust' is included as a package by using
the `packages` configuration field or the proper discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" and "data files" on setuptools
documentation page.
!!
check.warn(importable)
=============================DEBUG ASSISTANCE=============================
If you are seeing a compilation error please try the following steps to
successfully install cryptography:
1) Upgrade to the latest pip and try again. This will fix errors for most
users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
2) Read https://cryptography.io/en/latest/installation/ for specific
instructions for your platform.
3) Check our frequently asked questions for more information:
https://cryptography.io/en/latest/faq/
4) Ensure you have a recent Rust toolchain installed:
https://cryptography.io/en/latest/installation/#rust
Python: 3.7.3
platform: Linux-5.10.103-v7+-armv7l-with-debian-10.13
pip: n/a
setuptools: 67.0.0
setuptools_rust: 1.5.2
rustc: n/a
=============================DEBUG ASSISTANCE=============================
error: can't find Rust compiler
If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
To update pip, run:
pip install --upgrade pip
and then retry package installation.
If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
This package requires Rust >=1.48.0.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for cryptography
Failed to build cryptography
ERROR: Could not build wheels for cryptography, which is required to install pyproject.toml-based projects
I have tried different commands like:
pip3 install firebase-admin
or
python -m pip install firebase-admin
I have upgraded pip and tried again, but still it didn't work
I faced a similar problem long time ago and solved by installing openssl. Tried it this time, it didn't work
Also tried installing rust compiler and using it but it didn't work
I am using Raspbian OS Buster on my Raspberry Pi 3
I have tried fresh installs of the OS
Still the same problem occurs
We have some issues deploying our GCP Dataflow pipeline. After some analysis, found that the latest version of apache-beam has some issues while installing. To replicate the issue I created a virtualenv and ran the below
pip install apache-beam==2.32.0
Below errors started to pop while installing the 'orjson' dependency,
Using cached orjson-3.6.3.tar.gz (548 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... error
ERROR: Command errored out with exit status 1:
command: 'c:\temp\virtu\scripts\python.exe' 'c:\temp\virtu\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' prepare_metadata_for_build_wheel 'C:\Users\prasasaw\AppData\Local\Temp\tmpmekx1jjj'
cwd: C:\Users\prasasaw\AppData\Local\Temp\pip-install-0yofoe55\orjson_62f1ca2674934a7f8c45b08e87e05a4b
Complete output (6 lines):
Cargo, the Rust package manager, is not installed or is not on PATH.
This package requires Rust and Cargo to compile extensions. Install it through
the system's package manager or via https://rustup.rs/
Note that the previous versions of Apache beam like 2.30.0 does not have the dependency on 'orjson' and it works just fine when you do
pip install apache-beam==2.30.0
I tried to install RUST but it failed for some 'pysam' dependency. So would like to know what is the correct way to install the 'orjson' dependency
I saw this GitHub issue for orjson installation but could not find much from it.
https://github.com/readthedocs/readthedocs.org/issues/7687
//Prasad.
I'm a bit late to the party, but I ran into this issue as well today. I solved it by switching to a 64-bit Python environment (I accidently installed 32-bit).
The orjson dependency was introduced in https://github.com/apache/beam/pull/14690/files. According to the comment: orjson, only available on Python 3.6 and above. You may want to check your python version.
According to the orjson docs, you need to upgrade pip to be above 20.3:
pip install --upgrade "pip>=20.3" # manylinux_x_y, universal2 wheel support
pip install --upgrade orjson
(docs)
I want to install a package from GitHub that uses Cython (https://github.com/mlysy/kalmantv). I cloned the package locally, and after trying to install it with pip install ., I'm getting the following error:
DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
Processing /Users/me/Downloads/kalmantv
Installing build dependencies ... done
Getting requirements to build wheel ... error
ERROR: Command errored out with exit status 1:
command: /Users/me/.pyenv/versions/3.8.3/bin/python3.8 /Users/me/.pyenv/versions/3.8.3/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /var/folders/g1/0pjsd_bs24jgccrd6g0lzfvc0000gn/T/tmpxmrdlgnk
cwd: /Users/me/Downloads/kalmantv
Complete output (6 lines):
running egg_info
writing kalmantv.egg-info/PKG-INFO
writing dependency_links to kalmantv.egg-info/dependency_links.txt
writing requirements to kalmantv.egg-info/requires.txt
writing top-level names to kalmantv.egg-info/top_level.txt
error: package directory 'eigen-3.3.7' does not exist
----------------------------------------
WARNING: Discarding file:///Users/me/Downloads/kalmantv. Command errored out with exit status 1: /Users/me/.pyenv/versions/3.8.3/bin/python3.8 /Users/me/.pyenv/versions/3.8.3/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /var/folders/g1/0pjsd_bs24jgccrd6g0lzfvc0000gn/T/tmpxmrdlgnk Check the logs for full command output.
ERROR: Command errored out with exit status 1: /Users/me/.pyenv/versions/3.8.3/bin/python3.8 /Users/me/.pyenv/versions/3.8.3/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /var/folders/g1/0pjsd_bs24jgccrd6g0lzfvc0000gn/T/tmpxmrdlgnk Check the logs for full command output.
I tried adding --use-feature=in-tree-build from the deprecation warning but still got the error (without the initial warning).
I saw a number of suggestions such as using pip install --upgrade pip setuptools wheel, but nothing is doing the trick. I would guess that this has a simple fix but this stuff is a little over my head and I don't want to break anything else.
What do I need to do to safely correct this issue without causing other problems?
The setup.py file has the following lines:
# path to eigen library
EIGEN_PATH = r"eigen-3.3.7"
Strangely, it seems to expect the Eigen header library to be present at that location. You can install the library with brew install eigen, or sudo apt install libeigen3-dev. Note that this may not install version 3.3.7 of the library, which the project expects. I don't know if using a newer version would cause any issues.
If you want to install version 3.3.7, you can build and install it from source with the following link by following these instructions:
https://gitlab.com/libeigen/eigen/-/releases/3.3.7
Once that's done, change to the project directory, and create a symlink to the Eigen library:
If installed with brew: ln -s /usr/local/Cellar/eigen/*/include/eigen3 eigen-3.3.7
If installed with apt: ln -s /usr/include/eigen3 eigen-3.3.7
If installed from source: /usr/local/include/eigen3 eigen-3.3.7
Once the Eigen library has been installed and made available at ./eigen-3.3.7, pip install . should work, but for good measures you should run the following command first:
pip install -U pip setuptools wheel; pip install -U cython
Refer to Catalina C++: Using <cmath> headers yield error: no member named 'signbit' in the global namespace if you run into an error like: error: no member named 'signbit' in the global namespace.
On Ubuntu 16.04.4, I suspect some recent update of some Python system package of having broken my Python 2.7 configuration. Whatever package I try to install or reinstall with a basic sudo python setup.py install it always fails because of gitignore:
running install
running bdist_egg
running egg_info
[...]
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
error: Error: setup script specifies an absolute path:
/home/me/some_repo/.gitignore
setup() arguments must *always* be /-separated paths relative to the
setup.py directory, *never* absolute paths.
Lately I found a temporary workaround by manually cleaning the /usr/local/lib/python2.7/dist-packages/some_package directory manually before installing some_package. However tonight I'm facing the same issue on another repository and it keeps failing whatever I clean up. I also tried cleaning all compiled folders .egg-info/ build/ dist/ without success.
Notes: The setup script does not actually specify an absolute path to gitignore. An example of failing repo is https://github.com/philchristensen/python-artnet/blob/master/setup.py This repo has a setuptools_git entry which might lead to a clue, but other packages without this git entry point also fail because of the gitignore while a couple of months ago I never faced such issue with the same repos. Deleting the gitignore causes the setup to fail because of another non-py local file.
Any clue?
It looks like some other package that I had previously installed broke my system-wide Python.
Here's how I fixed in order to install package xyz:
Browsed /usr/local/lib/python2.7/dist-packages in search for occurences of "gitignore"
Deleted folders of all matching occurences (including setuptools_git itself that matches "gitignore")
pip install setuptools_git
In package xyz, rm -rf dist/ build/ *.egg-info/
Reinstall package xyz, which now succeeds to install
Use virtual envs as a lesson
To preface, I have already seen this question Is it possible to use pip to install a package from a private github repository?
I am trying to install a package from a private repository that I have access to using pip.
I am able to directly clone it like so:
(myenv)robbie#ubuntu:~/git$ git clone git#github.com:matherbk/django-messages.git
Cloning into 'django-messages'...
remote: Counting objects: 913, done.
remote: Compressing objects: 100% (345/345), done.
remote: Total 913 (delta 504), reused 913 (delta 504)
Receiving objects: 100% (913/913), 165.73 KiB, done.
Resolving deltas: 100% (504/504), done.
But when I try to install it via pip (my virtualenv is activated):
(myenv)robbie#ubuntu:~/git$ pip install git+https://git#github.com/matherbk/django-messages.gitDownloading/unpacking git+https://git#github.com/matherbk/django-messages.git
Cloning https://git#github.com/matherbk/django-messages.git to /tmp/pip-13ushS-build
Password for 'https://git#github.com':
fatal: Authentication failed
Complete output from command /usr/bin/git clone -q https://git#github.com/matherbk/django-messages.git /tmp/pip-13ushS-build:
----------------------------------------
Command /usr/bin/git clone -q https://git#github.com/matherbk/django-messages.git /tmp/pip-13ushS-build failed with error code 128 in None
Storing complete log in /home/robbie/.pip/pip.log
I tried typing in my password but it failed. However I am ssh authenticated for git#github.com:
(myenv)robbie#ubuntu:~/git$ ssh -T git#github.com
Hi robpodosek! You've successfully authenticated, but GitHub does not provide shell access.
I can switch git#github.com to robpodosek#github.com and it lets me install via pip just fine:
(myenv)robbie#ubuntu:~/git$ pip install git+https://robpodosek#github.com/matherbk/django-messages.git
Downloading/unpacking git+https://robpodosek#github.com/matherbk/django-messages.git
Cloning https://robpodosek#github.com/matherbk/django-messages.git to /tmp/pip-SqEan9-build
Password for 'https://robpodosek#github.com':
Running setup.py egg_info for package from git+https://robpodosek#github.com/matherbk/django-messages.git
warning: no files found matching 'README'
Installing collected packages: django-messages
Running setup.py install for django-messages
warning: no files found matching 'README'
Successfully installed django-messages
Cleaning up...
However I want to do what the first mentioned article does by using git#github.com so that I don't have to add my username into a requirements.txt file and add that to version control.
Any thoughts? I previously had this working but had to boot up a fresh image. Thanks ahead of time.
It worked by using oxyum's suggestion of changing the : to a /:
pip install git+ssh://git#github.com/matherbk/django-messages.git
Make sure you use github.com/account instead of github.com:account
see Git+SSH dependencies have subtle (yet critical) differences from git clone
Had virtualenv activated and had to install a series of applications from github.com from a text file.
(venv)$ cat requirements.txt
-e git://github.com/boto/botocore.git#develop#egg=botocore
-e git://github.com/boto/jmespath.git#develop#egg=jmespath
-e git://github.com/boto/s3transfer.git#develop#egg=s3transfer
nose==1.3.3
mock==1.3.0
wheel==0.24.0
unittest2==0.5.1; python_version == '2.6'
(venv)$ pip install -r requirements.txt
Ignoring unittest2: markers 'python_version == "2.6"' don't match your environment Obtaining botocore from git+git://github.com/boto/botocore.git#develop#egg=botocore (from -r requirements.txt (line 1))
Cloning git://github.com/boto/botocore.git (to develop) to ./venv/src/botocore
fatal: unable to connect to github.com:
github.com[0: 192.30.253.112]: errno=Connection timed out
github.com[1: 192.30.253.113]: errno=Connection timed out
Command "git clone -q git://github.com/boto/botocore.git
/home/ubuntu/utils/boto3/venv/src/botocore" failed with error code 128 in None
However, as #Robeezy suggested, edited the requirement.txt and changed from
-e git://github.com...
to
-e git+https://github.com...
That is the link provided if you were to clone from the site (only options were Clone or Download).
So, thank you! It did work finally.
If you're installing with pip install git+https://github.com/repo and getting this error, make sure your username and password are correct. I was getting this error because I was incorrectly entering my password.