I'm trying to pip install some python libraries in a virtual environment created by conda create, but for some packages, the installation were stuck on the step "Installing collected packages: .
Take pandas as an example:
My command and output are as follows:
pip install pandas --no-cache-dir
Collecting pandas
Downloading https://files.pythonhosted.org/packages/99/12/bf4c58eea94cea4f91ff931f284146337814fb8546e6eb0b52584446fd52/pandas-0.24.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (16.3MB)
100% |████████████████████████████████| 16.3MB 11.4MB/s
Requirement already satisfied: numpy>=1.12.0 in /anaconda/envs/testctds2/lib/python3.6/site-packages (from pandas) (1.16.1)
Requirement already satisfied: pytz>=2011k in /anaconda/envs/testctds2/lib/python3.6/site-packages (from pandas) (2018.9)
Requirement already satisfied: python-dateutil>=2.5.0 in /anaconda/envs/testctds2/lib/python3.6/site-packages (from pandas) (2.8.0)
Requirement already satisfied: six>=1.5 in /anaconda/envs/testctds2/lib/python3.6/site-packages (from python-dateutil>=2.5.0->pandas) (1.12.0)
Installing collected packages: pandas
The process just hang there (at least for 30 min) until I control+z to quit (control+c got no response).
What I have tried:
conda install pandas worked well, which is also the recommended way to install pandas. I just don't understand why pip install didn't work, as it's assumed to and this situation also happened to some other libraries such as numpy, scipy, and scikit-learn.
I also tried without --no-cache-dir or -vvv to see more details, but in either case there were no more information or error code after the line "Installing collected packages: pandas"
I tried the command in a new terminal window. Magically numpy can be installed very quickly by "pip install numpy", but it didn't work with pandas or scipy.
I see this may be a problem other users are having. Here is a github link describing the same problem. There are a few others on the Conda GitHub page. Some of the answers that come from that post are:
Make sure you are up to date on your root conda environment. try: conda upgrade conda
Create a brand new virtual env
Micheal Grant, who is a Director for Technical Consulting at Anaconda replied to that thread with this:
That said, when I look at the debug output, I'm finding that it's not able to prune back the package list very well. The more "old" packages it has to consider the higher the likelihood that this kind of solver stall happens. Thankfully it is a lot less likely than it used to be.
Related
import pandas
So I am working on a python project and want to import pandas library on https://replit.com/~
For some reasons, it shows some attribute error when I run the project.
Does anyone know ho do I fix or manually install pandas on replit?
Attaching Screenshot of an error herewith.
Usually packages have a lot of errors in replit but you can try this: -
Pandas does actually work on repl.it - you have to install it from the package manager. To do so, click the cube on the side navigation bar and type pandas into the search box. Then click on the pandas search entry and hit the plus sign. Tell me if this works!
Or
Broken package installs can usually be fixed by,
Updating pip and installing pandas from PyPI. By default, Repl.it comes with pip version 19.3.1, but the latest available version for python 3.8 is pip-21.1.1.
~/repl$ pip -V
pip 19.3.1 from /opt/virtualenvs/python3/lib/python3.8/site-packages/pip (python 3.8)
~/repl$ pip install pandas
Requirement already satisfied: numpy>=1.16.5 in /opt/virtualenvs/python3/lib/python3.8/site-packages (from pandas) (1.20.2)
Collecting pytz>=2017.3
Using cached https://files.pythonhosted.org/packages/70/94/784178ca5dd892a98f113cdd923372024dc04b8d40abe77ca76b5fb90ca6/pytz-2021.1-py2.py3-none-any.whl
Requirement already satisfied: python-dateutil>=2.7.3 in /opt/virtualenvs/python3/lib/python3.8/site-packages (from pandas) (2.8.1)
Requirement already satisfied: six>=1.5 in /opt/virtualenvs/python3/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas) (1.15.0)
Installing collected packages: pytz, pandas
Successfully installed pandas-1.2.4 pytz-2021.1
Pandas does not work on replit at all, because a lot of modules, which Pandas needs to work properly, do not work in replit. An online Compiler is also not the best choice for doing dataprocessing, it would be better if you install an Interpreter for python on your PC.
I want to install this library: https://github.com/stefanfoulis/django-phonenumber-field
I have done like this:
(venv) michael#michael:~/PycharmProjects/client$ pip install django-phonenumber-field[phonenumbers]
Requirement already satisfied: django-phonenumber-field[phonenumbers] in ./venv/lib/python3.8/site-packages (5.1.0)
Requirement already satisfied: Django>=2.2 in ./venv/lib/python3.8/site-packages (from django-phonenumber-field[phonenumbers]) (3.2.3)
Collecting phonenumbers>=7.0.2; extra == "phonenumbers"
Downloading phonenumbers-8.12.23-py2.py3-none-any.whl (2.6 MB)
|████████████████████████████████| 2.6 MB 2.8 MB/s
Requirement already satisfied: sqlparse>=0.2.2 in ./venv/lib/python3.8/site-packages (from Django>=2.2->django-phonenumber-field[phonenumbers]) (0.4.1)
Requirement already satisfied: asgiref<4,>=3.3.2 in ./venv/lib/python3.8/site-packages (from Django>=2.2->django-phonenumber-field[phonenumbers]) (3.3.4)
Requirement already satisfied: pytz in ./venv/lib/python3.8/site-packages (from Django>=2.2->django-phonenumber-field[phonenumbers]) (2021.1)
Installing collected packages: phonenumbers
Successfully installed phonenumbers-8.12.23
We can see that this library has installed its dependency: phonenumbers library.
But now I want to write in requirements.txt that I need django-phonenumber-field of a certain release. So that its dependency should be installed as appropriate.
In other words how to write requirements.txt for future pip install -r requirements.txt.
Could you help me here?
=====ADDED LATER=======
pip freeze > requirements.txt doesn't seem to be appropriate as it will contain all the dependencies. It will become overburden with dependencies. I will forget why a particular library is installed. Whether it is a dependency at all. If it is, what library depends on it.
I want to keep requirements.txt as clear as possible.
In this particular example I want just one directive: install django-phonenumber-field==5.1.0 and manage dependencies yourself.
Wrong way (bloated and unclear what to delete if I stop using the library):
django-phonenumber-field==5.1.0
phonenumbers==8.12.23
What I want:
django-phonenumber-field==5.1.0 [and all dependencies]
As #jonrsharpe said, you can use pip freeze which lists all your dependencies, and redirect the output to your requirements.txt file to update it.
pip freeze > requirements.txt
The simple way is to create a text file, type the shell commands in, save it and the rename it from .txt to .bat. Now you can double click the file and it installs the requirements. You could probably even execute the .bat
I am trying to update pandas within a lifecycle configuration, and following the example of AWS I have the next code:
#!/bin/bash
set -e
# OVERVIEW
# This script installs a single pip package in a single SageMaker conda environments.
sudo -u ec2-user -i <<EOF
# PARAMETERS
PACKAGE=pandas
ENVIRONMENT=python3
source /home/ec2-user/anaconda3/bin/activate "$ENVIRONMENT"
pip install --upgrade "$PACKAGE"==0.25.3
source /home/ec2-user/anaconda3/bin/deactivate
EOF
Then I attach it to a notebook and when I enter the notebook and open a notebook file, I see that pandas have not been updated. Using !pip show pandas I get:
Name: pandas
Version: 0.24.2
Summary: Powerful data structures for data analysis, time series, and statistics
Home-page: http://pandas.pydata.org
Author: None
Author-email: None
License: BSD
Location: /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages
Requires: pytz, python-dateutil, numpy
Required-by: sparkmagic, seaborn, odo, hdijupyterutils, autovizwidget
So we can see that I am indeed in the python3 env although the version is 0.24.
However, the log in cloudwatch shows that it has been installed:
Collecting pandas==0.25.3 Downloading https://files.pythonhosted.org/packages/52/3f/f6a428599e0d4497e1595030965b5ba455fd8ade6e977e3c819973c4b41d/pandas-0.25.3-cp36-cp36m-manylinux1_x86_64.whl (10.4MB)
2020-02-03T12:33:09.065+01:00
Requirement already satisfied, skipping upgrade: pytz>=2017.2 in ./anaconda3/lib/python3.6/site-packages (from pandas==0.25.3) (2018.4)
2020-02-03T12:33:09.065+01:00
Requirement already satisfied, skipping upgrade: python-dateutil>=2.6.1 in ./anaconda3/lib/python3.6/site-packages (from pandas==0.25.3) (2.7.3)
2020-02-03T12:33:09.065+01:00
Requirement already satisfied, skipping upgrade: numpy>=1.13.3 in ./anaconda3/lib/python3.6/site-packages (from pandas==0.25.3) (1.16.4)
2020-02-03T12:33:09.065+01:00
Requirement already satisfied, skipping upgrade: six>=1.5 in ./anaconda3/lib/python3.6/site-packages (from python-dateutil>=2.6.1->pandas==0.25.3) (1.13.0)
2020-02-03T12:33:09.065+01:00
Installing collected packages: pandas Found existing installation: pandas 0.24.2 Uninstalling pandas-0.24.2: Successfully uninstalled pandas-0.24.2
2020-02-03T12:33:12.066+01:00
Successfully installed pandas-0.25.3
What could be the problem?
if you want to install the packages only in for the python3 environment, use the following script in your Create Sagemaker Lifecycle configurations.
#!/bin/bash
sudo -u ec2-user -i <<'EOF'
# This will affect only the Jupyter kernel called "conda_python3".
source activate python3
# Replace myPackage with the name of the package you want to install.
pip install pandas==0.25.3
# You can also perform "conda install" here as well.
source deactivate
EOF
Reference : "Lifecycle Configuration Best Practices"
I have encountered the exact same problem when package was not available in the notebook while Lifecycle Cloudwatch indicated successful installation for the specific kernel. The solution that worked for me is to make sure installation completes before opening up notebook.
I am trying to install wxpython (I have python 3.5.2(32Bit) and Windows 10(64Bit))
I tried :pip install wx and i get: Failed building wheel for wxpython-phoenix 2 times and then a big error line-->
I also tried installing Anaconda, but it did not help.How can I overcome the problem ?
There's a lot of questions about failed installations on windows, probably because the packages are compiled when installed, and sometimes the compilation fail when the compiler is not properly configured. I admit I decided to let the specialized people handle it.
As a nice workaround, you can grab the already built distribution at the official wxpython website located here:
Python 3.5 32/64 bit windows:
https://wxpython.org/Phoenix/snapshot-builds/wxPython_Phoenix-3.0.3.dev2700+c524ed1-cp35-cp35m-win_amd64.whl
https://wxpython.org/Phoenix/snapshot-builds/wxPython_Phoenix-3.0.3.dev2700+c524ed1-cp35-cp35m-win32.whl
just download the .whl file and do for example:
C:\python35\scripts\pip install wxPython_Phoenix-3.0.3.dev2700+c524ed1-cp35-cp35m-win_amd64.whl
For linux you need to go here first
https://extras.wxpython.org/wxPython4/extras/linux/gtk3/
Then grab your linux version and do the install.
(base) fsdfsdfd#linux:~/Downloads$ pip install wxPython-4.1.1-cp38-cp38-linux_x86_64.whl
Processing ./wxPython-4.1.1-cp38-cp38-linux_x86_64.whl
Requirement already satisfied: pillow in /home/orangel/miniconda3/lib/python3.8/site-packages (from wxPython==4.1.1) (8.1.2)
Requirement already satisfied: numpy; python_version >= "3.0" in /home/orangel/miniconda3/lib/python3.8/site-packages (from wxPython==4.1.1) (1.20.1)
Requirement already satisfied: six in /home/orangel/miniconda3/lib/python3.8/site-packages (from wxPython==4.1.1) (1.15.0)
Installing collected packages: wxPython
Successfully installed wxPython-4.1.1
if does not works, please try this
sudo apt install libwxgtk3.0-gtk3-dev
I am using python jira package installed using PIP in a virtual environment. Recently my script started to complain about jira package version:
$ ./my_script.sh
jira/client.py:282: UserWarning: You are running an outdated version of JIRA Python 1.0.3. Current version is 1.0.6.dev20160420173258. Do not file any bugs against older versions.
I tried upgrade using pip like:
$ pip install --upgrade --no-cache-dir jira
Collecting jira
Downloading jira-1.0.3-py2.py3-none-any.whl (46kB)
100% |████████████████████████████████| 51kB 175kB/s
Requirement already up-to-date: requests>=2.6.0 in <...>/.virtualenvs/jira/lib/python3.4/site-packages (from jira)
Requirement already up-to-date: requests-oauthlib>=0.3.3 in <...>/.virtualenvs/jira/lib/python3.4/site-packages (from jira)
Requirement already up-to-date: six>=1.9.0 in <...>/.virtualenvs/jira/lib/python3.4/site-packages (from jira)
Requirement already up-to-date: requests-toolbelt in <...>/.virtualenvs/jira/lib/python3.4/site-packages (from jira)
Requirement already up-to-date: tlslite>=0.4.4 in <...>/.virtualenvs/jira/lib/python3.4/site-packages (from jira)
Requirement already up-to-date: oauthlib>=0.6.2 in <...>/.virtualenvs/jira/lib/python3.4/site-packages (from requests-oauthlib>=0.3.3->jira)
Installing collected packages: jira
Successfully installed jira-1.0.3
I tried to remove installed jira package and install it freshly with same result. Pip always installs only version 1.0.3 but scripts complain about newer version.
My assumption is that 1.0.6 is marked as released (the check is inside the package itself) but not published (I don´t know if this is the right word) for pip to download.
Any clue?
Regards,
JrBenito
It appears there is a 1.0.6.dev20160420173258 version but this isn't being downloaded when using pip install jira. It can be installed by using the workaround found on the issue #156 for this new version. pip install https://pypi.python.org/packages/f6/ea/2535e412ff76d85da20d2be6d1eaf9aa5de49481da94f2fe7e8830eedd35/jira-1.0.6.dev20160420173258-py2.py3-none-any.whl Which it appears you have already commented on, so hopefully they resolve the issue permanently.
I had this same issue, even after specifically downloading the 1.0.6.dev20160420173258 version. When the client.py file gets the version information, it doesn't get the git changeset correctly and so returns 1.0.6 instead of 1.0.6.dev20160420173258.
For now I made a workaround by hardcoding the version number pulled from https://pypi.python.org/pypi/jira/json
In /usr/lib/python2.7/site-packages/jira/client.py:
released_version = "1.0.6" # data['info']['version']
This is admittedly not a fix but lets hope it gets fixed.
It worked for me only after changed it to
/usr/lib/python2.7/site-packages/jira/client.py:
with
released_version = "1.0.6" # data['info']['version']