We Keep Coding

Pip install from private Git repo, with Personal access token in Git URL

I am trying to install a package from a private repository on Git.
I am using Personal Access Token in my Git URL in order to bypass the manual authentication step. (You can read about Personal Access Tokens here)
If I add this git URL in requirements file and then use the requirements file in pip to install build it works.
requirements.txt
<package name> # git+https://<Personal Access Token>#<git server address>/<username>/<repository name>.git#<branch name>#egg=<package name>
But, if I use the same URL directly it asks for password, how do I avoid this password prompt (as mentioned below):
pip install git+https://<Personal Access Token>#<git server address>/<username>/<repository name>.git#<branch name>#egg=<package name>
This issue is not observed on all machines that i tested on. It worked on Win 10 x64 and Win 10 x86. But it didn't work on Ubuntu x64. I made sure all the 3 systems has same Python version (3.8.0) and same Pip version (19.3.1).

Use environment variables with the syntax ${VARIABLE} (POSIX format, upper case and underscores allowed) so you're not hard-coding your secrets.
Pip will replace when installing from requirements.txt.
So you can refer to a token to clone the private repo, for example:
in requirements.txt
Github
git+https://${GITHUB_TOKEN}#github.com/user/project.git#{version}
Gitlab
git+https://${GITLAB_TOKEN_USER}:${GITLAB_TOKEN}#gitlab.com/user/project.git#{version}
Bitbucket
git+https://${BITBUCKET_USER}:${BITBUCKET_APP_PASSWORD}#bitbucket.org/user/project.git#{version}
More info here:
https://docs.readthedocs.io/en/stable/guides/private-python-packages.html

Go to GitLab profile settings and generate an read access token:
Select access tokens
give it a name (you can leave expiration date empty)
give it access to read all repositories you have access
generate it
Now edit your requirement file:
pandas==1.0.5
git+https://yourgitlabuser:<generated_token>#gitlab/group/repo#hash#egg=piplib
requests==2.24.0

I just had the same issue. In the end, I could install the package as follows.
from the command line:
pip install mypackagename --no-deps --index-url https://gitlab+deploy-token-mytokenname:tokenpassword#gitlab.mycompany.com/api/v4/projects/123456789/packages/pypi/simple
by specifying it in the requirements.txt file:
(Note that the flask and flask-cors package requirements in the example below are just an example, because it may seem really weird to a reader that the other lines in the example are really content that can be written in a requirements.txt.)
flask==1.1.1
flask-cors==3.0.8
--index-url https://pypi.org/simple --extra-index-url https://gitlab+deploy-token-mytokenname:tokenpassword#gitlab.mycompany.com/api/v4/projects/123456789/packages/pypi/simple
mypackagename
Then of course run pip install -r requirements.txt.
Note that both fragments above show how to provide your password, as you asked.

django Failed to build mysqlclient [duplicate]

I want to install MySqlclient on my windows system. I am Currently using Python 3.6. After going through the various post over Stackoverflow, I could Not find the correct way.
This is what I have done so far:
1) Installation by using pip pip install mysqlclient. Error:
Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools" http://landinghub.visualstudio.com/visual-cpp-build-tools
I already have Microsoft Visual C++ installed on my laptop. Some are saying you need 2015 edition.
2) Installation by using wheel file pip install mysqlclient-1.3.13-cp36-cp36m-win_amd64.whl. Error:
Requirement mysqlclient-1.3.13-cp36-cp36m-win_amd64.whl looks like a filename, but the file does not exist.
mysqlclient-1.3.13-cp36-cp36m-win_amd64.whl is not a supported wheel on this platform.
2.1) Changing the whl file to different version pip install mysqlclient-1.3.13-cp36-cp36m-win32.whl. Error:
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: 'C:\\Users\\Foxtrot\\Desktop\\finaltest\\mysqlclient-1.3.13-cp36-cp36m-win32.whl'
Other things that are done: updated setuptools, updated wheel.

Had the same problem, searched the web etc. Here this answer:
mysql-python install error: Cannot open include file 'config-win.h'
It has all the instructions. In short go to this site: https://www.lfd.uci.edu/~gohlke/pythonlibs/#mysqlclient:
At that website you will find
mysqlclient‑1.3.13‑cp36‑cp36m‑win32.whl
mysqlclient‑1.3.13‑cp36‑cp36m‑win_amd64.whl
Download the correct file for your platform.
Then use your downloaded wheels file with pip and you're done:
pip install c:\mysqlclient‑1.3.13‑cp36‑cp36m‑win_amd64.whl
The https://www.lfd.uci.edu/~gohlke/pythonlibs has lots of lots of compiled libraries to solve the problem of building them from source yourself. They even compile them for python 3.7 :)
Alternative Solution
You can also download Visual C++ Build Tools and then you should be able to install every (at least to my knowledge) version of mysqlclient with pip.
To do this go to this site: https://www.scivision.co/python-windows-visual-c++-14-required/ there you can find out which version of Build Tools you need and you can also find a link to download the installer. Be aware though Build Tools require more than 4GB of free disk space.

Tell pip not to use sources and use binary packages instead:
pip install --only-binary :all: mysqlclient
https://pip.pypa.io/en/stable/reference/pip_install/#install-only-binary

I can't find mysqlclient-1.3.13's whl file on PyPi. So you need to compile it from source. Unfortunately it's not easy. I'm not Windows guy, so I only can recommend guide like this

I am using python3.7 on Windows 10 operating system.
I had same issue and after a long research I had installed it successfully.
Install "Microsoft Visual C++ Build Tools"
AND
My OS is having 64 bit operating system but still then it need to install 32 bit version
"mysqlclient‑1.4.2‑cp37‑cp37m‑win32.whl"
Download binary wheels from "https://www.lfd.uci.edu/~gohlke/pythonlibs/" and run command
pip install [path_to_downloaded_file] eg: C:\Users\Ds\mysqlclient-1.4.2-cp37-cp37m-win32.whl
use pipenv instead of pip if you are using virtual environment.

The error means that the package has not yet been compiled for your versions of OS and Python. So pip tries to build it from the source for you.
There are two possible solutions.
The first option is to install the most recent version of Microsoft Visual C++ Build Tools. Just go ahead and download it from the Microsoft website. Then pip should be able to compile the package.
Another option is using an unofficial binary. As mentioned here, a resource proved to be useful is https://www.lfd.uci.edu/~gohlke/pythonlibs/#mysql-python . Just download the pre-compiled package and install it using
pip install c:\path-to-a-pre-compiled-package

Had the same problem just day.
Tried to install mysqlclient on a Windows Server R2.
[...]
Tl;dr
"MySQL Connector C 6.1" was installed in the wrong directory: "C:\Program Files\MySQL" instead of "C:\Program Files (x86)\MySQL" where it should be for me.
--> Copied "MySQL Connector C 6.1" to "C:\Program Files (x86)\MySQL" Directory.
"C:\Users\MoBoo\AppData\Local\Temp" was Read-Only: Therefore pip couldn't compile files into Temp dir.
--> Allow Write access to "C:\Users\MoBoo\AppData\Local\Temp" Directory.

Here is what worked for me. I uninstalled mysql and re-installed it.
pip uninstall mysqlclient
Then simply re-install, so it picked the current version "1.4.2.post1"
pip install mysqlclient
Which interestingly, works straightaway.

for this error, most of user's suggest to install vs build but there is an alternative which works perfectly in my case and is sure for you too.
Download latest MySQL client from here
mysqlclients
Here you can see many version but prefer to download the latest one which has 32 bit and 64-bit files.
download theme and past the file on your projects root folder then run the same command but with the full file name of downloaded mysqlclient.
like: pip install mysqlclient‑1.4.6‑cp38‑cp38‑win32.whl
in my case, the file is this
also if have already the XAMPP server then you can use its PHPMyAdmin with python.
You just need to change on your roots setting.py file for this.
Something like this
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mydjango',
'USER': 'root',
'PASSWORD':'',
'HOST':'localhost',
'PORT':'3306',
}
}
The port is the same which you see on xampp panel just before the start button of MySQL.
After changing this you just again start your server by hitting this command
python manage.py runserver
If you didn't see any error then congratulations you successfully connected with MySQL database.
Enjoy...

The easiest way to solve this problem is to download the correct version of MySQL client that supports the python version installed on your system.
MYSQLclient download link: https://pypi.org/project/mysqlclient/#files
Check the python version installed in your PC:

I was using Python version 3.7 and the same error was happening.
After trying all the possibilities, simply reinstalling the newest Python version (3.10.7 in my case) solved the issue.

python - pip: What are the files installed exactly?

I am trying to upgrade a python package called "bokeh", that is containing an "examples" directory in its ditribution files:
Here the link to the distribution file:
https://pypi.python.org/pypi/bokeh/0.12.10
And here an image showing the content of the tar.gz file:
I can see the "examples" directory that i am after is present.
However if i pip install --upgrade bokeh, only the directory "bokeh" (the first one in the image) is installed on my machine.
How can i have this "example" directory to be install as well eventually?

It is meant that way because of the sample data size needed for the examples provided. This is from the documentation:
Some of the Bokeh examples rely on sample data that is not included in
the Bokeh GitHub repository or released packages, due to their size.
Once Bokeh is installed, the sample data can be obtained by executing
the following command at a Bash or Windows prompt:
bokeh sampledata
or, run this in your python interpreter:
import bokeh.sampledata
bokeh.sampledata.download()

Normal pip installations are not supposed to be "looked into" by the user. Depending on your system they might end up in
/usr/local/lib/python2.7/site-packages
or
$PYTHON_HOME/lib/python2.7/site-packages
or somewhere entirely else.
Instead you should Git clone the repo (or download the tarball from there) and install the directory (an editable install)
git clone https://github.com/bokeh/bokeh.git
pip install -e bokeh/
that way the library you are using is directly next to the examples you are using:
import bokeh
directly imports the code you just downloaded.
Alternatively you can install bokeh from PyPI and just git clone or download the repository to get the examples. But you should be aware that
import bokeh
doesn't import the library you just downloaded, but the one you installed earlier.

Python pip install gives “Command ”python setup.py egg_info“ failed with error code 1”

Edit: Yes I know this question already exists, except my question is a bit different and none of the solutions fixed it.
I do most of my Python stuff when I'm at work and not on my personal machine, but I decided to install it on my personal computer as well. I fresh installed python 3.6.1, and created a virtual environment with virtualenv. Then within the virtualenv I tried to pip install urllib (or any module) and I received the error:
(pdbot) C:\Users\user\Documents\pdbot>pip install urllib
Collecting urllib
Using cached urllib-1.21.1.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\setup.py", line 191
s.connect((base64.b64decode(rip), 017620))
^
SyntaxError: invalid token
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\
I read elsewhere that this error had something to do with setuptools not being properly installed. So I ran this to attempt to fix the issue:
easy_install -U setuptools
I ended up receiving an even weirder error next:
(pdbot) C:\Users\zeke\Documents\pdbot>easy_install -U setuptools
Searching for setuptools
Reading https://pypi.python.org/simple/setuptools/
Downloading https://pypi.python.org/packages/a9/23/720c7558ba6ad3e0f5ad01e0d6ea2288b486da32f053c73e259f7c392042/setuptools-36.0.1.zip#md5=430eb106788183eefe9f444a300007f0
Best match: setuptools 36.0.1
Processing setuptools-36.0.1.zip
Writing C:\Users\zeke\AppData\Local\Temp\easy_install-jhg1val_\setuptools-36.0.1\setup.cfg
Running setuptools-36.0.1\setup.py -q bdist_egg --dist-dir C:\Users\zeke\AppData\Local\Temp\easy_install-jhg1val_\setuptools-36.0.1\egg-dist-tmp-8apak7kn
warning: no files found matching '*' under directory 'setuptools\_vendor'
Copying setuptools-36.0.1-py3.6.egg to c:\users\zeke\documents\pdbot\lib\site-packages
Adding setuptools 36.0.1 to easy-install.pth file
Installing easy_install-script.py script to c:\users\zeke\documents\pdbot\Scripts
Installing easy_install.exe script to c:\users\zeke\documents\pdbot\Scripts
error: [WinError 5] Access is denied: 'c:\\users\\zeke\\documents\\pdbot\\Scripts\\easy_install.exe'
This looks like a permissions error, but I ran these both in an administrator command prompt (Windows 10) and got the same result. I am the only user on this computer and I have all admin permissions. Is this virtualenv causing an issue? How do I remedy it?
EDIT: I was able to fix the permissions issue by leveraging the python executable like so:
python -m easy_install -U setuptools
But it didn't fix the python setup.py egg_info issue. I still get this error message when trying to pip install anything:
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\
I have tried both python -m pip install urllib and pip install urllib and neither work.

I had the same problem when trying to install urllib, but after doing a pip search urllib, I discovered that the problem was due to the version of urllib. From the search:
$ pip search urllib
...
> urllib5 (5.0.0) - Just increment the number and create a new lib. Never fix the original one.
At the end, a simple
pip install urllib5
within an elevated shell solved it.

Your problem has to do with permissions. The related/similar tools setup_tools, easy_install, and pip all tend to set a default set of permissions on files and folders they try to create in the package installation folder(s), rather than trying to match access permissions of the location they're installing in.
On Linux systems, where files and folders individually have permissions, this is frequently bypassed with the sudo command. On Windows, the equivalent is to run the installer as an Administrator. Since you're in the console, you have to open a console with Administrator privileges to run the pip command in.
Notable under Windows, the modules installed with pip from an Administrator console are still accessible to all users of the system that have the proper path in the PYTHONPATH system environment variable. Under Linux however, the problem is exacerbated by the fact that the files themselves may not be created with read and execute access for other users and may need to have their permissions manually modified after installation.
WARNING: urllib vs urllib2 vs urllibx
Both other answers claim that the problem is you're not specifying the correct "version" of the module in the call to pip. Neither is correct, as the error clearly indicates an installation folder access permissions violation causing the failure, but they also incorrectly recommended VERY unsafe behavior.
pip install urllib != pip install urllib5 these are two completely different packages.
The documentation for pip (https://packaging.python.org/tutorials/installing-packages/#id17) clearly says the way to specify a module version explicitly is pip install 'urllib==5'.
As part of how the package management engine implemented by pip works, running the command pip install urllib will always try to use the latest version of the urllib package, so you shouldn't need to specify the version unless you have some reason that you need a very specific version of the module.

There are two points to make in order to answer your question:
1. You are lucky you did not install that package!
The package you were trying to install was a maliciously created python package that was designed to look like a real package (in this case urllib3). If you had installed it, the package would have operated as normal except it would have sent some basic information about the system on which you installed the package to a URL (you can see more details on this here). You can read more about this fake package at either of the following links:
https://app.threatconnect.com/auth/incident/incident.xhtml?incident=5256822&owner=Common%20Community (you can sign up for a free account to view this one)
http://www.nbu.gov.sk/skcsirt-sa-20170909-pypi/index.html
Sending basic information about your systems to an unknown source isn't the worst thing you could do, but is certainly something you want to avoid when possible.
2. To properly install a package...
Specifically urllib:
To install urllib, you need to specify the version of the package you would like to install. For example, pip install urllib3.
Any package in general:
As #Elisabete Coelho suggested, you can use the pip search <package-name> feature to view the available packages. This is not perfect, however, as it may list malicious libraries like the one you were trying to install. A good guideline is that you should follow the installation instructions in a package's documentation closely to avoid any unforeseen issues. This is just an unfortunate necessity of living in a world where people make pretend python packages.

Updating the pip.conf file to use https

I am having trouble configuring my pip.conf file to stipulate that PIP should look for downloads from
https://pypi.python.org/simple/ rather than http://pypi.python.org/simple/
I have a related question posted PIP Could not find any downloads that satisfy the requirement SQLAlchemy where a couple of people diagnosed what was going wrong. However I am having trouble putting in place the solution.
I opend my pip.conf file located at /home/user/.pip/pip.conf using the pip config file documentation I added the below. There is nothing else in the file.
[global]
timeout = 60
find-links = https://pypi.python.org/simple/
[install]
find-links = https://pypi.python.org/simple/
After saving an exiting there is no change. I still cannot run commands such as pip install
$ sudo pip install SQLAlchemy
Downloading/unpacking SQLAlchemy
Cannot fetch index base URL http://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement SQLAlchemy
No distributions at all found for SQLAlchemy
Storing complete log in /home/user/.pip/pip.log
My setup
Ubunto 12.04 VM

You should use index-url = https://pypi.python.org/simple/ rather than find-links in your config. This will replace the default rather than just adding another option (which is what find-links does).

From latest guide: https://pip.pypa.io/en/stable/topics/configuration/
The new default configuration file is: $HOME/.config/pip/pip.conf
Legacy per-user configuration file which is also respected:
On Unix and macOS the configuration file is: $HOME/.pip/pip.conf

Update from 2011-04-04 version to latest one
pip version 1.0 is fairly old, pypi shows date 2011-04-04. Consider installing latest version.
Since 2011 there were some significant changes (security fixes, support for wheel format...)
I would first remove completely the pip installed by apt-get, incl. cleaning whatever is in directories around. And then install using get-pip.py as described on pypa.
I know, that it is often recommended following Linux distribution packages, but with pip it is simply not practical.

I think your changes are not being used.
pip install --find-links=https://pypi.python.org/simple/ SQLAlchemy works on my system.
. The log says it is checking https://pypi.python.org/simple.