We Keep Coding

Use setuptools to Install a Python package from a private Gitlab package repository

I created a private package for my employer. Since I’m forbidden to upload it to PyPI (it’s proprietary), I uploaded it to the packages index for my project on our private Gitlab hub. I can install it manually with:
$ pip install my-package --extra-index-url https://__token__:my-token-xxx#gitlab.company-domain.com/api/v4/projects/123456/packages/pypi/simple
Now I also want setuptools to be able to find it when listed in the install_requires argument to setup(). I tried:
setup(
install_requires=[
f"my-package # https://__token__:{API_TOKEN}#gitlab.company-domain.com/api/v4/projects/123456/packages/pypi/simple",
...
],
...
pip install -e . results in
ERROR: HTTP error 404 while getting https://__token__:****#gitlab.company-domain.com/api/v4/projects/123456/packages/pypi/simple
This is different than
my-package # git+https://user:password#gitlab.company-domain.com/..../my-package.git
That works, but I want to be able to download it as a pre-built wheel.
I’m not sure whether this is a setuptools issue or a Gitlab issue. The 404 response tells me that it might be a gitlab issue, yet the same URI works perfectly when used with the pip install CLI command.
This question is similar to Include python packages from gitlab's package registry and other external indexes directly into setup.py dependencies, but I don't think that one got sufficient response. I posted the same question to discuss.python.org, but that discussion is old and I think I might get a quicker response here.
I also found this response to a similar question, which wasn't encouraging. It recommends Poetry or Pipenv. I've tried both, and found each to be excruciatingly slow when resolving dependencies, so I fell back on setuptools.

Only include the package name in install_requires. Then, configure your (extra) index URL in your pip configuration (either environment variables or pip.conf/.pypirc or CLI argument). Then using pip install as normal will work.
For example:
In setup.py:
# ...
install_requires=[
'my-package-name',
# ...
],
# ...
Then the install command (assuming the environment variable API_TOKEN exists):
GITLAB_INDEX="https://__token__:${API_TOKEN}#gitlab.company-domain.com/api/v4/projects/123456/packages/pypi/simple"
pip install --extra-index-url "${GITLAB_INDEX}" -e .

'PyJWT' is not recognized as an internal or external command

I was able to decode JWT in either CMD or PowerShell and get a JSON result:
pyjwt decode --no-verify eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c
However, after re-installing Python a few times, this no longer works and I'm getting 'pyjwt' is not recognized as an internal or external command.
Things I've tried:
Reinstall Python 3.8.x from installer
Reinstall Python 3.8.x from Microsoft Store
Checked PATH Environment Variable to Python.exe
Reinstall PyJWT using pip install pyjwt
Tried installing pyjwt version 1.4.0, 1.7.1, and 2.0.1, same problem
Is there something I'm missing from my Environment Variables? I'm not sure why it's no longer working. http://pypi.org/project/PyJWT/1.7.1. shows this example under Decoding examples, and my command looks like the example.

Well
Try to add Python folder/Script to PATH by using instruction on this https://www.google.co.th/url?sa=t&source=web&rct=j&url=https://www.architectryan.com/2018/03/17/add-to-the-path-on-windows-10/&ved=2ahUKEwjM1fXu9pXwAhWyxTgGHeJKDpIQFjAKegQIGxAC&usg=AOvVaw0PG5BF3E1K_EAiJneUN07T

pip show pyjwt may show you where the package was installed. Then add whatever path that contains the pyjwt executable to your environment path variable

The command pyjwt has been dropped since 2.0.0:
https://github.com/jpadilla/pyjwt/blob/master/CHANGELOG.rst#v200
See "Drop CLI":
Dropped the included cli entry point

Pip install from private Git repo, with Personal access token in Git URL

I am trying to install a package from a private repository on Git.
I am using Personal Access Token in my Git URL in order to bypass the manual authentication step. (You can read about Personal Access Tokens here)
If I add this git URL in requirements file and then use the requirements file in pip to install build it works.
requirements.txt
<package name> # git+https://<Personal Access Token>#<git server address>/<username>/<repository name>.git#<branch name>#egg=<package name>
But, if I use the same URL directly it asks for password, how do I avoid this password prompt (as mentioned below):
pip install git+https://<Personal Access Token>#<git server address>/<username>/<repository name>.git#<branch name>#egg=<package name>
This issue is not observed on all machines that i tested on. It worked on Win 10 x64 and Win 10 x86. But it didn't work on Ubuntu x64. I made sure all the 3 systems has same Python version (3.8.0) and same Pip version (19.3.1).

Use environment variables with the syntax ${VARIABLE} (POSIX format, upper case and underscores allowed) so you're not hard-coding your secrets.
Pip will replace when installing from requirements.txt.
So you can refer to a token to clone the private repo, for example:
in requirements.txt
Github
git+https://${GITHUB_TOKEN}#github.com/user/project.git#{version}
Gitlab
git+https://${GITLAB_TOKEN_USER}:${GITLAB_TOKEN}#gitlab.com/user/project.git#{version}
Bitbucket
git+https://${BITBUCKET_USER}:${BITBUCKET_APP_PASSWORD}#bitbucket.org/user/project.git#{version}
More info here:
https://docs.readthedocs.io/en/stable/guides/private-python-packages.html

Go to GitLab profile settings and generate an read access token:
Select access tokens
give it a name (you can leave expiration date empty)
give it access to read all repositories you have access
generate it
Now edit your requirement file:
pandas==1.0.5
git+https://yourgitlabuser:<generated_token>#gitlab/group/repo#hash#egg=piplib
requests==2.24.0

I just had the same issue. In the end, I could install the package as follows.
from the command line:
pip install mypackagename --no-deps --index-url https://gitlab+deploy-token-mytokenname:tokenpassword#gitlab.mycompany.com/api/v4/projects/123456789/packages/pypi/simple
by specifying it in the requirements.txt file:
(Note that the flask and flask-cors package requirements in the example below are just an example, because it may seem really weird to a reader that the other lines in the example are really content that can be written in a requirements.txt.)
flask==1.1.1
flask-cors==3.0.8
--index-url https://pypi.org/simple --extra-index-url https://gitlab+deploy-token-mytokenname:tokenpassword#gitlab.mycompany.com/api/v4/projects/123456789/packages/pypi/simple
mypackagename
Then of course run pip install -r requirements.txt.
Note that both fragments above show how to provide your password, as you asked.

Conda environment from environment.yml - private package from git?

I use conda environment.yml for CircleCI continuous integration. It all works fine, but now I need to add a private package.
I install it from git via pip, but it can't access the page:
Could not install requirement SE from
https://github.com/organisation/research.git#egg=SE because of error 404
Client Error: Not Found for url:
https://github.com/organisation/research Could not install requirement
SE from https://github.com/organisation/research.git#egg=SE because of
HTTP error 404 Client Error: Not Found for url:
https://github.com/organisation/research for URL
http://github.com/organisation/research.git#egg=SE
So I wonder, how can I pass / setup github credentials to/in the conda environment?
Just to be clear, this package dependency is happily tested on CircleCI by itself...

Actually, CircleCi offers to add your SSH key in project settings.

Error by create secure_channel in grpc of python

grpc was installed using pip. I tried to use it and I got an error. I looked for errors, but I could not find a solution.
The environment through uname is as follows.
env
uname -s -> Linux
uname -r -> 3.10.65
uname -m -> aarch64
code
import grpc
creds = grpc.ssl_channel_credentials(open('roots.pem').read())
channel = grpc.secure_channel('myservice.example.com:443', creds)
error log
24061 ssl_transport_security.c:655] Could not set ephemeral ECDH key.
24061 security_connector.c:774] Handshaker factory creation failed with TSI_INTERNAL_ERROR.

(I love that xkcd comic.. I feel that way all the time)
I solved this on arm64, here's how:
Under-the-hood grpcio uses boringssl if an environment variable is not applied to use the system's ssl. I believe the issue seen is with boringssl actually. Assuming you have openssl, libssl1.0.0 and libssl-dev installed, and you have already installed grpcio via pip or pip3.. then, you would do the following:
First download the grpcio source from PyPi.
pip3 uninstall grpcio
tar -xvf grpcio-<version>.tar.gz
cd grpcio-<version>/
GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=True python3 setup.py install
I'm sure setting GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=True before your pip3 install would work also, but I haven't tested that. The above solution fixed my issue.