I've been following this (https://developer.ibm.com/tutorials/se-distributed-apps-zeromq-part2/) tutorial for setting up a ZeroMQ client/server setup which uses CurveZMQ to encrypt messages. The code works, however whenever I run either the client or the server it instructs me to 'Install tornado itself to use zmq with the tornado IOLoop.'
I've found that the warning appears in any file with the following import statement.
from zmq.eventloop.zmqstream import ZMQStream
I've looked in the code for zmq and noticed that it uses "minitornado", which is likely the problem, but I don't know much about tornado or how to use it.
You need to upgrade pyzmq. You can use this command to force your environment to reinstall updated version. (It worked for me!)
pip install --upgrade --force-reinstall pyzmq
I don't know if it's related to your problem. I googled about the same problem, but related to jupyter notebooks 500 internal server error, and found your question. I resolved my issue by downgrading tornado to version 5 (sudo -H pip3 install "tornado==5") and everything works again. I hope you find a solution by now or my hint helps you.
I use python for work and often need to install packages using pip but because the IT-department at work is using an https-man-in-the-middle every time i try to install packages while on the internal compagny network it failes with an ssl-certificate verification error.
Until recently I had a colleague (he left) who had found a command to disable the verification which enabled install with pip through the compagny network, it was something like SSL_...-VERIFY.. = FALSE ??? something like that very straight forward, but i don't remember it exactly and i cant seem to find it anywhere on the internet.
I find lots of solutions to the same problem but none of them work for me, here i'm thinking of solutions like '--trusted-host..' etc.
Please does anyone know the command I'm looking for?
I was able to get pip working by using both the --trusted-host flag and also the --cert flag to point it to the root certificate for the network. The certificate would be installed on any workstation subject to SSL MITM so you can export the certificate yourself or ask your IT department for it.
Example command that worked for me:
pip3 install ipython --trusted-host pypi.python.org --cert /path/to/cert/root_cert.cer
i am integrating paypalrestsdk in my AppEngine project. When, using my local dev_appserver, i try to create a payment on the PayPal sandbox, i have the following error:
SSLCertificateError: Invalid and/or missing SSL certificate for URL: https://api.sandbox.paypal.com/v1/oauth2/token
So, i have tried to provide the correct pem file, downloading it from here and setting up the correct ssl_option attribute:
# Setting up the correct path to the .pem file
cert = os.path.join(ROOT, 'certs/api.sandbox.paypal.com_SHA-2_01132018.pem')
logger.info("Using SSL certificate: %s", cert)
return Api(
mode=get_paypal_environment(), # sandbox or live
client_id=flask.current_app.config["PAYPAL_CLIENT_ID"],
client_secret=flask.current_app.config["PAYPAL_CLIENT_SECRET"],
ssl_options={"cert": cert}
)
Here there is the PayPalRestSDK documentation that gives details on how to provide certificate. I have double checked the path created is correct.
I have have found a bug report here that talks about a similar problem.
Also, i have tried the solution suggested here and still does not work.
On a live instance, on appspot, this all works perfectly.
Here's the relevant part of my requirements.txt:
Flask==0.10.1
itsdangerous==0.24
paramiko==1.15.1
pycrypto==2.6.1
Flask-OAuthlib==0.9.1
google-api-python-client==1.4.0
paypalrestsdk==1.11.1
requests[security]==2.9.1
Is anyone having the same issue ?
OK, I believed I've solved this one, at least in my case, which I'll describe below.
This seemed to be due to two issues:
Issue #1) PayPal is migrating to supporting only TLS 1.2, and has started by switching over the sandbox URLs, with the production URLs to come later. This explains why things are broken only when connecting from the sandbox, and why it used to work but doesn't now. More on this here.
Issue #2) My local install of Python didn't support TLS 1.2. Here is an easy way to check:
$ python
>>> import ssl
>>> print ssl._PROTOCOL_NAMES
If you don't see PROTOCOL_TLSv1_2 listed, this is definitely the issue. In my case, I was using the builtin version on Python on Mac OS X 10.11, which had a pretty old version on OpenSSL built in.
So how to fix it? Well, in my case, this worked pretty well (copied mostly from here):
$ brew update
$ brew install openssl
$ brew link openssl --force
$ brew install python --with-brewed-openssl
$ sudo ln -s /usr/local/Cellar/python/2.7.11/bin/python /usr/local/bin/python
Now if you run the test I listed above, you should see the 1.2 protocol listed.
This should make everything work again, good luck!
Whether I use PIP or download manually and run 'python setup.py install' on Pyramid I'm getting the same error: accessing standard pypi websites.
If I visit the sites in my browser it all works, but if I run the command it doesn't. Error as below.
I guess the fault will occur on most packages using these url's because I'm behind a proxy and my proxy is disallowing web access via python executable?
How do I specify pip or simply the apps setup.py to use my proxy, assuming this is the problem.
Installed c:\pymon\python34\lib\site-packages\pyramid-1.5.1-py3.4.egg
Processing dependencies for pyramid==1.5.1
Searching for PasteDeploy>=1.5.0
Reading https://pypi.python.org/simple/PasteDeploy/
Download error on https://pypi.python.org/simple/PasteDeploy/: timed out -- Some
packages may not be found!
Couldn't find index page for 'PasteDeploy' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: timed out -- Some packages may not be found!
No local packages or download links found for PasteDeploy>=1.5.0
error: Could not find suitable distribution for Requirement.parse('PasteDeploy>=
1.5.0')
My apologies I should have searched further:
Plone: pypi specific download issue behind a proxy server - can't install eggs
I created two environment variables: HTTP_PROXY and HTTPS_PROXY
Note the above comment regarding the --proxy argument also worked via pip.
I am trying to install a Python package from a private GitHub repository. For a public repository, I can issue the following command which works fine:
pip install git+git://github.com/django/django.git
However, if I try this for a private repository:
pip install git+git://github.com/echweb/echweb-utils.git
I get the following output:
Downloading/unpacking git+git://github.com/echweb/echweb-utils.git
Cloning Git repository git://github.com/echweb/echweb-utils.git to /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build
Complete output from command /usr/local/bin/git clone git://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build:
fatal: The remote end hung up unexpectedly
Cloning into /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build...
----------------------------------------
Command /usr/local/bin/git clone git://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build failed with error code 128
I guess this is because I am trying to access a private repository without providing any authentication. I therefore tried to use Git + ssh hoping that pip would use my SSH public key to authenticate:
pip install git+ssh://github.com/echweb/echweb-utils.git
This gives the following output:
Downloading/unpacking git+ssh://github.com/echweb/echweb-utils.git
Cloning Git repository ssh://github.com/echweb/echweb-utils.git to /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build
Complete output from command /usr/local/bin/git clone ssh://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build:
Cloning into /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build...
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
----------------------------------------
Command /usr/local/bin/git clone ssh://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build failed with error code 128
Is what I am trying to achieve even possible? If so, how can I do it?
You can use the git+ssh URI scheme, but you must set a username. Notice the git# part in the URI:
pip install git+ssh://git#github.com/echweb/echweb-utils.git
Also read about deploy keys.
PS: In my installation, the "git+ssh" URI scheme works only with "editable" requirements:
pip install -e URI#egg=EggName
Remember: Change the : character that git remote -v prints to a / character before using the remote's address in the pip command:
$ git remote -v
origin git#github.com:echweb/echweb-utils.git (fetch)
# ^ change this to a '/' character
If you forget, you will get this error:
ssh: Could not resolve hostname github.com:echweb:
nodename nor servname provided, or not known
As an additional technique, if you have the private repository cloned locally, you can do:
pip install git+file://c:/repo/directory
More modernly, you can just do this (and the -e will mean you don't have to commit changes before they're reflected):
pip install -e C:\repo\directory
You can do it directly with the HTTPS URL like this:
pip install git+https://github.com/username/repo.git
This also works just appending that line in the requirements.txt in a Django project, for instance.
It also works with Bitbucket:
pip install git+ssh://git#bitbucket.org/username/projectname.git
Pip will use your SSH keys in this case.
I found it much easier to use tokens than SSH keys. I couldn't find much good documentation on this, so I came across this solution mainly through trial and error. Further, installing from pip and setuptools have some subtle differences; but this way should work for both.
GitHub don't (currently, as of August 2016) offer an easy way to get the zip / tarball of private repositories. So you need to tell setuptools that you're pointing to a Git repository:
from setuptools import setup
import os
# Get the deploy key from https://help.github.com/articles/git-automation-with-oauth-tokens/
github_token = os.environ['GITHUB_TOKEN']
setup(
# ...
install_requires='package',
dependency_links = [
'git+https://{github_token}#github.com/user/{package}.git/#{version}#egg={package}-0'
.format(github_token=github_token, package=package, version=master)
]
A couple of notes here:
For private repositories, you need to authenticate with GitHub; the simplest way I found is to create an OAuth token, drop that into your environment, and then include it with the URL
You need to include some version number (here is 0) at the end of the link, even if there's isn't any package on PyPI. This has to be a actual number, not a word.
You need to preface with git+ to tell setuptools it's to clone the repository, rather than pointing at a zip / tarball
version can be a branch, a tag, or a commit hash
You need to supply --process-dependency-links if installing from pip
I figured out a way to automagically 'pip install' a GitLab private repository that requires no password prompt. This approach uses GitLab "Deploy Keys" and an SSH configuration file, so you can deploy using keys other than your personal SSH keys (in my case, for use by a 'bot). Perhaps someone kind soul can verify using GitHub.
Create a New SSH key:
ssh-keygen -t rsa -C "GitLab_Robot_Deploy_Key"
The file should show up as ~/.ssh/GitLab_Robot_Deploy_Key and ~/.ssh/GitLab_Robot_Deploy_Key.pub.
Copy and paste the contents of the ~/.ssh/GitLab_Robot_Deploy_Key.pub file into the GitLab "Deploy Keys" dialog.
Test the New Deploy Key
The following command tells SSH to use your new deploy key to set up the connection. On success, you should get the message: "Welcome to GitLab, UserName!"
ssh -T -i ~/.ssh/GitLab_Robot_Deploy_Key git#gitlab.mycorp.com
Create the SSH Configuration File
Next, use an editor to create a ~/.ssh/config file. Add the following contents. The 'Host' value can be anything you want (just remember it, because you'll be using it later). The HostName is the URL to your GitLab instance. The IdentifyFile is path to the SSH key file you created in the first step.
Host GitLab
HostName gitlab.mycorp.com
IdentityFile ~/.ssh/GitLab_Robot_Deploy_Key
Point SSH to the Configuration file
oxyum gave us the recipe for using pip with SSH:
pip install git+ssh://git#gitlab.mycorp.com/my_name/my_repo.git
We just need to modify it a bit to make SSH use our new Deploy Key. We do that by pointing SSH to the Host entry in the SSH configuration file. Just replace the 'gitlab.mycorp.com' in the command to the host name we used in the SSH configuration file:
pip install git+ssh://git#GitLab/my_name/my_repo.git
The package should now install without any password prompt.
Reference A
Reference B
If you want to install dependencies from a requirements file within a CI server, you can do this:
git config --global credential.helper 'cache'
echo "protocol=https
host=example.com
username=${GIT_USER}
password=${GIT_PASS}
" | git credential approve
pip install -r requirements.txt
In my case, I used GIT_USER=gitlab-ci-token and GIT_PASS=${CI_JOB_TOKEN}.
This method has a clear advantage. You have a single requirements file which contains all of your dependencies.
The syntax for the requirements file is given here:
https://pip.pypa.io/en/latest/reference/pip_install.html#requirements-file-format
So for example, use:
-e git+http://github.com/rwillmer/django-behave#egg=django-behave
if you want the source to stick around after installation.
Or just
git+http://github.com/rwillmer/django-behave#egg=django-behave
if you just want it to be installed.
If you need to do this in, say, a command line one-liner, it's also possible. I was able to do this for deployment on Google Colab:
Create a Personal Access Token: https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token
Run: pip install git+https://<USERNAME>:<PERSONAL ACCESS TOKEN>#github.com/<ACCOUNT>/<REPOSITORY>.git
My case was kind of more complicated than most of the ones described in the answers. I was the owner of two private repositories repo_A and repo_B in a Github organization and needed to pip install repo_A during the python unittests of repo_B, as a Github action.
Steps I followed to solve this task:
Created a Personal Access Token for my account. As for its permissions, I only needed to keep the default ones, .i.e. repo - Full control of private repositories.
Created a repository secret under repo_B, pasted my Personal Access Token in there and named it PERSONAL_ACCESS_TOKEN. This was important because, unlike the solution proposed by Jamie, I didn't need to explicitly expose my precious raw Personal Access Token inside the github action .yml file.
Finally, pip install the package from source via HTTPS (not SSH) as follows:
export PERSONAL_ACCESS_TOKEN=${{ secrets.PERSONAL_ACCESS_TOKEN }}
pip install git+https://${PERSONAL_ACCESS_TOKEN}#github.com/MY_ORG_NAME/repo_A.git
You can also install a private repository dependency via git+https://github.com/... URL by providing login credentials (login and password, or deploy token) for curl with the .netrc file:
echo "machine github.com login ei-grad password mypasswordshouldbehere" > ~/.netrc
pip install "git+https://github.com/ei-grad/my_private_repo.git#egg=my_private_repo"
If you don't want to use SSH, you could add the username and password in the HTTPS URL.
The code below assumes that you have a file called "pass" in the working directory that contains your password.
export PASS=$(cat pass)
pip install git+https://<username>:$PASS#github.com/echweb/echweb-utils.git
When I was installing from GitHub I was able to use:
pip install git+ssh://git#github.com/<username>/<projectname>.git#egg=<eggname>
But, since I had to run pip as sudo, the SSH keys were not working with GitHub any more, and "git clone" failed on "Permission denied (publickey)". Using git+https allowed me to run the command as sudo, and have GitHub ask me for my user/password.
sudo pip install git+https://github.com/<username>/<projectname>.git#egg=<eggname>
If you have your own library/package on GitHub, GitLab, etc., you have to add a tag to commit with a concrete version of the library, for example, v2.0, and then you can install your package:
pip install git+ssh://link/name/repo.git#v2.0
This works for me. Other solutions haven't worked for me.
Just copy the remote from the original git clone command (or from git remote -v). You will get something like this:
Bitbucket: git+ssh://git#bitbucket.org:your_account/my_pro.git
GitHub: git+ssh://git#github.com:your_account/my_pro.git
Next, you need to replace : with / next to the domain name.
So install using:
pip install git+ssh://git#bitbucket.org/your_account/my_pro.git
oxyum's solution is OK for this answer. I just want to point out that you need to be careful if you are installing using sudo as the keys must be stored for root too (for example, /root/.ssh).
Then you can type
sudo pip install git+ssh://git#github.com/echweb/echweb-utils.git
You may try
pip install git+git#gitlab.mycorp.com/my_name/my_repo.git
without ssh:.... That works for me.