We Keep Coding

Call speedtest.Speedtest() from Python using --secure (to avoid speedtest.ConfigRetrievalError: HTTP Error 403: Forbidden)

I have a small Python3-script like this:
import speedtest
# Speedtest
test = speedtest.Speedtest() # <--- line 4
test.get_servers()
best = test.get_best_server()
print(f"Found: {best['host']} located in {best['country']}")
The first time I run it, it works and everything is fine; it outputs:
Found: speedtest.witcom.cloud:8080 located in Germany
Happy days.
The second time (and subsequel times) that I run the script, I get this error:
Traceback (most recent call last):
File "/Users/zeth/Code/pinger/pinger.py", line 4, in <module>
test = speedtest.Speedtest()
File "/usr/local/lib/python3.9/site-packages/speedtest.py", line 1095, in __init__
self.get_config()
File "/usr/local/lib/python3.9/site-packages/speedtest.py", line 1127, in get_config
raise ConfigRetrievalError(e)
speedtest.ConfigRetrievalError: HTTP Error 403: Forbidden
When Googling around, I saw that I could also call this module straight from the command line, but just running this:
$ speedtest-cli
That gives me the same kind of error:
Retrieving speedtest.net configuration...
Cannot retrieve speedtest configuration
ERROR: HTTP Error 403: Forbidden
But if I run the direct cli-command: speedtest-cli --secure ( docs for the --secure-flag ), then it goes through and outputs this:
Retrieving speedtest.net configuration...
Testing from Deutsche Telekom AG (212.185.228.168)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by hotspot.koeln (Cologne) [3.44 km]: 28.805 ms
Testing download speed................................................................................
Download: 30.01 Mbit/s
Testing upload speed......................................................................................................
Upload: 8.68 Mbit/s
The question
I can't figure out how to change this Python-line: test = speedtest.Speedtest() to use a --secure-flag (nor via HTTPS).
The documentation for speedtest-cli is scarce.
Other attempts
I found this solution here: Python Speedtest facing problems with certification _ssl.c:1056, that suggests manually approving the certificates.
But in this directory: /Volumes/Macintosh HD/Applications/ I don't have anything called Python3.9. I have python3.9 installed via Brew. And I'm on a Mac.

I could do this:
test = speedtest.Speedtest(secure=True)
I looked into the source code myself, in this directory:
vim /usr/local/lib/python3.9/site-packages/speedtest.py
Where I would see the function was defined like this:
class Speedtest(object):
"""Class for performing standard speedtest.net testing operations"""
def __init__(self, config=None, source_address=None, timeout=10,
secure=False, shutdown_event=None):
self.config = {}
self._source_address = source_address
self._timeout = timeout
self._opener = build_opener(source_address, timeout)
self._secure = secure
...
...
...

Include python-dropbox in Yocto

My Python application uses Dropbox and I need to include this library in my Yocto image. I managed to generate a recipe for it with pipoe (pipoe -p dropbox). Here it is:
SUMMARY = "Official Dropbox API Client"
HOMEPAGE = "http://www.dropbox.com/developers"
AUTHOR = "Dropbox <dev-platform#dropbox.com>"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=ef3219362ea5e452a18031af12f35eb6"
SRC_URI = "https://files.pythonhosted.org/packages/85/33/bffd4a7596f3280f8bd2796b4f09c5c701b99d5c8e68715351cc2b3eeca8/dropbox-10.3.0.tar.gz"
SRC_URI[md5sum] = "f6d1a30af7e202237d660213d22e48a7"
SRC_URI[sha256sum] = "5f296f13ee7c358ab41779a73c4922ed81199447566c1a4c8fbf23dbcda25e20"
S = "${WORKDIR}/dropbox-10.3.0"
RDEPENDS_${PN} = "python-requests python-six "
inherit setuptools
When I run it, it says:
| ERROR: Do not try to fetch `pytest-runner' for building. Please add its native recipe to DEPENDS.
and then
| raise DistributionNotFound(req, requirers)
| pkg_resources.DistributionNotFound: The 'pytest-runner' distribution was not found and is required by the application
so I added the following line to the generated recipe:
DEPENDS = "python3-pytest-runner-native "
But I still get the same error. Any idea?

Probably an issue with inconsistent Python version.
You inherit setuptools which is for python2. The RDEPENDS are for python2 dependencies. But you add python3-pytest-runner-native in DEPENDS.
I'd suggest finding out which python version to use for this software and use the same for all RDEPENDS, inherited classes (setuptools3 for python3), and DEPENDS.

Integrate airbrake with Python

When I'm trying to integrate python logging with airbrake, I get following errors:
main.py
import pybrake
from config2.config import config
airbrake_handler = None
def filter_airbrake_msgs(notice):
if config.environment in ['production', 'staging']:
return notice
return None
def config_airbrake():
print(config.py_brake)
notifier = pybrake.Notifier(
project_id=config.py_brake.project_id,
project_key=config.py_brake.project_key
)
config_airbrake()
ENV=development python3 main.py
Error :
ERROR pybrake get_git_revision failed: [Errno 2] No such file or directory: '/user/xxx/xxx/xxx/.git/HEAD'

That looks like a log message coming from https://github.com/airbrake/pybrake/blob/master/pybrake/git.py#L12. I see why it is confusing but it is actually harmless and you can ignore it. I've created an issue to remove that log message. And feel free to use Github issues for such questions in future.
Overall pybrake checks if directory stored in context.rootDirectory contains a Git folder. If there is a git folder there it tries to extract some info like git revision, checkout date etc. Otherwise it logs the first error.

While using jenkins API, getting a failure on reconfig_job

I am using jenkins rest API to recurse through jobs and then reconfigure this one. All methods work except one. He's is my code :
def get_server_instance():
jenkins_url = 'xxxx'
#server = Jenkins(jenkins_url, username = '', password = '')
# Connect to instance - username and password are optional
server = jenkins.Jenkins(jenkins_url, username = '', password = '')
return server
def get_job_details():
# Refer Example #1 for definition of function 'get_server_instance'
server = get_server_instance()
for job in server.get_jobs_list():
if job == "GithubMigration":
configuration = server.get_job(job).get_config().encode('utf-8')
#server.reconfig_job(job, configuration)
if server.has_job("GithubMigration"):
server.reconfig_job('GithubMigration', config_xml)
It gets my configuration.xml, find the job as well but fails on server.reconfig_job('GithubMigration', config_xml) with the error , AttributeError: 'Jenkins' object has no attribute 'reconfig_job'
when obviously this functions exists in the jenkins rest API and yes I'm importing jenkins, from jenkinsapi.jenkins import Jenkins .
Edit 1 - I uninstalled Jenkinsapi and have only python-jenkins module and now it fails even before saying
AttributeError: 'module' object has no attribute 'Jenkins' for line : AttributeError: 'module' object has no attribute 'Jenkins'
Any ideas?
Edit 2 :
I tries solely python-jenkins API and tried their own example as you see here http://python-jenkins.readthedocs.org/en/latest/example.html
import jenkins
j = jenkins.Jenkins('http://your_url_here', 'username', 'password')
j.get_jobs()
j.create_job('empty', jenkins.EMPTY_CONFIG_XML)
j.disable_job('empty')
j.copy_job('empty', 'empty_copy')
j.enable_job('empty_copy')
j.reconfig_job('empty_copy', jenkins.RECONFIG_XML)
Even this fails at jenkins.Jenkins with attribute error at Jenkins - No module.
I am pretty sure the API is broken.

Your script is probably importing wrong module. You can check it as follows:
import jenkins
print jenkins.__file__
If printed path is other than installation path of jenkins module (eg. C:\Python27_32\lib\site-packages\jenkins\__init__.pyc), then you should check pythonpath:
import sys
print sys.path
Common problem is existence of python script with same name as imported module in current directory, which is at the first place in search path ''.
For more info on import order see module search path

Following #Chemik answer, I realized that the script I wrote was named jenkins.py and it was conflicting with python-jenkins import.
The library isn't broken. Check your script name.

had to add another solution, while running the same command
server = jenkins.Jenkins(jenkins_url, username = '', password = '')
i got the error:
'jenkins' has no attribute 'Jenkins'
my mistake was when installing the package, i installed package "jenkins" and the package i was needed is "python-jenkins".
docs can be found:
python-jenkins docs
so what i had to do is just
pip install python-jenkins

Download error on (…) hostname <proxy> doesn't match either of '*.c.ssl.fastly.net', (…) when running buildout behind proxy

I'm trying to build a project which uses Buildout and I'm behind a proxy but there's a problem with hostname verification. I couldn't find the root of the problem as passing multiple -v options to buildout command seems to not increase verbosity in this case. I couldn't find any information on buildout's interaction with proxies neither at http://www.buildout.org/en/latest/docs/index.html (which is version 1.2.1!?) nor at https://pypi.python.org/pypi/zc.buildout/2.2.1
pdobrogost#host:~/projects/projectx/projectx_buildout$ python bootstrap.py -c buildout-devel.cfg
Downloading http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg
Getting distribution for 'setuptools'.
/opt/python/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'src_root'
warnings.warn(msg)
Got setuptools 5.7.
Getting distribution for 'zc.buildout==1.4.3'.
Got zc.buildout 1.4.3.
pdobrogost#host:~/projects/projectx/projectx_buildout$ ./bin/buildout -vNc buildout-devel.cfg custom:cvsuser=pdobrogost
Installing 'mr.developer'.
We have no distributions for mr.developer that satisfies 'mr.developer'.
Download error on http://pypi.python.org/simple/mr.developer/: hostname 'proxy.site.local' doesn't match either of '*.c.ssl.fastly.net', 'c.ssl.fastly.net', '*.target.com', '*.vhx.tv', '*.snappytv.com', '*.atlassian.net', 'secure.lessthan3.com', '*.atlassian.com', 'a.sellpoint.net', 'cdn.upthere.com', '*.tissuu.com', '*.issuu.com', '*.kekofan.com', '*.python.org', '*.theverge.com', '*.sbnation.com', '*.polygon.com', '*.twobrightlights.com', '*.2brightlights.info', '*.vox.com', 'staging-cdn.upthere.com', '*.zeebox.com', '*.beamly.com', '*.aticpan.org', 'stream.svc.7digital.net', 'stream-test.svc.7digital.net', '*.articulate.com', 's.t.st', 'vid.thestreet.com', '*.planet-labs.com', '*.url2png.com', 'turn.com', 'www.turn.com', 'rivergathering.org', 'social.icfglobal2014-europe.org', '*.innogamescdn.com', '*.pathable.com', '*.staging.pathable.com', '*.kickstarter.com', 'sparkingchange.org', 'www.swedavia.se', 'www.swedavia.com', 'js-agent.newrelic.com', '*.fastly-streams.com', 'cdn.brandisty.com', 'fastly.hightailcdn.com', '*.fl.yelpcdn.com', '*.feedmagnet.com', 'api.contentbody.com', '*.acquia.com', '*.swarmapp.com', '*.pypa.io', 'pypa.io', 'static.qbranch.se', '*.krxd.net', '*.room.co', '*.metrological.com', 'room.co', 'cdn.evbuc.com', 'cdn.adagility.com', '*.bandpage.com', '*.ibmserviceengage.com', '*.quirky.com', '*.veez.co', '*.x.io', '*.otoycdn.net', '*.scribd.com', 'www.dwin1.com', 'api.imgur-ysports.com', 'i.imgur-ysports.com', '*.fxcm.co.jp', 'listora.com', '*.listora.com', 'blendle.nl', '*.blendle.nl', '*.modeanalytics.com', 'modeanalytics.com', 'krux.com', '*.krux.com', '*.udemy.com', '*.1stdibs.com', 'api.keep.com', 'www.piriform.com', '*.ustream.tv', 'www.zimbio.com', 'm.zimbio.com', 'www.stylebistro.com', 'm.stylebistro.com', 'm.lonny.com', 'www.lonny.com', 'assets.trabiancdn.com', '*.socialchorus.com', '*.heritagestatic.com', '*.theoutbound.com', 'img.rakuten.com', 'images.rakuten.com', 'img1.r10.io', 'ast1.r10.io', 'scribd.com' -- Some packages may not be found!
Couldn't find index page for 'mr.developer' (maybe misspelled?)
Download error on http://pypi.python.org/simple/: hostname 'proxy.site.local' doesn't match either of '*.c.ssl.fastly.net', 'c.ssl.fastly.net', '*.target.com', '*.vhx.tv', '*.snappytv.com', '*.atlassian.net', 'secure.lessthan3.com', '*.atlassian.com', 'a.sellpoint.net', 'cdn.upthere.com', '*.tissuu.com', '*.issuu.com', '*.kekofan.com', '*.python.org', '*.theverge.com', '*.sbnation.com', '*.polygon.com', '*.twobrightlights.com', '*.2brightlights.info', '*.vox.com', 'staging-cdn.upthere.com', '*.zeebox.com', '*.beamly.com', '*.aticpan.org', 'stream.svc.7digital.net', 'stream-test.svc.7digital.net', '*.articulate.com', 's.t.st', 'vid.thestreet.com', '*.planet-labs.com', '*.url2png.com', 'turn.com', 'www.turn.com', 'rivergathering.org', 'social.icfglobal2014-europe.org', '*.innogamescdn.com', '*.pathable.com', '*.staging.pathable.com', '*.kickstarter.com', 'sparkingchange.org', 'www.swedavia.se', 'www.swedavia.com', 'js-agent.newrelic.com', '*.fastly-streams.com', 'cdn.brandisty.com', 'fastly.hightailcdn.com', '*.fl.yelpcdn.com', '*.feedmagnet.com', 'api.contentbody.com', '*.acquia.com', '*.swarmapp.com', '*.pypa.io', 'pypa.io', 'static.qbranch.se', '*.krxd.net', '*.room.co', '*.metrological.com', 'room.co', 'cdn.evbuc.com', 'cdn.adagility.com', '*.bandpage.com', '*.ibmserviceengage.com', '*.quirky.com', '*.veez.co', '*.x.io', '*.otoycdn.net', '*.scribd.com', 'www.dwin1.com', 'api.imgur-ysports.com', 'i.imgur-ysports.com', '*.fxcm.co.jp', 'listora.com', '*.listora.com', 'blendle.nl', '*.blendle.nl', '*.modeanalytics.com', 'modeanalytics.com', 'krux.com', '*.krux.com', '*.udemy.com', '*.1stdibs.com', 'api.keep.com', 'www.piriform.com', '*.ustream.tv', 'www.zimbio.com', 'm.zimbio.com', 'www.stylebistro.com', 'm.stylebistro.com', 'm.lonny.com', 'www.lonny.com', 'assets.trabiancdn.com', '*.socialchorus.com', '*.heritagestatic.com', '*.theoutbound.com', 'img.rakuten.com', 'images.rakuten.com', 'img1.r10.io', 'ast1.r10.io', 'scribd.com' -- Some packages may not be found!
Getting distribution for 'mr.developer'.
While:
Installing.
Loading extensions.
Getting distribution for 'mr.developer'.
Error: Couldn't find a distribution for 'mr.developer'.
I have proxies set as following:
pdobrogost#host:~/projects/projectx/projectx_buildout$ env | grep proxy
http_proxy=http://proxy.site.local:8080
https_proxy=http://proxy.site.local:8080
no_proxy=localhost,127.0.0.1,.site.local,192.168.*
Also, buildout-devel.cfg extends buildout.cfg where the following versions are specified:
[versions]
zc.buildout = 1.4.3
zc.recipe.egg = 1.2.2
For comparision, with wget I get this:
pdobrogost#host:~/projects/projectx/projectx_buildout$ wget http://pypi.python.org/simple/mr.developer
--2014-09-05 10:56:20-- http://pypi.python.org/simple/mr.developer
Resolving proxy.site.local... 192.168.10.132
Connecting to proxy.site.local|192.168.10.132|:8080... connected.
Proxy request sent, awaiting response... 301 Moved Permanently
Location: https://pypi.python.org/simple/mr.developer [following]
--2014-09-05 10:56:20-- https://pypi.python.org/simple/mr.developer
Connecting to proxy.site.local|192.168.10.132|:8080... connected.
ERROR: certificate common name “*.c.ssl.fastly.net” doesn’t match requested host name “pypi.python.org”.
To connect to pypi.python.org insecurely, use ‘--no-check-certificate’.

Not sure if this is directly related, but your post came up when trying to find an answer to my problem. It looks like there may be an issue with some versions of Python (3.4 in my case) where it could not install dependencies behind a proxy. It looks like it's related to this: (https://github.com/shazow/urllib3/pull/385) It looks like the buildout script may be running into the same issue?
For a work-around in my case where I'm only using pip, I was able to install each dependency manually. So keep trying to install the main package with pip, then when it fails saying it can't find a package, pip install that package and try again.

You can try using http, by selecting different mirror that supports http as outlined in this guide -- http://jacobian.org/writing/when-pypi-goes-down/
For buildout, you will need to add the following as global declaration
[global]
index-url = http://pypi.python.org/simple
The main problem, however that pypi.python.org is now set up to serve https only and will still redirect you to https site even if your using http, so the only real solution is to create a mirror somewhere outside and serve pypi content via http.