We Keep Coding

Call speedtest.Speedtest() from Python using --secure (to avoid speedtest.ConfigRetrievalError: HTTP Error 403: Forbidden)

I have a small Python3-script like this:
import speedtest
# Speedtest
test = speedtest.Speedtest() # <--- line 4
test.get_servers()
best = test.get_best_server()
print(f"Found: {best['host']} located in {best['country']}")
The first time I run it, it works and everything is fine; it outputs:
Found: speedtest.witcom.cloud:8080 located in Germany
Happy days.
The second time (and subsequel times) that I run the script, I get this error:
Traceback (most recent call last):
File "/Users/zeth/Code/pinger/pinger.py", line 4, in <module>
test = speedtest.Speedtest()
File "/usr/local/lib/python3.9/site-packages/speedtest.py", line 1095, in __init__
self.get_config()
File "/usr/local/lib/python3.9/site-packages/speedtest.py", line 1127, in get_config
raise ConfigRetrievalError(e)
speedtest.ConfigRetrievalError: HTTP Error 403: Forbidden
When Googling around, I saw that I could also call this module straight from the command line, but just running this:
$ speedtest-cli
That gives me the same kind of error:
Retrieving speedtest.net configuration...
Cannot retrieve speedtest configuration
ERROR: HTTP Error 403: Forbidden
But if I run the direct cli-command: speedtest-cli --secure ( docs for the --secure-flag ), then it goes through and outputs this:
Retrieving speedtest.net configuration...
Testing from Deutsche Telekom AG (212.185.228.168)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by hotspot.koeln (Cologne) [3.44 km]: 28.805 ms
Testing download speed................................................................................
Download: 30.01 Mbit/s
Testing upload speed......................................................................................................
Upload: 8.68 Mbit/s
The question
I can't figure out how to change this Python-line: test = speedtest.Speedtest() to use a --secure-flag (nor via HTTPS).
The documentation for speedtest-cli is scarce.
Other attempts
I found this solution here: Python Speedtest facing problems with certification _ssl.c:1056, that suggests manually approving the certificates.
But in this directory: /Volumes/Macintosh HD/Applications/ I don't have anything called Python3.9. I have python3.9 installed via Brew. And I'm on a Mac.

I could do this:
test = speedtest.Speedtest(secure=True)
I looked into the source code myself, in this directory:
vim /usr/local/lib/python3.9/site-packages/speedtest.py
Where I would see the function was defined like this:
class Speedtest(object):
"""Class for performing standard speedtest.net testing operations"""
def __init__(self, config=None, source_address=None, timeout=10,
secure=False, shutdown_event=None):
self.config = {}
self._source_address = source_address
self._timeout = timeout
self._opener = build_opener(source_address, timeout)
self._secure = secure
...
...
...

How do I download a file from a website in python without the users username?

So I am trying to make a download tool to download stuff via a link here is a snippet from my code
if download_choice == "14":
print("Downloading test file to Desktop...")
myfile14 = requests.get(url14)
open('c:/users/%userprofile%/desktop/', 'wb').write(myfile14.content)
I want to make a program that can download stuff via a link. I want it to run on all PCs not only on mine. Here is the error:
Traceback (most recent call last): File "C:\Users\David\Desktop\Windows Download Tool\Python\WDT.py", line 100, in <module> open('c:/users/%userprofile%/desktop/', 'wb').write(myfile14.content) FileNotFoundError: [Errno 2] No such file or directory: 'c:/users/%userprofile%/desktop/'

Python does not use %userprofile% to get the username of the executing user.
To achieve this, you need to import the package os and run it's getlogin function. This returns a string with the current username
Example:
import os
username = os.getlogin()
if download_choice == "14":
print("Downloading test file to Desktop...")
myfile14 = requests.get(url14)
open(f'C:/Users/{username}/desktop/', 'wb').write(myfile14.content)
I am using an f-string for opening the file, since it is preferred by PEP-8 (it's just correct code-styling)
Attention: This only works on a windows machine

How to install packages with python scripts

I want to install a package with a python script. I have read the documentation about PackageManager API (http://doc.aldebaran.com/2-4/naoqi/core/packagemanager-api.html):
So I have packaged the app with choregraphe as it is described in http://doc.aldebaran.com/2-4/naoqi/core/packagemanager.html and I have tried to install it with a python script that looks like:
import qi
import sys
if __name__ == '__main__':
ip = "11.1.11.111"
port = 9559
session = qi.Session()
try:
session.connect("tcp://" + ip + ":" + str(port))
except RuntimeError:
print ("Can't connect to Naoqi at ip \"" + ip + "\" on port " + str(port))
sys.exit(1)
service = session.service("PackageManager")
package = "C:\\test_package_handlers_01-835a92-1.0.0.pkg"
# this is to see if the problem is that python can not locate the file
with open(package) as f:
print f
service.install(package)
And here is what I receive as an error:
# provided package could be opened
<open file 'C:\\test_package_handlers_01-835a92-1.0.0.pkg', mode 'r' at 0x02886288>
Traceback (most recent call last):
File "C:/test.py", line 24, in <module>
service.install(package)
RuntimeError: C:\test_package_handlers_01-835a92-1.0.0.pkg: no such file
I guess this is because the package must be uploaded on the robot and the package file path must be the one that is on the robot.
EDITED
I have added the package to a choreographe blank project and run this blank project on the robot. This way the package was saved to the robot with path /home/nao/.local/share/PackageManager/apps/.lastUploadedChoregrapheBehavior/test_package_handlers_01-835a92-1.0.0.pkg and when I have changed the path in my script ("C:\\test_package_handlers_01-835a92-1.0.0.pkg" with "/home/nao/.local/share/PackageManager/apps/.lastUploadedChoregrapheBehavior/test_package_handlers_01-835a92-1.0.0.pkg") the script worked as it was intended and the package was installed on the robot.
So is there a way to install packages from my PC without uploading them to the robot, because otherwise it is better to use Choregraphe to upload projects.
Maybe it is good to give the following explanation of what I want to achieve:
I have a folder on my PC with 20 packages for example
I want to install all those 20 packages with one python script
There is a python script that installs all the packages from the folder when it is invoked like this:
python package_installer.py path_to_packages_folder
EDITED_2
import qi
import ftplib
import os
ROBOT_URL = "10.80.129.90"
print "Uploading PKG"
pkg_file = "my-application-0.0.1.pkg"
pkg_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), pkg_file)
ftp = ftplib.FTP(ROBOT_URL)
ftp.login("nao", "nao")
with open(pkg_path) as pkg:
ftp.storbinary("STOR "+pkg_file, pkg)
print "Connecting NAOqi session"
app = qi.Application(url='tcp://'+ROBOT_URL+':9559')
app.start()
session = app.session
print "Installing app"
packagemgr = session.service("PackageManager")
packagemgr.install("/home/nao/"+pkg_file)
print "Cleaning robot"
ftp.delete(pkg_file)
ftp.quit()
print "End"
app.stop()
This piece of code ftp = ftplib.FTP(ROBOT_URL) throws the following exception:
Traceback (most recent call last):
File "C:/Stefan/DSK_PEPPER_clode_2/PythonScripts/_local_testing/uploading_and_installing_package.py", line 11, in <module>
ftp = ftplib.FTP(ROBOT_URL)
File "C:\Python27\lib\ftplib.py", line 120, in __init__
self.connect(host)
File "C:\Python27\lib\ftplib.py", line 135, in connect
self.sock = socket.create_connection((self.host, self.port), self.timeout)
File "C:\Python27\lib\socket.py", line 575, in create_connection
raise err
socket.error: [Errno 10061] No connection could be made because the target machine actively refused it
Also when I connect to the robot with username 'nao' and pass 'nao' as described in http://doc.aldebaran.com/2-5/dev/tools/opennao.html and then try to create a folder in /home/nao/.local/share/PackageManager/apps/ with sudo mkdir it informs me that: Sorry, user nao is not allowed to execute '/bin/mkdir dasdas' as root on Pepper.. If I use only mkdir here is what it tells me: mkdir: cannot create directory 'new_folder': Permission denied

Using qibuild, you can also directly install using:
qipkg deploy-package /path/to/my-package.pkg --url nao#10.10.23.45

You indeed need to upload the file before. You can use scp or sftp to do this. Once the .pkg is on the robot then you can use PackageManager.install.
Imagine something like:
import qi
import paramiko
import os
ROBOT_URL = "10.80.129.90"
print "Uploading PKG"
pkg_file = "my-application-0.0.1.pkg"
pkg_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), pkg_file)
transport = paramiko.Transport((ROBOT_URL, 22))
transport.connect(username="nao", password="nao")
sftp = paramiko.SFTPClient.from_transport(transport)
sftp.put(pkg_path, pkg_file)
print "Connecting NAOqi session"
app = qi.Application(url='tcp://'+ROBOT_URL+':9559')
app.start()
session = app.session
print "Installing app"
packagemgr = session.service("PackageManager")
packagemgr.install("/home/nao/"+pkg_file)
print "Cleaning robot"
sftp.remove(pkg_file)
sftp.close()
transport.close()
print "End"
app.stop()

Fixing SSL certificate error in exe compiled with py2exe (or PyInstaller)

I've just finished testing a Python programme which involves logging into a site and requires a CSRF cookie to be set. I've tried packaging it as an exe using py2exe and got a socket error. I have the same problem when I try with PyInstaller. Googling the Errno I found a few other people with the same problem and so I know the problem is to do with the location of SLL certificates.
This is my site_agent class including the logging calls.
class site_agent:
self.get_params()
URL = root_url + '/accounts/login/'
# Retrieve the CSRF token first
self.agent = requests.session()
self.agent.get(URL) # retrieves the cookie # This line throws the error
self.csrftoken = self.agent.cookies['csrftoken']
# Set up login data including the CSRF cookie
login_data = {'username': self.username,
'password': self.password,
'csrfmiddlewaretoken' : self.csrftoken}
# Log in
logging.info('Logging in')
response = self.agent.post(URL, data=login_data, headers=hdr)
The error comes at the self.agent.get(URL) line and the Traceback shows:
Traceback (most recent call last):
File "<string>", line 223, in <module>
File "<string>", line 198, in main
File "<string>", line 49, in __init__
File "C:\pyinstaller-2.0\pyinstaller-2.0\autoresponder\b
uild\pyi.win32\autoresponder\out00-PYZ.pyz\requests.sessions", line 350, in get
File "C:\pyinstaller-2.0\pyinstaller-2.0\autoresponder\b
uild\pyi.win32\autoresponder\out00-PYZ.pyz\requests.sessions", line 338, in requ
est
File "C:\pyinstaller-2.0\pyinstaller-2.0\autoresponder\b
uild\pyi.win32\autoresponder\out00-PYZ.pyz\requests.sessions", line 441, in send
File "C:\pyinstaller-2.0\pyinstaller-2.0\autoresponder\b
uild\pyi.win32\autoresponder\out00-PYZ.pyz\requests.adapters", line 331, in send
requests.exceptions.SSLError: [Errno 185090050] _ssl.c:336: error:0B084002:x509
certificate routines:X509_load_cert_crl_file:system lib
Does this mean that the problem is in requests.adapters?
If so, can I just edit it in my installed Python packages to look for cacert.pem somewhere else, rebuild my exe with py2exe or PyInstaller, then change it back in my installed version of Python?
EDIT
I now have the programme running after compiling with PyInstaller and setting verify=False in all requests.get() and requests.post() calls. But SSL is there for a a reason and I'd really like to be able to fix this error before letting anyone use the tool.

This can be solved easily if you are using "requests" module.
1) Place the below code in your main python file where "requests" module is used
os.environ['REQUESTS_CA_BUNDLE'] = "certifi/cacert.pem"
2) Within your distributable folder where exe is present, create a folder called "certifi" and place the "cacert.pem" file within it.
3) You can find the "cacert.pem" file by
pip install certifi
import certifi
certifi.where()
Awesome.. now your distributable includes the necessary certificates to validate ssl calls.

If using pyinstaller... create a hook-requests.py file in PyInstaller\hooks\ for the requests lib containing
from hookutils import collect_data_files
# Get the cacert.pem
datas = collect_data_files('requests')

In addition to the answer given by frmdstryr, I had to tell requests where the cacert.pem is. I added the following lines right after my imports:
# Get the base directory
if getattr( sys , 'frozen' , None ): # keyword 'frozen' is for setting basedir while in onefile mode in pyinstaller
basedir = sys._MEIPASS
else:
basedir = os.path.dirname( __file__ )
basedir = os.path.normpath( basedir )
# Locate the SSL certificate for requests
os.environ['REQUESTS_CA_BUNDLE'] = os.path.join(basedir , 'requests', 'cacert.pem')
This solution is derived from this link: http://kittyandbear.net/python/pyinstaller-request-sslerror-manual-cacert-solution.txt

I was struggling with this as well, The hook-requests.py provided by frmdstryr did not work for me. But this modified one did:
from PyInstaller.utils.hooks import collect_data_files
# Get the cacert.pem
datas = collect_data_files('certifi')

"OSERROR -10000 Apple event handler failed" when trying to change desktop wallpaper on Mac

I have written the following really simple python script to change the desktop wallpaper on my mac (based on this thread):
from appscript import app, mactypes
import sys
fileName = sys.argv[1:]
app('Finder').desktop_picture.set(mactypes.File(fileName))
However when I run it I get the following output:
Traceback (most recent call last):
File "../Source/SetWallPaper2.py",
line 6, in
app('Finder').desktop_picture.set(mactypes.File(fileName))
File
"/Library/Python/2.5/site-packages/appscript-0.19.0-py2.5-macosx-10.5-i386.egg/appscript/reference.py", line 513, in call
appscript.reference.CommandError:
Command failed: OSERROR: -10000
MESSAGE: Apple event handler failed.
COMMAND:
app(u'/System/Library/CoreServices/Finder.app').desktop_picture.set(mactypes.File(u"/Users/Daniel/Pictures/['test.jpg']"))
I've done some web searching but I can't find anything to help me figure out what OSERROR -10000 means or how to resolve the issue.

fileName = sys.argv[1]
instead of
fileName = sys.argv[1:]
mactypes.File(u"/Users/Daniel/Pictures/['test.jpg']")
See the square brackets and quotes around the filename?