We Keep Coding

Cannot call SOAP request behind proxy (Not supported proxy scheme None)

I'm unable to call a SOAP request from a simple Python script in a Windows Server 2016 environment with WinPython/VSCode:
from requests import Session
from zeep import Client
from zeep.transports import Transport
wsdl = "http://www.dneonline.com/calculator.asmx?wsdl"
#wsdl = "calculator.xml"
client = Client(wsdl=wsdl)
request_data={'intA' : 1 ,
'intB' : 2}
response=client.service.Add(**request_data)
print("response: " + response)
The output I get:
Traceback (most recent call last):
File ".\zeeptest.py", line 31, in <module>
client = Client(wsdl=wsdl)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\client.py", line 73, in __init__
self.wsdl = Document(wsdl, self.transport, settings=self.settings)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\wsdl\wsdl.py", line 92, in __init__
self.load(location)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\wsdl\wsdl.py", line 95, in load
document = self._get_xml_document(location)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\wsdl\wsdl.py", line 155, in _get_xml_document
return load_external(
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\loader.py", line 79, in load_external
content = transport.load(url)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\transports.py", line 122, in load
content = self._load_remote_data(url)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\transports.py", line 134, in _load_remote_data
response = self.session.get(url, timeout=self.load_timeout)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\requests\sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\requests\adapters.py", line 414, in send
raise InvalidURL(e, request=request)
requests.exceptions.InvalidURL: Not supported proxy scheme None
I tried to set the proxy manually with the following commands without success:
set http_proxy="http://<ip>:<port>"
set https_proxy="http://<ip>:<port>"

I resolved my problem by issuing the following command in a Powershell shell as an admin user:
netsh winhttp set proxy <ip>:<port>

eBay and Authlib Unconventional token type

I'm trying to use Authlib library to access new eBay REST API (as Authorization code grant)
Here is my code;
import json
import os
import webbrowser
from time import time
from authlib.integrations.requests_client import OAuth2Session
from rpi_order_data_sync import settings
def auth(seller):
def token_updater(token, seller=seller):
if not os.path.exists(seller):
open(seller, "w").close()
with open(seller, "w") as token_file:
json.dump(token, token_file)
scope = ["https://api.ebay.com/oauth/api_scope/sell.fulfillment.readonly"]
if not os.path.exists(seller):
ebay = OAuth2Session(
settings.E_APP_ID,
settings.E_CERT_ID,
redirect_uri=settings.E_RU_NAME,
scope=scope,
)
uri, state = ebay.create_authorization_url(
"https://auth.sandbox.ebay.com/oauth2/authorize",
)
print("Please go to {} and authorize access.".format(uri))
try:
webbrowser.open_new_tab(uri)
except webbrowser.Error:
pass
authorization_response = input("Please enter callback URL: ") # nosec
token = ebay.fetch_token(
"https://api.sandbox.ebay.com/identity/v1/oauth2/token",
authorization_response=authorization_response,
)
print(token)
token_updater(token)
return ebay
The problem is eBay's token response has an unconventional token type named "User Access Token" instead of "Bearer". Therefore I get this error;
Traceback (most recent call last):
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/authlib/integrations/requests_client/oauth2_session.py", line 37, in __call__
req.url, req.headers, req.body = self.prepare(
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/authlib/oauth2/auth.py", line 91, in prepare
sign = self.SIGN_METHODS[token_type.lower()]
KeyError: 'user access token'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/bin/rods", line 11, in <module>
load_entry_point('rpi-order-data-sync', 'console_scripts', 'rods')()
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/thiras/HDD/freelancer/contentassasin/rpi-order-data-sync/rpi_order_data_sync/main.py", line 132, in sync_ebay_orders
orders = ebay.get(
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/requests/sessions.py", line 543, in get
return self.request('GET', url, **kwargs)
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/authlib/integrations/requests_client/oauth2_session.py", line 113, in request
return super(OAuth2Session, self).request(
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/requests/sessions.py", line 516, in request
prep = self.prepare_request(req)
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/requests/sessions.py", line 449, in prepare_request
p.prepare(
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/requests/models.py", line 318, in prepare
self.prepare_auth(auth, url)
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/requests/models.py", line 549, in prepare_auth
r = auth(self)
File "/home/thiras/.local/share/virtualenvs/rpi-order-data-sync-tA0i1rrc/lib/python3.8/site-packages/authlib/integrations/requests_client/oauth2_session.py", line 41, in __call__
raise UnsupportedTokenTypeError(description=description)
authlib.integrations.base_client.errors.UnsupportedTokenTypeError: unsupported_token_type: Unsupported token_type: 'user access token'
I've noticed Compliance fix for non-standard section at Authlib documentation but couldn't figure out how to do this fix or even possible in this way.

I've found a solution and it also works with requests-oauthlib package. It seems working flawlessly so far. The main struggle was to create a fake request.Response model since request.Response has no setter for .text or .content attributes so modifying them was impossible.
So I've created a FakeResponse class that only mimics .json() method since it was the only method used by Authlib.
class FakeResponse:
""" Fake Class for Request Response class. """
def __init__(self, data):
self.data = data
def json(self):
""" Mocks requests.Response.json(). """
return self.data
After that I've created an access_token_response hook;
def non_compliant_token_type(resp):
data = resp.json()
data["token_type"] = "Bearer"
fake_resp = FakeResponse(data=data)
return fake_resp
Please let me know if you have a better answer or any recommendations to improve it.

Best way to use Python and Bit Bucket

I am having problems with Python and Bit Bucket. To display/pull/push do anything really.
I am looking # Two different libs, atlassian-python-api, and stashy, both seem to have problems my code is very simple:
from atlassian import Bitbucket
import getpass
username = input("What is your username: ")
password = getpass.getpass(prompt="Enter your password?: ")
bitbucket = Bitbucket(
url="https://website.com:port/projects/demo_projects/repppos/",
username=username,
password=password)
data = bitbucket.project_list()
both give me this error: using stashy and another library. I heard someone suggest to use Rest API but I have no experience with this?
Traceback (most recent call last):
File "C:/Users/User/PycharmProjects/ProjectName/terrafw_gui/test_no_gui.py", line 12, in <module>
data = bitbucket.project_list()
File "C:\Users\User\PycharmProjects\ProjectName\venv\lib\site-packages\atlassian\bitbucket.py", line 22, in project_list
return (self.get('rest/api/1.0/projects', params=params) or {}).get('values')
File "C:\Users\User\PycharmProjects\ProjectName\venv\lib\site-packages\atlassian\rest_client.py", line 208, in get
trailing=trailing)
File "C:\Users\User\PycharmProjects\ProjectName\venv\lib\site-packages\atlassian\rest_client.py", line 151, in request
files=files
File "C:\Users\User\PycharmProjects\ProjectName\venv\lib\site-packages\requests\sessions.py", line 279, in request
resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
File "C:\Users\User\PycharmProjects\ProjectName\venv\lib\site-packages\requests\sessions.py", line 374, in send
r = adapter.send(request, **kwargs)
File "C:\Users\User\PycharmProjects\ProjectName\venv\lib\site-packages\requests\adapters.py", line 174, in send
timeout=timeout
File "C:\Users\User\PycharmProjects\ProjectName\venv\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 417, in urlopen
conn = self._get_conn(timeout=pool_timeout)
File "C:\Users\User\PycharmProjects\ProjectName\venv\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 232, in _get_conn
return conn or self._new_conn()
File "C:\Users\User\PycharmProjects\ProjectName\venv\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 547, in _new_conn
strict=self.strict)
TypeError: __init__() got an unexpected keyword argument 'strict'
Process finished with exit code 1
I cannot figure out how come, or why I am getting these error messages without an attempted connection (the error is given immediately without any seconds for timeout).

requests.get() throws an exception in aws lambda

I'm not able to make HTTP calls from python based lambda function hosted on AWS and managed through Serverless framework.
I've tried using botocore.vendored requests module but it shows deprecation warning and suggested to use the requests module itself.
url = V2_URL + '/api/analytics/validate/' + smId
headers = {
'Content-Type':'application/json',
'Authorization': token
}
response = requests.get(url, headers=headers)
print('Result: ')
print(response.content)
In Cloudwatch, I see this stack trace:
[ERROR] UnboundLocalError: local variable 'response' referenced before assignment
Traceback (most recent call last):
File "/var/task/serverless_sdk/__init__.py", line 97, in wrapped_handler
return user_handler(event, context)
File "src/authorize.py", line 21, in validate
principal_id = verify_token(whole_auth_token, event['pathParameters']['smId'])
File "src/authorize.py", line 38, in verify_token
response = requests.get(url, headers=headers)
File "/var/task/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/var/task/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/var/task/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/var/task/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/var/task/requests/adapters.py", line 449, in send
timeout=timeout
File "/var/task/serverless_sdk/__init__.py", line 384, in wrapper
if response:

As #blhsing and #Mark A pointed out, there was a bug in version 3.1.1 of #serverless/enterprise-plugin package. Upgrading it to version 3.1.2 solved the issue for me. All I had to do was npm i -g serverless and it took care of itself.
Details of issue here: https://github.com/serverless/serverless/issues/6801

Can't POST file attachment and data parameters with Python requests lib

I'm trying to implement a simple Django view that accept a file attachment and some other parameters and proxy the request on a remote API call.
Please note: the proxy is not the point of my question :)
This is how I implemented the view:
def image_upload(request):
token = request.POST['token']
image_file = request.FILES.values()[0]
files = {'file': ('myupload.txt', image_file.read())}
client_id = request.POST['client_id']
folder_id = request.POST['folder_id']
advert_id = request.POST['advert_id']
image_type = request.POST['image_type']
crop_image = request.POST['crop_image']
api_base_url = settings.API_BASE_URL
file_post_data = {'client_id': client_id, 'folder_id': folder_id, 'advert_id': advert_id,
'image_type': image_type, 'crop_image': crop_image}
auth_header = {'Authorization': 'Token ' + token}
r = requests.post(api_base_url + 'assets/image/upload/',
data = json.dumps(file_post_data),
headers = auth_header,
files = files)
return r.json()
The problem is that when I test this view (I use Django Test Client to do it) I get an error on the line "files = files)" that says "ValueError: cannot encode objects that are not 2-tuples".
The complete trace log is this one:
======================================================================
ERROR: test_image_upload (fbx.tests.FbxTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/andrea/Documents/src/fbxapp/onboard/fbx/tests.py", line 18, in test_image_upload
'image_type': 'A', 'crop_image': False, 'attachment': fp})
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 449, in post
response = super(Client, self).post(path, data=data, content_type=content_type, **extra)
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 262, in post
return self.request(**r)
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/home/andrea/Documents/src/fbxapp/onboard/fbx/views.py", line 42, in image_upload
files = files)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 324, in request
prep = req.prepare()
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 225, in prepare
p.prepare_body(self.data, self.files)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 385, in prepare_body
(body, content_type) = self._encode_files(files, data)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 99, in _encode_files
fields = to_key_val_list(data or {})
File "/usr/local/lib/python2.7/dist-packages/requests/utils.py", line 136, in to_key_val_list
raise ValueError('cannot encode objects that are not 2-tuples')
ValueError: cannot encode objects that are not 2-tuples
I've also tried a quick test using this to read a file: files = {'file': ('myupload.txt', open('/tmp/mytmp.txt', 'rb'))}
but it doesn't work. Do tou have any idea about how to fix this?
Thanks!

You cannot simultaneously post JSON data and multipart/form-data which is in essence what you're trying to do here. json.dumps returns a string so you're sending a string that looks like
'{"client_id": 1, "folder_id": 2, "advert_id": 3, "image_type": "jpeg", "crop_image": true}'
And then telling requests you want to use that in combination with a multipart/form-data request by sending something in via the files parameter. That is impossible and could possibly raise a better exception.
Either add the file to the JSON data after reading it into memory and send that with the appropriate Content-Type header or send the entire thing as a multipart/form-data request without using json.dumps and simply passing the dictionary you're creating to data. Use one or the other but not both.
That aside, you say that your exception comes from the line that only has files=files) on it, but it is not that line alone that causes the exception. The exception is raised by the function which happens to end on that line. The fact that your exception arises from that too is mere coincidence. This is an wart in Python that may possibly be fixed in Python 3.4. You should upgrade, because 3.4 will be awesome and newer versions of Django support Python 3.x.