We Keep Coding

How to test python's http.client.HTTPResponse?

I'm trying to work with a third party API and I am having problems with sending the request when using the requests or even urllib.request.
Somehow when I use http.client I am successful sending and receiving the response I need.
To make life easier for me, I created an API class below:
class API:
def get_response_data(self, response: http.client.HTTPResponse) -> dict:
"""Get the response data."""
response_body = response.read()
response_data = json.loads(response_body.decode("utf-8"))
return response_data
The way I use it is like this:
api = API()
rest_api_host = "api.app.com"
connection = http.client.HTTPSConnection(rest_api_host)
token = "my_third_party_token"
data = {
"token":token
}
payload = json.loads(data)
headers = {
# some headers
}
connection.request("POST", "/some/endpoint/", payload, headers)
response = connection.getresponse()
response_data = api.get_response_data(response) # I get a dictionary response
This workflow works for me. Now I just want to write a test for the get_response_data method.
How do I instantiate a http.client.HTTPResponse with the desired output to be tested?
For example:
from . import API
from unittest import TestCase
class APITestCase(TestCase):
"""API test case."""
def setUp(self) -> None:
super().setUp()
api = API()
def test_get_response_data_returns_expected_response_data(self) -> None:
"""get_response_data() method returns expected response data in http.client.HTTPResponse"""
expected_response_data = {"token": "a_secret_token"}
# I want to do something like this
response = http.client.HTTPResponse(expected_response_data)
self.assertEqual(api.get_response_data(response), expected_response_data)
How can I do this?
From the http.client docs it says:
class http.client.HTTPResponse(sock, debuglevel=0, method=None, url=None)
Class whose instances are returned upon successful connection. Not instantiated directly by user.
I tried looking at socket for the sock argument in the instantiation but honestly, I don't understand it.
I tried reading the docs in
https://docs.python.org/3/library/http.client.html#http.client.HTTPResponse
https://docs.python.org/3/library/socket.html
Searched the internet on "how to test http.client.HTTPResponse" but I haven't found the answer I was looking for.

Does someone have an example of Unit-Testing a SOAP API with Python Zeep and Mock?

I'm building a Python app that accesses a 3rd party SOAP API with Python-zeep. I want to implement some unit-tests using mocked responses as I don't always have a live server to run my tests against.
I'm new to unit-testing and not quite sure where to start. I've seen examples of using mock with the requests library, but not really sure how to translate this into Zeep.
On one of my Models I have a method to get all DevicePools from a SOAP API. Here's an excerpt of the relevant code (I use a helper method to provide the service object as I plan on using this in many other methods).
# Get Zeep Service Object to make AXL API calls
service = get_axl_client(self)
# Get list of all DevicePools present in the cluster
resp = service.listDevicePool(searchCriteria={'name': '%'},
returnedTags={'name': '', 'uuid': ''})
I want to mock the resp object, however this is of type zeep.objects.ListDevicePoolRes (a dynamic type based on the parsing of the WSDL) and I can't just instantiate an object with static values.
Maybe I'm on the wrong track here and would have to go a bit deeper and actually mock some internals of the zeep library and replace the requests response before zeep parses the XML?
If someone had an example of something similar it would be greatly appreciated.

After looking through the Python Zeep source code, I found some examples of tests using the requests-mock library which I was able to work into my solution. Here's a working example in case anyone else is trying to do something similar.
I'm not doing any assertions in this example as I first wanted to prove the concept that I could get zeep to parse a mock response correctly.
# -*- coding: utf-8 -*-
from zeep import Client
from zeep.cache import SqliteCache
from zeep.transports import Transport
from zeep.exceptions import Fault
from zeep.plugins import HistoryPlugin
from requests import Session
from requests.auth import HTTPBasicAuth
from urllib3 import disable_warnings
from urllib3.exceptions import InsecureRequestWarning
from lxml import etree
import requests_mock
disable_warnings(InsecureRequestWarning)
username = 'administrator'
password = 'password'
host = 'cucm-pub'
wsdl = 'file://C:/path/to/axlsqltoolkit/schema/current/AXLAPI.wsdl'
location = 'https://{host}:8443/axl/'.format(host=host)
binding = "{http://www.cisco.com/AXLAPIService/}AXLAPIBinding"
session = Session()
session.verify = False
session.auth = HTTPBasicAuth(username, password)
transport = Transport(cache=SqliteCache(), session=session, timeout=20)
history = HistoryPlugin()
client = Client(wsdl=wsdl, transport=transport, plugins=[history])
service = client.create_service(binding, location)
def test_listPhone():
text = """<?xml version="1.0" ?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<ns:listDevicePoolResponse xmlns:ns="http://www.cisco.com/AXL/API/11.5">
<return>
<devicePool uuid="{1B1B9EB6-7803-11D3-BDF0-00108302EAD1}">
<name>AU_PER_DP</name>
</devicePool>
<devicePool uuid="{BEF5605B-1DB0-4157-0176-6C07814C47AE}">
<name>DE_BLN_DP</name>
</devicePool>
<devicePool uuid="{B4C65CAB-86CB-30FB-91BE-6F6E712CACB9}">
<name>US_NYC_DP</name>
</devicePool>
</return>
</ns:listDevicePoolResponse>
</soapenv:Body>
</soapenv:Envelope>
"""
with requests_mock.Mocker() as m:
m.post(location, text=text)
resp = service.listDevicePool(searchCriteria={'name': '%'},
returnedTags={'name': '',
'uuid': ''})
return resp
test_listPhone()
This then gives me the following result (have manually removed all the attributes with "none" that Zeep includes for brevity):
{
'return': {
'devicePool': [
{
'name': 'AU_PER_DP',
'uuid': '{1B1B9EB6-7803-11D3-BDF0-00108302EAD1}'
},
{
'name': 'DE_BLN_DP',
'uuid': '{BEF5605B-1DB0-4157-0176-6C07814C47AE}'
},
{
'name': 'US_NYC_DP',
'uuid': '{B4C65CAB-86CB-30FB-91BE-6F6E712CACB9}'
}
]
},
'sequence': None
}

I had the same need today and did a little research and started in zeeps test cases. The link (https://github.com/mvantellingen/python-zeep/blob/2f35b7d29355ba646f5e3c6e9925033d5d6df8bb/tests/test_client.py#L109) points to an excample that shows how easy it is to test that by mocking the GET/POST by defining the data returned by them.
This returned data then makes its way through zeep and ends with the same result as it would come from the remote source.

get the full path of web service's methods from wsdl file

I am working with suds library -python 2.7- and I would like to get the full path of all methods from a wsdl file.
I used this code to get the list of methods from a wsdl file :
from suds.client import Client
url = "http://www.webservicex.net/country.asmx?WSDL"
client = Client(url)
list_of_methods = [method for method in client.wsdl.services[0].ports[0].methods]
and I get something like this :
[GetCurrencyCodeByCurrencyName, GetCurrencyByCountry, GetCurrencyCode, GetCountries, GetCurrencies, GetISOCountryCodeByCountyName, GetISD, GetCountryByCurrencyCode, GetGMTbyCountry, GetCountryByCountryCode]
But I want to get something like this :
[http://www.webserviceX.NET/GetCountryByCountryCode, ...]
so, how can I do this ?

What about using a qname attribute of a port?
from suds.client import Client
url = "http://www.webservicex.net/country.asmx?WSDL"
client = Client(url)
port = client.wsdl.services[0].ports[0]
list_of_methods = [port.qname[1] + '/' + method for method in port.methods]
print(list_of_methods)

django/python the works of views and bringing in an API

I'm just beginning to learn about python/django. I know PHP, but I wanted to get to know about this framework. I'm trying to work with yelp's API. I'm trying to figure out what to do when someone brings in a new file into the project.
In their business.py they have this:
import json
import oauth2
import optparse
import urllib
import urllib2
parser = optparse.OptionParser()
parser.add_option('-c', '--consumer_key', dest='consumer_key', help='OAuth consumer key (REQUIRED)')
parser.add_option('-s', '--consumer_secret', dest='consumer_secret', help='OAuth consumer secret (REQUIRED)')
parser.add_option('-t', '--token', dest='token', help='OAuth token (REQUIRED)')
parser.add_option('-e', '--token_secret', dest='token_secret', help='OAuth token secret (REQUIRED)')
parser.add_option('-a', '--host', dest='host', help='Host', default='api.yelp.com')
parser.add_option('-i', '--id', dest='id', help='Business')
parser.add_option('-u', '--cc', dest='cc', help='Country code')
parser.add_option('-n', '--lang', dest='lang', help='Language code')
options, args = parser.parse_args()
# Required options
if not options.consumer_key:
parser.error('--consumer_key required')
if not options.consumer_secret:
parser.error('--consumer_secret required')
if not options.token:
parser.error('--token required')
if not options.token_secret:
parser.error('--token_secret required')
if not options.id:
parser.error('--id required')
url_params = {}
if options.cc:
url_params['cc'] = options.cc
if options.lang:
url_params['lang'] = options.lang
path = '/v2/business/%s' % (options.id,)
def request(host, path, url_params, consumer_key, consumer_secret, token, token_secret):
"""Returns response for API request."""
# Unsigned URL
encoded_params = ''
if url_params:
encoded_params = urllib.urlencode(url_params)
url = 'http://%s%s?%s' % (host, path, encoded_params)
print 'URL: %s' % (url,)
# Sign the URL
consumer = oauth2.Consumer(consumer_key, consumer_secret)
oauth_request = oauth2.Request('GET', url, {})
oauth_request.update({'oauth_nonce': oauth2.generate_nonce(),
'oauth_timestamp': oauth2.generate_timestamp(),
'oauth_token': token,
'oauth_consumer_key': consumer_key})
token = oauth2.Token(token, token_secret)
oauth_request.sign_request(oauth2.SignatureMethod_HMAC_SHA1(), consumer, token)
signed_url = oauth_request.to_url()
print 'Signed URL: %s\n' % (signed_url,)
# Connect
try:
conn = urllib2.urlopen(signed_url, None)
try:
response = json.loads(conn.read())
finally:
conn.close()
except urllib2.HTTPError, error:
response = json.loads(error.read())
return response
response = request(options.host, path, url_params, options.consumer_key, options.consumer_secret, options.token, options.token_secret)
print json.dumps(response, sort_keys=True, indent=2)
Its very lengthy, I appologize for that. But my concern is what do I do with this? They've set up a def request() in here, and I'm assuming that I have to import this into my views?
I've been following the django documentation of creating a new app. In the documentation they've set up a bunch of def inside the views.py file. I'm just confused as to how am I supposed to make this work with my project? If I wanted to search for a business in the URL, how would it send the data out?
Thanks for your help.

This is a command line script that makes http requests to the yelp api. You probably don't want to make such an external request within the context of a main request handler. Well, you could call a request handler that makes this call to yelp. Let's see ...
You could import the request function and instead of invoking it with command line options, call it yourself.
from yelp.business import request as yelp_req
def my_request_handler(request):
json_from_yelp = yelp_req(...
# do stuff and return a response
Making this kind of external call inside a request handler is pretty meh though (that is, making an http request to an external service within a request handler). If the call is in ajax, it may be ok for the ux.
This business.py is just an example showing you how to create a signed request with oauth2. You may be able to just import the request function and use it. OTOH, you may prefer to write your own (perhaps using the requests library). You probably want to use celery or some other async means to make the calls outside of your request handlers and/or cache the responses to avoid costly external http io with every request.

How to get the response headers from a suds request

I'm using the python suds module and would like to retrieve the response headers (specifically Last-Modified) from a suds response.

With more effort than ought be necessary is the answer.
I've got suds version 0.3.9 here. I had to subclass the transport class in use and wrap the send method to store the last received headers on in the transport class.
import logging
logging.basicConfig(level=logging.INFO)
#logging.getLogger('suds.client').setLevel(logging.DEBUG)
#logging.getLogger('suds.transport').setLevel(logging.DEBUG)
#logging.getLogger('suds.xsd.schema').setLevel(logging.DEBUG)
#logging.getLogger('suds.wsdl').setLevel(logging.DEBUG)
from suds.client import Client
from suds.xsd.doctor import ImportDoctor, Import
from suds.transport.https import HttpAuthenticated
class MyTransport(HttpAuthenticated):
def __init__(self,*args,**kwargs):
HttpAuthenticated.__init__(self, *args, **kwargs)
self.last_headers = None
def send(self,request):
result = HttpAuthenticated.send(self, request)
self.last_headers = result.headers
return result
doctor = ImportDoctor(Import('http://schemas.xmlsoap.org/soap/encoding/'))
svc_url = 'https://server/Service?wsdl'
svc_user = 'username'
svc_pass = 'password'
client = Client(svc_url,doctor=doctor,transport=MyTransport())
# For some reason I can't be bothered to investigate, setting the username and password in
# client kwargs doesn't pass them to the custom transport:
client.set_options(location=svc_url.partition('?')[0],username=svc_user,password=svc_pass)
# call a method
client.service.SomeMethod()
# look at headers
client.options.transport.last_headers