We Keep Coding

How to test python's http.client.HTTPResponse?

I'm trying to work with a third party API and I am having problems with sending the request when using the requests or even urllib.request.
Somehow when I use http.client I am successful sending and receiving the response I need.
To make life easier for me, I created an API class below:
class API:
def get_response_data(self, response: http.client.HTTPResponse) -> dict:
"""Get the response data."""
response_body = response.read()
response_data = json.loads(response_body.decode("utf-8"))
return response_data
The way I use it is like this:
api = API()
rest_api_host = "api.app.com"
connection = http.client.HTTPSConnection(rest_api_host)
token = "my_third_party_token"
data = {
"token":token
}
payload = json.loads(data)
headers = {
# some headers
}
connection.request("POST", "/some/endpoint/", payload, headers)
response = connection.getresponse()
response_data = api.get_response_data(response) # I get a dictionary response
This workflow works for me. Now I just want to write a test for the get_response_data method.
How do I instantiate a http.client.HTTPResponse with the desired output to be tested?
For example:
from . import API
from unittest import TestCase
class APITestCase(TestCase):
"""API test case."""
def setUp(self) -> None:
super().setUp()
api = API()
def test_get_response_data_returns_expected_response_data(self) -> None:
"""get_response_data() method returns expected response data in http.client.HTTPResponse"""
expected_response_data = {"token": "a_secret_token"}
# I want to do something like this
response = http.client.HTTPResponse(expected_response_data)
self.assertEqual(api.get_response_data(response), expected_response_data)
How can I do this?
From the http.client docs it says:
class http.client.HTTPResponse(sock, debuglevel=0, method=None, url=None)
Class whose instances are returned upon successful connection. Not instantiated directly by user.
I tried looking at socket for the sock argument in the instantiation but honestly, I don't understand it.
I tried reading the docs in
https://docs.python.org/3/library/http.client.html#http.client.HTTPResponse
https://docs.python.org/3/library/socket.html
Searched the internet on "how to test http.client.HTTPResponse" but I haven't found the answer I was looking for.

Can't change headers with DRF APIClient()

I use APIClient() for my tests.
I use Token auth, so I need to use THIS
If we dive into the source code we'll see next:
# rest_framework/test.py
class APIClient(APIRequestFactory, DjangoClient):
def __init__(self, enforce_csrf_checks=False, **defaults):
super().__init__(**defaults)
self.handler = ForceAuthClientHandler(enforce_csrf_checks)
self._credentials = {}
def credentials(self, **kwargs):
"""
Sets headers that will be used on every outgoing request.
"""
self._credentials = kwarg
Also I use APIClient() as a pytest fixture in my code:
#pytest.fixture(scope="function")
def _api():
"""API factory for anonymous and auth requests"""
def __api(token=None, field=None):
api_client = APIClient()
headers = {}
if token:
headers["HTTP_AUTHORIZATION"] = f"Token {token}"
if field:
headers["X-CUSTOM-HEADER"] = field
api_client.credentials(**headers)
return api_client
return __api
But if we create TestMiddleware to look for headers, we see next:
lass TestMiddleware:
def __init__(self, get_response):
self._get_response = get_response
def __call__(self, request):
header = request.headers.get("X-CUSTOM-HEADER") # None
header = request.META.get("X-CUSTOM-HEADER") # Works fine!
...
# Response processing
response = self._get_response(request)
return response
The question is: Have we any way to have access to the X-CUSTOM-HEADER with APIClient() ?
Also if I use Postman it obviously works fine with request.headers.get()

The kwargs passed to the credentials() method ends up feeding directly into the constructor for a WSGIRequest; this means the kwargs it accepts aren't HTTP headers, but WSGI environment variables. And HTTP headers passed as WSGI env vars are always prefixed with HTTP_ — e.g. the Authorization header is configured with HTTP_AUTHORIZATION. Also, underscores are used in place of dashes.
To have your X-Custom-Header header come out the other side in request.headers (not request.META, which is a copy of the WSGI env vars), pass HTTP_X_CUSTOM_HEADER instead of X-CUSTOM-HEADER.

Create a decorator to handle token validation in flask

I have two endpoints defined in my flask service python file.
The first endpoint returns a list of parent and child nodes from a mmap json file which it parses.
The second endpoint returns a specific child field from a mmap json file which it parses.
Each of these endpoints can only be used when the token has been validated. Thus I have the following implementations.
from flask import request
import requests
def check_token(self):
# Method to verify the token from the another python Service
token_header = request.headers['authorization']
# Remove the 'Basic" part from the token
auth_token = token_header.split(maxsplit=1)[1]
self.__payload = {'token' : auth_token}
# Append the token to the header by using the payload
response = requests.get(self.__url, params=self.__payload, verify=False)
return response
# 1. This endpoint returns a list of parent and child nodes from a mmap json file which it parses.
class SystemList(Resource):
def get(self, systemname):
token = check_token()
if token.ok:
# Open the mmap file, parse it, and jsonify the result and return it
# Invalid token present
else:
return make_response(
jsonify("Invalid access as token invalid.")
)
# 2. The endpoint returns a specific child field from a mmap json file which it parses.
class SystemChildList(Resource):
def get(self, systemname, id):
token = check_token()
if token.ok:
# Open the mmap file, parse it, and jsonify the result and return it
# Invalid token present
else:
return make_response(
jsonify("Invalid access as token invalid.")
)
The issue I have is that I want to use a decorator to handle the validation of the token.
I want to be able to add it before the get() method something like the following.
#validatetoken
def get(self, designation):
# I am not sure what goes here and what do I need to put here?
# I want to be able to have one decorator for both of the SystemList and SystemChildList
# resource shown above.
I am not sure on what goes in the decorator. I am really new to these concepts. Any help is appreciated.

You can use method_decorators parameter to achieve this
try,
from flask import request
from functools import wraps
import requests
def check_token(f):
#wraps(f)
def decorated(*args, **kwargs):
token_header = request.headers['authorization']
# Remove the 'Basic" part from the token
auth_token = token_header.split(maxsplit=1)[1]
__url = "url_for_token_validation"
__payload = {'token' : auth_token}
# Append the token to the header by using the payload
response = requests.get(__url, params=__payload, verify=False)
if response.status_code != requests.codes.ok:
return make_response(
jsonify("Invalid access as token invalid.")
)
return f(*args, **kwargs)
return decorated
# 1. This endpoint returns a list of parent and child nodes from a mmap json file which it parses.
class SystemList(Resource):
#check_token
def get(self, systemname):
# Open the mmap file, parse it, and jsonify the result and return it
# 2. The endpoint returns a specific child field from a mmap json file which it parses.
class SystemChildList(Resource):
#check_token
def get(self, systemname, id):
# Open the mmap file, parse it, and jsonify the result and return it

Google API Client for Python. Batch requests: how to access to a specific request in the callback

According to official doc, you can make batch request through Google API Client Libraries for Python. Here, there is an example
from apiclient.http import BatchHttpRequest
def list_animals(request_id, response, exception):
if exception is not None:
# Do something with the exception
pass
else:
# Do something with the response
pass
def list_farmers(request_id, response):
"""Do something with the farmers list response."""
pass
service = build('farm', 'v2')
batch = service.new_batch_http_request()
batch.add(service.animals().list(), callback=list_animals)
batch.add(service.farmers().list(), callback=list_farmers)
batch.execute(http=http)
Is there a way to access to the request in the callback. E.g., print the request (not the request_id) if there is an exception?

I ran into this same issue, as I needed the original request for using compute.instances().aggregatedList_next() function.
I ended up passing a unique request_id to batch.add() and created a dictionary to keep track of the original requests that were made. Then within the callback, I referenced that dictionary and pulled out the request by using the unique request_id.
So I did something along the following:
requests = {}
project = 'example-project'
batch = service.new_batch_http_request(callback=callback_callable)
new_request = compute.instances().aggregatedList(project=project)
requests[project] = new_request
batch.add(new_request, request_id=project)
Within the callable function:
def callback_callable(self, request_id, response, exception):
original_request = requests[request_id]
Note that the value passed to the request_id needs to be a string. If an int is passed it will raise a TypeError quote_from_bytes() expected bytes.
Hope this helps someone!

In case it helps anyone - I'm using google-api-python-client==2.65.0
and the call back is passed differently - it is specified like this:
def list_animals(request_id, response, exception=None):
if exception is None:
original_request = requests[request_id]
# process here...
batch = service.new_batch_http_request(callback=list_animals)
batch.add(service.animals().list())
batch.add(service.farmers().list())
batch.execute()

Create SoapRequest without sending them with Suds/Python

Is there anyway to get suds returning the SoapRequest (in XML) without sending it?
The idea is that the upper levels of my program can call my API with an additional boolean argument (simulation).
If simulation == false then process the other params and send the request via suds
If simulation == false then process the other params, create the XML using suds (or any other way) and return it to the caller without sending it to the host.
I already implemented a MessagePlugin follwing https://fedorahosted.org/suds/wiki/Documentation#MessagePlugin, but I am not able to get the XML, stop the request and send back the XML to the caller...
Regards

suds uses a "transport" class called HttpAuthenticated by default. That is where the actual send occurs. So theoretically you could try subclassing that:
from suds.client import Client
from suds.transport import Reply
from suds.transport.https import HttpAuthenticated
class HttpAuthenticatedWithSimulation(HttpAuthenticated):
def send(self, request):
is_simulation = request.headers.pop('simulation', False)
if is_simulation:
# don't actually send the SOAP request, just return its XML
return Reply(200, request.headers.dict, request.msg)
return HttpAuthenticated(request)
...
sim_transport = HttpAuthenticatedWithSimulation()
client = Client(url, transport=sim_transport,
headers={'simulation': is_simulation})
It's a little hacky. (For example, this relies on HTTP headers to pass the boolean simulation option down to the transport level.) But I hope this illustrates the idea.

The solution that I implemented is:
class CustomTransportClass(HttpTransport):
def __init__(self, *args, **kwargs):
HttpTransport.__init__(self, *args, **kwargs)
self.opener = MutualSSLHandler() # I use a special opener to enable a mutual SSL authentication
def send(self,request):
print "===================== 1-* request is going ===================="
is_simulation = request.headers['simulation']
if is_simulation == "true":
# don't actually send the SOAP request, just return its XML
print "This is a simulation :"
print request.message
return Reply(200, request.headers, request.message )
return HttpTransport.send(self,request)
sim_transport = CustomTransportClass()
client = Client(url, transport=sim_transport,
headers={'simulation': is_simulation})
Thanks for your help,