I'm trying to work with a third party API and I am having problems with sending the request when using the requests or even urllib.request.
Somehow when I use http.client I am successful sending and receiving the response I need.
To make life easier for me, I created an API class below:
class API:
def get_response_data(self, response: http.client.HTTPResponse) -> dict:
"""Get the response data."""
response_body = response.read()
response_data = json.loads(response_body.decode("utf-8"))
return response_data
The way I use it is like this:
api = API()
rest_api_host = "api.app.com"
connection = http.client.HTTPSConnection(rest_api_host)
token = "my_third_party_token"
data = {
"token":token
}
payload = json.loads(data)
headers = {
# some headers
}
connection.request("POST", "/some/endpoint/", payload, headers)
response = connection.getresponse()
response_data = api.get_response_data(response) # I get a dictionary response
This workflow works for me. Now I just want to write a test for the get_response_data method.
How do I instantiate a http.client.HTTPResponse with the desired output to be tested?
For example:
from . import API
from unittest import TestCase
class APITestCase(TestCase):
"""API test case."""
def setUp(self) -> None:
super().setUp()
api = API()
def test_get_response_data_returns_expected_response_data(self) -> None:
"""get_response_data() method returns expected response data in http.client.HTTPResponse"""
expected_response_data = {"token": "a_secret_token"}
# I want to do something like this
response = http.client.HTTPResponse(expected_response_data)
self.assertEqual(api.get_response_data(response), expected_response_data)
How can I do this?
From the http.client docs it says:
class http.client.HTTPResponse(sock, debuglevel=0, method=None, url=None)
Class whose instances are returned upon successful connection. Not instantiated directly by user.
I tried looking at socket for the sock argument in the instantiation but honestly, I don't understand it.
I tried reading the docs in
https://docs.python.org/3/library/http.client.html#http.client.HTTPResponse
https://docs.python.org/3/library/socket.html
Searched the internet on "how to test http.client.HTTPResponse" but I haven't found the answer I was looking for.
Related
I have a function like this:
def get_some_data(api_url, **kwargs)
# some logic on generating headers
# some more logic
response = requests.get(api_url, headers, params)
return response
I need to create a fake/mock "api_url", which, when made request to, would generate a valid response.
I understand how to mock the response:
def mock_response(data):
response = requests.Response()
response.status_code = 200
response._content = json.dumps(data)
return response
But i need to make the test call like this:
def test_get_some_data(api_url: some_magic_url_path_that_will_return_mock_response):
Any ideas on how to create an url path returning a response within the scope of the test (only standard Django, Python, pytest, unittest) would be very much appreciated
The documentation is very well written and more than clear on how to mock whatever you want. But, let say you have a service that makes the 3rd party API call:
def foo(url, params):
# some logic on generating headers
# some more logic
response = requests.get(url, headers, params)
return response
In your test you want to mock the return value of this service.
#patch("path_to_service.foo")
def test_api_call_response(self, mock_response):
mock_response.return_value = # Whatever the return value you want it to be
# Here you call the service as usual
response = foo(..., ...)
# Assert your response
I am trying to implement a type of custom authentication by using aiohttp something like the example in this link but I also need request body. Here is an example for requests:
class CustomAuth(AuthBase):
def __init__(self, secretkey):
self.secretkey = secretkey
def get_hash(self, request):
if request.body:
data = request.body.decode('utf-8')
else:
data = "{}"
signature = hmac.new(
str.encode(self.secretkey),
msg=str.encode(data),
digestmod=hashlib.sha256
).hexdigest().upper()
return signature
def __call__(self, request):
request.headers["CUSTOM-AUTH"] = self.get_hash(request)
return request
I've looked into tracing and BasicAuth but they are useless in my situation. On on_request_start request body is not ready, on on_request_chunk_sent headers have already been sent. A solution like BasicAuth don't have access the request data at all.
Do you have any idea?
Thanks in advance.
I have created a custom class, which push my logs to splunk, but somehow it is not working. Here is the class.
class Splunk(logging.StreamHandler):
def __init__(self, url, token):
super().__init__()
self.url = url
self.headers = {f'Authorization': f'Splunk {token}'}
self.propagate = False
def emit(self, record):
mydata = dict()
mydata['sourcetype'] = 'mysourcetype'
mydata['event'] = record.__dict__
response = requests.post(self.url, data=json.dumps(mydata), headers=self.headers)
return response
I call the class from my logger class, somehow like this (adding additional handler), so that it can log on console along with send to splunk
if splunk_config is not None:
splunk_handler = Splunk(splunk_config["url"], splunk_config["token"])
self.default_logger.addHandler(splunk_handler)
But somehow, I am not able to see any logs in splunk. Though I can see the logs in console.
When I try to run the strip down version of above logic from python3 terminal, it is successful.
import requests
import json
url = 'myurl'
token = 'mytoken'
headers = {'Authorization': 'Splunk mytoken'}
propagate = False
mydata = dict()
mydata['sourcetype'] = 'mysourcetype'
mydata['event'] = {'name': 'root', 'msg': 'this is a sample message'}
response = requests.post(url, data=json.dumps(mydata), headers=headers)
print(response.text)
Things I have already tried, making my dictionary data as JSON serializable using below link but it didn't helped.
https://pynative.com/make-python-class-json-serializable/
Any other things to try ?
I've successfully used this Python Class for Sending Events to Splunk HTTP Event Collector instead of writing a dedicated class
https://github.com/georgestarcher/Splunk-Class-httpevent
Advantage is that it implements batchEvent() and flushBatch() methods to submit multiple events at once across multiple threads.
The example here should get you started:
https://github.com/georgestarcher/Splunk-Class-httpevent/blob/master/example.py
If this answers your question, take a moment to accept the answer. This can be done by clicking on the check mark beside the answer to toggle it from greyed out to filled in!
The long story short is I am working on building a server that serves as something as a chat bot. The server uses google dialog flow. Right now I have an endpoint exposed that allows me to talk to my server, when I hit that endpoint, google auth, as well as google dialog flow gets called. I am attempting to mock the response of dialog flow while leaving the actual server to respond to the network call. As of now my test looks like this.
This is my base test file:
import unittest
import mock
class BaseTest(unittest.TestCase, object):
def __init__(self, *args, **kwargs):
super(BaseTest, self).__init__(*args, *kwargs)
def auto_patch(self, patch_target):
patcher = mock.patch(patch_target)
patched = patcher.start()
self.addCleanup(patcher.stop)
return patched
This is my test file:
import json
import uuid
from os import path
from tests.base_test import BaseTest
from agent.api_service import app
import requests_mock
import pytest
from hamcrest import assert_that, has_items, equal_to
CWD = path.dirname(path.realpath(__file__))
class TestAudio(BaseTest):
def test__interact__full_no_stt(self):
payload = json.load(open("tests/json_payloads/test__interact__full_audio.json"))
u_session_id = str(uuid.uuid1())
payload["session_id"] = u_session_id
#mock a 500 back from STT
with open("tests/json_payloads/stt_500.json", "r") as issues_file:
mock_response = issues_file.read()
with requests_mock.Mocker() as m:
m.register_uri('POST', 'https://speech.googleapis.com/v1/speech:recognize', text=mock_response)
request, response = app.test_client.post("/agent_service/interact", data=json.dumps(payload))
self.assertEqual(200, response.status)
This is my google stt file:
import json
import requests
from agent.exceptions import GoogleSTTException
from agent.integrations.google.google_auth_service import get_auth_token
from agent.integrations.google.google_stt_request import GoogleSTTRequest
from agent.integrations.google.google_stt_response import GoogleSTTResponse
def speech_to_text(audio_string):
try:
google_stt_request = GoogleSTTRequest(audio_string).to_payload()
request_headers = dict()
request_headers['Authorization'] = 'Bearer ' + get_auth_token()
request_headers['Content-Type'] = 'application/json'
url = 'https://speech.googleapis.com/v1/speech:recognize'
google_response = requests.post(url, data=json.dumps(google_stt_request), headers=request_headers)
response = GoogleSTTResponse(google_response.json())
return response
except Exception as e:
raise GoogleSTTException('Received an error invoking google stt {}'.format(e))
Does anyone have any ideas on how I can mock the response from the google stt call, without touching the google auth call or the server call itself? I have tried a handful of things and so far no luck. I either end up mocking nothing, or both the google stt and auth call.
So I ended up moving away from the original implementation, but this is what got me there.
#responses.activate
def test__interact__full_no_stt(self):
payload = json.load(open("tests/json_payloads/test__interact__full_audio.json"))
u_session_id = str(uuid.uuid1())
payload["session_id"] = u_session_id
#mock a 500 back from STT
responses.add(responses.POST,
'https://speech.googleapis.com/v1/speech:recognize',
json={'error': 'broken'}, status=500)
request, response = app.test_client.post("/agent_service/interact", data=json.dumps(payload))
self.assertEqual(200, response.status)
result = response.json
Responses makes this much easier, just be sure to include the annotation at the top of the test.
For testing purposes I'm trying to create a Response() object in python but it proves harder then it sounds.
i tried this:
from requests.models import Response
the_response = Response()
the_response.code = "expired"
the_response.error_type = "expired"
the_response.status_code = 400
but when I attempted the_response.json() i got an error because the function tries to get len(self.content) and a.content is null.
So I set a._content = "{}" but then I get an encoding error, so I have to change a.encoding, but then it fails to decode the content....
this goes on and on. Is there a simple way to create a Response object that's functional and has an arbitrary status_code and content?
That because the _content attribute on the Response objects (on python3) has to be bytes and not unicodes.
Here is how to do it:
from requests.models import Response
the_response = Response()
the_response.code = "expired"
the_response.error_type = "expired"
the_response.status_code = 400
the_response._content = b'{ "key" : "a" }'
print(the_response.json())
Create a mock object, rather than trying to build a real one:
from unittest.mock import Mock
from requests.models import Response
the_response = Mock(spec=Response)
the_response.json.return_value = {}
the_response.status_code = 400
Providing a spec ensures that the mock will complain if you try to access methods and attributes a real Response doesn't have.
Just use the responses library to do it for you:
import responses
#responses.activate
def test_my_api():
responses.add(responses.GET, 'http://whatever.org',
json={}, status=400)
...
This has the advantage that it intercepts a real request, rather than having to inject a response somewhere.
Another approach by using the requests_mock library, here with the provided fixture:
import requests
def test_response(requests_mock):
requests_mock.register_uri('POST', 'http://test.com/', text='data', headers={
'X-Something': '1',
})
response = requests.request('POST', 'http://test.com/', data='helloworld')
...