Unit testing for python function-using post requests and get requests - python

I am trying to test the below function using unit test mocks but i am not able to do it since the return type from request get/post is of type <class 'requests.models.Response'>. please let me know how to mock this and unit test this below function:
def response_check(base_url, headers, schedule_name, response):
"""
Function to get response from dataform API and check the job status
"""
run_url = base_url + "/" + response.json()["id"]
query_response = requests.get(run_url, headers=headers)
while query_response.json()["status"] == "RUNNING":
time.sleep(10)
print("Dataform job running")
query_response = requests.get(run_url, headers=headers)
if query_response.json()["status"] in ["FAILED", "CANCELLED", "TIMED_OUT"]:
raise AirflowException(
f'Dataform task {schedule_name} has been {response.json()["status"]} for reason {response.json()["runLogUrl"]}'
)
print(query_response.json())
return "Dataform job finished"
I need to check for different response status but i am not able to mock the return type of it

Please test using http module and the http codes.
For e.g. success is http.HTTPStatus.OK
Not found is http.HTTPStatus.NOT_FOUND
Bad input is http.HTTPStatus.BAD_REQUEST etc.

Related

Create a middleware which listens to localhost:3332 and prints the endpoints called

I am trying to create a system in java which listens to localhost:3332 and prints the endpoints. I have a application which runs on that port where I can apply various actions to a table.
I have tried to run this script :
url=url = 'http://127.0.0.1:3332/'
while True:
with requests.Session() as session:
response = requests.get(url)
if response.status_code == 200:
print("Succesful connection with API.")
// here I should print the endpoints
Unfortunately, it doesn't work. Any suggestion is more than welcome
The script doesn't work because the "with requests.Session() as session" command is missing parentheses, which are necessary for command execution. Correcting this will fix the issue.
Also, it's not clear what you mean by printing the endpoints. Depending on the application, you may need to modify the script in order to make an API call that will return the endpoints in this way:
url = "http://127.0.0.1:3333/endpoints"
with requests.Session() as session:
response = requests.get(url)
if response.status_code == 200:
print("Succesful connection with API.")
// here I should print the endpoints - assuming the API call gives json data
data = response.json()
if data:
for endpoint in data:
print("Endpoint:", endpoint)
Hope this helps.

How to mock a url path returning response in Django / Python?

I have a function like this:
def get_some_data(api_url, **kwargs)
# some logic on generating headers
# some more logic
response = requests.get(api_url, headers, params)
return response
I need to create a fake/mock "api_url", which, when made request to, would generate a valid response.
I understand how to mock the response:
def mock_response(data):
response = requests.Response()
response.status_code = 200
response._content = json.dumps(data)
return response
But i need to make the test call like this:
def test_get_some_data(api_url: some_magic_url_path_that_will_return_mock_response):
Any ideas on how to create an url path returning a response within the scope of the test (only standard Django, Python, pytest, unittest) would be very much appreciated
The documentation is very well written and more than clear on how to mock whatever you want. But, let say you have a service that makes the 3rd party API call:
def foo(url, params):
# some logic on generating headers
# some more logic
response = requests.get(url, headers, params)
return response
In your test you want to mock the return value of this service.
#patch("path_to_service.foo")
def test_api_call_response(self, mock_response):
mock_response.return_value = # Whatever the return value you want it to be
# Here you call the service as usual
response = foo(..., ...)
# Assert your response

Python3 mock response object

I have one function where I am calling API to post get the resource. Since my function does not return anything then its tough to write unit test for failure scenario. Here I want to force request.get() to return different HTTP status code.
Is there anyway to mock my function to return desired status code?
foo.py
def getData():
response = requests.get(run_task_status_url, headers=iics_job_header)
logging.debug(f"Activity Monitor API response: {response.json()}")
if 200 == response.status_code:
print("success")
else 401 == response.status_code:
print("401")

Facebook webhook making multiple calls for the same message?

I made and echo bot on AWS with Python and serverless.
I keep getting the same request again and again.
I read the faq where it says u have to deliver a status code 200 else it will keep retrying the webhook.
I'm not sure How I do this.
I have noticed that the sequence number is always the same for the calls so I assume the reply I sent was not acknowledged.
my code is here
import os
import json
import requests
import random
from datetime import datetime
######################
# helper functions
######################
##recursively look/return for an item in dict given key
def find_item(obj, key):
item = None
if key in obj: return obj[key]
for k, v in obj.items():
if isinstance(v,dict):
item = find_item(v, key)
if item is not None:
return item
##recursivley check for items in a dict given key
def keys_exist(obj, keys):
for key in keys:
if find_item(obj, key) is None:
return(False)
return(True)
##send txt via messenger to id
def send_message(send_id, msg_txt):
print("Send message called")
print (datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3])
params = {"statusCode": 200,"access_token": os.environment['accesstoken']}
headers = {"statusCode": "200","Content-Type": "application/json"}
data = json.dumps({"statusCode": "200","recipient": {"id": send_id},
"message": {"text": msg_txt}})
r = requests.post("https://graph.facebook.com/v2.9/me/messages", params=params, headers=headers, data=data)
print (r.text)
if r.status_code != 200:
print(r.status_code)
print(r.text)
#-----------------------------------------------------------
def hello(event, context):
#debug
event=json.loads(json.dumps(event))
print("event:" )
print(event)
# print("context")
# print(context)
#handle webhook challenge
try:
if keys_exist(event, ["queryStringParameters","hub.verify_token","hub.challenge"]):
print("subscribe to webhook invoked")
v_token = str(find_item(event, 'hub.verify_token'))
challenge = find_item(event, 'hub.challenge')
if ("strongtoken" == v_token):
response = {
"statusCode": 200,
"body": str(challenge)
}
print(challenge)
return response
#handle messaging events
if keys_exist(event, ['body']):
event_entry=json.loads(event['body'])
if ((len(event_entry['entry'])>0) & (keys_exist(event_entry['entry'][0],['messaging'])) ):
messaging_event = event_entry['entry'][0]['messaging'][0]
if (keys_exist(messaging_event,['message'])):
msg_txt = messaging_event['message']['text']
sender_id = messaging_event['sender']['id']
print(sender_id)
first_word = msg_txt.split(" ")[0]
send_message(sender_id, msg_txt)
else:
print("Did not send message")
pass
else:
print("Did not send message")
pass
else:
pass
except:
pass
I have given the status code 200 in soo many places and I'm not sure y I still keep getting the same problem.
If you are getting multiple messages your server did not return 200 status code to webhook request from Facebook server. This means an error occurred on your part otherwise 200 should be returned. It looks to me that problems are in following lines:
params = {"statusCode": 200,"access_token": os.environment['accesstoken']}
headers = {"statusCode": "200","Content-Type": "application/json"}
data = json.dumps({"statusCode": "200","recipient": {"id": send_id},
"message": {"text": msg_txt}})
Firstly you are passing statusCode in a data body of message and according to this documentation message response should not contain it.
Another problem could be sending status code inside params. I would remove status code completely from send_message method. I doubt its needed there. You are basically trying to return status 200 on wrong end. You are trying to return it on output instead of input (from Facebooks point of view).
So its quite possible you are correctly getting message from Facebook but you are still returning wrong status code because you are calling send_message method from inside messaging event and send_message method should return status "400 bad request" because you are sending faulty request. Thus your server also returns wrong response code.
Just make sure your code works correctly and 200 should be returned.
EDIT:
So i would try with following code:
params = {"access_token": os.environment['accesstoken']}
headers = {"Content-Type": "application/json"}
data = json.dumps({"recipient": {"id": send_id},
"message": {"text": msg_txt}})
I'm working on a Facebook Messenger chatbot with a Node/Express server and was having the same exact issue. The server was appropriately sending 200 responses and yet the issue persisted. I resolved the problem by caching message ID's and checking for duplicates before processing:
var NodeCache = require('node-cache');
var myCache = new NodeCache();
app.post('/webhook/', function(req, res) {
var messageID = req.body.entry[0].messaging[0].message.mid;
var checkForDupe = myCache.get(messageID);
if (checkForDupe == undefined) {
var setID = myCache.set(req.body.entry[0].messaging[0].message.mid, req.body.entry[0].messaging[0].message.mid);
//Handle message .....
I hope this helps someone. It was driving me nuts.
Your webhook callback should always return a 200 OK HTTP response when invoked by Facebook. Failing to do so may cause your webhook to be unsubscribed by the Messenger Platform. (copied from messenger docs.)
You have to search problem from your returning response. It is not with the sending data.
response = requests.post(POST_MESSAGE_URL, headers={
'Content-Type': 'application/json'
}, params={
'access_token': ACCESS_TOKEN
}, data=json.dumps(DATA_JSON)
this code working fine for me
I am from Java background but still I will try to help you.
In handle webhook challenge I can see you returning 200 status code
But in handle messaging block 200 is not returned You have set 200 status code in send message. But it will not work you have to return 200 from handle messaging block.
Also 200 should be returned even if any exception occurs in this block otherwise on exception it will get blocked. So may be in finally block you can return 200.
As an alternative, you can use chatbotproxy.com which always return 200 in 1ms and pass the exact same request to your endpoint. Since immediate response is very important for Facebook Messenger platform, 1 millisecond time is a very promising feature. If you need help to try, let me know.

requests library and http error

I'm currently using the python requests library to interact with an external api which uses json.
Each endpoint works via a method (of the api class) and uses the collect_data method.
However I want the scraper to continue running whenever it encounters a http error (and ideally output this to a log).
What's the best way to do this as currently it just breaks when I use http.raise_for_status()
It seems like I should be using a try/except in someway but not sure how best to do this here?
def scrape_full_address(self, house_no, postcode):
address_path = '/api/addresses'
address_url = self.api_source + address_path
payload = {
'houseNo': house_no,
'postcode': postcode,
}
return self.collect_data(url=address_url, method='get', payload=payload)
def collect_data(self, url, method, payload=None):
if method == 'get':
data = None
params = payload
elif method == 'post':
params = None
data = payload
response = getattr(requests, method)(url=url, params=params, json=data, headers=self.headers)
if response.status_code == 200:
return response.json()
else:
return response.raise_for_status()
When you call scrape_full_address() elsewhere in your code wrap that in a try statement.
For more info see: https://wiki.python.org/moin/HandlingExceptions
try:
scrape_full_address(659, 31052)
except HTTPError:
print "Oops! That caused an error. Try again..."

Categories