We Keep Coding

Can you get a more comprehensive http response using python requests?

I tried calling an API using both python and nodejs but the format in which the response is in is quite different.
For nodejs, it returns the statuscode, headers, body, request etc.
E.g.
(Only an example, not the actual response so you can ignore any syntax errors)
{
"statusCode" : 200,
"headers" : {
'xxxx' : 'xxxx"
},
"body" : {
"name" : "james",
"age" : 35
},
"request" {
"method" : "POST"
}
}
For python, it only returns the response body.
E.g.
(Only an example, not the actual response so you can ignore any syntax errors)
{
'name' : james,
'age' : 35
}
I am aware that i am able to get the headers etc using python request response object such as response.headers etc but is there a way that allows me to a similar response like nodejs. I know about response.dir but i need to forward this response so the format of response.dir is not accepted.
Thanks all!

You can compile your own response, similar to what you get from nodejs, something like
import json
import requests
url = 'http://date.jsontest.com/'
r = requests.get(url)
j = {
"statusCode": r.status_code,
"headers": dict(r.headers),
"body": r.json(),
"request": {
"method": r.request.method
}
}
print(json.dumps(j, indent=4))
Will produce:
{
"statusCode": 200,
"headers": {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
"X-Cloud-Trace-Context": "9b4e313aa5e6373ddc997dc963ee8f1b",
"Date": "Thu, 23 Jun 2022 03:11:27 GMT",
"Server": "Google Frontend",
"Content-Length": "100"
},
"body": {
"date": "06-23-2022",
"milliseconds_since_epoch": 1655953887719,
"time": "03:11:27 AM"
},
"request": {
"method": "GET"
}
}

API FedEX "INVALID.INPUT.EXCEPTION","message":"Invalid field value in the input"

I'm trying to validade an address in FedEX API using Python 3.8 and it returns an error of invalid field value
First I connect to the Auth API
payload={"grant_type": "client_credentials",'client_id':Client_id,'client_secret':Client_secret}
url = "https://apis-sandbox.fedex.com/oauth/token"
headers = {'Content-Type': "application/x-www-form-urlencoded"}
response=requests.post(url, data=(payload), headers=headers)
And it returns a message with the Auth token correctly
{"access_token":"eyJhbGciOiJSUzI1NiIsInRM5U0F2eUs1ZVFBVTFzS5k","token_type":"bearer","expires_in":3599,"scope":"CXS SECURE"}
Then I just get the token to use it in next transactions
token = json.loads(response.text)['access_token']
Then I prepare the next payload for address validation API
payload_valid_address = {
"addressesToValidate": [
{
"address":
{
"streetLines": ["7372 PARKRIDGE BLVD"],
"city": "Irving",
"stateOrProvinceCode": "TX",
"postalCode": "75063-8659",
"countryCode": "US"
}
}
]
}
And send the request to the new endpoint with the given token
url = "https://apis-sandbox.fedex.com/address/v1/addresses/resolve"
headers = {
'Content-Type': "application/json",
'X-locale': "en_US",
'Authorization': 'Bearer '+ token
}
response = requests.post(url, data=payload_valid_address, headers=headers)
print(response.text)
and get the error
<Response [422]>
{"transactionId":"50eae03e-0fec-4ec7-b068-d5c456b64fe5","errors":[{"code":"INVALID.INPUT.EXCEPTION","message":"Invalid field value in the input"}]}
I have made inumerous tests and I don't get the invalid field.
Anyone know what is happening and can help?

payload = json.dumps({input payload}) this works to prevent the response error 422
<Response [422]>

I have fixed it
For any reason, convert the string payload_valid_address to Json in 2 steps doesn't work
payload_valid_address = {....}
payload = json.dumps(payload_valid_address)
But if made in just one step it pass the API request
payload_valid_address = json.dumps({....})

I also had this error and additionally had to use json.loads(json_string) when the json data contained key words like false, true or null.
import json
json_string = r'''{
"accountNumber": {
"value": "802255209"
},
"requestedShipment": {
"shipper": {
"address": {
"postalCode": 75063,
"countryCode": "US"
}
},
"recipient": {
"address": {
"postalCode": "m1m1m1",
"countryCode": "CA"
}
},
"shipDateStamp": "2020-07-03",
"pickupType": "DROPOFF_AT_FEDEX_LOCATION",
"serviceType": "INTERNATIONAL_PRIORITY",
"rateRequestType": [
"LIST",
"ACCOUNT"
],
"customsClearanceDetail": {
"dutiesPayment": {
"paymentType": "SENDER",
"payor": {
"responsibleParty": null
}
},
"commodities": [
{
"description": "Camera",
"quantity": 1,
"quantityUnits": "PCS",
"weight": {
"units": "KG",
"value": 20
},
"customsValue": {
"amount": 100,
"currency": "USD"
}
}
]
},
"requestedPackageLineItems": [
{
"weight": {
"units": "KG",
"value": 20
}
}
]
}
}'''
data = json.loads(json_string)
data
response = requests.request("POST", url, data=json.dumps(data), headers=headers)

I have managed to get Fedex to accept my ayload, however i get this message:
"code":"INVALID.INPUT.EXCEPTION","message":"Validation failed for object='shipShipmentInputVO'. Error count: 1","parameterList":[{"key":"NotNull.shipShipmentInputVO.labelResponseOptions","value":"labelResponseOptions cannot be null"
and when I add the required parameter
'labelResponseOptions': "URL_ONLY"
I get this:
"code":"SYSTEM.UNEXPECTED.ERROR","message":"The system has experienced an unexpected problem and is unable to complete your request. Please try again later. We regret any inconvenience."
Is this really some Fedex internal error?

Is there a way to send a POST request to slack without using Webhook?

I have tried to send POST requests to my slack channel using webhooks to no avail.
It always returns a bad request no matter what I do. Is there a way to send a POST request to slack without using webhooks?
EDIT: Code that I'm using
import json
import urllib.request
#import botocore.requests as requests
def lambda_handler(event, context):
webhook=event['webhook']
#response = urllib.request.urlopen(message)
#print(response)
slack_URL = 'https://hooks.slack.com/services/mywebhookurl'
# req = urllib.request.Request(SLACK_URL, json.dumps(webhook).encode('utf-8'))
json=webhook
json=json.encode('utf-8')
headers={'Content-Type': 'application/json'}
#urllib.request.add_data(data)
req = urllib.request.Request(slack_URL, json, headers)
response = urllib.request.urlopen(req)

I think the problem arises when you encode your JSON in utf-8. Try the following script.
import json
import requests
# Generate your webhook url at https://my.slack.com/services/new/incoming-webhook/
webhook_url = "https://hooks.slack.com/services/YYYYYYYYY/XXXXXXXXXXX"
slack_data = {'text': "Hi Sarath Kaul"}
response = requests.post(webhook_url, data=json.dumps(slack_data),headers={'Content-Type': 'application/json'})
print response.status_code
If you want to use urllib
import json
import urllib.request
import urllib.parse
url = 'https://hooks.slack.com/services/YYYYYYYYY/XXXXXXXXXXX'
data = json.dumps({'text': "Sarath Kaul"}).encode('utf-8') #data should be in bytes
headers = {'Content-Type': 'application/json'}
req = urllib.request.Request(url, data, headers)
resp = urllib.request.urlopen(req)
response = resp.read()
print(response)

Without using any extra lib(like requests), one can still do GET/POST using urllib build-in python3 module. Below is the example code:
def sendSlack(message):
req_param= {"From":"","Time":"","message":message}
slack_data = {
"blocks": [
{
"type": "header",
"text": {
"type": "plain_text",
"text": "Message",
}
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*From:*\n{}".format(req_param['From'])
},
{
"type": "mrkdwn",
"text": "*Time:*\n{}".format(req_param['Time'])
}
]
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*Message:*\n{}".format(req_param['message'])
}
]
}
]}
req = request.Request("https://hooks.slack.com/services/<COMPLETE THE URL>", data=json.dumps(slack_data).encode('utf-8')) # this will make the method "POST"
resp = request.urlopen(req)
print(resp.read())
Make sure to send the right payload, on slack. This code will work like a charm for AWS LAMBDA too. Please see the example below:
import json
from urllib import request
def sendSlack(message):
req_param= {"From":"","StartTime":"now","DialWhomNumber":message}
slack_data = {
"blocks": [
{
"type": "header",
"text": {
"type": "plain_text",
"text": "Message",
}
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*From:*\n{}".format(req_param['From'])
},
{
"type": "mrkdwn",
"text": "*Time:*\n{}".format(req_param['Time'])
}
]
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*Message:*\n{}".format(req_param['message'])
}
]
}
]}
req = request.Request("https://hooks.slack.com/services/<COMPLETE THE URL>", data=json.dumps(slack_data).encode('utf-8')) # this will make the method "POST"
resp = request.urlopen(req)
print(resp.read())
def lambda_handler(event, context):
# TODO implement
try:
print("-->GOT A REQUEST",event['queryStringParameters'])
sendSlack(event['queryStringParameters']['message'])
return {'body':json.dumps({'status':200,'event':'accepted'})}
except Exception as e:
print("Exception happenned",e)
return {
'statusCode': 400,
'body': json.dumps({'status':400,'event':'Something went wrong'})
}

Json add additional payload in the request

I am very new to Python. I run into this problem and hope you can help. Let me explain what I try to do and let me know if I am confusing you.
I have this Python script and it works fine with creating an event.
# Set the request parameters
url = 'https://outlook.office365.com/api/v1.0/me/events?$Select=Start,End'
user = 'user1#domain.com'
pwd = getpass.getpass('Please enter your AD password: ')
# Create JSON payload
data = {
"Subject": "Testing Outlock Event",
"Body": {
"ContentType": "HTML",
"Content": "Test Content"
},
"Start": "2016-05-23T15:00:00.000Z",
"End": "2016-05-23T16:00:00.000Z",
"Attendees": [
{
"EmailAddress": {
"Address": "user1#domain.com",
"Name": "User1"
},
"Type": "Required" },
{
"EmailAddress": {
"Address": "user2#domain.com",
"Name": "User2"
},
"Type": "Optional" }
]
}
json_payload = json.dumps(data)
# Build the HTTP request
opener = urllib2.build_opener(urllib2.HTTPHandler)
request = urllib2.Request(url, data=json_payload)
auth = base64.encodestring('%s:%s' % (user, pwd)).replace('\n', '')
request.add_header('Authorization', 'Basic %s' % auth)
request.add_header('Content-Type', 'application/json')
request.add_header('Accept', 'application/json')
request.get_method = lambda: 'POST'
# Perform the request
result = opener.open(request)
Since other posts suggest to set properties for Json separately (here) for the attachment, so I include the data_attachment code below in addition to the request (see "data_attachment" and "json_payloadAttachment"). However, I am not sure how to add that in the request and make one POST.
# Set the request parameters
url = 'https://outlook.office365.com/api/v1.0/me/events?$Select=Start,End'
user = 'user1#domain.com'
pwd = getpass.getpass('Please enter your AD password: ')
# Create JSON payload
data = {
"Subject": "Testing Outlock Event",
"Body": {
"ContentType": "HTML",
"Content": "Test Content"
},
"Start": "2016-05-23T15:00:00.000Z",
"End": "2016-05-23T16:00:00.000Z",
"Attendees": [
{
"EmailAddress": {
"Address": "user1#domain.com",
"Name": "User1"
},
"Type": "Required" },
{
"EmailAddress": {
"Address": "user2#domain.com",
"Name": "User2"
},
"Type": "Optional" }
]
}
data_attachment = {
"#odata.type": "#Microsoft.OutlookServices.FileAttachment",
"Name": "test123.txt",
"ContentBytes": "VGVzdDEyMw=="
}
json_payload = json.dumps(data)
json_payloadAttachment = json.dumps(data_attachment)
# Build the HTTP request
opener = urllib2.build_opener(urllib2.HTTPHandler)
request = urllib2.Request(url, data=json_payload) # NOT Sure where to put the attachment payload here
auth = base64.encodestring('%s:%s' % (user, pwd)).replace('\n', '')
request.add_header('Authorization', 'Basic %s' % auth)
request.add_header('Content-Type', 'application/json')
request.add_header('Accept', 'application/json')
request.get_method = lambda: 'POST'
# Perform the request
result = opener.open(request)
Please help. Thanks in advance.

It appears that you need to merge the data; for example you can add another key to your data dictionary called Attachments which contains an array of dictionaries and merge them that way; then serialize your data to JSON.
You don't need json_payloadAttachment.
...
data["Attachments"] = [data_attachment]
json_payload = json.dumps(data)
You're also missing the HasAttachments key according to the link you posted.
data["HasAttachments"] = True

How to substitute value for a variable in a json in python?

#!/usr/bin/python
import requests
import uuid
random_uuid = uuid.uuid4()
print random_uuid
url = "http://192.168.54.214:8080/credential-store/domain/_/createCredentials"
payload = '''json={
"": "0",
"credentials": {
"scope": "GLOBAL",
"id": "random_uuid",
"username": "testuser3",
"password": "bar",
"description": "biz",
"$class": "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
}
}'''
headers = {
'content-type': "application/x-www-form-urlencoded",
}
response = requests.request("POST", url, data=payload, headers=headers)
print(response.text)
In the above script, I created a UUID and assigned it to the variable random_uuid. I want the UUID that was created to be substituted inside json for the value random_uuid for the key id. But, the above script is not substituting the value of random_uuid and just using the variable random_uuid itself.
Can anyone please tell me what I'm doing wrong here?
Thanks in advance.

You'll can use string formatting for that.
In your JSON string, replace random_uuid with %s, than do:
payload = payload % random_uuid
Another option is to use json.dumps to create the json:
payload_dict = {
'id': random_uuid,
...
}
payload = json.dumps(payload_dict)

Use str.format instead:
payload = '''json={
"": "0",
"credentials": {
"scope": "GLOBAL",
"id": "{0}",
"username": "testuser3",
"password": "bar",
"description": "biz",
"$class": "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
}
}'''.format(random_uuid)

You could use direct JSON entry into a dict:
payload = {
"": "0",
"credentials": {
"scope": "GLOBAL",
"id": random_uuid,
"username": "testuser3",
"password": "bar",
"description": "biz",
"$class": "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
}
}

This code may help.
#!/usr/bin/python
import requests
import uuid
random_uuid = uuid.uuid4()
print random_uuid
url = "http://192.168.54.214:8080/credential-store/domain/_/createCredentials"
payload = '''json={
"": "0",
"credentials": {
"scope": "GLOBAL",
"id": "%s",
"username": "testuser3",
"password": "bar",
"description": "biz",
"$class": "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
}
}''' % random_uuid
headers = {
'content-type': "application/x-www-form-urlencoded",
}
print payload
print(response.text)