We Keep Coding

How to read S3 object response body

I uploaded a file/csv to my Pivotal Cloud Foundry (PCF) ECS S3 storage. Which was created successfully.
import requests
url = "https://host/bucket/test.csv"
payload = {}
files = [
('file-name', open('test.csv','rb'))
]
headers = {
'X-Amz-Date': '20200622T112852Z',
'Authorization': 'AWS4-HMAC-SHA256 Credential=cred_endpoint, SignedHeaders=host;x-amz-date;x-amz-target, Signature=sigV4'
}
response = requests.put(url, headers=headers, data=payload, files=files)
When I try to access it I get a response like this
GET https://host/bucket/test.csv Status Code - 200
----------------------------017095580637364775189995
Content-Disposition: form-data; name="file-name"; filename="test.csv"
Content-Type: text/csv
rank,name
1,Michael
2,Jim
3,Dwight
----------------------------017095580637364775189995--
Which is the correct data but it's in this weird format with boundaries in the body, what would be the most robust way to read this kind of response body and generate the exact same or similar file/csv as the one upload?
Also am I even doing this correctly? Because all the docs imply there should be a [181(whatever the file size) bytes of object data] in the body instead of the actual content of the file.

Slack post payload in wrong format

How to get payload printed in json format
Getting payload like %7B%22type%22%3A%22interactive_message%22%2C%22actions%22%3A%5B%7B%22
2020-02-17 14:09:22,624 200 POST /slack/message_actions HTTP/1.1 Host: myipteventsbot.tunnel.com X-Real-IP: 3.93.14.114 Content-Length: 5159 User-Agent: Slackbot 1.0 (+https://api.slack.com/robots) Accept-Encoding: gzip,deflate Accept: application/json,*/*
X-Slack-Signature: v0=96ab9dd1e89047f6cea117313a17989eb25849f7a86ad1f1eff8a0be692238cf X-Slack-Request-Timestamp: 1581970162 Content-Type: application/x-www-form-urlencoded
payload=%7B%22type%22%3A%22interactive_message%22%2C%22actions%22%3A%5B%7B%22name%22%3A%22fullfilment_list%22%2C%22type%22%3A%22select%22%2C%22selected_options%22%3A%5B%7B%22value%22%3A%22spu%22%7D%5D%7D%5D%2C%22callback_id%22%3A%22test%22%2C%22team%22%3A%7B%22id%22%3A%22TFHGD33RV%22%2C%22domain%22%3A%22bby-sandbox-general%22%2C%22enterprise_id%22%3A%22EFGBDRLNP%22%2C%22enterprise_name%22%3A%22Best+Buy+Test%22%7D%2C%22channel%22%3A%7B%22id%22%3A%22DTHBLNXD0%22%2C%22name%22%3A%22directmessage%22%7D%2C%22user%22%3A%7B%22id%22%3A%22WLCLZ2LDP%22%2C%22name%22%3A%22a6002043%22%2C%
My code snippet
#app.route("/slack/message_actions", methods=["POST"])
def message_actions():
# Parse the request payload
form_json = json.loads(request.form["payload"])
# Verify that the request came from Slack
verify_slack_token(form_json["token"])
# Check to see what the user's selection was and update the message accordingly
selection = form_json["actions"][0]["selected_options"][0]["value"]
if selection == "spu":
message_text = "spu"
else:
message_text = "sth"
response = slack_client.api_call(
"chat.update",
channel=form_json["channel"]["id"],
ts=form_json["message_ts"],
text="One {} order coming up! :coffee:".format(message_text),
attachments=[] # empty `attachments` to clear the existing massage attachments
)
# Send an HTTP 200 response with empty body so Slack knows we're done here
return make_response("", 200)

figured out i need update my code as below
request.form.get('payload')

HTTP post Json 400 Error

I am trying to post data to my server from my microcontroller. I need to send raw http data from my controller and this is what I am sending below:
POST /postpage HTTP/1.1
Host: https://example.com
Accept: */*
Content-Length: 18
Content-Type: application/json
{"cage":"abcdefg"}
My server requires JSON encoding and not form encoded request.
For the above request sent, I get an 400 error from the server, HTTP/1.1 400 Bad Request
However, when I try to reach the post to my server via a python script via my laptop, I am able to get a proper response.
import requests
url='https://example.com'
mycode = 'abcdefg'
def enter():
value = requests.post('url/postpage',
params={'cage': mycode})
print vars(value)
enter()
Can anyone please let me know where I could be going wrong in the raw http data I'm sending above ?

HTTP specifies the separator between headers as a single newline, and requires a double newline before the content:
POST /postpage HTTP/1.1
Host: https://example.com
Accept: */*
Content-Length: 18
Content-Type: application/json
{"cage":"abcdefg"}
If you don’t think you’ve got all of the request right, try seeing what was sent by Python:
response = ...
request = response.request # request is a PreparedRequest.
headers = request.headers
url = request.url
Read the docs for PreparedRequest for more information.
To pass a parameter, use this Python:
REQUEST = 'POST /postpage%s HTTP/1.1\r\nHost: example.com\r\nContent-Length: 0\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.4.3 CPython/2.7.9 Linux/4.4.11-v7+\r\n\r\n';
query = ''
for k, v in params.items():
query += '&' + k + '=' + v # URL-encode here if you want.
if len(query): query = '?' + query[1:]
return REQUEST % query

HTTP response body is obfuscated

I'm using httplib.HTTPConnection to submit an HTTP POST request. I get a 200 response status but the response data looks obfuscated or something.
When I submit the request in Firefox, the response is displayed fine.
conn = httplib.HTTPConnection("www.foo.com")
conn.request('POST', '/foo', postdata, headers)
resp = conn.getresponse()
conn.close()
print resp.read()

If the response is unexpectedly 'binary', look at the Content-Encoding header. Most likely you are being served a compressed response; it can be either gzip or deflate.
If you the encoding is gzip, decode it with:
import zlib
decompressor = zlib.decompressobj(16 + zlib.MAX_WBITS)
data = decompressor.decompress(response_body)
For deflate, you'd have to try both a default compressor and one with -zlib.MAX_WBITS:
try:
decompressor = zlib.decompressobj()
data = decompressor.decompress(response_body)
except zlib.error:
decompressor = zlib.decompressobj(-zlib.MAX_WBITS)
data = decompressor.decompress(response_body)

In addition to the other answer, you could probably disable encoding by setting the Accept-Encoding header to identity.
headers = {
# ...
"Accept-Encoding": "identity",
# ...
}

Office 365 REST API (Python) Mark Email as Read

I'm sure I'm doing something simple wrong, but I can't for the life of me figure out how to set the "IsRead" property to true. It's the last step of my process that gets a filtered list of messagesa and stores and processes any attachments.
According to the docs "IsRead" is writable: http://msdn.microsoft.com/office%5Coffice365%5CAPi/complex-types-for-mail-contacts-calendar#ResourcesMessage
http://msdn.microsoft.com/office%5Coffice365%5CAPi/mail-rest-operations#MessageoperationsUpdatemessages
I'm using python 2.7 and the requests module:
# once file acquired mark the email as read
params = {'IsRead':'True'}
base_email_url = u'https://outlook.office365.com/api/v1.0/me/messages/{0}'.format( msgId )
response = requests.patch(base_email_url, params, auth=(email,pwd))
log.debug( response )
The response that comes back is this:
{"error":{"code":"ErrorInvalidRequest","message":"Cannot read the request body."}}
What's the problem with my request?

At first glance it looks OK. I wonder if the Content-Type header isn't being set to "application/json" or something along those lines. Try getting a network trace and verify that the request looks something like:
PATCH https://outlook.office365.com/api/v1.0/Me/Messages('msgid') HTTP/1.1
Accept: application/json;odata.metadata=full
Authorization: Bearer <token>
Content-Type: application/json;odata.metadata=full
Host: outlook.office365.com
Content-Length: 24
Expect: 100-continue
Connection: Keep-Alive
{
"IsRead": "true"
}

Well I have an answer for myself and it is indeed a simple matter.
It was a mistake to not fully read how PATCH is different from GET or POST.
In short it's important to make sure your headers are set for the right content-type.
Here is the working code:
# once file acquired mark the email as read
changes = {u'IsRead':u'True'}
headers = {'Content-Type': 'application/json'}
json_changes = json.dumps(changes)
base_email_url = u'https://outlook.office365.com/api/v1.0/me/messages/{0}'.format( msgId )
response = requests.patch(base_email_url, data=json_changes, auth=__AUTH, headers=headers)
log.debug( response )