We Keep Coding

Django view doesn't receive Json data?

Using a simple Python script, i want to send a request, with Python-Requests, to a Django view. The Django view should receive the json data inside the request and should print it to my console; here is what i tried:
This is how i send the request:
url = 'http://127.0.0.1:8000/myview/view'
client = requests.session()
csrftoken = requests.get(url).cookies['csrftoken']
data = json.dumps({'data': 'test-value'})
header = {'X-CSRFToken': csrftoken}
cookies = {'csrftoken': csrftoken}
resp = requests.post(url, data=data, headers=header, cookies=cookies)
And this is how the Django view receives it:
def myview(request):
if request.method == 'POST':
data = request.POST.get('data')
print(data)
print('received.')
response = HttpResponse(get_token(request))
return response
The problem with my current code is that print(data) will throw the following output:
None
received.
[06/Jan/2020 21:23:57] "POST /myview/view HTTP/1.1" 200 64
So, instead of printing test-value, it prints nothing. I don't understand whether the error is in my Django view or in how i'm sending the request. Any advice is appreciated!

The problem is with your request, and entirely caused by this line:
data = json.dumps({'data': 'test-value'})
You simply want
data = {'data': 'test-value'}
The POST data should be sent as a simple dictionary, not a JSON string - see the documentation and example here.

Facebook webhook making multiple calls for the same message?

I made and echo bot on AWS with Python and serverless.
I keep getting the same request again and again.
I read the faq where it says u have to deliver a status code 200 else it will keep retrying the webhook.
I'm not sure How I do this.
I have noticed that the sequence number is always the same for the calls so I assume the reply I sent was not acknowledged.
my code is here
import os
import json
import requests
import random
from datetime import datetime
######################
# helper functions
######################
##recursively look/return for an item in dict given key
def find_item(obj, key):
item = None
if key in obj: return obj[key]
for k, v in obj.items():
if isinstance(v,dict):
item = find_item(v, key)
if item is not None:
return item
##recursivley check for items in a dict given key
def keys_exist(obj, keys):
for key in keys:
if find_item(obj, key) is None:
return(False)
return(True)
##send txt via messenger to id
def send_message(send_id, msg_txt):
print("Send message called")
print (datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3])
params = {"statusCode": 200,"access_token": os.environment['accesstoken']}
headers = {"statusCode": "200","Content-Type": "application/json"}
data = json.dumps({"statusCode": "200","recipient": {"id": send_id},
"message": {"text": msg_txt}})
r = requests.post("https://graph.facebook.com/v2.9/me/messages", params=params, headers=headers, data=data)
print (r.text)
if r.status_code != 200:
print(r.status_code)
print(r.text)
#-----------------------------------------------------------
def hello(event, context):
#debug
event=json.loads(json.dumps(event))
print("event:" )
print(event)
# print("context")
# print(context)
#handle webhook challenge
try:
if keys_exist(event, ["queryStringParameters","hub.verify_token","hub.challenge"]):
print("subscribe to webhook invoked")
v_token = str(find_item(event, 'hub.verify_token'))
challenge = find_item(event, 'hub.challenge')
if ("strongtoken" == v_token):
response = {
"statusCode": 200,
"body": str(challenge)
}
print(challenge)
return response
#handle messaging events
if keys_exist(event, ['body']):
event_entry=json.loads(event['body'])
if ((len(event_entry['entry'])>0) & (keys_exist(event_entry['entry'][0],['messaging'])) ):
messaging_event = event_entry['entry'][0]['messaging'][0]
if (keys_exist(messaging_event,['message'])):
msg_txt = messaging_event['message']['text']
sender_id = messaging_event['sender']['id']
print(sender_id)
first_word = msg_txt.split(" ")[0]
send_message(sender_id, msg_txt)
else:
print("Did not send message")
pass
else:
print("Did not send message")
pass
else:
pass
except:
pass
I have given the status code 200 in soo many places and I'm not sure y I still keep getting the same problem.

If you are getting multiple messages your server did not return 200 status code to webhook request from Facebook server. This means an error occurred on your part otherwise 200 should be returned. It looks to me that problems are in following lines:
params = {"statusCode": 200,"access_token": os.environment['accesstoken']}
headers = {"statusCode": "200","Content-Type": "application/json"}
data = json.dumps({"statusCode": "200","recipient": {"id": send_id},
"message": {"text": msg_txt}})
Firstly you are passing statusCode in a data body of message and according to this documentation message response should not contain it.
Another problem could be sending status code inside params. I would remove status code completely from send_message method. I doubt its needed there. You are basically trying to return status 200 on wrong end. You are trying to return it on output instead of input (from Facebooks point of view).
So its quite possible you are correctly getting message from Facebook but you are still returning wrong status code because you are calling send_message method from inside messaging event and send_message method should return status "400 bad request" because you are sending faulty request. Thus your server also returns wrong response code.
Just make sure your code works correctly and 200 should be returned.
EDIT:
So i would try with following code:
params = {"access_token": os.environment['accesstoken']}
headers = {"Content-Type": "application/json"}
data = json.dumps({"recipient": {"id": send_id},
"message": {"text": msg_txt}})

I'm working on a Facebook Messenger chatbot with a Node/Express server and was having the same exact issue. The server was appropriately sending 200 responses and yet the issue persisted. I resolved the problem by caching message ID's and checking for duplicates before processing:
var NodeCache = require('node-cache');
var myCache = new NodeCache();
app.post('/webhook/', function(req, res) {
var messageID = req.body.entry[0].messaging[0].message.mid;
var checkForDupe = myCache.get(messageID);
if (checkForDupe == undefined) {
var setID = myCache.set(req.body.entry[0].messaging[0].message.mid, req.body.entry[0].messaging[0].message.mid);
//Handle message .....
I hope this helps someone. It was driving me nuts.

Your webhook callback should always return a 200 OK HTTP response when invoked by Facebook. Failing to do so may cause your webhook to be unsubscribed by the Messenger Platform. (copied from messenger docs.)
You have to search problem from your returning response. It is not with the sending data.
response = requests.post(POST_MESSAGE_URL, headers={
'Content-Type': 'application/json'
}, params={
'access_token': ACCESS_TOKEN
}, data=json.dumps(DATA_JSON)
this code working fine for me

I am from Java background but still I will try to help you.
In handle webhook challenge I can see you returning 200 status code
But in handle messaging block 200 is not returned You have set 200 status code in send message. But it will not work you have to return 200 from handle messaging block.
Also 200 should be returned even if any exception occurs in this block otherwise on exception it will get blocked. So may be in finally block you can return 200.

As an alternative, you can use chatbotproxy.com which always return 200 in 1ms and pass the exact same request to your endpoint. Since immediate response is very important for Facebook Messenger platform, 1 millisecond time is a very promising feature. If you need help to try, let me know.

requests library and http error

I'm currently using the python requests library to interact with an external api which uses json.
Each endpoint works via a method (of the api class) and uses the collect_data method.
However I want the scraper to continue running whenever it encounters a http error (and ideally output this to a log).
What's the best way to do this as currently it just breaks when I use http.raise_for_status()
It seems like I should be using a try/except in someway but not sure how best to do this here?
def scrape_full_address(self, house_no, postcode):
address_path = '/api/addresses'
address_url = self.api_source + address_path
payload = {
'houseNo': house_no,
'postcode': postcode,
}
return self.collect_data(url=address_url, method='get', payload=payload)
def collect_data(self, url, method, payload=None):
if method == 'get':
data = None
params = payload
elif method == 'post':
params = None
data = payload
response = getattr(requests, method)(url=url, params=params, json=data, headers=self.headers)
if response.status_code == 200:
return response.json()
else:
return response.raise_for_status()

When you call scrape_full_address() elsewhere in your code wrap that in a try statement.
For more info see: https://wiki.python.org/moin/HandlingExceptions
try:
scrape_full_address(659, 31052)
except HTTPError:
print "Oops! That caused an error. Try again..."

HTTP POST using urllib2

IN the code below, I am trying to make a POST of data with urllib2. However, I am getting a HTTP 400 bad request error. Can anyone help me with why this might be the case? The URL is reachable from my computer and all relevant ports are open.
data = {'operation' : 'all'}
results = an.post(an.get_cookie(), 'http://{}:8080/api/v1/data/controller/core/action/switch/update-host-stats'.format(an.TARGET), data)
print results
def post(session_cookie, url, payload):
data = urllib.urlencode(payload)
req = urllib2.Request(url, data)
req.add_header('Cookie','session_cookie=' + session_cookie)
try:
returnedData = urllib2.urlopen(req, data, timeout = 30)
data = json.load(returnedData)
except urllib2.URLError, e:
print e.code
print 'URL ERROR'
return {}
return data

The following code works for me:
import json
import urllib2
import logging
def post_json_request(url, post_data, optional_headers = {}):
"""
HTTP POST to server with json as parameter
#param url: url to post the data to
#param post_data: JSON formatted data
#return: response as raw data
"""
response = ""
try:
req = urllib2.Request(url, post_data, optional_headers)
jsonDump = json.dumps(post_data)
response = urllib2.urlopen(req, jsonDump)
except Exception, e:
logging.fatal("Exception while trying to post data to server - %s", e)
return response
I'm using it in various stubborn platforms that insist to retrieve data on a specific method.
Hope it will help,
Liron

Return a requests.Response object from Flask

I'm trying to build a simple proxy using Flask and requests. The code is as follows:
#app.route('/es/<string:index>/<string:type>/<string:id>',
methods=['GET', 'POST', 'PUT']):
def es(index, type, id):
elasticsearch = find_out_where_elasticsearch_lives()
# also handle some authentication
url = '%s%s%s%s' % (elasticsearch, index, type, id)
esreq = requests.Request(method=request.method, url=url,
headers=request.headers, data=request.data)
resp = requests.Session().send(esreq.prepare())
return resp.text
This works, except that it loses the status code from Elasticsearch. I tried returning resp (a requests.models.Response) directly, but this fails with
TypeError: 'Response' object is not callable
Is there another, simple, way to return a requests.models.Response from Flask?

Ok, found it:
If a tuple is returned the items in the tuple can provide extra information. Such tuples have to be in the form (response, status, headers). The status value will override the status code and headers can be a list or dictionary of additional header values.
(Flask docs.)
So
return (resp.text, resp.status_code, resp.headers.items())
seems to do the trick.

Using text or content property of the Response object will not work if the server returns encoded data (such as content-encoding: gzip) and you return the headers unchanged. This happens because text and content have been decoded, so there will be a mismatch between the header-reported encoding and the actual encoding.
According to the documentation:
In the rare case that you’d like to get the raw socket response from the server, you can access r.raw. If you want to do this, make sure you set stream=True in your initial request.
and
Response.raw is a raw stream of bytes – it does not transform the response content.
So, the following works for gzipped data too:
esreq = requests.Request(method=request.method, url=url,
headers=request.headers, data=request.data)
resp = requests.Session().send(esreq.prepare(), stream=True)
return resp.raw.read(), resp.status_code, resp.headers.items()
If you use a shortcut method such as get, it's just:
resp = requests.get(url, stream=True)
return resp.raw.read(), resp.status_code, resp.headers.items()

Flask can return an object of type flask.wrappers.Response.
You can create one of these from your requests.models.Response object r like this:
from flask import Response
return Response(
response=r.reason,
status=r.status_code,
headers=dict(r.headers)
)

I ran into the same scenario, except that in my case my requests.models.Response contained an attachment. This is how I got it to work:
return send_file(BytesIO(result.content), mimetype=result.headers['Content-Type'], as_attachment=True)

My use case is to call another API in my own Flask API. I'm just propagating unsuccessful requests.get calls through my Flask response. Here's my successful approach:
headers = {
'Authorization': 'Bearer Muh Token'
}
try:
response = requests.get(
'{domain}/users/{id}'\
.format(domain=USERS_API_URL, id=hit['id']),
headers=headers)
response.raise_for_status()
except HTTPError as err:
logging.error(err)
flask.abort(flask.Response(response=response.content, status=response.status_code, headers=response.headers.items()))