I am using AWS SQS to store information coming in from an external server and then sending it to a Lambda function to process it and dequeue the information.
The information that I am sending in is in the form of a JSON and is being used as a python dictionary.
def lambda_handler(event, context):
for record in event['Records']:
messageHandler(record)
return {
'statusCode': 200,
'body': json.dumps('Batch Processed')
}
Assuming that the code for the messageHandler is working and properly implemented, how do I catch the messages from the queue in their batches. This is all being deployed by AWS Chalice without the use of CLI.
I am well out of my depth right now and have no idea why this is not working when I deploy it but is working when I trigger a normal Lambda Function in the AWS Console through the SQS Send/Recieve Message feature. As far as I know the triggers are set up correctly and they should have no issue.
If you have any questions please let me know.
The event that you are processing will look something like this:
{
"Records": [
{
"messageId": "11d6ee51-4cc7-4302-9e22-7cd8afdaadf5",
"receiptHandle": "AQEBBX8nesZEXmkhsmZeyIE8iQAMig7qw...",
"body": "Test message.",
"attributes": {
"ApproximateReceiveCount": "1",
"SentTimestamp": "1573251510774",
"SequenceNumber": "18849496460467696128",
"MessageGroupId": "1",
"SenderId": "AIDAIO23YVJENQZJOL4VO",
"MessageDeduplicationId": "1",
"ApproximateFirstReceiveTimestamp": "1573251510774"
},
"messageAttributes": {},
"md5OfBody": "e4e68fb7bd0e697a0ae8f1bb342846b3",
"eventSource": "aws:sqs",
"eventSourceARN": "arn:aws:sqs:us-east-2:123456789012:fifo.fifo",
"awsRegion": "us-east-2"
}
]
}
where the "body" is your json encoded message. You'll want your message handler function to do something like this:
def message_handler(event):
message = json.loads(event["body"])
# do stuff...
The return value from the lambda is pretty pointless if it is being used as the event target from sqs.
I am going to use Firebase Auth and Database modules to create my web app. However, not all things that I want my app to do is possible to achieve on only front end. So I want to also use backend with Python's Bottle framework to handle requests and Pyrebase to get access to Firebase Database.
Let's say that after logging in I need to go to mainpage and see personalized content, for example my notes. They are structured this way in DB:
{
"notes": [{
"id": "1",
"title": "X",
"author": "user1"
},
{
"id": "2",
"title": "Y",
"author": "user2"
} and so on... ]
}
So how it's possible to implement showing only my articles on main page?
I understand that I need to filter my notes based on author value, but how to let Bottle understand who is currently logged in?
I've read there, that I should somehow send unique token to backend server to authenticate current user, but how to do that? Inserting Token in every link as GET parameter seems to be silly, but I see no other way to implement that.
Start by organizing your database so that each note becomes a child object:
{
"notes": {
"id1": {
"id": "id1",
"title": "X",
"author": "user1",
},
"id2": {
}
}
}
Then this particular interaction can be implemented entirely in the client-side. Just execute a query to filter the notes you want. For example in a JS client:
var uid = firebase.auth().currentUser.uid;
var query = ref.orderByChild('author').equalTo(uid);
// Listen for query value events
If you want to run this on a backend server, and you want to ensure that only logged in users are allowed to execute it, then you must pass the ID token from the client app to the server on each request. Here's how to implement the server-side logic using the Python Admin SDK:
import firebase_admin
from firebase_admin import auth
from firebase_admin import db
token = '....' # Extract from the client request
try:
decoded = auth.verify_id_token(token)
uid = decoded.uid
ref = db.reference('path/to/notes')
notes = ref.order_by_child('author').equal_to(uid).get()
# Process notes response
except ValueError as ex:
print(ex)
# Send error to client
I would like to constantly send data using Django WebSockets. At this moment I try in the way shown below.
routing.py:
channel_routing = [
route("websocket.connect", ws_connect),
consumers.py:
def ws_connect(message):
message.reply_channel.send({"accept": True})
while True:
message.reply_channel.send({
"text": json.dumps({
'message': 'My response2'
})
})
Unfortunately it doesn't work properly. Obviously without while True it send data but only once. Any ideas how can I solve it?
Hi i am copying parts of the github project multichat from the creator of django channels.
I am making slight changes to the code like not using jquery, renaming of some consumers and such.
I have literally no errors when running the code however when i join the page and the JS creates a websocket it says simply
[2017/08/03 13:13:48] WebSocket HANDSHAKING /chat/stream [127.0.0.1:37070]
[2017/08/03 13:13:48] WebSocket CONNECT /chat/stream [127.0.0.1:37070]
Which one would think is fine ofcourse... However i'n my connect function i have a print("********CONNECTED**********"), wich is nowhere to be seen in the console. It simply doesn't run the function i have told it to when someone connects but it still says the person connected and it throws no errors.
This is the main routing:
channel_routing = [
include("crypto_chat.routing.websocket_routing", path=r"^/chat-stream/$"),
include("crypto_chat.routing.chat_routing"),
]
Routing from app:
websocket_routing = [
route("websocket.connect", ws_connect),
route("websocket.receive", ws_receive),
route("websocket.disconnect", ws_disconnect),
]
chat_routing = [
route("chat.receive", chat_send, command="^send$"),
route("chat.receive", user_online, command="^online$"),
Connect Consumer:
#channel_session_user_from_http
def ws_connect(message):
# only accept connection if you have any rooms to join
print("******************CONNECT*************************''")
message.reply_channel.send({"accept": True})
# init rooms - add user to the groups and pk num to the session
message.channel_session['rooms'] = []
for room in Room.objects.get(users=message.user):
room.websocket_group.add(message.reply_channel)
message.channel_session['rooms'].append(room.pk)
print(message.channel_session['rooms'])
Heres JS (note: i am using the JS extension that is available on the project website also):
function send_msg(){
var msg=document.getElementById('msg_input').value;
console.log("sending msg" + msg);
webSocketBridge.send({
"command": "send",
"room": "1",
"message": msg
});
}
// logging
var ws_path = "/chat/stream";
console.log("connecting to " + ws_path);
// connect
var webSocketBridge = new channels.WebSocketBridge();
webSocketBridge.connect(ws_path);
// listen loop
webSocketBridge.listen(function(data)
{
// read json file and act accordingly
if(data.error){
// post error message in chat
console.log("Error - " + data.error);
return;
}
// handle if the user comes back online
if(data.online){
console.log("User is online");
}
else if(data.offline){
console.log("User offline");
}
else if(data.message){
console.log("Got message");
}
else{ console.log("Unknown message type"); }
});
// Helpful debugging
webSocketBridge.socket.onopen = function () {
console.log("Connected to chat socket");
};
webSocketBridge.socket.onclose = function () {
console.log("Disconnected from chat socket");
}
Websocket paths should match on server and client side. On server side, you have /chat-stream/ and on client side /chat/stream. These should match. Also, make sure you don't forget the trailing slash as django explicitly requires it.
I'm trying to get the FB messenger API working using Python's Flask, adapting the following instructions: https://developers.facebook.com/docs/messenger-platform/quickstart
So far, things have been going pretty well. I have verified my callback and am able to receive the messages I send using Messenger on my page, as in the logs in my heroku server indicate the appropriate packets of data are being received by my server. Right now I'm struggling a bit to send responses to the client messenging my app. In particular, I am not sure how to perform the following segment from the tutorial in Flask:
var token = "<page_access_token>";
function sendTextMessage(sender, text) {
messageData = {
text:text
}
request({
url: 'https://graph.facebook.com/v2.6/me/messages',
qs: {access_token:token},
method: 'POST',
json: {
recipient: {id:sender},
message: messageData,
}
}, function(error, response, body) {
if (error) {
console.log('Error sending message: ', error);
} else if (response.body.error) {
console.log('Error: ', response.body.error);
}
});
}
So far, I have this bit in my server-side Flask module:
#app.route('/', methods=["GET", "POST"])
def chatbot_response():
data = json.loads(req_data)
sender_id = data["entry"][0]["messaging"][0]["sender"]["id"]
url = "https://graph.facebook.com/v2.6/me/messages"
qs_value = {"access_token": TOKEN_OMITTED}
json_response = {"recipient": {"id": sender_id}, "message": "this is a test response message"}
response = ("my response text", 200, {"url": url, "qs": qs_value, "method": "POST", "json": json_response})
return response
However, running this, I find that while I can process what someone send my Page, it does not send a response back (i.e. nothing shows up in the messenger chat box). I'm new to Flask so any help would be greatly appreciated in doing the equivalent of the Javascript bit above in Flask.
Thanks!
This is the code that works for me:
data = json.loads(request.data)['entry'][0]['messaging']
for m in data:
resp_id = m['sender']['id']
resp_mess = {
'recipient': {
'id': resp_id,
},
'message': {
'text': m['message']['text'],
}
}
fb_response = requests.post(FB_MESSAGES_ENDPOINT,
params={"access_token": FB_TOKEN},
data=json.dumps(resp_mess),
headers = {'content-type': 'application/json'})
key differences:
message needs a text key for the actual response message, and you need to add the application/json content-type header.
Without the content-type header you get the The parameter recipient is required error response, and without the text key under message you get the param message must be non-empty error response.
This is the Flask example using fbmq library that works for me:
echo example :
from flask import Flask, request
from fbmq import Page
page = fbmq.Page(PAGE_ACCESS_TOKEN)
#app.route('/webhook', methods=['POST'])
def webhook():
page.handle_webhook(request.get_data(as_text=True))
return "ok"
#page.handle_message
def message_handler(event):
page.send(event.sender_id, event.message_text)
In that scenario in your tutorial, the node.js application is sending an HTTP POST request back to Facebook's servers, which then forwards the content on to the client.
So far, sounds like your Flask app is only receiving (AKA serving) HTTP requests. The reason is that that's what the Flask library is all about, and it's the only thing that Flask does.
To send an HTTP request back to Facebook, you can use any Python HTTP client library you like. There is one called urllib in the standard library, but it's a bit clunky to use... try the Requests library.
Since your request handler is delegating to an outgoing HTTP call, you need to look at the response to this sub-request also, to make sure everything went as planned.
Your handler may end up looking something like
import json
import os
from flask import app, request
# confusingly similar name, keep these straight in your head
import requests
FB_MESSAGES_ENDPOINT = "https://graph.facebook.com/v2.6/me/messages"
# good practice: don't keep secrets in files, one day you'll accidentally
# commit it and push it to github and then you'll be sad. in bash:
# $ export FB_ACCESS_TOKEN=my-secret-fb-token
FB_TOKEN = os.environ['FB_ACCESS_TOKEN']
#app.route('/', method="POST")
def chatbot_response():
data = request.json() # flasks's request object
sender_id = data["entry"][0]["messaging"][0]["sender"]["id"]
send_back_to_fb = {
"recipient": {
"id": sender_id,
},
"message": "this is a test response message"
}
# the big change: use another library to send an HTTP request back to FB
fb_response = requests.post(FB_MESSAGES_ENDPOINT,
params={"access_token": FB_TOKEN},
data=json.dumps(send_back_to_fb))
# handle the response to the subrequest you made
if not fb_response.ok:
# log some useful info for yourself, for debugging
print 'jeepers. %s: %s' % (fb_response.status_code, fb_response.text)
# always return 200 to Facebook's original POST request so they know you
# handled their request
return "OK", 200
When doing responses in Flask, you have to be careful. Simply doing a return statement won't return anything to the requester.
In your case, you might want to look at jsonify(). It will take a Python dictionary and return it to your browser as a JSON object.
from flask import jsonify
return jsonify({"url": url, "qs": qs_value, "method": "POST", "json": json_response})
If you want more control over the responses, like setting codes, take a look at make_response()