We Keep Coding

How to add a message object to API with FLASK

I have a task to create a REST API that will be responsible for handling messages. As a part of the functionality, I should be able to write a message. This message should contain the following fields:
id
sender
receiver
message itself
subject
creation date
So, as I expected to do this is to have a route that should handle the object that I will send as an argument. But I am not sure I can do so. What would you recommend in this case?
For now, I can handle it somehow like this:
#app.route('/new_message/<string:sender>/<string:receiver>/<string:message>/', methods=['POST'])
def get_message(sender, receiver, message):
sender = sender
receiver = receiver
message = message
# Code that will add the data or to the database or to the json file
# if I decide not to implement DB for this task
return 'Sent successfully'
Thanks for your advice!

I am suggesting you to use JSON request body instead of the path parameters for the POST method.
Here is the example,
from flask import request, Flask
app = Flask(__name__)
#app.route('/new_message', methods=['POST'])
def get_message():
payload = request.json()
sender = payload['sender']
receiver = payload['receiver']
message = payload['message']
return 'Sent successfully'
if __name__ == "__main__":
app.run()
Now, If you want to add message as object then you can add it in JSON body. Not only message object you can add any number of fields if required.

Problem to pass `http_conn_id` to SlackWebhookOperator

I try create a SlackWebhookOperator, and using may HTTP connection, but he still traing use the http_default.
failed_alert = SlackWebhookOperator(
task_id='slack_test',
http_conn_id='slack_conn',
webhook_token=slack_webhook_token,
message=slack_msg,
username='airflow')
failed_alert.execute(context=context)
[2019-07-21 13:14:57,415] {{init.py:1625}} ERROR - Failed at executing callback
[2019-07-21 13:14:57,415] {{init.py:1626}} ERROR - The conn_id http_default isn't defined

I think its a known issue with 1.10.3: https://github.com/apache/airflow/pull/5066
My workaround is this:
def task_fail_slack_alert_hook(url, context):
""" This is a webhook utility which will push an error message to a given slack channel using a URL """
slack_msg = """
:red_circle: Task Failed.
*Task*: {task}
*Dag*: {dag}
*Execution Time*: {exec_date}
*Log Url*: {log_url}
<!channel>
""".format(
task=context.get("task_instance").task_id,
dag=context.get("task_instance").dag_id,
ti=context.get("task_instance"),
exec_date=context.get("execution_date"),
log_url=context.get("task_instance").log_url,
)
slack_data = {"text": slack_msg}
return requests.post(
url,
data=json.dumps(slack_data),
headers={"Content-Type": "application/json"},
)
You will have to put the whole webhook URL in the host though, rather than splitting host and password up.
You could also have a look at the slack client instead

Waiting for websocket.receive consumer in django-channels

How to wait for a response from client after sending the client something using django-channels?
Whenever Group.send() is called from function send_to_client() and upon receiving the message by client, send_to_client() is expecting a response back from client which is getting received on websocket.receive channel.
Is there a way to return the response_msg in send_to_client() function?
Now I have reached here
Sample Code for consumers.py:
def ws_receive(message):
response_msg = message.content['text']
return response_msg
def send_to_client(param1, param2, param3):
Group.send({
"text" : json.dumps({
"First" : param1,
"Second" : param2,
})
})
So once the message reaches at the client side, the client will send a response back to the server which will be received by the ws_receive(message) function through the websocket.receive channel which is defined in the urls.py file,
channel_patterns = [
route("websocket.receive", ws_receive),
...
]
Is there a way to do this so that my function would look like this?
def send_to_client(...):
Group.send(...)
response_msg = #response message from client

Since you are recieving via a websocket, I am not sure if you would be able to even tell if the recieving thing is directly as a response for your request. I would rather put an id variable or something in the ongoing request, and maybe ask the client to put that id in the response as well. That might require both the sender and reciever to know the value of id as well (probably store in the db?)
Also, it do not seem logical to be blocked waiting for the response from websocket as well.

Run particular Django script in background

I am developing a Gmail extracting app and using Gmail API to fetch mail from server. the problem lies in the fact that fetch time for mails is too large even though I used threading in back end framework. now I am going to implement one feature which will suggest user opting for bulk download that "once your download is ready, we will mail you" but for that i want to run download.py mentioned below in app tree in background and once the fetch is over it will get terminated.
And in the very bottom of the code i want to mail user that their download is ready but its not working though i have defined the mail server in settings.py .
download.py
import httplib2, base64
from stripogram import html2text
from oauth2client.django_orm import Storage
from apiclient.discovery import build
from oauth2client import client
from django.contrib.auth.models import User
from .models import CredentialsModel
from django.conf import settings
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import authentication, permissions
from gextracto import models
from gextracto.models import UserData
from django.core.mail import EmailMessage
from django.core import mail
connection = mail.get_connection()
class ListMails(APIView):
"""
Gets a list of a specified number mail ids for a particular label
Extracts the email in the form of plain/text
The API returns all the extracted mails
"""
authentication_classes = (authentication.SessionAuthentication,)
permission_classes = (permissions.IsAuthenticated,)
def extract_headers(self, message):
"""
Extract the headers for a single mail and returns it
{To, From, Subject}
"""
needed_fields = ('From', 'To', 'Subject')
return {i['name']:i['value'] for i in message['payload']['headers'] if i['name'] in needed_fields}
def get_message_body(self, message):
"""
Get the body of an email
Recursively look for the body for different mimetypes
Returns the body as text/plain
"""
if 'payload' in message:
return self.get_message_body(message['payload'])
elif 'parts' in message:
return self.get_message_body(message['parts'][0])
else:
data = base64.urlsafe_b64decode(message['body']['data'].encode('ASCII'))
markdown_data = html2text(data)#.decode('utf-8', "replace")
data = data.replace("\n", "<br/>")
# return {markdown, html}
return {'markdown':unicode( markdown_data,"ISO-8859-1"), 'html':unicode(data,"ISO-8859-1")} if markdown_data else {'html':unicode(data,"ISO-8859-1")}
def message_content_html(self, userId, message_id, service):
"""
Make queries to get the content for a mail given its message id
Returns all the content
"""
content = {'id':message_id}
# try
message = service.users().messages().get(userId=userId, id=message_id).execute()
mimetype = message['payload']['mimeType']
if mimetype == 'text/html':
return {}
#
else:
body = self.get_message_body(message)
if body == "":
body = "<empty message>"
headers = self.extract_headers(message)
content['body'] = body
content.update(headers)
return content
def collect_mails(self, user, messages, service):
"""
Collect the content for all the mails currently downloaded
"""
all_messages = []
try:
for message in messages:
content = self.message_content_html(user.username, message['id'], service)
if content:
all_messages.append(content)
return all_messages
# return empty list if no messages were downloaded
except KeyError:
return []
def get(self, request, format=None):
"""
Handles the GET request to get all the mails for a label
Paginages through the GAPI content if required
API returns all the messages
{To, From, Subject, body}
"""
user = request.user
storage = Storage(CredentialsModel, 'id', user, 'credential')
credentials = storage.get()
http_auth = credentials.authorize(httplib2.Http())
service = build('gmail', 'v1', http=http_auth)
user_Id = user.username
label_id = request.GET['label']
# try
# call Google API with a request to get a list of all the labels
response = service.users().messages().list(userId=user_Id, labelIds=label_id, maxResults=100).execute()
all_messages = self.collect_mails(user, response['messages'], service)
if not all_messages:
return Response([])
else:
if 'nextPageToken' in response:
page_token_flag = True
# request more more mails if the download limit has not yet been satisfied
while(page_token_flag):
response = service.users().messages().list(userId=user_Id, pageToken=response['nextPageToken'], maxResults=100).execute()
all_messages.append(self.collect_mails(user, response['messages'], service))
print(all_messages)
#for x in range(0,len(all_messages)):
#b=all_messages[10]
#instance= UserData(user_id=user ,label=label_id, sender = b['From'] , subject=b['Subject'] , body=b['body'])
#instance.save()
page_token_flag = 'nextPageToken' in response
##
for x in range(0,len(all_messages)):
b=all_messages[10]
instance= UserData(user_id=user ,label=label_id, sender = b['From'] , subject=b['Subject'] , body=b['body'])
instance.save()
print ("Hi i am here!!!")
email = EmailMessage('Your Download Ready!', 'http://127.0.0.1:8000/admin/gextracto/userdata/', to=[user], connection=connection)
email.send()
connection.close()
return Response(all_messages)
Please tell me the way to run it in background. if need any other info please do ask. Thanks

Don't know the exact requirements but I'll think about Celery to run background tasks. This approach allows to manage all post-script activities in native Django manner.
Also you can think about running the Django script using cron (as manage.py command) - but it can lead to some limitations.
What about sending emails failure - believe, you don't need to close connection after sending email. Usually I use send_mail()/send_mass_mail() functions - please, check their code to get an idea.

How to process JSON in Flask?

I have asked a few questions about this before, but still haven't solved my problem.
I am trying to allow Salesforce to remotely send commands to a Raspberry Pi via JSON (REST API). The Raspberry Pi controls the power of some RF Plugs via an RF Transmitter called a TellStick. This is all setup, and I can use Python to send these commands. All I need to do now is make the Pi accept JSON, then work out how to send the commands from Salesforce.
Someone kindly forked my repo on GitHub, and provided me with some code which should make it work. But unfortunately it still isn't working.
Here is the previous question: How to accept a JSON POST?
And here is the forked repo: https://github.com/bfagundez/RemotePiControl/blob/master/power.py
What do I need to do? I have sent test JSON messages n the Postman extension and in cURL but keep getting errors.
I just want to be able to send various variables, and let the script work the rest out.
I can currently post to a .py script I have with some URL variables, so /python.py?power=on&device=1&time=10&pass=whatever and it figures it out. Surely there's a simple way to send this in JSON?
Here is the power.py code:
# add flask here
from flask import Flask
app = Flask(__name__)
app.debug = True
# keep your code
import time
import cgi
from tellcore.telldus import TelldusCore
core = TelldusCore()
devices = core.devices()
# define a "power ON api endpoint"
#app.route("/API/v1.0/power-on/<deviceId>",methods=['POST'])
def powerOnDevice(deviceId):
payload = {}
#get the device by id somehow
device = devices[deviceId]
# get some extra parameters
# let's say how long to stay on
params = request.get_json()
try:
device.turn_on()
payload['success'] = True
return payload
except:
payload['success'] = False
# add an exception description here
return payload
# define a "power OFF api endpoint"
#app.route("/API/v1.0/power-off/<deviceId>",methods=['POST'])
def powerOffDevice(deviceId):
payload = {}
#get the device by id somehow
device = devices[deviceId]
try:
device.turn_off()
payload['success'] = True
return payload
except:
payload['success'] = False
# add an exception description here
return payload
app.run()

Your deviceID variable is a string, not an integer; it contains a '1' digit, but that's not yet an integer.
You can either convert it explicitly:
device = devices[int(deviceId)]
or tell Flask you wanted an integer parameter in the route:
#app.route("/API/v1.0/power-on/<int:deviceId>", methods=['POST'])
def powerOnDevice(deviceId):
where the int: part is a URL route converter.
Your views should return a response object, a string or a tuple instead of a dictionary (as you do now), see About Responses. If you wanted to return JSON, use the flask.json.jsonify() function:
# define a "power ON api endpoint"
#app.route("/API/v1.0/power-on/<int:deviceId>", methods=['POST'])
def powerOnDevice(deviceId):
device = devices[deviceId]
# get some extra parameters
# let's say how long to stay on
params = request.get_json()
try:
device.turn_on()
return jsonify(success=True)
except SomeSpecificException as exc:
return jsonify(success=False, exception=str(exc))
where I also altered the exception handler to handle a specific exception only; try to avoid Pokemon exception handling; do not try to catch them all!

To retrieve the Json Post values you must use request.json
if request.json and 'email' in request.json:
request.json['email']