We Keep Coding

Is there a way to decode the string returned from AWS lambda to meta's whatsapp business api?

I am trying to setup a webhook in AWS Lambda (using API Gateway) for Meta's WhatsApp Business API. They have the following guidelines:
Whenever your endpoint receives a verification request, it must:
Verify that the hub.verify_token value matches the string you set in the Verify Token field when you configure Webhooks in your App Dashboard (you haven't set up this token string yet). Respond with the hub.challenge value."
I have setup all the query strings it needs in the API gateway. Here is what my lambda handler looks like:
def lambda_handler(event, context):
response = {
"status": 400,
"body" : "failed"
}
print(str(event))
print("Received context" + str(context))
if(len(event) != 0 and (event['hub.verify_token'] == "some value")):
response['status'] = 200
response['body'] = event['hub.challenge']
return event['hub.challenge']
#print(response)
#returnResponse = re.findall('[0-9]+', event['hub.challenge'])
#return returnResponse[0]
else:
return(response)
the event looks like:
{
"hub.mode" : "some value",
"hub.verify_token": "some value",
"hub.challenge": "1158201444"
}
The response in AWS console looks like "1158201444" but on meta's end, the response looks like "\"1158201444\"" and the webhook confirmation fails in Meta's dashboard.
How can remove the extra characters and decode the string? I have already tried regex and still getting the extra characters (\"\").

So what worked for me is was that I passed hub.challenge as a integer format instead of string. I just converted it to an integer before returning it.

AWS lambda depending on event source, push the payload to different buckets/deliverystreams

My Lambda function ge[enter image description here][1]t's event form our backend solution. Right now it sends all the event.body to one delivery-stream which sends the data to a bucket. I would like to have different prefixes to my objects, so that i can create tables with a crawler for the same event-type data. When i crawl the data that comes in right now the schema of the table looks like this.
And when i open the payload struct it looks liek this inside
[enter image description here][2]
So in general i would like to only put the payload to the delivery-stream.
I was thinking of referring to the type columnname as this gives me information, where the data has been originally created.
Now to my question.
How can i refer to it in my lambda code and based on the value appearing in the type columnname put it to a specific delivery-stream
So I need to implement a if, elif structure and only put the payload to the delivery stream form the incoming data
import json
import boto3
from datetime import datetime
client = boto3.client('firehose')
def handler(event):
print(json.dumps(event, indent=4))
try:
response = client.put_record(
DeliveryStreamName='string',
Record={
'Data': event.body
}
)
except Exception as e:
return {
'statusCode': 400,
'body': json.dumps(f'Cannot process event: {e}')
}
return {
'statusCode': 200,
'body': json.dumps({ 'ack': datetime.now().strftime('%H:%M:%S') })
}
so instead of
'Data': event.body
i need toput the payload only, but before puting the payload i need to specify to which bucket.
I don't know how to or where to add the if, elif structure and how to refer to the type column which is in the event body
[1]: https://i.stack.imgur.com/8DKyw.png
[2]: https://i.stack.imgur.com/TAjFM.png

AioHttp: How can I create a data form for an aiohttp post

This is what the data looks like when I'm using requests and it works fine.
data ={
"srt": srt,
"firstname" : firstname,
"lastname" : lastname,
"Email" : email,
"password" : password,
"promotion" : "true",
"action" : {"name":"EMAIL_REG_FORM_SUBMIT"},
"ri" : "NORU",
"ets" : ets
}
I'm trying to convert it so that it works with aiohttp and this is what I have. I think I'm getting an error because of line: "action" : {"name":"EMAIL_REG_FORM_SUBMIT"},
data = aiohttp.FormData()
data.add_field("srt", srt)
data.add_field("firstname", firstname)
data.add_field("lastname", lastname)
data.add_field("Email", email)
data.add_field("password", password)
data.add_field("promotion", 'true')
data.add_field("action", {"name":"EMAIL_REG_FORM_SUBMIT"})
data.add_field("ri", 'NORU')
data.add_field("ets", ets)
If anyone has any ideas on how to make this work pls leave a comment. Essentially I need an async requests with a session, if you know how to do that pls let me know.

I was able to get a submission of a full dictionary in a field by simply converting it to a JSON string:
data.add_field("action", json.dumps({"name":"EMAIL_REG_FORM_SUBMIT"}))
Depending on the data in the dictionary, you might need to add a serialization class to json.dumps to handle "special" data types that json.dumps default serialization class can't handle or for which you need to serialize to JSON following some special format (for example converting a DateTime with time zone to some special text format the server is expecting)
You can also add files with additional calls to add_field, specifying the name of the file you have in the form field(s) as the name field of your data.add_field() call.
Under the hood, FormData tries to transform all your fields and files in a properly formatted multipart/form-data payload.
(I found out most of this after fighting a whole day with both the aiohttp client docs and the server I was submitting data to that was doing "unsmart" things)

POST request remake from Java to python gets null

I am totally new to python flask and encountered a problem when writing some code using the requests and flask modules.
I am working on a project using the web API offered by the Panther platform. The project provided an example using Apache Java.
The source code is as below (see more for details).
public class TestProject {
public static void main(String args[]) throws Exception {
CloseableHttpClient httpclient = HttpClients.createDefault();
try {
HttpPost httppost = new HttpPost("http://pantherdb.org/webservices/garuda/tools/enrichment/VER_2/enrichment.jsp?");
StringBody organism = new StringBody("Homo sapiens", ContentType.TEXT_PLAIN);
FileBody fileData = new FileBody(new File("c:\\data_files\\gene_expression_files\\7_data\\humanEnsembl"), ContentType.TEXT_PLAIN);
StringBody enrichmentType = new StringBody("process", ContentType.TEXT_PLAIN);
StringBody testType = new StringBody("FISHER", ContentType.TEXT_PLAIN);
//StringBody cor = new StringBody("FDR", ContentType.TEXT_PLAIN);
//StringBody cor = new StringBody("BONFERRONI", ContentType.TEXT_PLAIN);
//StringBody cor = new StringBody("NONE", ContentType.TEXT_PLAIN);
StringBody type = new StringBody("enrichment", ContentType.TEXT_PLAIN);
HttpEntity reqEntity = MultipartEntityBuilder.create()
.addPart("organism", organism)
.addPart("geneList", fileData)
.addPart("enrichmentType", enrichmentType)
.addPart("test_type", testType)
.addPart("type", type)
//.addPart("correction", cor)
.build();
httppost.setEntity(reqEntity);
CloseableHttpResponse response = httpclient.execute(httppost);
try {
//System.out.println("----------------------------------------");
//System.out.println(response.getStatusLine());
HttpEntity resEntity = response.getEntity();
if (resEntity != null) {
System.out.println(IOUtils.toString(resEntity.getContent(), StandardCharsets.UTF_8));
}
EntityUtils.consume(resEntity);
} finally {
response.close();
}
} finally {
httpclient.close();
}
}
}
The part I am mostly interested in is .addPart("organism", organism) and all the other code with similar structures. They will help pass the parameters from a third-party website to the web API offered by Panther.
I remade the JAVA code into python3 using requests. The code is as follows:
uploadTemp = {'file':open('./app/static/data_temp/temp.txt','rb')}
url="http://pantherdb.org/webservices/garuda/tools/enrichment/VER_2/enrichment.jsp?"
params = {"organism":organism,"geneList":pantherName,"enrichmentType":"fullGO_process","test_type":"BINOMIAL","type":"enrichment","correction":"BONFERRONI"}
# or params = {"organism":organism,"geneList":uploadTemp,"enrichmentType":"fullGO_process","test_type":"BINOMIAL","type":"enrichment","correction":"BONFERRONI"}
Pantherpost= requests.post(url, params = params)
print(Pantherpost.text)
I am expecting an XML object from the web API including some basic biological information. However, the result I got was null (or \n\n\rnull\n when I print Pantherpost.content)
It seems that the parameters I have got from my own web were not correctly sent to the web API.
In addition to this getting null problem, as a beginner, I am also not quite sure about whether the "geneList" part should be receiving a plain-text object or a file. The manual says it is expecting a file, however, it may have been reformatted into plain-text by this command
FileBody fileData = new FileBody(new File("c:\\data_files\\gene_expression_files\\7_data\\humanEnsembl"), ContentType.TEXT_PLAIN);
Anyway, I did try both interpretations: pantherName is a list with name correctly formatted in plain-text and uploadTemp is a .txt file generated for the project. There must be some extra bugs in my code since it returned null in both cases.
Can someone please help out? Thank you very much.

I've found the following issues with your python code:
One. If you want to POST a file using requests, you should use keyword files=.
Two. Keys in files object should match respective parameters of the request (you're using file instead).
Three. You put your parameters in the wrong place of the request by writing params=params.
Function annotation from requests source code:
:param params: (optional) Dictionary or bytes to be sent in the query string for the :class:Request.
In example Java code StringBody is used to create parameters, which implies that parameters should be placed inside the body of HTTP request, not query string. So you should use data= keyword instead. If you use params=, output will be null.
SO article on difference between data and params keywords in requests.
So I've spent some time reading thier manual and made a test script:
import requests
url = "http://pantherdb.org/webservices/garuda/tools/enrichment/VER_2/enrichment.jsp?"
filepath = "C:\\data\\YOUR_DATA.txt" # change to your file location
# all required parameters according to manual, except geneList which is a file (see below)
params = { # using defaults from manual
"type": "enrichment",
"organism": "Homo sapiens",
"enrichmentType": "process",
"test_type": "FISHER",
"correction": "FDR",
}
# note that the key here is the name of paramter: geneList
files = {'geneList': open(filepath, 'rb')}
# it outputs null, when 'params=params' is used
r = requests.post(url, data=params, files=files)
print(r.status_code)
print(r.text)
Output:
200
Id Name GeneId raw P-value FDR

API call fails because of additional escape character in json

I am trying to make an API call to service running locally. The service expects post to have json data like this:
{"tool" : "name", "version" : "1", "payload" : "{"branch" : "main"}"
All these are member of a class where variables are all string (tool, version and payload) and I initialize them with respective strings: "name", "version" and "{\"branch\" : \"main\"}"
Now when I dump the class variable using json.dumps(self.__dict__), I get following json, which I send to my api call:
json_upload = {"tool" : "name", "version" : "1", "payload" : "{\\"branch\\" : \\"main\\"}"
I added a line json_upload.replace("\\\\", "\\") but every time API call fails with 415 and I see the data sent to the api endpoint has double escape character. This is really frustrating, as I don't understand how to strip that extra "\\". How do I fix this? I am using standard library json for encoding.

Seems like the data got escaped twice.
Now when I dump the class variable using json.dumps(self.dict), I get following json, which I send to my api call:
Which client library are you using? e.g. with requests you just pass the object and it will be converted to JSON:
r = requests.post('http://example.com/post', json={"key": "value"})
And if you give it a string, it will escape the string.