We Keep Coding

Parsing JSON in AWS Lambda Python

For a personal project I'm trying to write an AWS Lambda in Python3.9 that will delete a newly created user, if the creator is not myself. For this, the logs in CloudWatch Logs will trigger (via CloudTrail and EventBridge) my Lambda. Therefore, I will receive the JSON request as my event in :
def lambdaHandler(event, context)
But I have trouble to parse it...
If I print the event, I get that :
{'version': '1.0', 'invokingEvent': '{
"configurationItemDiff": {
"changedProperties": {},
"changeType": "CREATE"
},
"configurationItem": {
"relatedEvents": [],
"relationships": [],
"configuration": {
"path": "/",
"userName": "newUser",
"userId": "xxx",
"arn": "xxx",
"createDate": "2022-11-23T09:02:49.000Z",
"userPolicyList": [],
"groupList": [],
"attachedManagedPolicies": [],
"permissionsBoundary": null,
"tags": []
},
"supplementaryConfiguration": {},
"tags": {},
"configurationItemVersion": "1.3",
"configurationItemCaptureTime": "2022-11-23T09:04:40.659Z",
"configurationStateId": 1669194280659,
"awsAccountId": "141372946428",
"configurationItemStatus": "ResourceDiscovered",
"resourceType": "AWS::IAM::User",
"resourceId": "xxx",
"resourceName": "newUser",
"ARN": "arn:aws:iam::xxx:user/newUser",
"awsRegion": "global",
"availabilityZone": "Not Applicable",
"configurationStateMd5Hash": "",
"resourceCreationTime": "2022-11-23T09:02:49.000Z"
},
"notificationCreationTime": "2022-11-23T09:04:41.317Z",
"messageType": "ConfigurationItemChangeNotification",
"recordVersion": "1.3"
}', 'ruleParameters': '{
"badUser": "arn:aws:iam::xxx:user/badUser"
}', 'resultToken': 'xxx=', 'eventLeftScope': False, 'executionRoleArn': 'arn:aws:iam: : xxx:role/aws-service-role/config.amazonaws.com/AWSServiceRoleForConfig', 'configRuleArn': 'arn:aws:config:eu-west-1: xxx:config-rule/config-rule-q3nmvt', 'configRuleName': 'UserCreatedRule', 'configRuleId': 'config-rule-q3nmvt', 'accountId': 'xxx'
}
And for my purpose, I'd like to get the "changeType": "CREATE" value to say that if it is CREATE, I check the creator and if it is not myself, I delete newUser.
So the weird thing is that I copy/paste that event into VSCode and format it in a .json document and it says that there are errors (line 1 : version and invokingEvent should be double quote for example, but well).
For now I only try to reach and print the
"changeType": "CREATE"
by doing :
import json
import boto3
import logging
iam = boto3.client('iam')
def lambda_handler(event, context):
"""
Triggered if a user is created
Check the creator - if not myself :
- delete new user and remove from groups if necessary
"""
try:
print(event['invokingEvent']["configurationItemDiff"]["changeType"])
except Exception as e:
print("Error because :")
print(e)
And get the error string indices must be integers - it happens for ["configurationItemDiff"].
I understand the error already (I'm new to python though so maybe not completely) and tried many things like :
print(event['invokingEvent']['configurationItemDiff']) : swapping double quote by simple quote but doesnt change anything
print(event['invokingEvent'][0]) : but it gives me the index { and [2] gives me the c not the whole value.
At this point I'm stuck and need help because I can't find any solution on this. I don't use SNS, maybe should I ? Because I saw that with it, the JSON document would not be the same and we can access through ["Records"][...] ? I don't know, please help

What you are printing is a python dict, it looks sort of like JSON but is not JSON, it is the representation of a python dict. That means it will have True / False instead of true / false, it will have ' instead of ", etc.
You could do print(json.dumps(event)) instead.
Anyway, the actual problem is that invokingEvent is yet another JSON, but in its string form, you need to to json.loads that nested JSON string. You can see that because the value after invokingEvent is inside another set of '...', therefore it is a string, not a parsed dict already.
invoking_event = json.loads(event['invokingEvent'])
change_type = invoking_event["configurationItemDiff"]["changeType"]
ruleParameters would be another nested JSON which needs parsing first if you wanted to use it.

Query for a specific item in firebase from python

I am getting this error
Index not defined, add ".indexOn": "Number", for path "/public_access", to the rules
and have no idea what the issue is. All I'd like to do is retrieve (Check if the entry exists in my db) to see if the tag has already been use
My rules look like this
{
"rules": {
"public_access": {
"all_users": {
".indexOn": "Numbers"
},
".read": "auth.uid === 'regular_user'",
".write": true,
},
"connection_testing": {
".read": true,
".write": "auth.uid === 'admin_user'"
},
"login_data": {
".read": true,
".write": true
},
}
}
And the python code I am trying to use is this
ref = db.reference('/public_access/all_users')
snapshot = ref.order_by_child('Number').equal_to('AAAAAA').get()
But it just errors, I've tried copying so many different examples and I've been stuck on it for hours. I know the solution will be something stupidly simple but I just can't apply it to what I want to do.
I don't need the number value at all, it was just an attempt to get it to find 'AAAAAA'.
The end goal is to see if AAAAAA exists by searching for it, and I have not been successful at achieving this

You don't need order_by_child or database rules here. If you want to search for "AAAAAA", just access it by the child method:
ref = db.reference('/public_access/all_users')
data = ref.child('AAAAAA').get()
print(data['Date_of_join']) # '12/34/34'
print(data['Number']) # 2

Python - custom keyboard for Telegram bot

I'm trying to write up a Telegram bot from scratch using Python and Flask, without existing libraries, for learning sake.
I'm still stuck with custom keyboard and its syntax. Can anyone show me the right method with a simple example? Starting from here:
#!/usr/bin/env python
import json, requests
keyboard = json.dumps({'inline_keyboard': [[ "<user1>", "<user2>" ]]})
headers = {'Accept': 'application/json'}
url = "https://api.telegram.org/bot<bot-id>/sendMessage"
kbd = {'text':'Whatever','chat_id':'<channel>','reply_markup': keyboard}
send_kbd = requests.post(url,data=kbd,headers=headers)
Thanks in advance.

Telegram responses are well informative on the issues.
InlineKeyboardMarkup is an Array of Array of InlineKeyboardButton which any element of outer array defines a set of buttons in a row. InlineKeyboardButton by itself must be json object, requires a text key and also requires exactly one of url, callback_data, switch_inline_query, switch_inline_query_current_chat, callback_game set of keys. for more information InlineKeyboardButton.
Change keyboard to:
keyboard = '{
"inline_keyboard": [
[
{"text":"baa boo", "url":"http://some.url"},
{"text":"boo baa", "switch_inline_query":""}
]
]
}'
and you may need to replace <bot-id> and <channel> to correct codes, if you did not.

A correctly formatted (JSON) message with inline buttons looks like this:
{
"text" : "Whatever text",
"chat_id" : 0123456798,
"reply_markup": {
"inline_keyboard": [
[
{
"text": "text",
"callback_data": "click_data"
}
],
[
{
"text": "text",
"callback_data": "click_data"
}
]
]
}
}
Please note that a inline button must be an object.
You are not allowed to send text-only buttons by Telegram.

hellow guys, i found a soloution in python, here it is:
first of all you need to send a text with it.
urlrequest='https://api.telegram.org/bot<TOKEN>/sendmessage?chat_id=<chat_id>&text=<your text>'
keyboard={'keyboard':[[{'text':'text1'}],[{'text':'text2'}]]}
key=json.JSONEncoder().encode(keyboard)
url=urlrequest+'&reply_markup='+key
response=urlopen(url)
it worked for me, try it maybe worked for you.
thanks of your attention.

Access GlobalParameters in Azure ML Python script

How can one access the global parameters ("GlobalParameters") sent from a web service in a Python script on Azure ML?
I tried:
if 'GlobalParameters' in globals():
myparam = GlobalParameters['myparam']
but with no success.
EDIT: Example
In my case, I'm sending a sound file over the web service (as a list of samples). I would also like to send a sample rate and the number of bits per sample. I've successfully configured the web service (I think) to take these parameters, so the GlobalParameters now look like:
"GlobalParameters": {
"sampleRate": "44100",
"bitsPerSample": "16",
}
However, I cannot access these variables from the Python script, neither as GlobalParameters["sampleRate"] nor as sampleRate. Is it possible? Where are they stored?

based on our understanding of your question, here may has a miss conception that Azure ML parameters are not “Global Parameters”, as a matter of fact they are just parameter substitution tied to a particular module. So in affect there are no global parameters that are accessible throughout the experiment you have mentioned. Such being the case, we think the experiment below accomplishes what you are asking for:
Please add an “Enter Data” module to the experiment and add Data in csv format. Then for the Data click the parameter to create a web service parameter. Add in the CSV data which will be substituted from data passed by the client application. I.e.
Please add an “Execute Python” module and hook up the “Enter Data” output to the “Execute Python” input1. Add the python code to take the dataframe1 and add it to a python list. Once you have it in a list you can use it anywhere in your python code.
Python code snippet
def azureml_main(dataframe1 = None, dataframe2 = None):
import pandas as pd
global_list = []
for g in dataframe1["Col3"]:
global_list.append(g)
df_global = pd.DataFrame(global_list)
print('Input pandas.DataFrame:\r\n\r\n{0}'.format(df_global))
return [df_global]
Once you publish your experiment, you can add in new values in the “Data”: “”, section below with the new values that you was substituted for the “Enter Data” values in the experiment.
data = {
"Inputs": {
"input1":
{
"ColumnNames": ["Col1", "Col2", "Col3"],
"Values": [ [ "0", "value", "0" ], [ "0", "value", "0" ], ]
}, },
"GlobalParameters": {
"Data": "1,sampleRate,44500\\n2,bitsPerSample,20",
}
}
Please feel free to let us know if this makes sense.

The GlobalParameters parameter can not be used in a Python script. It is used to override certain parameters in other modules.
If you, for example, take the 'Split Data' module, you'll find an option to turn a parameter into a web service parameter:
Once you click that, a new section appears titled "Web Service Parameters". There you can change the default parameter name to one of your choosing.
If you deploy your project as a web service, you can override that parameter by putting it in the GlobalParameters parameter:
"GlobalParameters": {
"myFraction": 0.7
}
I hope that clears things up a bit.

Although it is not possible to use GlobalParameters in the Python script (see my previous answer), you can however hack/abuse the second input of the Python script to pass in other parameters. In my example I call them metadata parameters.
To start, I added:
a Web service input module with name: "realdata" (for your real data off course)
a Web service input module with name: "metadata" (we will abuse this one to pass parameters to our Python).
a Web service output module with name: "computedMetadata"
Connect the modules as follows:
As you can see, I also added a real data set (Restaurant ratings) as wel as a dummy metadata csv (the Enter Data Manually) module.
In this manual data you will have to predefine your metadata parameters as if they were a csv with a header and a only a single row to hold the data:
In the example both sampleRate and bitsPerSample are set to 0.
My Python scripts then takes in that fake csv as metadata, does some dummy calculation with it and returns it as column name:
import pandas as pd
def azureml_main(realdata = None, metadata = None):
theSum = metadata["sampleRate"][0] + metadata["bitsPerSample"][0]
outputString = "The sum of the sampleRate and the bitsPerSecond is " + str(theSum)
print(outputString)
return pd.DataFrame([outputString])
I then published this as a web service and called it using Node.js like this:
httpreq.post('https://ussouthcentral.services.azureml.net/workspaces/xxx/services/xxx', {
headers: {
Authorization: 'Bearer xxx'
},
json: {
"Inputs": {
"realdata": {
"ColumnNames": [
"userID",
"placeID",
"rating"
],
"Values": [
[
"100",
"101",
"102"
],
[
"200",
"201",
"202"
]
]
},
"metadata": {
"ColumnNames": [
"sampleRate",
"bitsPerSample"
],
"Values": [
[
44100,
16
]
]
}
},
"GlobalParameters": {}
}
}, (err, res) => {
if(err) return console.log(err);
console.log(JSON.parse(res.body));
});
The output was as expected:
{ Results:
{ computedMetadata:
{ type: 'table',
value:
{ ColumnNames: [ '0' ],
ColumnTypes: [ 'String' ],
Values:
[ [ 'The sum of the sampleRate and the bitsPerSecond is 44116' ] ] } } } }
Good luck!

Organizing my config variable for webapp2

For simplicity I think I need to rewrite this to just one statement
config = {'webapp2_extras.jinja2': {'template_path': 'templates',
'filters': {
'timesince': filters.timesince,
'datetimeformat': filters.datetimeformat},
'environment_args': {'extensions': ['jinja2.ext.i18n']}}}
config['webapp2_extras.sessions'] = \
{'secret_key': 'my-secret-key'}
Then I want to know where to put it if I use multiple files with multiple request handlers. Should I just put it in one file and import it to the others? Since the session code is secret, what are your recommendation for handling it via source control? To always change the secret before or after committing to source control?
Thank you

Just add 'webapp2_extras.sessions' to your dict initializer:
config = {'webapp2_extras.jinja2': {'template_path': 'templates',
'filters': {
'timesince': filters.timesince,
'datetimeformat': filters.datetimeformat},
'environment_args': {'extensions': ['jinja2.ext.i18n']}},
'webapp2_extras.sessions': {'secret_key': 'my-secret-key'}}
This would be clearer if the nesting were explicit, though:
config = {
'webapp2_extras.jinja2': {
'template_path': 'templates',
'filters': {
'timesince': filters.timesince,
'datetimeformat': filters.datetimeformat
},
'environment_args': {'extensions': ['jinja2.ext.i18n']},
},
'webapp2_extras.sessions': {'secret_key': 'my-secret-key'}
}

I would recommend storing those in a datastore Entity for more flexibility and caching them in the instance memory at startup.
You could also consider having a config.py file excluded from the source control, if you want to get things done quickly.