We Keep Coding

python variable extracted from nested yaml

Trying to find a way to iterate over the roleprivs and having issues getting to that level of the yaml from python.
testrole.yaml
info:
rolename: "testDeveloper"
desc: "Test Developer Role"
roletype: "user"
roleprivs:
admin-appliance:
name: "Administrate Appliance" # Informational Only Not used in code
description: "admin-appliance" # Informational Only Not used in code
code: "admin-appliance"
access: "full"
admin-backupSettings:
name: "Administrate Backup Settings" # Informational Only Not used in code
description: "admin-appliance" # Informational Only Not used in code
code: "admin-backupSettings"
access: "full"
I have a few different needs / use cases.
Part 1 of the script below - grab all the files in a directory and take the rolename, desc, and roletype and create a role.
Get the Role ID of the newly created role that was above.
HELP Needed - going back to the original yaml file and iterating over it and getting only the roleprivs..code and roleprivs..code --> role type would be something like admin-appliance. Keeping in mind that there are like 50 some odd features that need to be updated with the type of access.
The question:
How do i get the code and access in the yaml file into python variables?
def genericRoleCreate(baseURL, bearerToken):
print("initial")
files = glob.glob(ROLES_DIR)
logger.debug('Roles Dir '+ROLES_DIR)
for file in files:
yaml_file = file
logger.debug(yaml_file)
with open(yaml_file) as f:
try:
result=yaml.safe_load(f)
authority = result['info']['rolename']
desc = result['info']['desc']
roletype = result['info']['roletype']
url = baseURL+"/api/roles"
payload= json.dumps({"role":{"authority": authority, "description": desc, "roletype": roletype}})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
roleResult = requests.request("POST", url, verify=False, headers=headers, data=payload)
logger.debug(roleResult.text)
except yaml.YAMLError as exc:
logger.error(exc)
# Getting Role ID
try:
with open(yaml_file) as f:
result = yaml.safe_load(f)
authority = result['info']['rolename']
url = baseURL+"/api/roles?phrase="+authority
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
roleResult = requests.request("GET", url, verify=False, headers=headers )
#print(roleResult.text)
roleID = json.loads(roleResult.text)
role = roleID['roles'][0]['id']
#logger.debug(role)
logger.info("Get Role ID")
print(role)
#return role
#logger.debug("Role ID: "+role)
except Exception as e:
logger.error('Exception occurred', exc_info=True)
logger.error('Error getting roleID')
# Start Updating
#role = getRoleId(baseURL, bearerToken)
try:
with open(yaml_file) as f:
result = yaml.safe_load(f)
except Exception as e:
logger.error("Broken")
strRoleID = str(role)
url = baseURL+"/api/roles/"+strRoleID+"/update-permission"
#logger.debug(result)
keys = list(result.keys())
for features in keys:
#logger.debug(keys)
code = result[features]['code']
access = result[features]['access']
payload = json.dumps({
"permissionCode": code,
"access": access
})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
requests.request("PUT", url, verify=False, headers=headers, data=payload)
lets keep in mind i do know that i should be breaking that big nasty thing into multiple functions - i have it broken down in other areas - but compiling everything in a single function at the time.
I have been trying multiple iterations of how to get to the feature level. I have looked at many examples and can't seem to figure out how to drop to a level.
update 1
try:
with open(yaml_file, 'r') as f:
result = yaml.safe_load(f)
except Exception as e:
logger.error("Broken")
strRoleID = str(role)
url = baseURL+"/api/roles/"+strRoleID+"/update-permission"
#logger.debug(result)
keys = list(result['roleprivs'].keys())
#code2 = {roleprivs for roleprivs in result['roleprivs'].keys()}
#print(code2)
#return inventory, sites
for features in keys:
print(features)
The code above produces the output:
admin-appliance
admin-backupSettings
now the question is how do i go one step deeper in the chain and get code and access into a variable in python.

I ended up solving the problem after a few hours of testing and researching...The main problem i was encountering was how do i get to the next level after roleprivs. I could easily print the main elements under the roleprivs, but getting the code and access elements were a bit of a challenge.
I also kept running into an error where the indices needed to be an integer. This cleans up some of the keys that i was doing before and puts it into a one liner. Should help clean up a few for loops that i have been working with.
for k, v in result['roleprivs'].items():
print("Code "+result['roleprivs'][k]['code'])
print("Access: "+result['roleprivs'][k]['access'])
access = result['roleprivs'][k]['access']
code = result['roleprivs'][k]['code']
payload = json.dumps({
"permissionCode": code,
"access": access
})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
requests.request("PUT", url, verify=False, headers=headers, data=payload)
From the original code i may have multiple roles in the ./config/roles/ directory. I needed to make sure i can read all and iterate in a for loop for each one. This solved it for me.
Final output:
2021-10-13 01:25:50,212:84:logger:role:genericRoleCreate:DEBUG:./config/roles/testDeveloper.yaml
2021-10-13 01:25:50,487:110:logger:role:genericRoleCreate:INFO:Get Role ID
8
Code admin-appliance
Access: full
Code admin-backupSettings
Access: full

Python API call - loop / nextpagetoken

I'm brand new to python and api as well.
I'm trying to use a endpoint we have at work.
We have an API we are using a lot, we also have an UI. But using the UI we can only extract 10.000 records at the time.
There is no limit on the api.
I have found a small piece of code - but i need to add a nextpagetoken.
My code looks like this:
login_url = 'https://api.ubsend.io/v1/auth/login'
username = 'xxxxx'
password = 'xxxxx'
omitClaims = "true"
session = requests.Session()
session.headers['Accept'] = "application/json; charset=UTF-8"
response = session.post(
login_url,
json={'username': username, 'password': password},
headers={'VERSION': '3'},
)
response.raise_for_status()
response_data = response.json()
print(response_data)
This gives me the AccessToken.
Then I call:
getevents = 'https://api.ubsend.io/v1/reporting/shipments?'
data ={'client_id': 13490, 'created_after': '2020-05-01T00:00', 'created_before': '2021-05-02T00:00'} req.prepare_url(getevents, data)
events = requests.get(req.url, headers={'Authorization' : 'Bearer ' + response_data['accessToken'], Content-Type': 'application/json'})
events.json()
Which returns:
'nextPageToken': 'NjA4ZDc3YzNkMjBjODgyYjBhMWVkMTVkLDE2MTk4ODM5NzA3MDE='}
So I want to loop my script - until nextPageToken is blank ....
Any thoughts?

Edit thanks for the update. I think this might be the solution we're looking for. You might have to do some poking around to figure out exactly what the name of the page_token URL parameter should be.
has_next = True
getevents = 'https://api.ubsend.io/v1/reporting/shipments?'
token = None
while has_next:
data ={'client_id': 13490, 'created_after': '2020-05-01T00:00', 'created_before': '2021-05-02T00:00'}
if token:
# I don't know the proper name for this URL parameter.
data['page_token'] = token
req.prepare_url(getevents, data)
events = requests.get(req.url, headers={'Authorization' : 'Bearer ' + response_data['accessToken'], Content-Type: 'application/json'})
token = events.json().get('nextPageToken')
if not token:
has_next = False
I made a slight typo. It should be events.json().get('nextPageToken') I believe.
Let me know if this works.

Sending fcm from PYTHON(flask) but not receiving on android side

Here are three different ways i try to send data message fcm.
1. With pyfcm
def send_fcm(fcm_tokens, title=None, body=None, data_message=None):
push_service = FCMNotification(api_key=app.config['FCM_KEY'])
try:
if type(fcm_tokens) is list:
print(fcm_tokens, data_message)
result = push_service.notify_multiple_devices(registration_ids=fcm_tokens, data_message=data_message)
print(result, '++++++++++++++', flush=True)
else:
print(fcm_tokens, 'single device', data_message)
result = push_service.notify_single_device(registration_id=fcm_tokens, data_message=data_message)
print(result, flush=True)
except errors.InvalidDataError as e:
print(e, flush=True)
2. With firebase_admin sdk
def send_fcm_with_sdk(tokens, data_message=None):
if type(tokens) is list:
message = messaging.MulticastMessage(
data=data_message,
tokens=tokens, )
response = messaging.send_multicast(message)
print('{0} messages were sent successfully'.format(response.success_count), flush=True)
else:
message = messaging.Message(
data=data_message,
token=tokens,
)
print(message)
response = messaging.send(message)
print('Successfully sent message:', response)
3. With rest api
def send_fcm_with_rest(tokens, title=None, body=None, image=None, data_message=None):
headers = {
'Content-Type': 'application/json',
'Authorization': 'key=xxxxxxxKEY',
'Content-Type': 'application/json'
}
url = 'https://fcm.googleapis.com/fcm/send'
if type(tokens) is list:
payload = {
"registration_ids": tokens,
"collapse_key": "type_a",
"data": data_message
}
else:
payload = {
"to": tokens,
"collapse_key": "type_a",
"data": data_message
}
print(json.dumps(payload))
resp = requests.post(url, headers=headers, data=json.dumps(payload))
print(resp.text.encode('utf8'), flush=True)
return resp
The strange thing is all three of them run correctly and show a success message, with success_id, but none of fcm is received on the android side.
I tried sending from POSTMAN, and from postman, fcm are received on Andriod side.
Can anyone please tell me what is the issue in my code?

Your code is OK, however you should use flask==1.1.2 version as it is the stable release.

All the three methods for sending fcm is correct, later we figure out it was the issue from the frontend guy. So anyone can use these three methods if they want to.

How can I post an image on Facebook using Python requests and Graph API?

1) I require a function which should publish a post with the given message and photo.
One can use the page ID and access tokens provided in self.page_id and self.page_access_token
def __init__(self):
self.page_id = Facebook.get_access_token('FACEBOOK_PAGE_ID')
self.page_access_token = Facebook.get_access_token('FACEBOOK_PAGE_ACCESS_TOKEN')
2) To find which API to hit, check out developer section of the Facebook API: https://developers.facebook.com/docs/graph-api/reference/page/photos/#example-2
3) The function should not return anything.
def publish_photo_msg(self, message, image_url):
#your code here
return
Please help.

python
params = (
('access_token', self.page_access_token),
)
Next, let us fix the data dictionary:
python
data = {
'url': image_url,
'caption': 'this is the caption'
}
Now, let’s proceed to the URL. There are many ways to set this. Here is one way:
python
url = 'https://graph.facebook.com/v5.0/' + self.page_id + '/photos?access_token=' + self.page_access_token
Since we have stored the access_token in the params tuple, let’s make use of it in the requests.post() call.
python
url = 'https://graph.facebook.com/v5.0/' + self.page_id + '/photos'
response = requests.post(url=url, params=params, data=data)
Lastly, you can also verify if your requests.post() call was successful by checking the value of the response variable:
python
print response.status_code, response.json()
For easy reference, here is the full implementation of the publish_photo_msg function with all the suggestions incorporated:
python
params = (
('access_token', self.page_access_token),
)
data = {
'url': image_url,
'caption': message
}
url = 'https://graph.facebook.com/v5.0/' + self.page_id + '/photos'
response = requests.post(url=url, params=params, data=data)
print(response.status_code, response.json())

Creating an Epic from a Github Issue with Zenhub API

This is a follow up question to How to set an issue pipeline with zenhub.
I'm attempting to convert an issue to an epic in a Python script. I can convert the issue to an Epic, but I get an error when I attempt to add issues when creating the epic.
This works:
zenhub_headers = {"X-Authentication-Token": "%s" % token}
target_zh_issues_url = '%s/p1/repositories/%d/issues' % (zh_api_endpoint, target_repo_id)
params = {}
response = requests.post(target_zh_issues_url + '/%s/convert_to_epic' % issue, headers=zenhub_headers, data=params)
The code also works when I set params = {"issues":[]}
But when I attempt to add an issue with params = {"issues": [{"repo_id": 280565, "issue_number": 17}]}
I get a 400 error, b'{"message":"Invalid Field for issues: [object Object],[object Object]"}'
I then tried using the /update_issues API to add issues to the epics I'd created.
target_zh_epics_url = '%s/p1/repositories/%d/epics' % (zh_api_endpoint, target_repo_id)
params = {"add_issues": [{"repo_id": 280565, "issue_number": 17}]}
response = requests.post(target_zh_epics_url + '/%s/update_issues' % issue, headers=zenhub_headers, data=params)
This resulted in a 400 error, b'{"message":"Invalid Field for addIssues: repo_id,issue_number"}'. Those fields are as described in the API doc.

I got this to work by adding 'Content-Type': 'application/json' to my headers and dumping the JSON body to a string, params = json.dumps({"issues": [{"repo_id": 280565, "issue_number": 17}]})
My code now looks like:
zenhub_headers = {"X-Authentication-Token": "%s" % token, 'Content-Type': 'application/json'}
target_zh_issues_url = '%s/p1/repositories/%d/issues' % (zh_api_endpoint, target_repo_id)
params = json.dumps({"issues": [{"repo_id": 280565, "issue_number": 17}]})
response = requests.post(target_zh_issues_url + '/%s/convert_to_epic' % issue, headers=zenhub_headers, data=params)
Though I'm not sure why the call with a body of unstringified {"issues":[]} was succeeding.