python variable extracted from nested yaml - python

Trying to find a way to iterate over the roleprivs and having issues getting to that level of the yaml from python.
testrole.yaml
info:
rolename: "testDeveloper"
desc: "Test Developer Role"
roletype: "user"
roleprivs:
admin-appliance:
name: "Administrate Appliance" # Informational Only Not used in code
description: "admin-appliance" # Informational Only Not used in code
code: "admin-appliance"
access: "full"
admin-backupSettings:
name: "Administrate Backup Settings" # Informational Only Not used in code
description: "admin-appliance" # Informational Only Not used in code
code: "admin-backupSettings"
access: "full"
I have a few different needs / use cases.
Part 1 of the script below - grab all the files in a directory and take the rolename, desc, and roletype and create a role.
Get the Role ID of the newly created role that was above.
HELP Needed - going back to the original yaml file and iterating over it and getting only the roleprivs..code and roleprivs..code --> role type would be something like admin-appliance. Keeping in mind that there are like 50 some odd features that need to be updated with the type of access.
The question:
How do i get the code and access in the yaml file into python variables?
def genericRoleCreate(baseURL, bearerToken):
print("initial")
files = glob.glob(ROLES_DIR)
logger.debug('Roles Dir '+ROLES_DIR)
for file in files:
yaml_file = file
logger.debug(yaml_file)
with open(yaml_file) as f:
try:
result=yaml.safe_load(f)
authority = result['info']['rolename']
desc = result['info']['desc']
roletype = result['info']['roletype']
url = baseURL+"/api/roles"
payload= json.dumps({"role":{"authority": authority, "description": desc, "roletype": roletype}})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
roleResult = requests.request("POST", url, verify=False, headers=headers, data=payload)
logger.debug(roleResult.text)
except yaml.YAMLError as exc:
logger.error(exc)
# Getting Role ID
try:
with open(yaml_file) as f:
result = yaml.safe_load(f)
authority = result['info']['rolename']
url = baseURL+"/api/roles?phrase="+authority
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
roleResult = requests.request("GET", url, verify=False, headers=headers )
#print(roleResult.text)
roleID = json.loads(roleResult.text)
role = roleID['roles'][0]['id']
#logger.debug(role)
logger.info("Get Role ID")
print(role)
#return role
#logger.debug("Role ID: "+role)
except Exception as e:
logger.error('Exception occurred', exc_info=True)
logger.error('Error getting roleID')
# Start Updating
#role = getRoleId(baseURL, bearerToken)
try:
with open(yaml_file) as f:
result = yaml.safe_load(f)
except Exception as e:
logger.error("Broken")
strRoleID = str(role)
url = baseURL+"/api/roles/"+strRoleID+"/update-permission"
#logger.debug(result)
keys = list(result.keys())
for features in keys:
#logger.debug(keys)
code = result[features]['code']
access = result[features]['access']
payload = json.dumps({
"permissionCode": code,
"access": access
})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
requests.request("PUT", url, verify=False, headers=headers, data=payload)
lets keep in mind i do know that i should be breaking that big nasty thing into multiple functions - i have it broken down in other areas - but compiling everything in a single function at the time.
I have been trying multiple iterations of how to get to the feature level. I have looked at many examples and can't seem to figure out how to drop to a level.
update 1
try:
with open(yaml_file, 'r') as f:
result = yaml.safe_load(f)
except Exception as e:
logger.error("Broken")
strRoleID = str(role)
url = baseURL+"/api/roles/"+strRoleID+"/update-permission"
#logger.debug(result)
keys = list(result['roleprivs'].keys())
#code2 = {roleprivs for roleprivs in result['roleprivs'].keys()}
#print(code2)
#return inventory, sites
for features in keys:
print(features)
The code above produces the output:
admin-appliance
admin-backupSettings
now the question is how do i go one step deeper in the chain and get code and access into a variable in python.

I ended up solving the problem after a few hours of testing and researching...The main problem i was encountering was how do i get to the next level after roleprivs. I could easily print the main elements under the roleprivs, but getting the code and access elements were a bit of a challenge.
I also kept running into an error where the indices needed to be an integer. This cleans up some of the keys that i was doing before and puts it into a one liner. Should help clean up a few for loops that i have been working with.
for k, v in result['roleprivs'].items():
print("Code "+result['roleprivs'][k]['code'])
print("Access: "+result['roleprivs'][k]['access'])
access = result['roleprivs'][k]['access']
code = result['roleprivs'][k]['code']
payload = json.dumps({
"permissionCode": code,
"access": access
})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
requests.request("PUT", url, verify=False, headers=headers, data=payload)
From the original code i may have multiple roles in the ./config/roles/ directory. I needed to make sure i can read all and iterate in a for loop for each one. This solved it for me.
Final output:
2021-10-13 01:25:50,212:84:logger:role:genericRoleCreate:DEBUG:./config/roles/testDeveloper.yaml
2021-10-13 01:25:50,487:110:logger:role:genericRoleCreate:INFO:Get Role ID
8
Code admin-appliance
Access: full
Code admin-backupSettings
Access: full

Related

Python : Passing form-data directly in fast-api with UI

I am calling a API of OmniDocs, I am able to Add Document using Postman, passing it as form-data.
When I am trying to do the same in python it is returning Bad request.
NGOAddDocumentBDO Value
<NGOAddDocumentBDO>
<cabinetName>samplecabinet</cabinetName>
<folderIndex>3185</folderIndex>
<documentName>Restweb postman</documentName>
<userDBId></userDBId>
<volumeId>1</volumeId>
<accessType>S</accessType>
<createdByAppName>txt</createdByAppName>
<enableLog>Y</enableLog>
<versionFlag>N</versionFlag>
<textAlsoFlag></textAlsoFlag>
<ownerType>U</ownerType>
<ownerIndex>2</ownerIndex>
<nameLength></nameLength>
<thumbNailFlag>N</thumbNailFlag>
<imageData></imageData>
<encrFlag>N</encrFlag>
<passAlgoType>MD5</passAlgoType>
<userName>test123</userName>
<userPassword>Test#1234</userPassword>
<comment></comment>
<locale>en_US</locale>
<NGOAddDocDataDefCriterionBDO>
<dataDefIndex>22</dataDefIndex>
<dataDefName>DIGI2</dataDefName>
<NGOAddDocDataDefCriteriaDataBDO>
<indexId>43</indexId>
<indexType>I</indexType>
<indexValue>123</indexValue>
</NGOAddDocDataDefCriteriaDataBDO>
</NGOAddDocDataDefCriterionBDO>
<NGOAddDocKeywordsCriterionBDO>
<keyword></keyword>
</NGOAddDocKeywordsCriterionBDO>
</NGOAddDocumentBDO>
Postman output
Using fastapi in python, I am passing 'content-type': 'multipart/form-data', but not able to determine where the request is going wrong, if it is working fine with postman, not able to figure out the problem in fast-api.
lstr_add_document_response = "Empty"
lxml_add_document = """
<NGOAddDocumentBDO>
<cabinetName>samplecabinet</cabinetName>
<folderIndex>3185</folderIndex>
<documentName>Restweb postman</documentName>
<userDBId></userDBId>
<volumeId>1</volumeId>
<accessType>S</accessType>
<createdByAppName>txt</createdByAppName>
<enableLog>Y</enableLog>
<versionFlag>N</versionFlag>
<textAlsoFlag></textAlsoFlag>
<ownerType>U</ownerType>
<ownerIndex>2</ownerIndex>
<nameLength></nameLength>
<thumbNailFlag>N</thumbNailFlag>
<imageData></imageData>
<encrFlag>N</encrFlag>
<passAlgoType>MD5</passAlgoType>
<userName>test123</userName>
<userPassword>Test#1234</userPassword>
<comment></comment>
<locale>en_US</locale>
<NGOAddDocDataDefCriterionBDO>
<dataDefIndex>22</dataDefIndex>
<dataDefName>DIGI2</dataDefName>
<NGOAddDocDataDefCriteriaDataBDO>
<indexId>43</indexId>
<indexType>I</indexType>
<indexValue>123</indexValue>
</NGOAddDocDataDefCriteriaDataBDO>
</NGOAddDocDataDefCriterionBDO>
<NGOAddDocKeywordsCriterionBDO>
<keyword></keyword>
</NGOAddDocKeywordsCriterionBDO>
</NGOAddDocumentBDO>
"""
lstr_headers = {'content-type': 'multipart/form-data'}
lstr_data = {'NGOAddDocumentBDO': lxml_add_document, 'file': open('/home/donny/Desktop/omnidocs/output-document/8VQUI_something.pdf', "wb+")}
try:
lstr_add_document_response = requests.post(gstr_omnidocs_add_document_service,
data=lstr_data,
headers=lstr_headers)
print(gstr_omnidocs_add_document_service)
print(lstr_data)
print(lstr_headers)
except Exception as e:
logger.error(str(e), exc_info=True)
print("Response text=", lstr_add_document_response.text.encode('utf8'))
print(lstr_add_document_response.content)
# replace < with <
new_lstr_add_document_response = (lstr_add_document_response.text).replace("<", "<")
# get the response from the add document response
lstr_soup_response = BeautifulSoup(new_lstr_add_document_response, features="xml")
return lstr_soup_response
except Exception as e:
logger.error(str(e), exc_info=True)
Postman on right side should have icon </> to open function Code snippet which can generate code for different languages - even for Python (requests and http.client)
It gives me code which sends file in files=...
import requests
url = "http://10.10.2.41:8003/OmniBook"
payload={'NGOAddDocumentBDO': '<XML>'}
files=[
('file',('file',open('/path/to/file','rb'),'application/octet-stream'))
]
headers = {}
response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.text)
But I couldn't test it.

Using Python Requests to POST data to REST API

I have been working with the FHIR REST API for a while but haven't had any experience with Python. As my first python project I am attempting to create a simple python script that can read and write to an open API. I am able to read but I am stuck on creating a successful POST due to the following: error [TypeError("unhashable type: 'dict'")]. I don't fully understand how the python dictionary works and attempted to use a tuple but get the same error.
import requests #REST Access to FHIR Server
print('Search patient by MRN to find existing appointment')
MRN = input("Enter patient's MRN -try CT12181 :")
url = 'http://hapi.fhir.org/baseR4/Patient?identifier='+MRN
print('Searching for Patient by MRN...#'+url)
response = requests.get(url)
json_response = response.json()
try:
key='entry'
EntryArray=json_response[key]
FirstEntry=EntryArray[0]
key='resource'
resource=FirstEntry['resource']
id=resource['id']
PatientServerId= id
patientName = resource['name'][0]['given'][0] + ' ' +resource['name'][0]['family']
print('Patient Found')
print('Patient Id:'+id)
#Searching for assertppointments
url='http://hapi.fhir.org/baseR4/Appointment?patient='+id #fhir server endpoint
#Print appointment data
print('Now Searching for Appointments...#'+url)
appt_response = requests.get(url).json()
key='entry'
EntryArray=appt_response[key]
print (f'Appointment(s) found for the patient {patientName}')
for entry in EntryArray:
appt=entry['resource']
# print('-------------------------')
# Date=appt['start']
# Status=appt['status']
# print(appt_response)
#print ('AppointmentStartDate/Time: ' ,appt['start'])
print ('Status: ' ,appt['status'])
print ('ID: ' ,appt['id'])
print('Search for open general practice slot?')
option = input('Enter yes or no: ')
while not(option == 'yes'):
print('Please search a different paitent')
option = input('Enter yes or no: ')
url = 'http://hapi.fhir.org/baseR4/Slot?service-type=57' #fhir server endpoint
print('Searching for General Practice Slot...#'+url)
slot_response = requests.get(url).json()
key='entry'
EntryArray=slot_response[key]
print ('Slot(s) found for the service type General Practice')
for entry in EntryArray:
slot=entry['resource']
#print('-------------------------')
#slotDate=slot['start']
#slotStatus=slot['status']
print (f'SlotID: ' +slot['id'])
#print (f'Status: ' +slot['status'])
print('Book a slot?')
option = input('Enter yes or no: ')
while not(option == 'yes'):
print('Please search a different paitent')
option = input('Enter yes or no: ')
#Book slot
slotID = input("Enter slot ID :")
url = 'http://hapi.fhir.org/baseR4/Appointment' #fhir server endpoint
print('Booking slot...#'+url)
headers = {"Content-Type": "application/fhir+json;charset=utf-8"}
data = {{"resourceType": "Appointment","status": "booked","slot": tuple({"reference":"Slot/104602"}),"participant": tuple({"actor": {"reference":"Patient/1229151","status": "accepted"}}),"reasonCode": tuple({"text": "I have a cramp"})}}
#fhir server json header content
# headers = {"Content-Type": "application/fhir+json;charset=utf-8"}
response = requests.post(url=url,headers=headers,data=data)
print(response)
print(response.json())
except Exception as e:
print ('error' ,[e])
I was expecting the JSON data to successfully write to the API. I am able to use the same JSON data in Postman to make a call, but I am not as familiar on how this should work within Python.
It looks like the Appointment POST endpoint accepts a simple payload like:
{
"resourceType": "Appointment"
}
Which then returns a corresponding ID, according to the API docs.
This differs from what you seem to be attempting in your code, where you try to pass other details to this endpoint:
ata = {{"resourceType": "Appointment","status": "booked","slot": tuple({"reference":"Slot/104602"}),"participant": tuple({"actor": {"reference":"Patient/1229151","status": "accepted"}}),"reasonCode": tuple({"text": "I have a cramp"})}}
However, to make a POST request to the endpoint as documented in the docs, perhaps try the json argument to requests.post. Something along the lines of:
>>> import requests
>>> headers = {"Content-Type": "application/fhir+json;charset=utf-8"}
>>> json_payload = {
... "resourceType": "Appointment"
... }
>>> url = 'http://hapi.fhir.org/baseR4/Appointment'
>>> r = requests.post(url, headers=headers, json=json_payload)
>>> r
<Response [201]>
>>> r.json()
{'resourceType': 'Appointment', 'id': '2261980', 'meta': {'versionId': '1', 'lastUpdated': '2022-03-25T23:40:42.621+00:00'}}
>>>
If you're already familiar with this API, then perhaps this might help. I suspect you then need to send another POST or PATCH request to another endpoint, using the ID returned in your first request to enter the relevant data.

Python API call - loop / nextpagetoken

I'm brand new to python and api as well.
I'm trying to use a endpoint we have at work.
We have an API we are using a lot, we also have an UI. But using the UI we can only extract 10.000 records at the time.
There is no limit on the api.
I have found a small piece of code - but i need to add a nextpagetoken.
My code looks like this:
login_url = 'https://api.ubsend.io/v1/auth/login'
username = 'xxxxx'
password = 'xxxxx'
omitClaims = "true"
session = requests.Session()
session.headers['Accept'] = "application/json; charset=UTF-8"
response = session.post(
login_url,
json={'username': username, 'password': password},
headers={'VERSION': '3'},
)
response.raise_for_status()
response_data = response.json()
print(response_data)
This gives me the AccessToken.
Then I call:
getevents = 'https://api.ubsend.io/v1/reporting/shipments?'
data ={'client_id': 13490, 'created_after': '2020-05-01T00:00', 'created_before': '2021-05-02T00:00'} req.prepare_url(getevents, data)
events = requests.get(req.url, headers={'Authorization' : 'Bearer ' + response_data['accessToken'], Content-Type': 'application/json'})
events.json()
Which returns:
'nextPageToken': 'NjA4ZDc3YzNkMjBjODgyYjBhMWVkMTVkLDE2MTk4ODM5NzA3MDE='}
So I want to loop my script - until nextPageToken is blank ....
Any thoughts?
Edit thanks for the update. I think this might be the solution we're looking for. You might have to do some poking around to figure out exactly what the name of the page_token URL parameter should be.
has_next = True
getevents = 'https://api.ubsend.io/v1/reporting/shipments?'
token = None
while has_next:
data ={'client_id': 13490, 'created_after': '2020-05-01T00:00', 'created_before': '2021-05-02T00:00'}
if token:
# I don't know the proper name for this URL parameter.
data['page_token'] = token
req.prepare_url(getevents, data)
events = requests.get(req.url, headers={'Authorization' : 'Bearer ' + response_data['accessToken'], Content-Type: 'application/json'})
token = events.json().get('nextPageToken')
if not token:
has_next = False
I made a slight typo. It should be events.json().get('nextPageToken') I believe.
Let me know if this works.

Python passing a vaiable into payload 400 error

Hi I'm trying to loop through some values in a for loop in python3.8 (lambda) to create a few bitbucket pipline variables. To start with, just to prove my code works i run the following which goes through a loop of 2 values, creates a variable called "acc" and gives a 201 response and then for the second time in the loop the variable acc already exists so it returns a 409 which is great as i expect that
accounts=['123', '12345']
for acc in accounts:
http = urllib3.PoolManager()
payload='{"key": "acc","value":"apply","secured":"false"}'
data = payload
url = 'https://api.bitbucket.org/2.0/repositories/<workspace>/<project>/pipelines_config/variables/'
headers = urllib3.util.make_headers(basic_auth='user' + ':' + 'Password')
print(headers)
headers['Content-type'] = 'application/json'
print(headers)
r = http.request('POST', url, headers=headers, body=data)
now if i change "acc" in the payload to actually be the variable im passing in it fails with a 400 error.
accounts=['123', '12345']
for **acc** in accounts:
http = urllib3.PoolManager()
payload='{"key": "'+ acc +'","value":"apply","secured":"false"}'
data = payload
url = 'https://api.bitbucket.org/2.0/repositories/<workspace>/<project>/pipelines_config/variables/'
headers = urllib3.util.make_headers(basic_auth='user' + ':' + 'Password')
print(headers)
headers['Content-type'] = 'application/json'
print(headers)
r = http.request('POST', url, headers=headers, body=data)
i've tried nearly every combination i can think of including escaping / using double single quotes etc but gett no where.
how can i pass a variable from a loop into this payload ?
Maybe i'm just not getting it, but why don't you simply use your acc variable instead of making it a string in your first example?
accounts=['123', '12345']
for acc in accounts:
http = urllib3.PoolManager()
payload='{"key": "'+ acc +'","value":"apply","secured":"false"}'
data = payload
...
I've just figured it out after hours of trying.
BitBucket doesn't allow variables to begin with numbers.
would have been nice if their documentation advised this of course !

Creating an Epic from a Github Issue with Zenhub API

This is a follow up question to How to set an issue pipeline with zenhub.
I'm attempting to convert an issue to an epic in a Python script. I can convert the issue to an Epic, but I get an error when I attempt to add issues when creating the epic.
This works:
zenhub_headers = {"X-Authentication-Token": "%s" % token}
target_zh_issues_url = '%s/p1/repositories/%d/issues' % (zh_api_endpoint, target_repo_id)
params = {}
response = requests.post(target_zh_issues_url + '/%s/convert_to_epic' % issue, headers=zenhub_headers, data=params)
The code also works when I set params = {"issues":[]}
But when I attempt to add an issue with params = {"issues": [{"repo_id": 280565, "issue_number": 17}]}
I get a 400 error, b'{"message":"Invalid Field for issues: [object Object],[object Object]"}'
I then tried using the /update_issues API to add issues to the epics I'd created.
target_zh_epics_url = '%s/p1/repositories/%d/epics' % (zh_api_endpoint, target_repo_id)
params = {"add_issues": [{"repo_id": 280565, "issue_number": 17}]}
response = requests.post(target_zh_epics_url + '/%s/update_issues' % issue, headers=zenhub_headers, data=params)
This resulted in a 400 error, b'{"message":"Invalid Field for addIssues: repo_id,issue_number"}'. Those fields are as described in the API doc.
I got this to work by adding 'Content-Type': 'application/json' to my headers and dumping the JSON body to a string, params = json.dumps({"issues": [{"repo_id": 280565, "issue_number": 17}]})
My code now looks like:
zenhub_headers = {"X-Authentication-Token": "%s" % token, 'Content-Type': 'application/json'}
target_zh_issues_url = '%s/p1/repositories/%d/issues' % (zh_api_endpoint, target_repo_id)
params = json.dumps({"issues": [{"repo_id": 280565, "issue_number": 17}]})
response = requests.post(target_zh_issues_url + '/%s/convert_to_epic' % issue, headers=zenhub_headers, data=params)
Though I'm not sure why the call with a body of unstringified {"issues":[]} was succeeding.

Categories