Hi I'm trying to loop through some values in a for loop in python3.8 (lambda) to create a few bitbucket pipline variables. To start with, just to prove my code works i run the following which goes through a loop of 2 values, creates a variable called "acc" and gives a 201 response and then for the second time in the loop the variable acc already exists so it returns a 409 which is great as i expect that
accounts=['123', '12345']
for acc in accounts:
http = urllib3.PoolManager()
payload='{"key": "acc","value":"apply","secured":"false"}'
data = payload
url = 'https://api.bitbucket.org/2.0/repositories/<workspace>/<project>/pipelines_config/variables/'
headers = urllib3.util.make_headers(basic_auth='user' + ':' + 'Password')
print(headers)
headers['Content-type'] = 'application/json'
print(headers)
r = http.request('POST', url, headers=headers, body=data)
now if i change "acc" in the payload to actually be the variable im passing in it fails with a 400 error.
accounts=['123', '12345']
for **acc** in accounts:
http = urllib3.PoolManager()
payload='{"key": "'+ acc +'","value":"apply","secured":"false"}'
data = payload
url = 'https://api.bitbucket.org/2.0/repositories/<workspace>/<project>/pipelines_config/variables/'
headers = urllib3.util.make_headers(basic_auth='user' + ':' + 'Password')
print(headers)
headers['Content-type'] = 'application/json'
print(headers)
r = http.request('POST', url, headers=headers, body=data)
i've tried nearly every combination i can think of including escaping / using double single quotes etc but gett no where.
how can i pass a variable from a loop into this payload ?
Maybe i'm just not getting it, but why don't you simply use your acc variable instead of making it a string in your first example?
accounts=['123', '12345']
for acc in accounts:
http = urllib3.PoolManager()
payload='{"key": "'+ acc +'","value":"apply","secured":"false"}'
data = payload
...
I've just figured it out after hours of trying.
BitBucket doesn't allow variables to begin with numbers.
would have been nice if their documentation advised this of course !
Related
Trying to find a way to iterate over the roleprivs and having issues getting to that level of the yaml from python.
testrole.yaml
info:
rolename: "testDeveloper"
desc: "Test Developer Role"
roletype: "user"
roleprivs:
admin-appliance:
name: "Administrate Appliance" # Informational Only Not used in code
description: "admin-appliance" # Informational Only Not used in code
code: "admin-appliance"
access: "full"
admin-backupSettings:
name: "Administrate Backup Settings" # Informational Only Not used in code
description: "admin-appliance" # Informational Only Not used in code
code: "admin-backupSettings"
access: "full"
I have a few different needs / use cases.
Part 1 of the script below - grab all the files in a directory and take the rolename, desc, and roletype and create a role.
Get the Role ID of the newly created role that was above.
HELP Needed - going back to the original yaml file and iterating over it and getting only the roleprivs..code and roleprivs..code --> role type would be something like admin-appliance. Keeping in mind that there are like 50 some odd features that need to be updated with the type of access.
The question:
How do i get the code and access in the yaml file into python variables?
def genericRoleCreate(baseURL, bearerToken):
print("initial")
files = glob.glob(ROLES_DIR)
logger.debug('Roles Dir '+ROLES_DIR)
for file in files:
yaml_file = file
logger.debug(yaml_file)
with open(yaml_file) as f:
try:
result=yaml.safe_load(f)
authority = result['info']['rolename']
desc = result['info']['desc']
roletype = result['info']['roletype']
url = baseURL+"/api/roles"
payload= json.dumps({"role":{"authority": authority, "description": desc, "roletype": roletype}})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
roleResult = requests.request("POST", url, verify=False, headers=headers, data=payload)
logger.debug(roleResult.text)
except yaml.YAMLError as exc:
logger.error(exc)
# Getting Role ID
try:
with open(yaml_file) as f:
result = yaml.safe_load(f)
authority = result['info']['rolename']
url = baseURL+"/api/roles?phrase="+authority
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
roleResult = requests.request("GET", url, verify=False, headers=headers )
#print(roleResult.text)
roleID = json.loads(roleResult.text)
role = roleID['roles'][0]['id']
#logger.debug(role)
logger.info("Get Role ID")
print(role)
#return role
#logger.debug("Role ID: "+role)
except Exception as e:
logger.error('Exception occurred', exc_info=True)
logger.error('Error getting roleID')
# Start Updating
#role = getRoleId(baseURL, bearerToken)
try:
with open(yaml_file) as f:
result = yaml.safe_load(f)
except Exception as e:
logger.error("Broken")
strRoleID = str(role)
url = baseURL+"/api/roles/"+strRoleID+"/update-permission"
#logger.debug(result)
keys = list(result.keys())
for features in keys:
#logger.debug(keys)
code = result[features]['code']
access = result[features]['access']
payload = json.dumps({
"permissionCode": code,
"access": access
})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
requests.request("PUT", url, verify=False, headers=headers, data=payload)
lets keep in mind i do know that i should be breaking that big nasty thing into multiple functions - i have it broken down in other areas - but compiling everything in a single function at the time.
I have been trying multiple iterations of how to get to the feature level. I have looked at many examples and can't seem to figure out how to drop to a level.
update 1
try:
with open(yaml_file, 'r') as f:
result = yaml.safe_load(f)
except Exception as e:
logger.error("Broken")
strRoleID = str(role)
url = baseURL+"/api/roles/"+strRoleID+"/update-permission"
#logger.debug(result)
keys = list(result['roleprivs'].keys())
#code2 = {roleprivs for roleprivs in result['roleprivs'].keys()}
#print(code2)
#return inventory, sites
for features in keys:
print(features)
The code above produces the output:
admin-appliance
admin-backupSettings
now the question is how do i go one step deeper in the chain and get code and access into a variable in python.
I ended up solving the problem after a few hours of testing and researching...The main problem i was encountering was how do i get to the next level after roleprivs. I could easily print the main elements under the roleprivs, but getting the code and access elements were a bit of a challenge.
I also kept running into an error where the indices needed to be an integer. This cleans up some of the keys that i was doing before and puts it into a one liner. Should help clean up a few for loops that i have been working with.
for k, v in result['roleprivs'].items():
print("Code "+result['roleprivs'][k]['code'])
print("Access: "+result['roleprivs'][k]['access'])
access = result['roleprivs'][k]['access']
code = result['roleprivs'][k]['code']
payload = json.dumps({
"permissionCode": code,
"access": access
})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
requests.request("PUT", url, verify=False, headers=headers, data=payload)
From the original code i may have multiple roles in the ./config/roles/ directory. I needed to make sure i can read all and iterate in a for loop for each one. This solved it for me.
Final output:
2021-10-13 01:25:50,212:84:logger:role:genericRoleCreate:DEBUG:./config/roles/testDeveloper.yaml
2021-10-13 01:25:50,487:110:logger:role:genericRoleCreate:INFO:Get Role ID
8
Code admin-appliance
Access: full
Code admin-backupSettings
Access: full
I'm brand new to python and api as well.
I'm trying to use a endpoint we have at work.
We have an API we are using a lot, we also have an UI. But using the UI we can only extract 10.000 records at the time.
There is no limit on the api.
I have found a small piece of code - but i need to add a nextpagetoken.
My code looks like this:
login_url = 'https://api.ubsend.io/v1/auth/login'
username = 'xxxxx'
password = 'xxxxx'
omitClaims = "true"
session = requests.Session()
session.headers['Accept'] = "application/json; charset=UTF-8"
response = session.post(
login_url,
json={'username': username, 'password': password},
headers={'VERSION': '3'},
)
response.raise_for_status()
response_data = response.json()
print(response_data)
This gives me the AccessToken.
Then I call:
getevents = 'https://api.ubsend.io/v1/reporting/shipments?'
data ={'client_id': 13490, 'created_after': '2020-05-01T00:00', 'created_before': '2021-05-02T00:00'} req.prepare_url(getevents, data)
events = requests.get(req.url, headers={'Authorization' : 'Bearer ' + response_data['accessToken'], Content-Type': 'application/json'})
events.json()
Which returns:
'nextPageToken': 'NjA4ZDc3YzNkMjBjODgyYjBhMWVkMTVkLDE2MTk4ODM5NzA3MDE='}
So I want to loop my script - until nextPageToken is blank ....
Any thoughts?
Edit thanks for the update. I think this might be the solution we're looking for. You might have to do some poking around to figure out exactly what the name of the page_token URL parameter should be.
has_next = True
getevents = 'https://api.ubsend.io/v1/reporting/shipments?'
token = None
while has_next:
data ={'client_id': 13490, 'created_after': '2020-05-01T00:00', 'created_before': '2021-05-02T00:00'}
if token:
# I don't know the proper name for this URL parameter.
data['page_token'] = token
req.prepare_url(getevents, data)
events = requests.get(req.url, headers={'Authorization' : 'Bearer ' + response_data['accessToken'], Content-Type: 'application/json'})
token = events.json().get('nextPageToken')
if not token:
has_next = False
I made a slight typo. It should be events.json().get('nextPageToken') I believe.
Let me know if this works.
1) I require a function which should publish a post with the given message and photo.
One can use the page ID and access tokens provided in self.page_id and self.page_access_token
def __init__(self):
self.page_id = Facebook.get_access_token('FACEBOOK_PAGE_ID')
self.page_access_token = Facebook.get_access_token('FACEBOOK_PAGE_ACCESS_TOKEN')
2) To find which API to hit, check out developer section of the Facebook API: https://developers.facebook.com/docs/graph-api/reference/page/photos/#example-2
3) The function should not return anything.
def publish_photo_msg(self, message, image_url):
#your code here
return
Please help.
python
params = (
('access_token', self.page_access_token),
)
Next, let us fix the data dictionary:
python
data = {
'url': image_url,
'caption': 'this is the caption'
}
Now, let’s proceed to the URL. There are many ways to set this. Here is one way:
python
url = 'https://graph.facebook.com/v5.0/' + self.page_id + '/photos?access_token=' + self.page_access_token
Since we have stored the access_token in the params tuple, let’s make use of it in the requests.post() call.
python
url = 'https://graph.facebook.com/v5.0/' + self.page_id + '/photos'
response = requests.post(url=url, params=params, data=data)
Lastly, you can also verify if your requests.post() call was successful by checking the value of the response variable:
python
print response.status_code, response.json()
For easy reference, here is the full implementation of the publish_photo_msg function with all the suggestions incorporated:
python
params = (
('access_token', self.page_access_token),
)
data = {
'url': image_url,
'caption': message
}
url = 'https://graph.facebook.com/v5.0/' + self.page_id + '/photos'
response = requests.post(url=url, params=params, data=data)
print(response.status_code, response.json())
In my API call defined below to retrieve the last 24 hrs of data, the normal request url would be:
https://api.foobar.com/data
That is why I have set the next_page parameter default to NULL.
However, sometimes the API will return a unique URL at the end of the json (such as https://api.foobar.com/data?page%237hfaj39), which indicates another page exists and another get_data request needs to be made to retrieve the remainder.
In that case, the {next_page} parameter will be set to whatever this unique url returned would be.
My problem is after adding the {next_page} parameter, the default get_data url somehow gets 4 unwanted characters - %7B%7D appended so that the request looks like
https://api.foobar.com/data%7B%7D and of course the API does not respond.
In UTF-8 encoding %7B%7D are two brackets {}
Why does this happen and what am I doing wrong here in terms of formatting? Using None in place of {} also does not work.
The code:
def make_request(url, params={}, headers={}):
r = requests.get(url, params=params, headers=headers)
print r.url
if(not r.status_code is 200):
print "Error access API" + r.text
exit()
return r.json()
def get_data(access_token, next_page={}):
end_time = int(round(time.time() * 1000))
start_time = end_time - (seconds_in_day * 1000)
headers = {'Authorization': 'Bearer ' + access_token, 'start_time': str(start_time), 'end_time': str(end_time)}
url = 'https://api.foobar.com/data{next_page}'.format(next_page=next_page)
return make_request(url, headers=headers)
Note: the API call works when the next_page parameter is removed
With next_page={}, you will get unexpected formatting results. If you try the following:
>>> '{}'.format({})
'{}'
As you can see, instead of the desired '', you get a string with two brackets. This is because:
>>> str({})
'{}'
A similar thing happens with None:
>>> '{}'.format(None)
'None'
>>> str(None)
'None'
To fix this, instead of next_page={}, try next_page='', because .format() will do this:
>>> '{}'.format('')
''
This is a follow up question to How to set an issue pipeline with zenhub.
I'm attempting to convert an issue to an epic in a Python script. I can convert the issue to an Epic, but I get an error when I attempt to add issues when creating the epic.
This works:
zenhub_headers = {"X-Authentication-Token": "%s" % token}
target_zh_issues_url = '%s/p1/repositories/%d/issues' % (zh_api_endpoint, target_repo_id)
params = {}
response = requests.post(target_zh_issues_url + '/%s/convert_to_epic' % issue, headers=zenhub_headers, data=params)
The code also works when I set params = {"issues":[]}
But when I attempt to add an issue with params = {"issues": [{"repo_id": 280565, "issue_number": 17}]}
I get a 400 error, b'{"message":"Invalid Field for issues: [object Object],[object Object]"}'
I then tried using the /update_issues API to add issues to the epics I'd created.
target_zh_epics_url = '%s/p1/repositories/%d/epics' % (zh_api_endpoint, target_repo_id)
params = {"add_issues": [{"repo_id": 280565, "issue_number": 17}]}
response = requests.post(target_zh_epics_url + '/%s/update_issues' % issue, headers=zenhub_headers, data=params)
This resulted in a 400 error, b'{"message":"Invalid Field for addIssues: repo_id,issue_number"}'. Those fields are as described in the API doc.
I got this to work by adding 'Content-Type': 'application/json' to my headers and dumping the JSON body to a string, params = json.dumps({"issues": [{"repo_id": 280565, "issue_number": 17}]})
My code now looks like:
zenhub_headers = {"X-Authentication-Token": "%s" % token, 'Content-Type': 'application/json'}
target_zh_issues_url = '%s/p1/repositories/%d/issues' % (zh_api_endpoint, target_repo_id)
params = json.dumps({"issues": [{"repo_id": 280565, "issue_number": 17}]})
response = requests.post(target_zh_issues_url + '/%s/convert_to_epic' % issue, headers=zenhub_headers, data=params)
Though I'm not sure why the call with a body of unstringified {"issues":[]} was succeeding.