I'm brand new to python and api as well.
I'm trying to use a endpoint we have at work.
We have an API we are using a lot, we also have an UI. But using the UI we can only extract 10.000 records at the time.
There is no limit on the api.
I have found a small piece of code - but i need to add a nextpagetoken.
My code looks like this:
login_url = 'https://api.ubsend.io/v1/auth/login'
username = 'xxxxx'
password = 'xxxxx'
omitClaims = "true"
session = requests.Session()
session.headers['Accept'] = "application/json; charset=UTF-8"
response = session.post(
login_url,
json={'username': username, 'password': password},
headers={'VERSION': '3'},
)
response.raise_for_status()
response_data = response.json()
print(response_data)
This gives me the AccessToken.
Then I call:
getevents = 'https://api.ubsend.io/v1/reporting/shipments?'
data ={'client_id': 13490, 'created_after': '2020-05-01T00:00', 'created_before': '2021-05-02T00:00'} req.prepare_url(getevents, data)
events = requests.get(req.url, headers={'Authorization' : 'Bearer ' + response_data['accessToken'], Content-Type': 'application/json'})
events.json()
Which returns:
'nextPageToken': 'NjA4ZDc3YzNkMjBjODgyYjBhMWVkMTVkLDE2MTk4ODM5NzA3MDE='}
So I want to loop my script - until nextPageToken is blank ....
Any thoughts?
Edit thanks for the update. I think this might be the solution we're looking for. You might have to do some poking around to figure out exactly what the name of the page_token URL parameter should be.
has_next = True
getevents = 'https://api.ubsend.io/v1/reporting/shipments?'
token = None
while has_next:
data ={'client_id': 13490, 'created_after': '2020-05-01T00:00', 'created_before': '2021-05-02T00:00'}
if token:
# I don't know the proper name for this URL parameter.
data['page_token'] = token
req.prepare_url(getevents, data)
events = requests.get(req.url, headers={'Authorization' : 'Bearer ' + response_data['accessToken'], Content-Type: 'application/json'})
token = events.json().get('nextPageToken')
if not token:
has_next = False
I made a slight typo. It should be events.json().get('nextPageToken') I believe.
Let me know if this works.
Related
I get the Auth0 management access token by:
conn = http.client.HTTPSConnection("{env.get("AUTH0_DOMAIN")}")
payload = "{\"client_id\":\"id\",\"client_secret\":\"secret\",\"audience\":\"https://{env.get("AUTH0_DOMAIN")}/api/v2/\",\"grant_type\":\"client_credentials\"}"
headers = { 'content-type': "application/json" }
conn.request("POST", "/oauth/token", payload, headers)
res = conn.getresponse()
data = res.read()
token_json = data.decode('utf-8').replace("'", '"')
token = json.loads(token_json)
AUTH0_ACCESS_TOKEN = token["access_token"]
which is successful and I get the management API access token.
The problem arises when I try to get user information during callback:
#app.route("/callback", methods=["GET", "POST"])
def callback():
token = oauth.auth0.authorize_access_token()
session["user"] = token
id = session['user']['userinfo']['sub']
payload = ''
conn = http.client.HTTPConnection("{env.get("AUTH0_DOMAIN")}")
headers = { 'authorization': "Bearer {}".format(AUTH0_ACCESS_TOKEN)}
conn.request("GET", f"/api/v2/users/{id}", payload, headers=headers)
res = conn.getresponse()
data = res.read()
user_json = data.decode('utf-8').replace("'", '"')
user_data = json.loads(user_json)
The error occurs with the last line. json.loads yields an error: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
When I checked the user_json, I find that it is empty meaning I didn't get a response from the Management API.
What am I doing wrong? Is there a formatting mistake? Am I not using the right user_id? For ID, I tried both session['user']['userinfo']['sub'] and session['user']['userinfo']['sub'].split('|', 1)[1] to exclude the prefix in the ID but still getting the same error.
I think I figured out what was wrong with getting user information in callback.
This
conn = http.client.HTTPConnection("{env.get("AUTH0_DOMAIN")}")
should be instead
conn = http.client.HTTPSConnection("{env.get("AUTH0_DOMAIN")}")
The quickstart guide in Auth0's console for a Machine to Machine app said to use HTTPConnection so that might need to be fixed.
Trying to find a way to iterate over the roleprivs and having issues getting to that level of the yaml from python.
testrole.yaml
info:
rolename: "testDeveloper"
desc: "Test Developer Role"
roletype: "user"
roleprivs:
admin-appliance:
name: "Administrate Appliance" # Informational Only Not used in code
description: "admin-appliance" # Informational Only Not used in code
code: "admin-appliance"
access: "full"
admin-backupSettings:
name: "Administrate Backup Settings" # Informational Only Not used in code
description: "admin-appliance" # Informational Only Not used in code
code: "admin-backupSettings"
access: "full"
I have a few different needs / use cases.
Part 1 of the script below - grab all the files in a directory and take the rolename, desc, and roletype and create a role.
Get the Role ID of the newly created role that was above.
HELP Needed - going back to the original yaml file and iterating over it and getting only the roleprivs..code and roleprivs..code --> role type would be something like admin-appliance. Keeping in mind that there are like 50 some odd features that need to be updated with the type of access.
The question:
How do i get the code and access in the yaml file into python variables?
def genericRoleCreate(baseURL, bearerToken):
print("initial")
files = glob.glob(ROLES_DIR)
logger.debug('Roles Dir '+ROLES_DIR)
for file in files:
yaml_file = file
logger.debug(yaml_file)
with open(yaml_file) as f:
try:
result=yaml.safe_load(f)
authority = result['info']['rolename']
desc = result['info']['desc']
roletype = result['info']['roletype']
url = baseURL+"/api/roles"
payload= json.dumps({"role":{"authority": authority, "description": desc, "roletype": roletype}})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
roleResult = requests.request("POST", url, verify=False, headers=headers, data=payload)
logger.debug(roleResult.text)
except yaml.YAMLError as exc:
logger.error(exc)
# Getting Role ID
try:
with open(yaml_file) as f:
result = yaml.safe_load(f)
authority = result['info']['rolename']
url = baseURL+"/api/roles?phrase="+authority
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
roleResult = requests.request("GET", url, verify=False, headers=headers )
#print(roleResult.text)
roleID = json.loads(roleResult.text)
role = roleID['roles'][0]['id']
#logger.debug(role)
logger.info("Get Role ID")
print(role)
#return role
#logger.debug("Role ID: "+role)
except Exception as e:
logger.error('Exception occurred', exc_info=True)
logger.error('Error getting roleID')
# Start Updating
#role = getRoleId(baseURL, bearerToken)
try:
with open(yaml_file) as f:
result = yaml.safe_load(f)
except Exception as e:
logger.error("Broken")
strRoleID = str(role)
url = baseURL+"/api/roles/"+strRoleID+"/update-permission"
#logger.debug(result)
keys = list(result.keys())
for features in keys:
#logger.debug(keys)
code = result[features]['code']
access = result[features]['access']
payload = json.dumps({
"permissionCode": code,
"access": access
})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
requests.request("PUT", url, verify=False, headers=headers, data=payload)
lets keep in mind i do know that i should be breaking that big nasty thing into multiple functions - i have it broken down in other areas - but compiling everything in a single function at the time.
I have been trying multiple iterations of how to get to the feature level. I have looked at many examples and can't seem to figure out how to drop to a level.
update 1
try:
with open(yaml_file, 'r') as f:
result = yaml.safe_load(f)
except Exception as e:
logger.error("Broken")
strRoleID = str(role)
url = baseURL+"/api/roles/"+strRoleID+"/update-permission"
#logger.debug(result)
keys = list(result['roleprivs'].keys())
#code2 = {roleprivs for roleprivs in result['roleprivs'].keys()}
#print(code2)
#return inventory, sites
for features in keys:
print(features)
The code above produces the output:
admin-appliance
admin-backupSettings
now the question is how do i go one step deeper in the chain and get code and access into a variable in python.
I ended up solving the problem after a few hours of testing and researching...The main problem i was encountering was how do i get to the next level after roleprivs. I could easily print the main elements under the roleprivs, but getting the code and access elements were a bit of a challenge.
I also kept running into an error where the indices needed to be an integer. This cleans up some of the keys that i was doing before and puts it into a one liner. Should help clean up a few for loops that i have been working with.
for k, v in result['roleprivs'].items():
print("Code "+result['roleprivs'][k]['code'])
print("Access: "+result['roleprivs'][k]['access'])
access = result['roleprivs'][k]['access']
code = result['roleprivs'][k]['code']
payload = json.dumps({
"permissionCode": code,
"access": access
})
headers = {'Content-Type': 'application/json','Authorization': 'Bearer ' +bearerToken}
requests.request("PUT", url, verify=False, headers=headers, data=payload)
From the original code i may have multiple roles in the ./config/roles/ directory. I needed to make sure i can read all and iterate in a for loop for each one. This solved it for me.
Final output:
2021-10-13 01:25:50,212:84:logger:role:genericRoleCreate:DEBUG:./config/roles/testDeveloper.yaml
2021-10-13 01:25:50,487:110:logger:role:genericRoleCreate:INFO:Get Role ID
8
Code admin-appliance
Access: full
Code admin-backupSettings
Access: full
Hi I'm trying to loop through some values in a for loop in python3.8 (lambda) to create a few bitbucket pipline variables. To start with, just to prove my code works i run the following which goes through a loop of 2 values, creates a variable called "acc" and gives a 201 response and then for the second time in the loop the variable acc already exists so it returns a 409 which is great as i expect that
accounts=['123', '12345']
for acc in accounts:
http = urllib3.PoolManager()
payload='{"key": "acc","value":"apply","secured":"false"}'
data = payload
url = 'https://api.bitbucket.org/2.0/repositories/<workspace>/<project>/pipelines_config/variables/'
headers = urllib3.util.make_headers(basic_auth='user' + ':' + 'Password')
print(headers)
headers['Content-type'] = 'application/json'
print(headers)
r = http.request('POST', url, headers=headers, body=data)
now if i change "acc" in the payload to actually be the variable im passing in it fails with a 400 error.
accounts=['123', '12345']
for **acc** in accounts:
http = urllib3.PoolManager()
payload='{"key": "'+ acc +'","value":"apply","secured":"false"}'
data = payload
url = 'https://api.bitbucket.org/2.0/repositories/<workspace>/<project>/pipelines_config/variables/'
headers = urllib3.util.make_headers(basic_auth='user' + ':' + 'Password')
print(headers)
headers['Content-type'] = 'application/json'
print(headers)
r = http.request('POST', url, headers=headers, body=data)
i've tried nearly every combination i can think of including escaping / using double single quotes etc but gett no where.
how can i pass a variable from a loop into this payload ?
Maybe i'm just not getting it, but why don't you simply use your acc variable instead of making it a string in your first example?
accounts=['123', '12345']
for acc in accounts:
http = urllib3.PoolManager()
payload='{"key": "'+ acc +'","value":"apply","secured":"false"}'
data = payload
...
I've just figured it out after hours of trying.
BitBucket doesn't allow variables to begin with numbers.
would have been nice if their documentation advised this of course !
1) I require a function which should publish a post with the given message and photo.
One can use the page ID and access tokens provided in self.page_id and self.page_access_token
def __init__(self):
self.page_id = Facebook.get_access_token('FACEBOOK_PAGE_ID')
self.page_access_token = Facebook.get_access_token('FACEBOOK_PAGE_ACCESS_TOKEN')
2) To find which API to hit, check out developer section of the Facebook API: https://developers.facebook.com/docs/graph-api/reference/page/photos/#example-2
3) The function should not return anything.
def publish_photo_msg(self, message, image_url):
#your code here
return
Please help.
python
params = (
('access_token', self.page_access_token),
)
Next, let us fix the data dictionary:
python
data = {
'url': image_url,
'caption': 'this is the caption'
}
Now, let’s proceed to the URL. There are many ways to set this. Here is one way:
python
url = 'https://graph.facebook.com/v5.0/' + self.page_id + '/photos?access_token=' + self.page_access_token
Since we have stored the access_token in the params tuple, let’s make use of it in the requests.post() call.
python
url = 'https://graph.facebook.com/v5.0/' + self.page_id + '/photos'
response = requests.post(url=url, params=params, data=data)
Lastly, you can also verify if your requests.post() call was successful by checking the value of the response variable:
python
print response.status_code, response.json()
For easy reference, here is the full implementation of the publish_photo_msg function with all the suggestions incorporated:
python
params = (
('access_token', self.page_access_token),
)
data = {
'url': image_url,
'caption': message
}
url = 'https://graph.facebook.com/v5.0/' + self.page_id + '/photos'
response = requests.post(url=url, params=params, data=data)
print(response.status_code, response.json())
Okay so I'm using code very similar to this (https://gist.github.com/metadaddy-sfdc/1374762)
to get authentication token and do simple query's using the libur2 for the rest api in python for a sales force database, but when I tried to follow the instructions which were given in this answer How to make HTTP DELETE method using urllib2?,
I cannot get it to work so that I can use delete, both codes use liburl but they seem to be in different format, so that I don't know how to apply the solution offered on stack exchange, to my code, as you can tell I am a beginner so any help would be greatly appreciated
edit:
here is the code I'm using with keys/passwords blanked
import urllib
import urllib2
import json
import pprint
import re
import subprocess
def authorise():
consumer_key = '**********************'
consumer_secret = '**************'
username = '***********'
password = '*****************'
login_server = 'https://login.salesforce.com'
token_url = login_server+'/services/oauth2/token'
params = urllib.urlencode({
'grant_type': 'password',
'client_id': consumer_key,
'client_secret': consumer_secret,
'username': username,
'password': password
})
data = urllib2.urlopen(token_url, params).read()
oauth = json.loads(data)
return oauth
def country_id_query(params):
query_url = oauth['instance_url']+'/services/data/v23.0/query?%s' % params
headers = {
'Authorization': 'OAuth '+oauth['access_token']
}
req = urllib2.Request(query_url, None, headers)
data = urllib2.urlopen(req).read()
result = json.loads(data)
id = result['records'][0]['Id']
return id
oauth = authorise()
token = oauth['access_token']
print "\ntoken is = " + token
params = urllib.urlencode({
'q': 'SELECT id from Country__c WHERE name = \'A New Found Land\''
})
id = country_id_query(params)
print "\nCountry id is "+id + "\n"
I am looking to find out what I need to add to this to get DELETE working
Okay, found the solution to above for anyone with a similar problem:
def delete_country(id):
query_url = oauth['instance_url']+'/services/data/v23.0/sobjects/Country__c/%s' % id + '/'
headers = {
'Authorization': 'OAuth '+oauth['access_token']
}
opener = urllib2.build_opener(urllib2.HTTPHandler)
req = urllib2.Request(query_url, None, headers)
req.get_method = lambda: 'DELETE' # creates the delete method
url = urllib2.urlopen(req) # deletes database item