I tried to fetch list of API Keys created in my aws account. There are 50 Api keys. I want to list 10 Api Keys per request. Here is the code snippet.
import boto3
client = boto3.client('apigateway')
response = client.get_api_keys(
limit=10,
position="1",
)
print response
When I run the script, It is showing an error,
Traceback (most recent call last):
File "test.py", line 7, in <module>
position="1",
File "C:\Users\Murthy\workspace\my_project\venv\lib\site-packages\botocore\client.py", line 310, in _api_call
return self._make_api_call(operation_name, kwargs)
File "C:\Users\Murthy\workspace\my_project\venv\lib\site-packages\botocore\client.py", line 599, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.BadRequestException: An error occurred (BadRequestException) when calling the GetApiKeys operation: Invalid position parameter
Am I doing wrong ?
Also maximum results per query is 500. How do we know total number of results to perform the pagination ?
The position variable acts as a token. It is to be used to get the next page of your request.
In the first call to retrieve the usage plans, the position should be set to null, then to get the next page, you have to assign the position the value of the position in the previous response. The position variable is more like a token.
import boto3
client = boto3.client('apigateway')
response = client.get_api_keys(
limit=10,
position=null,
)
print response
Your response would be something like this
{
"position": "vWQ3Uzc4bAY2YQ%3D%3D",
"items": [
{
"id": "fqewi1",
"name": "3rd Party",
"description": null,
"apiStages": [],
"throttle": {
"burstLimit": 2,
"rateLimit": 1.0
},
.
.
.
}
Your next request would be in this format
import boto3
client = boto3.client('apigateway')
response = client.get_api_keys(
limit=10,
position="vWQ3Uzc4bAY2YQ%3D%3D",
)
print response
This would return the next page. For the last page, the position would be null.
Related
using boto3 I am trying to get a object in a dynamodb table.
following this stack overflow post the correct syntax is
client = boto3.client('dynamodb')
response = client.get_item(TableName='Garbage_collector_table', Key={'topic':{'S':str(my_topic)}})
Not able to get_item from AWS dynamodb using python?
http://boto3.readthedocs.io/en/latest/reference/services/dynamodb.html
I have tried various iterations to get proper syntax my current is this
if event['hcpcs_codes'] != None:
# codes must be in numerical, alphabetical order
# multi codes should be seperated by a comma with no spaces
client = boto3.client('dynamodb')
payload = {
"HCPCSCodeSet": event['hcpcs_codes']
}
response = client.get_item(TableName="mce-hcpcs-associations-table-dev",
Key={'HCPCSCodeSet':{'S': payload}})
print('here comes the tacos brah')
print(response)
Im not sure what this thing wants. What is the proper syntax?
Invalid type for parameter Key.HCPCSCodeSet, value: tacoman, type: <class 'str'>, valid types: <class 'dict'>
Traceback (most recent call last):
File "/var/task/lambdafile.py", line 18, in lambda_handler
Key= payload)
File "/var/task/botocore/client.py", line 415, in _api_call
the primary key name for the dynamodb database is
Partition key
HCPCSCodeSet (String)
This code:
payload = {
"HCPCSCodeSet": event['hcpcs_codes']
}
response = client.get_item(TableName="mce-hcpcs-associations-table-dev",
Key={'HCPCSCodeSet':{'S': payload}})
Ends up trying to send the following to DynamoDB:
Key={'HCPCSCodeSet':{'S': {
"HCPCSCodeSet": event['hcpcs_codes']
}}}
Which is obviously not correct. It's unclear why you are building the payload object at all. You could just do this:
response = client.get_item(TableName="mce-hcpcs-associations-table-dev",
Key={'HCPCSCodeSet':{'S': event['hcpcs_codes']}})
I am working on a Braintree subscription search in order to retrieve customer IDs and subscription prices tied to these IDs. In my code I am following the suggestions from this post.
Here is an excerpt of my code:
gateway = braintree.BraintreeGateway(
braintree.Configuration(
environment=braintree.Environment.Production,
merchant_id= 'our_merchant_id',
public_key='our_public_key',
private_key='our_private_key'
)
)
subscriptions = gateway.subscription.search(
braintree.SubscriptionSearch.status.in_list(
braintree.Subscription.Status.Active,
braintree.Subscription.Status.PastDue,
braintree.Subscription.Status.Pending
)
)
result = {}
for subscription in subscriptions.items:
payment_method = gateway.payment_method.find(subscription.payment_method_token)
result[payment_method.customer_id] = subscription.price
"""do something with result """
This approach works fine in the BT sandbox and on small queries around 100 records. However, whenever I try to query for more than about 120 subscriptions the BT server responds consistently with a 504 error. I would like to query for around 5000 subscriptions at a time in production. Any suggestions?
Traceback:
Traceback (most recent call last):
File "BT_subscrAmount.py", line 22, in <module>
for subscription in subscriptions.items:
File "/home/karen/miniconda3/lib/python3.6/site-
packages/braintree/resource_collection.py", line 38, in items
for item in self.__method(self.__query, batch):
File "/home/karen/miniconda3/lib/python3.6/site-
packages/braintree/subscription_gateway.py", line 79, in __fetch
response = self.config.http().post(self.config.base_merchant_path() +
"/subscriptions/advanced_search", {"search": criteria})
File "/home/karen/miniconda3/lib/python3.6/site-
packages/braintree/util/http.py", line 56, in post
return self.__http_do("POST", path, Http.ContentType.Xml, params)
File "/home/karen/miniconda3/lib/python3.6/site-
packages/braintree/util/http.py", line 86, in __http_do
Http.raise_exception_from_status(status)
File "/home/karen/miniconda3/lib/python3.6/site-
packages/braintree/util/http.py", line 49, in raise_exception_from_status
raise UnexpectedError("Unexpected HTTP_RESPONSE " + str(status))
braintree.exceptions.unexpected_error.UnexpectedError: Unexpected
HTTP_RESPONSE 504
With the help from an amazing BT support engineer, we were finally able to figure out a solution to our problem.
The explanation:
When BT returns search results, they gather all associated data and return a large (serialized) response object to the client. In our case some of the subscriptions had a large number of associated transactions, which makes the process of fetching, serializing and returning them to the client slow. When making a request, the BT gateway returns pages in groups of 50 with a default timeout of 60 seconds. Any group with several large response objects will likely time out and produce a 504 error.
The solution:
To avoid this delay, instead of using the items method, we can use the id method from the ResourceCollections class and return just the ids of the objects. With the individual subscription id, we can fetch the individual subscription and only the attributes we need, like so:
subscriptions = gateway.subscription.search(
braintree.SubscriptionSearch.status.in_list(
braintree.Subscription.Status.Active
)
)
subscription_ids = subscriptions.ids
def find_customer_id(id):
subscription = gateway.subscription.find(id)
payment_method = gateway.payment_method.find(subscription.payment_method_token)
return (payment_method.customer_id, subscription.price)
Hope this will help someone else!
Split your queries into small batches (5000+ subscriptions to become 50 calls of 100 subscriptions) and then aggregate as you get the responses back. Many APIs have rate-limits and response limits hard-coded.
I'm Trying to list the users of my domain smartsheet to backup their data one by one.
For that i need to list them then for every id i retrieve user Home with ss.Home.list_all_content()
the first one is ok, but the second give me this error when i try to retrieve his home or just make a ss.Users.get_User(his id) see below error:
Traceback (most recent call last):
File "<pyshell#84>", line 1, in <module>
ss.Users.list_users(include_all=True)
File "D:\Users\maxime.champain\AppData\Local\Programs\Python\Python35-32\lib\site-packages\smartsheet\users.py", line 277, in list_users
response = self._base.request(prepped_request, expected, _op)
File "D:\Users\maxime.champain\AppData\Local\Programs\Python\Python35-32\lib\site-packages\smartsheet\smartsheet.py", line 218, in request
raise the_ex(native, str(native.result.code) + ': ' + native.result.message)
smartsheet.exceptions.ApiError: {"result": {"shouldRetry": false, "refId": null, "code": 5349, "recommendation": "Do not retry without fixing the problem. ", "statusCode": 400, "message": "You must agree to the Smartsheet User Agreement before using Smartsheet. These terms can be viewed the next time you log in to https://app.smartsheet.com from a desktop browser.", "name": "ApiError"}, "requestResponse": null}
Question
How can I simply bypass this error to continue the program.
The exception of this error is handled by the smartsheet api sdk but i don't know how to call it.
regards,
If you want to catch this exception and continue, you'll need to use the try and except keywords.
There are many tutorials on the web, here's one: https://docs.python.org/3/tutorial/errors.html#handling-exceptions
I'm trying to retrieve sheets with Users.list_org_sheets(include_all=True) from the Python SDK API.
With That I can see all the things I need :
lstorg = ss.Users.list_org_sheets(include_all=True)
for entry in lstorg.data:
print(entry.id)
print(entry.name)
ss is the smartsheet.Smartsheet(token).
So with that i can see all the sheets in my domain, but when i try to download them with get_sheet_as_excel(id, path) I got that error:
print(ss.Sheets.get_sheet('6157394402142084'))
Traceback (most recent call last):
File "<pyshell#7>", line 1, in <module>
print(ss.Sheets.get_sheet('6157394402142084')
File "E:\Python\lib\site-packages\smartsheet\sheets.py", line 525, in get_sheet
response = self._base.request(prepped_request, expected, _op)
File "E:\Python\lib\site-packages\smartsheet\smartsheet.py", line 218, in request
raise the_ex(native, str(native.result.code) + ': ' + native.result.message)
smartsheet.exceptions.ApiError: {"requestResponse": null, "result": {"code": 1006, "message": "Not Found", "name": "ApiError", "recommendation": "Do not retry without fixing the problem. ", "refId": null, "shouldRetry": false, "statusCode": 404}}
I guess that this error mean that he didn't find the file (Error 404) but I don't know why, just before i make the command to get a list of all sheets and then when i pick one of that (that i'm not owner with this token but the token is SysAdmin)
Thanks For the help
The situation you're describing is due to how permissions work in Smartsheet. Using the SysAdmin token, you can successfully get the list of all Sheets owned by members of the account, but the SysAdmin token will not allow you to access the contents of those sheets, unless that SysAdmin user has been explicitly granted access to the sheets. You're getting the "not found" error when you try to retrieve the sheet because the SysAdmin user hasn't explicitly been granted access to that sheet in Smartsheet.
To actually retrieve a sheet that's listed in the "List Org Sheets" response, you'll need to add the Assume-User header to the "Get Sheet" API request to impersonate the sheet owner. You can find documentation about the Assume-User header here: https://smartsheet-platform.github.io/api-docs/#http-headers.
I'm unable to use the AWS boto API to change name servers for my AWS Route53 Registered Domain. In the following Python code, I get
boto.exception.JSONResponseError: JSONResponseError: 400 Bad Request
{u'Message': u'Expected null', u'__type': u'SerializationException'}
even though I'm using the API as documented, passing a list of strings such as
['ns-705.awsdns-21.net', 'ns-1401.awsdns-24.org', 'ns-1107.awsdns-11.co.uk', 'ns-242.awsdns-75.com']
as the second argument.
How can I change nameservers from Python?
def createhz(domain=None, verbose=False):
"""Create a Hosted Zone for the specified domain and update nameservers for Route53 Registered Domain"""
r53 = boto.route53.connection.Route53Connection()
if r53.get_hosted_zone_by_name(domain + '.'):
print('WARNING: Hosted Zone for {} already exists.'.format(domain))
hz = r53.get_zone(domain + '.')
else:
if verbose:
print('Creating Hosted Zone for {}.'.format(domain))
hz = r53.create_zone(domain + '.')
nameservers = hz.get_nameservers()
if verbose:
print('Hosted Zone has nameservers:')
for ns in nameservers:
print(' {}'.format(ns))
registered_domains = boto.route53.domains.layer1.Route53DomainsConnection()
try:
registered_domains.get_domain_detail(domain)
if verbose:
print('Updating nameservers for Route53 Registered Domain.'.format(domain))
# THE FOLLOWING LINE FAILS
registered_domains.update_domain_nameservers(domain, nameservers)
except Exception as e:
if e.message == 'Domain not found.':
print('WARNING: No Route53 Registered Domain for {}.'.format(domain))
print('Set the nameservers at your domain registrar to:.'.format(domain))
for ns in nameservers:
print(' {}'.format(ns))
else:
raise e
return
Traceback (most recent call last):
File "manage.py", line 362, in <module>
manager.run()
File "/usr/local/lib/python2.7/site-packages/flask_script/__init__.py", line 412, in run
result = self.handle(sys.argv[0], sys.argv[1:])
File "/usr/local/lib/python2.7/site-packages/flask_script/__init__.py", line 383, in handle
res = handle(*args, **config)
File "/usr/local/lib/python2.7/site-packages/flask_script/commands.py", line 216, in __call__
return self.run(*args, **kwargs)
File "manage.py", line 336, in createhz
raise e
boto.exception.JSONResponseError: JSONResponseError: 400 Bad Request
{u'Message': u'Expected null', u'__type': u'SerializationException'}
The support for the Route53 domain functionality is pretty new in boto and not very well documented. By looking at the service API I see that the update_domain_servers method expects the nameservers parameter to be a list of dictionaries of the form:
[{"Name": "ns1.example.com"}, {"Name": "ns2.example.com"}]
There is also an optional key called GlueIps. It doesn't sound like that would be necessary for your case but you can find more information here
I just went through this myself so here's what I did (based on your example above).. btw, this is based on Boto v2, fyi:
The below line:
nameservers = hz.get_nameservers()
Actually returns a list of NSs, such as:
[u'ns-1069.awsdns-05.org.', u'ns-1755.awsdns-27.co.uk.', u'ns-399.awsdns-49.com.', u'ns-816.awsdns-38.net.']
BUT, when you want to run registered_domains.update_domain_nameservers(domain, nameservers) you have to pass in a JSON list with key values of "Name", such as:
[{u'Name': u'ns-1515.awsdns-61.org'}, {u'Name': u'ns-576.awsdns-08.net'}]
To convert, you can do something like this
ns = [dict(Name=pn) for pn in nameservers]
And then you can update the NS records:
registered_domains.update_domain_nameservers(domain, ns)
Hope this helps!
Sorry I am posting from 2 accounts - fixed!