I am using a simple boto3 script to retrieve a parameter from SSM param store in my aws account. The python script looks like below:
client = get_boto3_client('ssm', 'us-east-1')
try:
response = client.get_parameter(Name='my_param_name',WithDecryption=True)
except Exception as e:
logging.error("retrieve param error: {0}".format(e))
raise e
return response
If the given parameter is not available, I get a generic error in the response like below:
An error occurred (ParameterNotFound) when calling the GetParameter operation: Parameter my_param_name not found.
I have verified method signature from boto3 ssm docs. Related AWS API Docs confirms to return a 400 response when parameter does not exist in the param store.
My question is that how do I verify if the exception caught in the response is actually a 400 status code so that I can handle it accordingly.
You can try catching client.exceptions.ParameterNotFound:
client = get_boto3_client('ssm', 'us-east-1')
try:
response = client.get_parameter(Name='my_param_name',WithDecryption=True)
except client.exceptions.ParameterNotFound:
logging.error("not found")
You can look at the status via response['Error']['Code'], but since there are multiple reasons for a 400, I would recommend a better approach:
response = client.get_parameter(Name='my_param_name',WithDecryption=True)
if 'Parameters' not in response:
raise ValueError('Response did not contain parameters key')
else:
return response
Related
My service bus queue trigger will get triggered every time when a new file is landed in blob storage, and the trigger function will perform a post request to Airflow to trigger an airflow job, but doesn't matter if it is a good request with 200 or bad request 404, the trigger will mark as success automatically even when the airflow job does not get triggered, hence, we will lost that message and there is no way to retrieve it back.
I thought 2 solutions, firstly is catch error in Exception, secondly is manually failed the job if code != 200, but none of the solution works.
As we can see if the code, status = 404 and trigger function still marked succeeded. Where did I do wrong in my code? How could I make my logic work?
try:
result = requests.post(
f"http://localhost:8080/api/v1/dags/{dag_id}/dagRunss",
data = json.dumps(data),
headers = header,
auth = ("airflow", "airflow"))
logging.info(result.content.decode('utf-8'))
if result.status_code != 200:
raise ValueError('This is the exception you expect to handle')
except Exception as err:
logging.error(err)
I have modified your code with handling the response status code
try:
result = requests.post(
f"http://localhost:8080/api/v1/dags/{dag_id}/dagRunss",
data = json.dumps(data),
headers = header,
auth = ("airflow", "airflow"))
logging.info(result.content.decode('utf-8'))
except Exception as err:
logging.error(err)
# Check the status code and trigger your airflow job
if result.status_code == 200:
# do your steps further like trigger your airflow job
return "return your airflow job response "
# If status_code != 200 do your other jobs or not trigger the airflow job
else:
logging.error(f' Status Code: {result.status_code} ')
#here you will get other response status code like 40*,50*
return "return your not 200 response and dont call the airflow job"
else:
return " your airflow job / !=200 response"
if you got 404 response Status_Code, it will execute the code under first else: logging.error(f' Status Code: {result.status_code} '), return "return your not 200 response and dont call the airflow job".
Refer here for more information
I created a lambda function in AWS and apparently it is throwing an error. Here it is:
import json
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
import boto3
#Create a SSM Client to access parameter store
ssm = boto3.client('ssm')
def lambda_handler(event, context):
# TODO implement
#return {
# 'statusCode': 200,
# 'body': json.dumps('Hello from Lambda!')
#}
slack_message = {
'text' = f'Hello World'
}
#retrieve webhook url from parameter store
webhook_url = ssm.get_parameter(Name='slackwebhookurl', WithDecryption=True)
#make request to the API
req = Request(webhook_url['Parameter']['Value'],
json.dumps(slack_message).encode('utf-8'))
try:
response = urlopen(req)
response.read()
print("Messge posted to Slack")
except HTTPError as e:
print(f'Request failed: {e.code} {e.reason})
except URLError as e:
print(f'Server Connection failed: {e.reason})
It's triggered by an AWS SNS notification. It's supposed to grab a webhook url for a slack channel and then send the notification to Slack.
Can anyone see what the problem is?
If it's not obvious, can someone direct me to a tutorial on how to test AWS Lambda functions?
Thanks.
AWS Lambda has an in-built Test functionality. You can click the Test button and configure an input in case the function uses values from event.
Logs will be displayed within the Lambda console.
Also, ensure that the IAM Role associated with the AWS Lambda function has the AWSLambdaBasicExecutionRole permission policy so that it can write to CloudWatch Logs. Then, you can go to the Monitoring tab of the function and click View logs in CloudWatch to see past logs.
You can add print() statements in your code, which will appear in the Logs.
API http://admin.wechat.com/wiki/index.php?title=Transferring_Multimedia_Files
import requests
r='https://api.wechat.com/cgi-bin/token?grant_type=client_credential&appid=wx82c6ebdb6e33ad33&secret=c2861ec348b3c94087c4b64cbe166fbb' #credentials sharing no problem
a=(requests.get(r).json()['access_token'])
print(a)
params = (
('access_token', a),
('type', 'image'),
)
import os
files = {
'media': ('1.jpg', open('static/1.jpg', 'rb'),'image/jpg',),
}
print()
response = requests.post('http://file.api.wechat.com/cgi-bin/media/upload', params=params, files=files)
def uprint(x,file=None):
try:
pass
if x:
print(x.encode('cp65001',errors='backslashreplace').decode('cp1252'),file=file)
except Exception as e:
return f'{x}\n{e}'
def prin(*a):print(ascii(*a))
print(response.text,file=open('z.html','a',encoding="utf-8"))
print(response.headers)
Looks like you are using http for the upload call. I have seen this error returned from sites for this reason before.
The HTTP error 503 means "Service Unavailable". It is usually returned by a server when it is unable to handle the request due to a temporary overloading or maintenance of the server.
After checking the API documentation for wechat, I notided this:
This API must be used via HTTPS.
And then I noticed this in the Q&A:
Q: Which server should I send API requests to?
A: If you have an International Official Account, use api.wechat.com.
If you have a China Official Account, use api.weixin.qq.com.
So, in your case I think you need to use https, and the domain api.wechat.com, like this:
response = requests.post('https://api.wechat.com/cgi-bin/media/upload', params=params, files=files)
According to official doc, you can make batch request through Google API Client Libraries for Python. Here, there is an example
from apiclient.http import BatchHttpRequest
def list_animals(request_id, response, exception):
if exception is not None:
# Do something with the exception
pass
else:
# Do something with the response
pass
def list_farmers(request_id, response):
"""Do something with the farmers list response."""
pass
service = build('farm', 'v2')
batch = service.new_batch_http_request()
batch.add(service.animals().list(), callback=list_animals)
batch.add(service.farmers().list(), callback=list_farmers)
batch.execute(http=http)
Is there a way to access to the request in the callback. E.g., print the request (not the request_id) if there is an exception?
I ran into this same issue, as I needed the original request for using compute.instances().aggregatedList_next() function.
I ended up passing a unique request_id to batch.add() and created a dictionary to keep track of the original requests that were made. Then within the callback, I referenced that dictionary and pulled out the request by using the unique request_id.
So I did something along the following:
requests = {}
project = 'example-project'
batch = service.new_batch_http_request(callback=callback_callable)
new_request = compute.instances().aggregatedList(project=project)
requests[project] = new_request
batch.add(new_request, request_id=project)
Within the callable function:
def callback_callable(self, request_id, response, exception):
original_request = requests[request_id]
Note that the value passed to the request_id needs to be a string. If an int is passed it will raise a TypeError quote_from_bytes() expected bytes.
Hope this helps someone!
In case it helps anyone - I'm using google-api-python-client==2.65.0
and the call back is passed differently - it is specified like this:
def list_animals(request_id, response, exception=None):
if exception is None:
original_request = requests[request_id]
# process here...
batch = service.new_batch_http_request(callback=list_animals)
batch.add(service.animals().list())
batch.add(service.farmers().list())
batch.execute()
Using Python 2.7, Django on Google App Engine. I'm trying to do some simple URL checking, including checking a JSON data payload, and return a meaningful error to the user. What I have coded is basically this:
from django.core.exceptions import SuspiciousOperation
...
def check(self, request):
json_data = json.loads(request.body)
if not json_data:
raise SuspiciousOperation('Required JSON data not found in the POST request.')
...
But, when I test this in debug mode (DEBUG = True in settings.py) by omitting the JSON data, instead of returning a HTTP 400 as I expect from SuspiciousOperation, I get an HTTP 500 that contains my error message "Required JSON data not found in the POST request." The same thing occurs if I check for a valiud URL with URLValidator(): I can correctly test for a good or bad URL with the URLValidator(), but if I try to raise a custom message on a bad URL with SuspiciousOperation I get HTTP 500 instead of 400.
How can I return a meaningful error to my caller without the server error obfuscating everything when Debug is turned back off and crashing the process in the process? Is SuspiciousOperation not supported by GAE?
There was an issue raised about this on Django's bug tracker and it looks like it was fixed in 1.6 but not backported. Indeed, SuspiciousOperation is handled by a catch-all in 1.5.11 (django/django/core/handlers/base.py line 173):
except: # Handle everything else, including SuspiciousOperation, etc.
# Get the exception info now, in case another exception is thrown later.
signals.got_request_exception.send(sender=self.__class__, request=request)
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())