We Keep Coding

Locust load test a "not secure" site

I would like to test a site with APIs that at this stage of development is classified as "not secure".
In the Locust documentation is said that:
Safe mode:
The HTTP client is configured to run in safe_mode. What this does is that any request that fails due to a connection error, timeout, or similar will not raise an exception, but rather return an empty dummy Response object. The request will be reported as a failure in User’s statistics. The returned dummy Response’s content attribute will be set to None, and its status_code will be 0.
I would like to know if there is a configuration that let you disable this option.
Thank you for your time

safe_mode has nothing to do with SSL/TLS, it refers to requests not throwing an exception when the request fails (for whatever reason).
To make HttpUser/requests ignore SSL issues, add verify=False to your call.
——
You're looking at a very old Locust documentation, right? I dont think the text you mention has been there for quite a while.
safe_mode has been removed from requests (it used to look like this https://github.com/psf/requests/pull/604) but the same behaviour is implemented by Locust here https://github.com/locustio/locust/blob/51b1d5038a2be6e2823b2576c4436f2ff9f7c7c2/locust/clients.py#L195

Properly catch boto3 Errors

I am developing a django app which communicates with several Amazon Web Services.
So far I am having trouble dealing with and catching exceptions thrown by the boto3 client. What I am doing seems unnecessarily tedious:
Example:
client = boto3.client('sns')
client.create_platform_endpoint(PlatformApplicationArn=SNS_APP_ARN, Token=token)
this might throw an botocore.errorfactory.InvalidParameterException if e.g. the token is bad.
client.get_endpoint_attributes(EndpointArn=endpoint_arn)
might throw an botocore.errorfactory.NotFoundException.
First, I can't find these Errors anywhere in code, so they are probably generated somewhere. Bottom line: I can't import it and catch it as usual.
Second, I found one way to catch the error here using:
try:
# boto3 stuff
except botocore.exceptions.ClientError as e:
if e.response['Error']['Code'] == 'NotFound':
# handle exception
else:
raise e
But I have to remove the Exception part of the error name. Seems very random and I have no clue whether I would remove the Error in botocore.exceptions.ParamValidationError if I wanted to catch that one. So it's hard to generalize.
Another way to catch the error is using the boto3 client object I got:
try:
# boto3 stuff
except client.exceptions.NotFoundException as e:
# handle exception
This seems the cleanest way so far. But I don't always have the boto3 client object at hand where I want to catch the error. Also I am still only trying things out, so it's mostly guess work.
Does anybody know how boto3 errors are supposed to be handled?
Or can point me towards some coherent documentation which mentions the errors above? Thanks

You've summarized the situation well. The old boto had a simple hardcoded approach to supporting AWS APIs. boto3, in what appears to be an attempt to reduce the overhead of keeping Python client synced with evolving features on the various apis, has been more squishy around exceptions, so the ClientError approach you outlined above used to be the canonical way.
In 2017 they introduced the second mechanism you highlight: 'modeled' exceptions available on the client.
I am not familiar with SNS but in my experience with other AWS products, the ClientError naming matches up with the HTTP apis, which tend to be well documented. So I would start here: https://docs.aws.amazon.com/sns/latest/api/Welcome.html
It looks like the new-style modeled exceptions are generated from service definition files that live in botocore module. I can't find any documentation about it, but go browse around the AWS service models in https://github.com/boto/botocore/tree/master/botocore/data.
Also, it's good to know that if you are not (in contrast to OP's code) dealing directly with the low-level client, but instead are using a high-level AWS ServiceResource object, a low-level client is still easily available at my_service_resource.meta.client so you can handle exceptions like this:
try:
my_service_resource.do_stuff()
except my_service_resource.meta.client.exceptions.NotFoundException as e:
# handle exception

Use Boto3 exceptions: https://www.programcreek.com/python/example/97944/boto3.exceptions
client = boto3.client('logs')
try:
client.create_log_group(logGroupName=LOG_GROUP)
except client.exceptions.ResourceAlreadyExistsException:
pass

Flask restful - Exception handling traceback?

I am using Flask Restful to build an API. I have a number of model classes with methods that may raise custom exceptions (for example: AuthFailed exception on my User model class). I am using the custom error handling, documented here, to handle this (so that when auth fails, an appropriate response is sent). So far so good. However, I notice that when the exception is raised although the correct response JSON and status is sent back, I still get a traceback which is not ideal. Usually, if I handle an error (outside of flask) with a try-except block, the except can catch the error and handle it (preventing the traceback). So what is the correct approach here? Am I misunderstanding how to use the errors feature?

Unfortunately for you, it is handled this way "by design" of the Flask-RESTful APIs errors functionality.
The exceptions which are thrown are logged and the corresponding response defined in the errors dict is returned.
However, you can change the level of log output by modifying the log level of Flask's logger like this:
app = Flask(__name__)
app.logger.setLevel(logging.CRITICAL)
I think you would actually have to set it to CRITICAL because these errors are still getting logged even on log level ERROR as far as I know.
Furthermore, both Flask and Flask-RESTful are open-source. That being said, after looking at the code I found the function of a Flask app that is responsible for adding the exception traceback to the log (Flask version 0.11.1).
Of course you could just create your own App class (extending the original class of Flask) which overrides this method (or a caller of it) and does something else instead. However, you should be careful when updating your Flask version if you make use of undocumented stuff like this.

Softlayer API -- exception happened when calling softlayer api

I got an exception: "TransportError: TransportError(0): ('Connection aborted.', error(110, 'Connection timed out'))" when I called the api: Virtual_Guest::getBandwidthTotal.
It happened in this situation:
one same softlayer-api username and key
I called the functions concurrently thousands times at one moment.
So I do not know the exception happened due to "huge concurrent api callings" or just a network problem, or some other reasons.
If it causes since "huge concurrent api callings", here is an additional question:
As I says before that I called with one same username and key, if I calls concurrently with different username and key, will this exception happen as well?

The timeout errors are usually generated when the client is waiting a response of the API, this situation is documented here, in your case you can try to increase the timeout of your client, if you are using the Softlayer Python client please see the documentation to increase the timeout here, and aslo please review that you network connection is fine.
Regards

There is a limit on the number of API calls that can be made by an account per second. I believe this limit is per username, however I would not recommend using a bunch of different users to get around this limit.
My suggestion would be to use an objectMask to get as much data as possible in one API call, instead of making numerous api calls.
Instead of calling Virtual_Guest::getBandwidthTotal on every virtual guest on your account, you could call
SoftLayer_Account::getVirtualGuests(mask="mask[inboundPrivateBandwidthUsage,inboundPublicBandwidthUsage,outboundPrivateBandwidthUsage,outboundPublicBandwidthUsage]")
You might also need to use result Limits so that one big call doesn't time out as well.

Which exception should I raise for a REST API error?

I am writing a simple API client in Python and I am wondering what exception should I raise if the remote server is not happy.
The API itself is very poorly documented (I don't know all the possible error messages so I can't just define custom classes for all of them), I am letting Requests handle HTTP-level errors which raises HTTPError in those cases, but what should I raise if the server just supplies an error key in the JSON response?
I am currently using Exception but that feels quite broad, I'd like to know whether there's a better alternative.
Regards.

Yes, just raising Exception is likely too broad.
Usually a module should define his own base exception class:
class MyAPIClientError(Exception):
pass
And then you can subclass that:
class RemoteServerNotHappyError(MyAPIClientError):
pass
RemoteServerNotHappyError should probably mention something about what the server's json returned that was not expected by your API client class. Add in the relevant error messages as you see fit.
This means users of your library can catch these specific exceptions (which they might know how to handle), without having to catch every exception (they surely don't know how to handle every possible failure mode).