Error occurred when I try download data from s3 - python

It is full log
File "/home/workspace/viv_data_handler/s3manager/s3_downloader.py", line 18, in download_file
self.resource.meta.client.download_file(bucket, key, dest)
File "/usr/local/lib/python3.5/site-packages/boto3/s3/inject.py", line 172, in download_file
extra_args=ExtraArgs, callback=Callback)
File "/usr/local/lib/python3.5/site-packages/boto3/s3/transfer.py", line 307, in download_file
future.result()
File "/usr/local/lib/python3.5/site-packages/s3transfer/futures.py", line 73, in result
return self._coordinator.result()
File "/usr/local/lib/python3.5/site-packages/s3transfer/futures.py", line 233, in result
raise self._exception
File "/usr/local/lib/python3.5/site-packages/s3transfer/tasks.py", line 255, in _main
self._submit(transfer_future=transfer_future, **kwargs)
File "/usr/local/lib/python3.5/site-packages/s3transfer/download.py", line 353, in _submit
**transfer_future.meta.call_args.extra_args
File "/usr/local/lib/python3.5/site-packages/botocore/client.py", line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.5/site-packages/botocore/client.py", line 599, in _make_api_call
operation_model, request_dict)
File "/usr/local/lib/python3.5/site-packages/botocore/endpoint.py", line 148, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/local/lib/python3.5/site-packages/botocore/endpoint.py", line 177, in _send_request
success_response, exception):
File "/usr/local/lib/python3.5/site-packages/botocore/endpoint.py", line 273, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/usr/local/lib/python3.5/site-packages/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "/usr/local/lib/python3.5/site-packages/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "/usr/local/lib/python3.5/site-packages/botocore/retryhandler.py", line 183, in __call__
if self._checker(attempts, response, caught_exception):
File "/usr/local/lib/python3.5/site-packages/botocore/retryhandler.py", line 251, in __call__
caught_exception)
File "/usr/local/lib/python3.5/site-packages/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/usr/local/lib/python3.5/site-packages/botocore/retryhandler.py", line 317, in __call__
caught_exception)
File "/usr/local/lib/python3.5/site-packages/botocore/retryhandler.py", line 223, in __call__
attempt_number, caught_exception)
File "/usr/local/lib/python3.5/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "/usr/local/lib/python3.5/site-packages/botocore/endpoint.py", line 222, in _get_response
proxies=self.proxies, timeout=self.timeout)
File "/usr/local/lib/python3.5/site-packages/botocore/vendored/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.5/site-packages/botocore/vendored/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
botocore.vendored.requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:719)
Hello I am developing a program using S3 and I have a problem
botocore.vendored.requests.exceptions.SSLError error occurred when I try download data using boto3 from S3
It works when another server use the same code. and It works when use aws cli on the same server(boto3 failed server).
I don't know how to solve it.
please help me. thank you.

For any boto3.Session.client or boto3.Session.resource object, you can pass the verify=False parameter to skip SSL certificate verification. For example, when you create the client or resource, do this:
import boto3
boto3_session = boto3.Session(profile_name=some_profile_you_configured)
s3_resource = boto3_session.resource(service_name='s3', verify=False)
s3_client = boto3_session.client(service_name='s3', verify=False)
[insert your code here using the s3_resource or s3_client objects]
The boto3 documentation is your best friend here.

Related

python request error. sys.exit(-(256 - signum))

try:
r = requests.get(url)
except requests.exceptions.Timeout:
_LOG.info(f"Loop {i} timeout")
continue
I got error as below,
return request(\'get\', url, params=params, **kwargs)\n
File "/venv/lib/python3.6/site-packages/requests/api.py", line 61, in request\n
return session.request(method=method, url=url, **kwargs)\n
File "/venv/lib/python3.6/site-packages/requests/sessions.py", line 542, in request\n resp = self.send(prep, **send_kwargs)\n
File "/venv/lib/python3.6/site-packages/requests/sessions.py", line 655, in send\n r = adapter.send(request, **kwargs)\n
File "/venv/lib/python3.6/site-packages/requests/adapters.py", line 449, in send\n timeout=timeout\n
File "/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 706, in urlopen\n chunked=chunked,\n
File "/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 382, in _make_request\n self._validate_conn(conn)\n
File "/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn\n conn.connect()\n
File "/venv/lib/python3.6/site-packages/urllib3/connection.py", line 421, in connect\n tls_in_tls=tls_in_tls,\n
File "/venv/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 450, in ssl_wrap_socket\n sock, context, tls_in_tls, server_hostname=server_hostname\n
File "/venv/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl\n
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)\n
File "/usr/local/lib/python3.6/ssl.py", line 401, in wrap_socket\n _context=self, _session=session)\n
File "/usr/local/lib/python3.6/ssl.py", line 808, in __init__\n self.do_handshake()\n
File "/usr/local/lib/python3.6/ssl.py", line 1061, in do_handshake\n self._sslobj.do_handshake()\n File "/usr/local/lib/python3.6/ssl.py", line 683, in do_handshake\n self._sslobj.do_handshake()\n
File "/venv/lib/python3.6/site-packages/billiard/common.py", line 125, in _shutdown_cleanup\n
sys.exit(-(256 - signum))\n
File "/venv/lib/python3.6/site-packages/billiard/pool.py", line 280, in exit\n
return _exit()\nSystemExit\n'
Can anyone let me know what is the issue above, I am using celery and request, I got ^^ error. I am confused that if this is celery error or request error. Thanks.
Also my sqlalchemy got the same issue !
File "/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 976, in execute\n
return self._execute_text(object_, multiparams, params)\n
File "/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1151, in _execute_text\n parameters,\n
File "/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1288, in _execute_context\n e, statement, parameters, cursor, context\n
File "/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1485, in _handle_dbapi_exception\n util.raise_(exc_info[1], with_traceback=exc_info[2])\n
File "/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_\n raise exception\n
File "/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context\n cursor, statement, parameters, context\n
File "/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 588, in do_execute\n cursor.execute(statement, parameters)\n
File "/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 247, in execute\n res = self._query(query)\n
File "/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 411, in _query\n rowcount = self._do_query(q)\n
File "/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 375, in _do_query\n self._do_get_result()\n
File "/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 184, in _do_get_result\n self._result = self._get_result()\n
File "/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 408, in _get_result\n return self._get_db().store_result()\n
File "/venv/lib/python3.6/site-packages/billiard/common.py", line 125, in _shutdown_cleanup\n sys.exit(-(256 - signum))\n
File "/venv/lib/python3.6/site-packages/billiard/pool.py", line 280, in exit\n
return _exit()\nSystemExit\n'
If you go this issue, please try to update the celery version, sometimes it is from your deprecated celery.
I successfully solved this issue after updating to celery==4.4.2.
Also the message queue retry sometimes need to be disable from sql, it might cause some wait lock timeout issue if celery unexpected retry.

“SSL: CERTIFICATE_VERIFY_FAILED” Error when publish MQTT, AWS IoT

I am getting the following error:
[ERROR] SSLError: SSL validation failed for https://data.iot.ap-northeast-2.amazonaws.com/topics/app%2Ftest%2Fresponse?qos=1 [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1124)
Traceback (most recent call last):
File "/var/task/app.py", line 197, in lambda_handler
mqttcli.test('test', '11111', {}, 1, 200)
File "/opt/python/lib/python3.8/site-packages/connectors/MQTTClient.py", line 40, in test
response = self._iot_client.publish(
File "/var/task/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/var/task/botocore/client.py", line 662, in _make_api_call
http, parsed_response = self._make_request(
File "/var/task/botocore/client.py", line 682, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/var/task/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/var/task/botocore/endpoint.py", line 136, in _send_request
while self._needs_retry(attempts, operation_model, request_dict,
File "/var/task/botocore/endpoint.py", line 253, in _needs_retry
responses = self._event_emitter.emit(
File "/var/task/botocore/hooks.py", line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/var/task/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/var/task/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/var/task/botocore/retryhandler.py", line 183, in __call__
if self._checker(attempts, response, caught_exception):
File "/var/task/botocore/retryhandler.py", line 250, in __call__
should_retry = self._should_retry(attempt_number, response,
File "/var/task/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/var/task/botocore/retryhandler.py", line 316, in __call__
checker_response = checker(attempt_number, response,
File "/var/task/botocore/retryhandler.py", line 222, in __call__
return self._check_caught_exception(
File "/var/task/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "/var/task/botocore/endpoint.py", line 200, in _do_get_response
http_response = self._send(request)
File "/var/task/botocore/endpoint.py", line 269, in _send
return self.http_session.send(request)
File "/var/task/botocore/httpsession.py", line 281, in send
raise SSLError(endpoint_url=request.url, error=e)
This is the code that is causing this error:
_iot_client = boto3.client('iot-data',
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
region_name= REGION_NAME)
response = _iot_client.publish(
topic = "app/test/response",
qos = 1,
payload = json.dumps(
{
'msgid': msgid,
'status': status,
'data': payload
}
)
)
There is no error in S3 or other services through boto3. only iot-data.
It works without any problems when i run the .py.
but an error occurs when running after deploy to lambda.
There was no error until recently.
We also are experiencing this issue, in our case, an update in the "certifi" library (requests dependency) was causing some conflict with boto3 iot publish, rolling back the version solved the problem, although we are not entirely sure what exactly was failing.
You need to get the "Data-ATS" endpoint instead of the untrusted "Symantec" endpoint that's built-in. Try this:
import boto3
def get_aws_iot_ats_endpoint():
"""
Get the "Data-ATS" endpoint instead of the
untrusted "Symantec" endpoint that's built-in.
"""
iot_client = boto3.client(
"iot",
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
region_name= REGION_NAME,
verify=True
)
details = iot_client.describe_endpoint(endpointType="iot:Data-ATS")
host = details.get("endpointAddress")
return f"https://{host}"
IOT_DATA_ENDPOINT = get_aws_iot_ats_endpoint()
client_iot = boto3.client(
"iot-data",
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
region_name= REGION_NAME,
verify=True,
endpoint_url=IOT_DATA_ENDPOINT
)
response = client_iot.publish(
topic = "app/test/response",
qos = 1,
payload = json.dumps(
{
'msgid': msgid,
'status': status,
'data': payload
}
)
)
I had the same error.
Simply insert this at the beginning of your code:`
from botocore.exceptions import ClientError
It should work.
Best.

ConnectionRefusedError: [Errno 61] Connection refused & Could not connect to the endpoint URL: http://127.0.0.1:3001

I am trying to run integration tests using boto3 and aws. It seems the tests are unable to connect to the localhost endpoint so the tests are failing. I'm quite sure I've configured my aws environment variables correctly. I suspect that maybe there is no local server running but I'm still unsure how to fix the problem. Any help would be greatly appreciated.
I'm running on Mac Mojave, I've tried changing the ports that the server is attempting to connect to localhost:3000. I've also tried connecting to a server that I'm running using AWS Step Functions (https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local-config-options.html) but with no luck.
Error messages from Log:
ConnectionRefusedError: [Errno 61] Connection refused
and
raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://127.0.0.1:3001/2015-03-31/functions/AthenaDatabaseProvider/invocations"
Complete Log:
Traceback (most recent call last):
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/httpsession.py", line 262, in send
chunked=self._chunked(request.headers),
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/urllib3/connectionpool.py", line 641, in urlopen
_stacktrace=sys.exc_info()[2])
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/urllib3/util/retry.py", line 344, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/urllib3/packages/six.py", line 686, in reraise
raise value
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/urllib3/connectionpool.py", line 603, in urlopen
chunked=chunked)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/urllib3/connectionpool.py", line 355, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1244, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/awsrequest.py", line 125, in _send_request
method, url, body, headers, *args, **kwargs)
File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1290, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1239, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/awsrequest.py", line 152, in _send_output
self.send(msg)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/awsrequest.py", line 236, in send
return super(AWSConnection, self).send(str)
File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/urllib3/connection.py", line 183, in connect
conn = self._new_conn()
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/urllib3/connection.py", line 169, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPConnection object at 0x11ea50d90>: Failed to establish a new connection: [Errno 61] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/patrickward/repositories/JAM/hobbes/transformations/tests/integration/lambdas/athena_database_provider_test.py", line 97, in test_drop_with_stack_false_keeps_database_on_delete
invoke_fn(cloud_formation_event('Delete', DatabaseName=existent_database, DropWithStack='False'))
File "/Users/patrickward/repositories/JAM/hobbes/transformations/tests/integration/lambdas/athena_database_provider_test.py", line 54, in _exec
Payload=json.dumps(event)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/client.py", line 648, in _make_api_call
operation_model, request_dict, request_context)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/client.py", line 667, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/endpoint.py", line 137, in _send_request
success_response, exception):
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/endpoint.py", line 231, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/hooks.py", line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/retryhandler.py", line 183, in __call__
if self._checker(attempts, response, caught_exception):
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/retryhandler.py", line 251, in __call__
caught_exception)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/retryhandler.py", line 317, in __call__
caught_exception)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/retryhandler.py", line 223, in __call__
attempt_number, caught_exception)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/endpoint.py", line 200, in _do_get_response
http_response = self._send(request)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/endpoint.py", line 244, in _send
return self.http_session.send(request)
File "/Users/patrickward/.local/share/virtualenvs/hobbes-iOpBOaJm/lib/python3.7/site-packages/botocore/httpsession.py", line 282, in send
raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://127.0.0.1:3001/2015-03-31/functions/AthenaDatabaseProvider/invocations"
Turns out I need to have the docker app running when I run tests. Hope this can help someone who might have run into the same problem!

botocore.exceptions.NoCredentialsError: Unable to locate credentials in jenkins

I have configured the awscli in my linux server. I have set the credential file. Jenkins is also installed in it.
I have a python code for placing a file from the linux server to AWS s3 bucket.
When I run the script in the linux server it works fine. I get the correct output.
But when I run the same script for a job in jenkins execute shell, it throws the following error:
"File "/usr/lib/python2.7/site-packages/boto3/resources/factory.py", line 520, in do_action
response = action(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/boto3/resources/action.py", line 83, in __call__
response = getattr(parent.meta.client, operation_name)(**params)
File "/usr/lib/python2.7/site-packages/botocore/client.py", line 312, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/lib/python2.7/site-packages/botocore/client.py", line 588, in _make_api_call
operation_model, request_dict)
File "/usr/lib/python2.7/site-packages/botocore/endpoint.py", line 141, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/lib/python2.7/site-packages/botocore/endpoint.py", line 166, in _send_request
request = self.create_request(request_dict, operation_model)
File "/usr/lib/python2.7/site-packages/botocore/endpoint.py", line 150, in create_request
operation_name=operation_model.name)
File "/usr/lib/python2.7/site-packages/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "/usr/lib/python2.7/site-packages/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "/usr/lib/python2.7/site-packages/botocore/signers.py", line 90, in handler
return self.sign(operation_name, request)
File "/usr/lib/python2.7/site-packages/botocore/signers.py", line 154, in sign
auth.add_auth(request)
File "/usr/lib/python2.7/site-packages/botocore/auth.py", line 408, in add_auth
super(S3SigV4Auth, self).add_auth(request)
File "/usr/lib/python2.7/site-packages/botocore/auth.py", line 340, in add_auth
raise NoCredentialsError
botocore.exceptions.NoCredentialsError: Unable to locate credentials
Build step 'Execute shell' marked build as failure
Finished: FAILURE"
Any solution????

Exception in Boto3 - botocore.exceptions.EndpointConnectionError

When trying to connect to a database with this python code to test the connection.
import boto3
s3 = boto3.resource('s3')
for b in s3.buckets.all():
print b.name
I am getting this error thrown at me.
Traceback (most recent call last):
File "boto3_test.py", line 4, in <module>
for b in s3.buckets.all():
File "/usr/local/lib/python2.7/dist-packages/boto3/resources/collection.py", line 83, in __iter__
for page in self.pages():
File "/usr/local/lib/python2.7/dist-packages/boto3/resources/collection.py", line 161, in pages
pages = [getattr(client, self._py_operation_name)(**params)]
File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 310, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 385, in _make_api_call
operation_model, request_dict)
File "/usr/local/lib/python2.7/dist-packages/botocore/endpoint.py", line 111, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/local/lib/python2.7/dist-packages/botocore/endpoint.py", line 140, in _send_request
success_response, exception):
File "/usr/local/lib/python2.7/dist-packages/botocore/endpoint.py", line 213, in _needs_retry
caught_exception=caught_exception)
File "/usr/local/lib/python2.7/dist-packages/botocore/hooks.py", line 226, in emit
return self._emit(event_name, kwargs)
File "/usr/local/lib/python2.7/dist-packages/botocore/hooks.py", line 209, in _emit
response = handler(**kwargs)
File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 183, in __call__
if self._checker(attempts, response, caught_exception):
File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 250, in __call__
caught_exception)
File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 273, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 313, in __call__
caught_exception)
File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 222, in __call__
return self._check_caught_exception(attempt_number, caught_exception)
File "/usr/local/lib/python2.7/dist-packages/botocore/retryhandler.py", line 355, in _check_caught_exception
raise caught_exception
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://s3.us-east-5.amazonaws.com/"
The strange part is that the connection was working earlier in the day and then at it stopped working the same thing happened to me yesterday at the same time so I am assuming it is a network configuration issue. This question is the only one that is similar the solution the original poster stated was to unset both HTTP_PROXY and HTTPS_PROXY which I attempted but it did not work.
You have configured AWS env incorrectly. Check your ~/.aws/config file. You must have configured it incorrectly as us-east-5 which is an invalid region. Change it to us-east-1. Or run aws configure command again and specify the correct region.
[default]
region = us-east-1
We got this issue (with status code 500 returned) out of the blue.
After some research we found out that there was some infra update on AWS which caused the service to be down.
You can find the current down\up services per zone and upcoming down time here
Got the same boto3 error.
In my case a config.lock and credentials.lock were created behind the scenes:
Removing them solved the issue.

Categories