Can we add the unix user credentials to SQS client?
Situation :
I have a user in Unix system (without 'sudo'/root privileges). And this Unix user's permissions does not allow me to connect to amazonaws.com.
Can I pass the username and password of this Unix user to boto3 client (SQS) ?
OR, can we pass boto3 client receive message object in urllib2/3?
What I already tried :
import boto3
# Create SQS client
sqs = boto3.client('sqs',region_name='eu-west-1')
queue_url='https://sqs.eu-west-1.amazonaws.com/XXXX'
# Receive message from SQS queue
response = sqs.receive_message(
QueueUrl=queue_url,
AttributeNames=[
'SentTimestamp'
],
MaxNumberOfMessages=10,
MessageAttributeNames=[
'All'
],
VisibilityTimeout=0,
WaitTimeSeconds=0
)
I can access any URL using urllib2 to scrape it. And I am able to pass Unix credentials while doing it.
As a root user I am able to get messages from amazonSQS using boto3 as mentioned in the above code.
No. Not possible. From your code, it looks like the credentials are stored under root user's directory which cannot be accessed by an user without sudo privileges. This is a Linux issue, nothing to do with Boto3.
Find a way to let the user access the aws credentials. Like a soft link with read permissions on the credentials file.
Related
I am trying to use the documentation on https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.Python.html. Right now I am stuck at session = boto3.session(profile_name='RDSCreds'). What is profile_name and how do I find that in my RDS?
import sys
import boto3
ENDPOINT="mysqldb.123456789012.us-east-1.rds.amazonaws.com"
PORT="3306"
USR="jane_doe"
REGION="us-east-1"
os.environ['LIBMYSQL_ENABLE_CLEARTEXT_PLUGIN'] = '1'
#gets the credentials from .aws/credentials
session = boto3.Session(profile_name='RDSCreds')
client = session.client('rds')
token = client.generate_db_auth_token(DBHostname=ENDPOINT, Port=PORT, DBUsername=USR, Region=REGION)
session = boto3.Session(profile_name='RDSCreds')
profile_name here means the name of the profile you have configured to use for your aws cli.
usually when you run aws configure it creates a default profile.But sometime users want to manage aws cli with another account credentials or amange request for another region so they configure separate profile. docs for creating configuring multiple profiles
aws configure --profile RDSCreds #enter your access keys for this profile
in case if you think you have already created RDSCreds profile to check that profile less ~/.aws/config
the documentation which you have mentioned for rds using boto3 also says "The code examples use profiles for shared credentials. For information about the specifying credentials, see Credentials in the AWS SDK for Python (Boto3) documentation."
I would like to use boto3 to get temporary credentials for access AWS services. The use case is this: A user in my Cognito User Pool logs in to my server and I want the server code to provide that user with temporary credentials to access other AWS services.
I have a Cognito User Pool where my users are stored. I have a Cognito Identity Pool that does NOT allow unauthorized access, only access by users from the Cognito User Pool.
So here is the code I am starting with:
import boto3
client = boto3.client('cognito-identity','us-west-2')
resp = client.get_id(AccountId='<ACCNTID>',
IdentityPoolId='<IDPOOLID>')
However, just running these three lines of code throws an exception:
botocore.errorfactory.NotAuthorizedException: An error
occurred (NotAuthorizedException) when calling
the GetId operation: Unauthenticated access is not
supported for this identity pool.
Since my Cognito Identity Pool is not set up for unauthenticated access, it seems that I cannot call get_id until I somehow authenticate somewhere.
How do I solve this? What exactly do I need to do to authenticate so I can call get_id?
UPDATE: Looks like I need to pass a Logins field and data to the get_id function call, but to do that I need the login JWT token. If I am running this inside a webapp (eg a Django backend) where I use the AWS Cognito prepackaged login screens, then yes I can get this from the homepage URL after redirection from successful login. But now I am writing some test scripts that have nothing to do with a website. Is there a way to use boto or boto3 or some other python package to login with username and password and get JWT token?
Just to add to the answer from Arka Mukherjee above, to get the token I do this:
auth_data = { 'USERNAME':username , 'PASSWORD':password }
provider_client=boto3.client('cognito-idp', region_name=region)
resp = provider_client.admin_initiate_auth(UserPoolId=user_pool_id, AuthFlow='ADMIN_NO_SRP_AUTH', AuthParameters=auth_data, ClientId=client_id)
token = resp['AuthenticationResult']['IdToken']
Here I have to use the username and password of the Cognito user, client_id is the app client id for the app client that I set up thru Cognito, and user_pool_id is the user pool id.
Note that my app client has this option checked/selected: Enable sign-in API for server-based authentication (ADMIN_NO_SRP_AUTH) and I created that app client with no secret key (apparently that is important for web clients especially).
To pass the Cognito User Pool JWT Token, you would need to use the Logins Map in the GetId API call. You could try the following Python code out on your end, after replacing the necessary placeholders.
response = client.get_id(
AccountId='string',
IdentityPoolId='string',
Logins={
'cognito-idp.<region>.amazonaws.com/<YOUR_USER_POOL_ID>': '<JWT ID Token>'
}
)
If you do not provide a Logins Map, Amazon Cognito treats the authentication event as Unauthenticated, and hence, you are facing this error.
I have been stuck on the following issue for quite some time now. Within Python I want users to retrieve a token based upon their username and password from the AWS cognito-identity-pool making use of srp authentication. With this token I want the users to upload data to s3.
This is part of the code I use (from the warrant library): https://github.com/capless/warrant
self.client = boto3.client('cognito-idp', region_name="us-east-1")
response = boto_client.initiate_auth(
AuthFlow='USER_SRP_AUTH',
AuthParameters=auth_params,
ClientId=self.client_id
)
def get_auth_params(self):
auth_params = {'USERNAME': self.username,
'SRP_A': long_to_hex(self.large_a_value)}
if self.client_secret is not None:
auth_params.update({
"SECRET_HASH":
self.get_secret_hash(self.username,self.client_id, self.client_secret)})
return auth_params
However, I keep on getting:
botocore\auth.py", line 352, in add_auth raise NoCredentialsError
botocore.exceptions.NoCredentialsError: Unable to locate credentials
I was able to get rid of this error by adding credentials in the .aws/credentials file. But this is not in line with the purpose of this program. It seems like there is a mistake in the warrant or botocore library and the it keeps on attempting to use the AWS Access Key ID and AWS Secret Access Key from the credentials file, rather than that the given credentials (username and password) are used.
Any help is appreciated
I am on to Cognito team. initiate auth is an unauthenticated call so it shouldn't require you to provide AWS credentials. The service endpoint will not validate the sigv4 signature for these calls.
That being said, some client libraries have certain peculiarities in the sense that you need to provide some dummy credentials otherwise the client library will throw an exception. However you can provide anything for the credentials.
I too ran into this, using warrant.
The problem is that the boto3 libraries are trying to sign the request to aws, but this request is not supposed to be signed. To prevent that, create the identity pool client with a config that specifies no signing.
import boto3
from botocore import UNSIGNED
from botocore.config import Config
client = boto3.client('cognito-idp', region_name='us-east-1', config=Config(signature_version=UNSIGNED))
AWS Access Key ID and AWS Secret Access Key are totally different from username and password.
The Boto3 client has to connect to the AWS service endpoint (in your case: cognito-idp.us-east-1.amazonaws.com) to execute any API. Before executing an API, the API credentials (key+secret) have to provided to authenticate your AWS account. Without autheticating your account, you cannot call cognito-idp APIs.
There is one AWS account (key/secret) but there can be multiple users (username/password).
I am retrieving a ChromeOS device MAC address via the Google Admin Directory API using the device's Serial Number as reference, and am making my calls through
apiclient.
service = discovery.build('admin', 'directory_v1', developerKey=settings.API_KEY)
Here are the calls available for ChromeOS devices; my issue is that I require a Device ID in order to execute the following:
service.chromeosdevices().get(customerId=settings.CID, deviceId=obtained_id, projection=None).execute()
I can send a GET query via the following format:
https://www.googleapis.com/admin/directory/v1/customer/my_customer/devices/chromeos?projection=full&query=id:" + serial + "&orderBy=status&sortOrder=ascending&maxResults=10", "GET")
... but I'm trying to avoid using OAuth2 and just use my API key. Passing the key in a GET request doesn't work either, as it still returns a "Login Required" notice.
How do I squeeze the above query into an apiclient-friendly format? The only option I found via the above calls was to request every device we have (via list), then sift through the mountain of data for the matching Serial number, which seems silly and excessive.
I did notice I could call apiclient.http.HttpRequests, but I couldn't find a way to pass the API key through it either. There's new_batch_http_request, but I can't discern from the docs how to simply pass a URL to it.
Thank you!
Got it!
You can't use just a key for Directory API queries, you need a Service account.
I'm using google-auth (see here) since oauth2client is deprecated.
You also need to:
Delegate the necessary permissions for your service account (mine has the role of Viewer and has scope access to https://www.googleapis.com/auth/admin.directory.device.chromeos.readonly)
Delegate API access to it separately in the Admin Console (Security -> Advanced Settings -> Authentication)
Get your json client secret key and place it with your app (don't include it in your VCS)
Obtain your credentials like this:
credentials = service_account.Credentials.from_service_account_file(
settings.CLIENT_KEY,
scopes=settings.SCOPES,
subject=settings.ADMIN_USER)
where ADMIN_USER is the email address of an authorized Domain admin.
Then you send a GET request like so:
authed_session = AuthorizedSession(credentials)
response = authed_session.get(request_id_url)
This returns a Requests object you can read via response.content.
Hope it helps someone else!
Toying around with Soundclouds SDK for Python with an impact of TKinter as GUI. Now I want to generate a access token for each user so that I could access more API-endpoints.
I have created an applicaton in Soundclouds Developer portal with a link to my callback.
There is nothing corresponding to generating a access token for an desktop application. Only for server-side application. I tried this code below:
import soundcloud
# create client object with app credentials
client = soundcloud.Client(client_id='YOUR_CLIENT_ID',
client_secret='YOUR_CLIENT_SECRET',
redirect_uri='REDIRECT_URL')
# redirect user to authorize URL
redirect client.authorize_url()
I have set my keys, and redirect_uri as the callback on my webserver. When I run my python file from the terminal, I get this:
File "token.py", line 9
redirect client.authorize_url()
^
SyntaxError: invalid syntax
Using Python 2.7.5+
What is causing this? I want to generete my access token and print in later on.
The solution might be that I need to create an instance of an web browser window, make the user accept the app using Soundcloud connect. The I grab the url and sort out the "code" in the url. Exchanges the code against an access-token and stores it inside a text-file. So that I could grab it later on.
A simple way of obtaining an access token is by first authenticating via the User Credentials flow, which exchanges your username and password for an access token:
client = soundcloud.Client(client_id = 'CLIENT_ID',
client_secret = 'CLIENT_SECRET',
username = 'USERNAME',
password = 'PASSWORD')
print client.access_token
try:
redirect(client.authorize_url())