We Keep Coding

Possible to use boto3/SDK service resource cross account? [duplicate]

I'm trying to use the AssumeRole in such a way that i'm traversing multiple accounts and retrieving assets for those accounts. I've made it to this point:
import boto3
stsclient = boto3.client('sts')
assumedRoleObject = sts_client.assume_role(
RoleArn="arn:aws:iam::account-of-role-to-assume:role/name-of-role",
RoleSessionName="AssumeRoleSession1")
Great, i have the assumedRoleObject. But now i want to use that to list things like ELBs or something that isn't a built-in low level resource.
How does one go about doing that? If i may ask - please code out a full example, so that everyone can benefit.

Here's a code snippet from the official AWS documentation where an s3 resource is created for listing all s3 buckets. boto3 resources or clients for other services can be built in a similar fashion.
# create an STS client object that represents a live connection to the
# STS service
sts_client = boto3.client('sts')
# Call the assume_role method of the STSConnection object and pass the role
# ARN and a role session name.
assumed_role_object=sts_client.assume_role(
RoleArn="arn:aws:iam::account-of-role-to-assume:role/name-of-role",
RoleSessionName="AssumeRoleSession1"
)
# From the response that contains the assumed role, get the temporary
# credentials that can be used to make subsequent API calls
credentials=assumed_role_object['Credentials']
# Use the temporary credentials that AssumeRole returns to make a
# connection to Amazon S3
s3_resource=boto3.resource(
's3',
aws_access_key_id=credentials['AccessKeyId'],
aws_secret_access_key=credentials['SecretAccessKey'],
aws_session_token=credentials['SessionToken'],
)
# Use the Amazon S3 resource object that is now configured with the
# credentials to access your S3 buckets.
for bucket in s3_resource.buckets.all():
print(bucket.name)

To get a session with an assumed role:
import botocore
import boto3
import datetime
from dateutil.tz import tzlocal
assume_role_cache: dict = {}
def assumed_role_session(role_arn: str, base_session: botocore.session.Session = None):
base_session = base_session or boto3.session.Session()._session
fetcher = botocore.credentials.AssumeRoleCredentialFetcher(
client_creator = base_session.create_client,
source_credentials = base_session.get_credentials(),
role_arn = role_arn,
extra_args = {
# 'RoleSessionName': None # set this if you want something non-default
}
)
creds = botocore.credentials.DeferredRefreshableCredentials(
method = 'assume-role',
refresh_using = fetcher.fetch_credentials,
time_fetcher = lambda: datetime.datetime.now(tzlocal())
)
botocore_session = botocore.session.Session()
botocore_session._credentials = creds
return boto3.Session(botocore_session = botocore_session)
# usage:
session = assumed_role_session('arn:aws:iam::ACCOUNTID:role/ROLE_NAME')
ec2 = session.client('ec2') # ... etc.
The resulting session's credentials will be automatically refreshed when required which is quite nice.
Note: my previous answer was outright wrong but I can't delete it, so I've replaced it with a better and working answer.

You can assume role using STS token, like:
class Boto3STSService(object):
def __init__(self, arn):
sess = Session(aws_access_key_id=ARN_ACCESS_KEY,
aws_secret_access_key=ARN_SECRET_KEY)
sts_connection = sess.client('sts')
assume_role_object = sts_connection.assume_role(
RoleArn=arn, RoleSessionName=ARN_ROLE_SESSION_NAME,
DurationSeconds=3600)
self.credentials = assume_role_object['Credentials']
This will give you temporary access key and secret keys, with session token. With these temporary credentials, you can access any service. For Eg, if you want to access ELB, you can use the below code:
self.tmp_credentials = Boto3STSService(arn).credentials
def get_boto3_session(self):
tmp_access_key = self.tmp_credentials['AccessKeyId']
tmp_secret_key = self.tmp_credentials['SecretAccessKey']
security_token = self.tmp_credentials['SessionToken']
boto3_session = Session(
aws_access_key_id=tmp_access_key,
aws_secret_access_key=tmp_secret_key, aws_session_token=security_token
)
return boto3_session
def get_elb_boto3_connection(self, region):
sess = self.get_boto3_session()
elb_conn = sess.client(service_name='elb', region_name=region)
return elb_conn

with reference to the solution by #jarrad which is not working as of Feb 2021, and as a solution that does not use STS explicitly please see the following
import boto3
import botocore.session
from botocore.credentials import AssumeRoleCredentialFetcher, DeferredRefreshableCredentials
def get_boto3_session(assume_role_arn=None):
session = boto3.Session(aws_access_key_id="abc", aws_secret_access_key="def")
if not assume_role_arn:
return session
fetcher = AssumeRoleCredentialFetcher(
client_creator=_get_client_creator(session),
source_credentials=session.get_credentials(),
role_arn=assume_role_arn,
)
botocore_session = botocore.session.Session()
botocore_session._credentials = DeferredRefreshableCredentials(
method='assume-role',
refresh_using=fetcher.fetch_credentials
)
return boto3.Session(botocore_session=botocore_session)
def _get_client_creator(session):
def client_creator(service_name, **kwargs):
return session.client(service_name, **kwargs)
return client_creator
the function can be called as follows
ec2_client = get_boto3_session(role_arn='my_role_arn').client('ec2', region_name='us-east-1')

If you want a functional implementation, this is what I settled on:
def filter_none_values(kwargs: dict) -> dict:
"""Returns a new dictionary excluding items where value was None"""
return {k: v for k, v in kwargs.items() if v is not None}
def assume_session(
role_session_name: str,
role_arn: str,
duration_seconds: Optional[int] = None,
region_name: Optional[str] = None,
) -> boto3.Session:
"""
Returns a session with the given name and role.
If not specified, duration will be set by AWS, probably at 1 hour.
If not specified, region will be left unset.
Region can be overridden by each client or resource spawned from this session.
"""
assume_role_kwargs = filter_none_values(
{
"RoleSessionName": role_session_name,
"RoleArn": role_arn,
"DurationSeconds": duration_seconds,
}
)
credentials = boto3.client("sts").assume_role(**assume_role_kwargs)["Credentials"]
create_session_kwargs = filter_none_values(
{
"aws_access_key_id": credentials["AccessKeyId"],
"aws_secret_access_key": credentials["SecretAccessKey"],
"aws_session_token": credentials["SessionToken"],
"region_name": region_name,
}
)
return boto3.Session(**create_session_kwargs)
def main() -> None:
session = assume_session(
"MyCustomSessionName",
"arn:aws:iam::XXXXXXXXXXXX:role/TheRoleIWantToAssume",
region_name="us-east-1",
)
client = session.client(service_name="ec2")
print(client.describe_key_pairs())

import json
import boto3
roleARN = 'arn:aws:iam::account-of-role-to-assume:role/name-of-role'
client = boto3.client('sts')
response = client.assume_role(RoleArn=roleARN,
RoleSessionName='RoleSessionName',
DurationSeconds=900)
dynamodb_client = boto3.client('dynamodb', region_name='us-east-1',
aws_access_key_id=response['Credentials']['AccessKeyId'],
aws_secret_access_key=response['Credentials']['SecretAccessKey'],
aws_session_token = response['Credentials']['SessionToken'])
response = dynamodb_client.get_item(
Key={
'key1': {
'S': '1',
},
'key2': {
'S': '2',
},
},
TableName='TestTable')
print(response)

#!/usr/bin/env python3
import boto3
sts_client = boto3.client('sts')
assumed_role = sts_client.assume_role(RoleArn = "arn:aws:iam::123456789012:role/example_role",
RoleSessionName = "AssumeRoleSession1",
DurationSeconds = 1800)
session = boto3.Session(
aws_access_key_id = assumed_role['Credentials']['AccessKeyId'],
aws_secret_access_key = assumed_role['Credentials']['SecretAccessKey'],
aws_session_token = assumed_role['Credentials']['SessionToken'],
region_name = 'us-west-1'
)
# now we make use of the role to retrieve a parameter from SSM
client = session.client('ssm')
response = client.get_parameter(
Name = '/this/is/a/path/parameter',
WithDecryption = True
)
print(response)

Assuming that 1) the ~/.aws/config or ~/.aws/credentials file is populated with each of the roles that you wish to assume and that 2) the default role has AssumeRole defined in its IAM policy for each of those roles, then you can simply (in pseudo-code) do the following and not have to fuss with STS:
import boto3
# get all of the roles from the AWS config/credentials file using a config file parser
profiles = get_profiles()
for profile in profiles:
# this is only used to fetch the available regions
initial_session = boto3.Session(profile_name=profile)
# get the regions
regions = boto3.Session.get_available_regions('ec2')
# cycle through the regions, setting up session, resource and client objects
for region in regions:
boto3_session = boto3.Session(profile_name=profile, region_name=region)
boto3_resource = boto3_session.resource(service_name='s3', region_name=region)
boto3_client = boto3_session.client(service_name='s3', region_name=region)
[ do something interesting with your session/resource/client here ]
Credential Setup (boto3 - Shared Credentials File)
Assume Role Setup (AWS)

After a few days of searching, this is the simplest solution I have found. explained here but does not have a usage example.
import boto3
for profile in boto3.Session().available_profiles:
boto3.DEFAULT_SESSION = boto3.session.Session(profile_name=profile)
s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
print(bucket)
This will switch the default role you will be using. To not make the profile the default, just do not assign it to boto3.DEFAULT_SESSION. but instead, do the following.
testing_profile = boto3.session.Session(profile_name='mainTesting')
s3 = testing_profile.resource('s3')
for bucket in s3.buckets.all():
print(bucket)
Important to note that the .aws credentials need to be set in a specific way.
[default]
aws_access_key_id = default_access_id
aws_secret_access_key = default_access_key
[main]
aws_access_key_id = main_profile_access_id
aws_secret_access_key = main_profile_access_key
[mainTesting]
source_profile = main
role_arn = Testing role arn
mfa_serial = mfa_arn_for_main_role
[mainProduction]
source_profile = main
role_arn = Production role arn
mfa_serial = mfa_arn_for_main_role
I don't know why but the mfa_serial key has to be on the roles for this to work instead of the source account which would make more sense.

Here's the code snippet I used
sts_client = boto3.client('sts')
assumed_role_object = sts_client.assume_role(
RoleArn=<arn of the role to assume>,
RoleSessionName="<role session name>"
)
print(assumed_role_object)
credentials = assumed_role_object['Credentials']
session = Session(
aws_access_key_id=credentials['AccessKeyId'],
aws_secret_access_key=credentials['SecretAccessKey'],
aws_session_token=credentials['SessionToken']
)
self.s3 = session.client('s3')

GOG Galaxy 2.0 PSN account (offline)

I have been trying to connect my PSN account to Galaxy 2.0 for a while now and it keeps telling me that it's (offline), I have tried the solutions that come up first on a google search and they didn't work for me.
All the solutions I found have a different code (I don't have it) than the one the app installs by default or the one I can find on Github.
I will provide the code that I have which is the one the app installed by default.
If you have any solutions or know how to solve this please help me, thanks in advance.
This is the log in case it is needed
https://www.mediafire.com/file/3b3921wgyq9357m/plugin-psn-38087aea-3c30-439f-867d-ddf9fae8fe6f.log/file
import sys
from typing import List, Any, AsyncGenerator
from galaxy.api.consts import Platform, LicenseType
from galaxy.api.errors import InvalidCredentials
from galaxy.api.plugin import Plugin, create_and_run_plugin
from galaxy.api.types import Authentication, Game, NextStep, SubscriptionGame, \
Subscription, LicenseInfo
from http_client import HttpClient
from http_client import OAUTH_LOGIN_URL, OAUTH_LOGIN_REDIRECT_URL
from psn_client import PSNClient
from version import __version__
AUTH_PARAMS = {
"window_title": "Login to My PlayStation\u2122",
"window_width": 536,
"window_height": 675,
"start_uri": OAUTH_LOGIN_URL,
"end_uri_regex": "^" + OAUTH_LOGIN_REDIRECT_URL + ".*"
}
logger = logging.getLogger(__name__)
class PSNPlugin(Plugin):
def __init__(self, reader, writer, token):
super().__init__(Platform.Psn, __version__, reader, writer, token)
self._http_client = HttpClient()
self._psn_client = PSNClient(self._http_client)
logging.getLogger("urllib3").setLevel(logging.FATAL)
async def _do_auth(self, cookies):
if not cookies:
raise InvalidCredentials()
self._http_client.set_cookies_updated_callback(self._update_stored_cookies)
self._http_client.update_cookies(cookies)
await self._http_client.refresh_cookies()
user_id, user_name = await self._psn_client.async_get_own_user_info()
if user_id == "":
raise InvalidCredentials()
return Authentication(user_id=user_id, user_name=user_name)
async def authenticate(self, stored_credentials=None):
stored_cookies = stored_credentials.get("cookies") if stored_credentials else None
if not stored_cookies:
return NextStep("web_session", AUTH_PARAMS)
auth_info = await self._do_auth(stored_cookies)
return auth_info
async def pass_login_credentials(self, step, credentials, cookies):
cookies = {cookie["name"]: cookie["value"] for cookie in cookies}
self._store_cookies(cookies)
return await self._do_auth(cookies)
def _store_cookies(self, cookies):
credentials = {
"cookies": cookies
}
self.store_credentials(credentials)
def _update_stored_cookies(self, morsels):
cookies = {}
for morsel in morsels:
cookies[morsel.key] = morsel.value
self._store_cookies(cookies)
async def get_subscriptions(self) -> List[Subscription]:
is_plus_active = await self._psn_client.get_psplus_status()
return [Subscription(subscription_name="PlayStation PLUS", end_time=None, owned=is_plus_active)]
async def get_subscription_games(self, subscription_name: str, context: Any) -> AsyncGenerator[List[SubscriptionGame], None]:
yield await self._psn_client.get_subscription_games()
async def get_owned_games(self):
def game_parser(title):
return Game(
game_id=title["titleId"],
game_title=title["name"],
dlcs=[],
license_info=LicenseInfo(LicenseType.SinglePurchase, None)
)
def parse_played_games(titles):
return [{"titleId": title["titleId"], "name": title["name"]} for title in titles]
purchased_games = await self._psn_client.async_get_purchased_games()
played_games = parse_played_games(await self._psn_client.async_get_played_games())
unique_all_games = {game['titleId']: game for game in played_games + purchased_games}.values()
return [game_parser(game) for game in unique_all_games]
async def shutdown(self):
await self._http_client.close()
def main():
create_and_run_plugin(PSNPlugin, sys.argv)
if __name__ == "__main__":
main()```

fastapi - optional OAuth2 authentication

I need to create a API with a route that is able to recognize if the current user is the one indicated in the request or not (also no auth should be valid)
For the others paths I followed https://fastapi.tiangolo.com/tutorial/security/oauth2-jwt/ and everything work with Bearer with JWT tokens like this
user: User = Depends(get_current_active_user)
Modifying the methods provided by the docs I tried with
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="auth")
async def get_user_or_none(db: Session = Depends(get_db), token: str = Depends(oauth2_scheme)):
"""
Return the current active user if is present (using the token Bearer) or None
"""
try:
payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
username: str = payload.get("sub")
if username is None:
return None
except JWTError:
return None
# check user in db
user = crud.get_user(db, username)
if user is None:
return None
return user
#router.get("/api/{user_id}/structure")
async def get_user_structure(
user_id: int,
user = Depends(get_user_or_none),
db: Session = Depends(get_db)
):
# do_something() if user.id == user_id else do_something_else()
but I receive an error
401 Error: Unauthorized {
"detail": "Not authenticated"
}

You need to use a different OAuth2PasswordBearer for these optionally authenticated endpoints with auto_error=False, e.g.
optional_oauth2_scheme = OAuth2PasswordBearer(tokenUrl="auth", auto_error=False)
async def get_user_or_none(db: Session = Depends(get_db), token: str = Depends(optional_oauth2_scheme)):
...
Now the token will be None when the Authorization header isn't provided, resulting in your desired behavior.

Create a Discord channel upon receiving DMs

So I'm working on a bot, and I'd like it to have it's own ticket support system.
What I'd like it to do is that, upon receiving a DM it:
- Creates a new channel named after the DM author (such as #David0001) in a "ticket" category
- Sets up permissions for that channel to be only accessible to read and write by the DM author (and admins ofc)
- Restranscribes whatever the DM author wrote in his DM
I'm currently using the latest Async discord.py branch (I know I should probably be using rewrite but oh well)
#client.event
async def on_message(message):
if message.server is None and message.author != client.user:
server = client.get_server("serverid")
for channel in server.channels:
if channel.name == str(message.author):
await client.send_message(message.author, "Hey you already have a support ticket open!")
break
else:
await client.create_channel(server, str(message.author), type=discord.ChannelType.text)
overwrite = discord.PermissionOverwrite()
overwrite.read_messages = True
overwrite.send_messages = True
overwrite.ban_members = False
for channel in server.channels:
if channel.name == str(message.author):
await client.edit_channel_permissions(channel.id, message.author, overwrite)
await client.send_message(channel.id, message.content)
break
else:
break
break
await client.process_commands(message)
I'd also like it to verify first if a support channel with the users name doesn't already exist and if so to send a little message like "hey you already have a support ticket channel open"
This code seems to work at first but not well, it does create a "david0001" channel upon DM, however it doesn't set up the permissions properly, doesn't set it up in a pre-existing ticket catgeory (cause I don't know how to do that), it does not retranscribe whatever the user wrote in the DMs and it does not verify if the user has a open channel, it just keeps making a new one

There is multiple solutions that can be proposed for your situation :
As the old async branch is depreciated, here is the up to date code.
Through a command :
I assume you're using a cog system :
This is a command designed for your request, when the user uses it, it inspect the support server, if there is no Support category, it creates one, and creates a channel for the user. If the requirement already exist it just send the message stored in request into the correct channel.
import asyncio, discord
from discord.ext import commands
from discord.utils import get
class Cmd_support(commands.Cog):
def __init__(self, client):
self.client = client
#commands.command()
async def support(self, ctx, *, request: str):
user = ctx.message.author
guild_id = 000000000000000000 # fill it up with your support guild id
support_server = self.client.get_guild(guild_id)
# try to match with a channel name
match = False
for channel in support_server.text_channels:
await asyncio.sleep(0)
if channel.name == user.name.lower(): # it's a match
match = True
user_support = get(support_server.text_channels, name = user.name.lower()) # get the user support channel as lower case, just like channels name
break
# end for
if not match: # if the channel doesn't exist, we create it
support_category_name = 'Support' # defines the Support category name, case sensitive
support_category = get(support_server.categories, name = support_category_name) # get the support category
user_support = get(support_server.text_channels, name = user.name.lower()) # get the user support channel as lower case, just like channels name
if support_category == None: # if the category is not found, we create it
# setting up the category permissions
support_category_permissions = {
support_server.default_role : discord.PermissionOverwrite(send_messages = False)
}
await support_server.create_category(name = support_category_name, overwrites = support_category_permissions)
support_category = get(support_server.categories, name = support_category_name) # redefine the variable with the new category
if user_support == None: # if the channel doesn't exist
# setting up the permissions for the channel
user_channel_permissions = {
support_server.default_role : discord.PermissionOverwrite(read_messages = False, send_messages = False), # othe users cannot see the channels
support_server.me : discord.PermissionOverwrite(read_messages = True, send_messages = True),
user : discord.PermissionOverwrite(read_messages = True, send_messages = True)
}
# now we create the channel
await support_server.create_text_channel(name = user.name, overwrites = user_channel_permissions, category = support_category)
user_support = get(support_server.text_channels, name = user.name.lower()) # redefine the support channel
# now send the message to the new channel
await user_support.send(request) # sends what the user sent to the command
def setup(client):
client.add_cog(Cmd_support(client))
Using DMs :
It's slightly different from the command solution, request is just replaced by message.content and we add the condition :
Please note that it will create a channel and send message into it no matter the message content. You can filter it by using a condition like : if message.content.startswith("something"):
if type(message.channel) == discord.DMchannel:
Here's the code :
#commands.Cog.listener() # equivalent to discord.Event
async def on_message(self, message):
if type(message.channel) == discord.DMChannel:
user = message.author
guild_id = 000000000000000000 # fill it up with your support guild id
support_server = self.client.get_guild(guild_id)
# try to match with a channel name
match = False
for channel in support_server.text_channels:
await asyncio.sleep(0)
if channel.name == user.name.lower(): # it's a match
match = True
user_support = get(support_server.text_channels, name = user.name.lower()) # get the user support channel as lower case, just like channels name
break
# end for
if not match: # if the channel doesn't exist, we create it
support_category_name = 'Support' # defines the Support category name, case sensitive
support_category = get(support_server.categories, name = support_category_name) # get the support category
user_support = get(support_server.text_channels, name = user.name.lower()) # get the user support channel as lower case, just like channels name
if support_category == None: # if the category is not found, we create it
# setting up the category permissions
support_category_permissions = {
support_server.default_role : discord.PermissionOverwrite(send_messages = False)
}
await support_server.create_category(name = support_category_name, overwrites = support_category_permissions)
support_category = get(support_server.categories, name = support_category_name) # redefine the variable with the new category
if user_support == None: # if the channel doesn't exist
# setting up the permissions for the channel
user_channel_permissions = {
support_server.default_role : discord.PermissionOverwrite(read_messages = False, send_messages = False), # othe users cannot see the channels
support_server.me : discord.PermissionOverwrite(read_messages = True, send_messages = True),
user : discord.PermissionOverwrite(read_messages = True, send_messages = True)
}
# now we create the channel
await support_server.create_text_channel(name = user.name, overwrites = user_channel_permissions, category = support_category)
user_support = get(support_server.text_channels, name = user.name.lower()) # redefine the support channel
# now send the message to the new channel
await user_support.send(message.content) # sends what the user sent to the command
def setup(client):
client.add_cog(Event_on_message(client))
As you can see it's not that different from the first solution I've proposed.
Get the latest discord.py
Well, you should really update your discord.py version to the latest, the old async branch doesn't exist anymore and has been replaced by the rewrite version they were working on.
Try this little script in your terminal :
>>> import discord
>>> discord.__version__
'1.2.3'
If your version is like 0.X.X you're not up to date !
Think about pip install discord.py --upgrade to get the latest version (it's so much better than the old async one)
Usefull link : discord.py - Migrating to v1.0 (I've migrated to the new version in few hours, it wasn't too long. You should better migrate now, before your code becomes too big)
Hope it helped !
Have a nice day !

How do I invoke a lambda function inside a views function in Django?

I have a Django views to create a sort of AWS pack (contains account and new vpc creation), I am trying to invoke a lambda function in between that will delete the default vpc in the newly created account. Here is my snippet :
def createlab(request):
if request.method == 'POST':
lab_name = request.POST.get('labname')
cidr = request.POST.get('cidr')
bill = request.POST.get('budget')
email = request.POST.get('email')
#CREATE ACCOUNT
org = boto3.client('organizations')
acc = org.create_account(
Email=email,
AccountName=lab_name,
IamUserAccessToBilling='ALLOW'
)
time.sleep(35)
cid = acc['CreateAccountStatus']['Id']
#GET ACCOUNT DETAILS
status = org.describe_create_account_status(
CreateAccountRequestId=cid
)
print(status)
time.sleep(17)
while True:
status = org.describe_create_account_status(CreateAccountRequestId=cid)
try:
accid = status['CreateAccountStatus']['AccountId']
break
except KeyError:
time.sleep(40)
accid = status['CreateAccountStatus']['State']
#CREATE VPC
session =
boto3.Session
(aws_access_key_id=acc_key,
aws_secret_access_key=sec_key,aws_session_token=token)
ec2 = session.resource('ec2',region_name=region_ec2_sub)
vpc = ec2.create_vpc(CidrBlock=cidr)
id = vpc.id
#DELETE DEFAULT VPC
client = boto3.client('lambda',region_name='us-west-2')
deletevpc=
client.invoke(FunctionName='default_vpc',
InvocationType='RequestResponse')
print (deletevpc)
I have included a portion of my views function. I have added the #DELETE DEFAULT VPC snippet in the end. The lambda function is not getting executed. What am I doing wrong here ?