We Keep Coding

Python boto3 how to parse role_arn from AWS_CONFIG_FILE?

I have an AWS config file that my boto3 session has access to, via the AWS_CONFIG_FILE environment variable.
The config file looks like this: (multi-account environment)
[profile profile1]
credential_source Environment
region=us-east-whatever
role_arn=arn:aws:iam:<ACCOUNT NUMBER 1>:role/all-profiles-same-role-name
[profile profile2]
credential_source Environment
region=us-east-whatever
role_arn=arn:aws:iam:<ACCOUNT NUMBER 2>:role/all-profiles-same-role-name
[profile profileN]
credential_source Environment
region=us-east-whatever
role_arn=arn:aws:iam:<ACCOUNT NUMBER N>:role/all-profiles-same-role-name
In my Python code, I am trying to setup RefreshableCredentails (boto3 method) using somethign like this: (excluding full code because I think the issue is mainly about parsing the aws_config_file):
def __get_session_credentials(self):
# hardcode one role_arn for now but need to variablize
session_ttl=3000
aws_role_arn="arn:aws:iam::<ACCOUNT NUM>:role/all-profiles-same-role-name
...
Can I somehow parse the "role_arn" from the config file on a per profile basis to make that function more extensible? How would I do that?

You can use the configparser module from the standard library:
import configparser
from pathlib import Path
def main():
path_to_config = Path(Path.home(), ".aws", "config")
parser = configparser.ConfigParser()
parser.read(path_to_config)
for profile in parser.sections():
if "role_arn" in parser[profile]:
print(
"Found profile", profile, "with role_arn", parser[profile]["role_arn"]
)
if __name__ == "__main__":
main()
I'm not going to share the output here, though ;-)

I think that your problem could be solved if you configure a new environment variable AWS_PROFILE
According to boto3 docs[1]:
AWS_PROFILE
The default profile to use, if any. If no value is specified, Boto3 attempts to search the shared credentials file and the config file for the default profile.
And aws-cli (just for reference) docs[2]:
AWS_PROFILE
Specifies the name of the AWS CLI profile with the credentials and options to use. This can be the name of a profile stored in a credentials or config file, or the value default to use the default profile.
If defined, this environment variable overrides the behavior of using the profile named [default] in the configuration file. You can override this environment variable by using the --profile command line parameter.
So, just set this environment variable to the profileN

pass extra file argument to azureml inference config class

Currently crating inf_conf from entry script (score.py) and environment however, I have a json file that i also want to include in this.
Is there a way i can do this?
I have seen source_directory argument but json file is not in the same folder as score.py file. https://learn.microsoft.com/en-us/python/api/azureml-core/azureml.core.model.inferenceconfig?view=azure-ml-py
inf_conf = InferenceConfig(entry_script="score.py",environment=environment)

Currently, it is required that all the necessary files and objects related to the endpoint be placed in the source_directory:
inference_config = InferenceConfig(
environment=env,
source_directory='./endpoint_source',
entry_script="./score.py",
)
One workaround is to upload your JSON file somewhere else, e.g., on the Blob Storage, and download it in the init() function of your entry script. For example:
score.py:
import requests
def init():
"""
This function is called when the container is initialized/started,
typically after create/update of the deployment.
"""
global model
# things related to initializing the model
model = ...
# download your JSON file
json_file_rul = 'https://sampleazurestorage.blob.core.windows.net/data/my-configs.json'
response = requests.get(json_file_rul)
open('my_configs.json', "wb").write(response.content)

Name error when trying to import API key from .env

I am trying to store my API Keys in a .env file
I created the file as a File containing settings for editor file type. Stored my APIKeys
TWILIO_ACCOUNT_SID=***
TWILIO_AUTH_TOKEN=***
TWIML_APPLICATION_SID=***
TWILIO_API_KEY=***
TWILIO_API_SECRET=***
Installed decouple, imported and used config to retrieve my API tokens in my settings.py file
from decouple import config
...
TWILIO_ACCOUNT_SID = config(TWILIO_ACCOUNT_SID)
TWILIO_AUTH_TOKEN = config(TWILIO_AUTH_TOKEN)
TWIML_APPLICATION_SID = config(TWIML_APPLICATION_SID)
TWILIO_API_KEY = config(TWILIO_API_KEY)
TWILIO_API_SECRET = config(TWILIO_API_SECRET)
I am however getting the error message:
TWILIO_ACCOUNT_SID = config(TWILIO_ACCOUNT_SID)
NameError: name 'TWILIO_ACCOUNT_SID' is not defined

You don't need to use the decouple library to read your environment variables.
Firstly download the .env plug-in supporter for PyCharm (if that's what you're using)
https://www.codestudyblog.com/cs2112pyc/1224021812.html
This will allow you to set and get variables from your file. Make sure your configuration has the correct .env file set.
my .env file has the variable set to:
TWILIO_ACCOUNT_SID=SUPER SECRET KEY
Then all you need to is:
import os
twilio_key = os.environ.get('TWILIO_ACCOUNT_SID')
print(twilio_key)
>>>SUPER SECRET KEY
Process finished with exit code 0

passing a list in python to generate url

I have to import my config.py file in my code and then basically pass the REPOS_TO_CHECK values to a function which should be in the form of a list such that when I pass the list to my build_url function which generates a url when the REPOS_TO_CHECK variable is passed to it from the config.py file
I have to generate the url like GET /repos/:owner/:repo
GITHUB_URL = 'https://testurl.com'
how do I pass the REPOS_TO_CHECK parameter from the config.py file as a list so that when I pass
def build_url(['owner','repo']):
the url generated will be https://testurl.com/:owner/:repo
:owner, :repo are in the REPOS_TO_CHECK in the config.py file.
I can access the config.py file by importing config.py in my code and then accessing the values by using the config. for example: config.GITHUB_URL ,gives me 'https://testurl.com'
This is my config file:
GITHUB_URL = 'https://testurl.com'
GITHUB_ACCESS_TOKEN = 'access token'
REPOS_TO_CHECK = [
('owner1', 'repo1'),
('owner2', 'repo2'),]

You can import your config.py as any other file given it's in the same folder: (otherwise navigate to it)
import config
Then you can call the variable like config.REPOS_TO_CHECK
To generate the url you can simply use the variables given in the function call:
If you're using python 3.6 use f strings:
def generate_url(list_here):
return f'http://test.com/{list_here[0]}/{list_here[1]}'
Else use .format()
def generate_url(list_here):
return 'http://test.com/{0}/{1}'.format(list_here[0], list_here[1])

Retrieving config from a blueprint in Sanic app

I have a Sanic application, and want to retrieve app.config from a blueprint as it holds MONGO_URL, and I will pass it to a repository class from the blueprint.
However, I could not find how to get app.config in a blueprint. I have also checked Flask solutions, but they are not applicable to Sanic.
My app.py:
from sanic import Sanic
from routes.authentication import auth_route
from routes.user import user_route
app = Sanic(__name__)
app.blueprint(auth_route, url_prefix="/auth")
app.blueprint(user_route, url_prefix="/user")
app.config.from_envvar('TWEETBOX_CONFIG')
app.run(host='127.0.0.1', port=8000, debug=True)
My auth blueprint:
import jwt
from sanic import Blueprint
from sanic.response import json, redirect
from domain.user import User
from repository.user_repository import UserRepository
...
auth_route = Blueprint('authentication')
mongo_url = ?????
user_repository = UserRepository(mongo_url)
...
#auth_route.route('/signin')
async def redirect_user(request):
...

The Sanic way...
Inside a view method, you can access the app instance from the request object. And, therefore access your configuration.
#auth_route.route('/signin')
async def redirect_user(request):
configuration = request.app.config
2021-10-10 Update
There are two newer ways to get to the configuration values (or, perhaps more accuratlely getting the application instance from which you can get the configuration). The first version might be more on point to answering the question of how to get to the config from the blueprint. However, the second option is probably the preferred method since it is precisely intended for this kind of use.
Alternative #1
Blueprints have access to the Sanic applications they are attached to beginning with v21.3.
Therefore, if you have a blueprint object, you can trace that back to the application instance, and therefore also the config value.
app = Sanic("MyApp")
bp = Blueprint("MyBlueprint")
app.blueprint(bp)
assert bp.apps[0] is app
The Blueprint.apps property is a set because it is possible to attach a single blueprint to multiple applications.
Alternative #2
Sanic has a built-in method for retrieving an application instance from the global scope beginning in v20.12. This means that once an application has been instantiated, you can retrieve it using: Sanic.get_app().
app = Sanic("MyApp")
assert Sanic.get_app() is app
This method will only work if there is a single Sanic instance available. If you have multiple application instances, you will need to use the optional name argument:
app1 = Sanic("MyApp")
app2 = Sanic("MyOtherApp")
assert Sanic.get_app("MyApp") is app1

I would suggest a slightly different approach, based on the 12 Factor App (very interesting read which, among others, provides a nice guideline on how to protect and isolate your sensitive info).
The general idea is to place your sensitive and configuration variables in a file that is going to be gitignored and therefore will only be available locally.
I will try to present the method I tend to use in order to be as close as possible to the 12 Factor guidelines:
Create a .env file with your project variables in it:
MONGO_URL=http://no_peeking_this_is_secret:port/
SENSITIVE_PASSWORD=for_your_eyes_only
CONFIG_OPTION_1=config_this
DEBUG=True
...
(Important) Add .env and .env.* on your .gitignore file, thus protecting your sensitive info from been uploaded to GitHub.
Create an env.example (be careful not to name it with a . in the beginning, because it will get ignored).
In that file, you can put an example of the expected configuration in order to be reproducible by simply copy, paste, rename to .env.
In a file named settings.py, use decouple.config to read your config file into variables:
from decouple import config
MONGO_URL = config('MONGO_URL')
CONFIG_OPTION_1 = config('CONFIG_OPTION_1', default='')
DEBUG = config('DEBUG', cast=bool, default=True)
...
Now you can use these variables wherever is necessary for your implementation:
myblueprint.py:
import settings
...
auth_route = Blueprint('authentication')
mongo_url = settings.MONGO_URL
user_repository = UserRepository(mongo_url)
...
As a finisher, I would like to point out that this method is framework (and even language) agnostic so you can use it on Sanic as well as Flask and everywhere you need it!

I think you can create a config.py to save your configuration, just like
config.py
config = {
'MONGO_URL'：'127.0.0.1:27017'
}
and use it in app.py
from config import config
mongo_url = config['MONGO_URL']

There is a variable named current_app in Flask. You can use current_app.config["MONGO_URL"].
But I am not familiar with Sanic.