Bases on the MinIO documentation at page 14 following API is mentioned /minio/v2/metrics/cluster and /minio/v2/metrics/node.
But looks like these API available only for "Prometheus". And I have no clue to get those using python.
Of course if I try send request to http://<host>:<port>/minio/v2/metrics/cluster I get 403 error:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AllAccessDisabled</Code><Message>All access to this bucket has been disabled.</Message><Resource>/minio/v2/metrics/cluster</Resource><RequestId></RequestId><HostId><uuid-here></HostId></Error>
I've tried to generate presigned URL (like for GET object from bucket) but even with signature result is same as above.
Maybe someone know to reach that API(s)?
And how to do proper authentication for these endpoint without "Prometheus"?
Prometheus just requests an HTTP resource, which presents metrics in plain text format. You can grab the same URL that prometheus would use and then parse it yourself.
If you look at this document, it describes how to configure authentication for the metrics endpoint.
Configure authentication type for Prometheus metrics
MinIO supports two authentication modes for Prometheus either jwt or
public, by default MinIO runs in jwt mode. To allow public access
without authentication for prometheus metrics set environment as
follows.
export MINIO_PROMETHEUS_AUTH_TYPE="public" minio server ~/test
If you're running it in jwt mode (the default), the following
sections tells you how to generate a configuration that will show you
the appropriate bearer token to use for authentication:
3.1 Authenticated Prometheus config
If MinIO is configured to expose metrics without authentication, you
don't need to use mc to generate prometheus config. You can skip
reading further and move to 3.2 section.
The Prometheus endpoint in MinIO requires authentication by default.
Prometheus supports a bearer token approach to authenticate
prometheus scrape requests, override the default Prometheus config
with the one generated using mc. To generate a Prometheus config for
an alias, use mc as follows mc admin prometheus generate .
The command will generate the scrape_configs section of the
prometheus.yml as follows:
scrape_configs:
- job_name: minio-job
bearer_token: <secret>
metrics_path: /minio/v2/metrics/cluster
scheme: http
static_configs:
- targets: ['localhost:9000']
You can just use the bearer token and path from that configuration to
access your metrics. The bearer token should go into an
Authorization header (Authorization: Bearer <token>).
Solution:
import jwt # pyJWT
import requests
import datetime
expired_at = (datetime.datetime.now() + datetime.timedelta(days=1)).timestamp()
token = jwt.encode(
payload={
"exp": expired_at,
"sub": "<AccessKey>",
"iss": "prometheus",
},
key="<SecretKey>",
algorithm="HS512"
)
headers = {
"Authorization": f"Bearer {token.decode()}"
}
response = requests.get(
url="<schema>://<host>:<port>/minio/prometheus/metrics",
headers=headers
)
How mc utility generates bearer token I found at https://github.com/minio/mc/blob/master/cmd/admin-prometheus-generate.go
URL /minio/prometheus/metrics was found during call to utility mc admin prometheus generate test-minio
Related
I used azure-ad-verify-token 0.2.1 on Django-rest backend to validate a jwt token from Microsoft Azure, where the user is authenticated on the frontend with React.
According to the documentation, this library should do everything on its own.
from azure_ad_verify_token import verify_jwt
azure_ad_app_id = 'my app id'
azure_ad_issuer = 'https://exampletenant.b2clogin.com/0867afa-24e7-40e9-9d27-74bb598zzzzc/v2.0/'
azure_ad_jwks_uri = 'https://exampletenant.b2clogin.com/exampletenant.onmicrosoft.com/B2C_1_app_sign_in/discovery/v2.0/keys'
payload = verify_jwt(
token='<AZURE_JWT_TO_VERIFY_HERE>',
valid_audiences=[azure_ad_app_id],
issuer=azure_ad_issuer,
jwks_uri=azure_ad_jwks_uri,
verify=True,
)
print(payload)
I don't understand the line token='<AZURE_JWT_TO_VERIFY_HERE>', how can I put the token there?
Authorization from Azure on React is successful, and I get a access jwt-token that I can extract:
token = request.headers['Authorization']
But I need to validate it and somehow insert it into a string token='<AZURE_JWT_TO_VERIFY_HERE>', but it doesn't recognize the request here.
How can I put a token= from the header?
And in general, is this the right way? Or am I missing something? Any help and hints would be very helpful and would be greatly appreciated. Or advise another library for token validation in Python.
azure-ad-verify-token This is used to verify the tokens received from azure ad.
You have to get auth tokens from azure using MSAL python library and the azure-ad-verify-token will then verify the token.
To retrieve the tokens, you will need MSAL python library, and it
will also take clientid and tenentd as arguments.
test_app=PublicClientApplication(client_id=client_id,authority="https://login.microsoftonline.com/"+tenant_id)
test_tokens=test_app.acquire_token_interactive(scopes=scopes)
Now you can take the token you just received and use it in theazure-ad-verify-token.
token=test_tokens['access_token']
Reference:
MSAL Python
Authenticate Python apps by using the Azure SDK for Python
I tried so many methods, but none seem to work. Help me make a connection with LinkedIn using python. Issue in generating Access Token I received CODE but it doesn't work. I have python 3.9 Please post a sample of basic code that establishes a connection and gets a access Token. And which redirectUri I have to use. Can i use any website link for rediectUri.
I tried to check API through curl and Postman but didn't get solution its say Unauthorized Accesss.
https://github.com/ozgur/python-linkedin <---This is where I got some idea how to use API .To recievd Access token .
First solution valid for any (including free) applications, it useses so-called 3-Legged OAuth 2.0 Authentication:
Login to your account in the browser.
Create new application by this link.
If you already have application you may use it by selecting it here and changing its options if needed.
In application credentials copy Client ID and Client Secret, you'll need them later.
On your application's server side create Authorization request URL by next code and send/redirect it to client. If your Python code runs locally you may just open this URL in your browser with import webbrowser; webbrowser.open(url) code. Fill in all fields with your values too. There is redirect_uri in the code, this is URL where authorization response is sent back, for locally running script you have to run Python HTTP web server to retrieve result.
# Needs: python -m pip install requests
import requests, secrets
url = requests.Request(
'GET',
'https://www.linkedin.com/oauth/v2/authorization',
params = {
'response_type': 'code', # Always should equal to fixed string "code"
# ClientID of your created application
'client_id': 'REPLACE_WITH_YOUR_CLIENT_ID',
# The URI your users are sent back to after authorization.
# This value must match one of the OAuth 2.0 Authorized Redirect
# URLs defined in your application configuration.
# This is basically URL of your server that processes authorized requests like:
# https://your.server.com/linkedin_authorized_callback
'redirect_uri': 'REPLACE_WITH_REDIRECT_URL', # Replace this with your value
# state, any unique non-secret randomly generated string like DCEeFWf45A53sdfKef424
# that identifies current authorization request on server side.
# One way of generating such state is by using standard "secrets" module like below.
# Store generated state string on your server for further identifying this authorization session.
'state': secrets.token_hex(8).upper(),
# Requested permissions, below is just example, change them to what you need.
# List of possible permissions is here:
# https://learn.microsoft.com/en-us/linkedin/shared/references/migrations/default-scopes-migration#scope-to-consent-message-mapping
'scope': ' '.join(['r_liteprofile', 'r_emailaddress', 'w_member_social']),
},
).prepare().url
# You may now send this url from server to user
# Or if code runs locally just open browser like below
import webbrowser
webbrowser.open(url)
After user authorized your app by previous URL his browser will be redirected to redirect_uri and two fields code and state will be attached to this URL, code is unique authorization code that you should store on server, code expires after 30 minutes if not used, state is a copy of state from previous code above, this state is like unique id of your current authorization session, use same state string only once and generate it randomly each time, also state is not a secret thing because you send it to user inside authorization URL, but should be unique and quite long. Example of full redirected URL is https://your.server.com/linkedin_authorized_callback?code=987ab12uiu98onvokm56&state=D5B1C1348F110D7C.
Next you have to exchange code obtained previously to access_token by next code, next code should be run on your server or where your application is running, because it uses client_secret of your application and this is a secret value, you shouldn't show it to public, never share ClientSecret with anyone except maybe some trusted people, because such people will have ability to pretend (fake) to be your application while they are not.
# Needs: python -m pip install requests
import requests
access_token = requests.post(
'https://www.linkedin.com/oauth/v2/accessToken',
params = {
'grant_type': 'authorization_code',
# This is code obtained on previous step by Python script.
'code': 'REPLACE_WITH_CODE',
# This should be same as 'redirect_uri' field value of previous Python script.
'redirect_uri': 'REPLACE_WITH_REDIRECT_URL',
# Client ID of your created application
'client_id': 'REPLACE_WITH_YOUR_CLIENT_ID',
# Client Secret of your created application
'client_secret': 'REPLACE_WITH_YOUR_CLIENT_SECRET',
},
).json()['access_token']
print(access_token)
access_token obtained by previous script is valid for 60 days! So quite long period. If you're planning to use your application for yourself only or your friends then you can just pre-generate manually once in two months by hands several tokens for several people without need for servers.
Next use access_token for any API calls on behalf of just authorized above user of LinkedIn. Include Authorization: Bearer ACCESS_TOKEN HTTP header in all calls. Example of one such API code below:
import requests
print(requests.get(
'https://api.linkedin.com/v2/jobs',
params = {
# Any API params go here
},
headers = {
'Authorization': 'Bearer ' + access_token,
# Any other needed HTTP headers go here
},
).json())
More details can be read here. Regarding how your application is organized, there are 3 options:
Your application is running fully on remote server, meaning both authentication and running application (API calls) are done on some dedicated remote server. Then there is no problem with security, server doesn't share any secrets like client_secret, code, access_token.
Your application is running locally on user's machine while authentication is runned once in a while by your server, also some other things like storing necessary data in DataBase can be done by server. Then your server doesn't need to share client_secret, code, but shares access_token which is sent back to application to user's machine. It is also OK, then your server can keep track of what users are using your application, also will be able to revoke some or all of access_tokens if needed to block user.
Your application is fully run on local user's machine, no dedicated server is used at all. In this case all of client_secret, code, access_token are stored on user's machine. In this case you can't revoke access to your application of some specific users, you can only revoke all of them by regenerating client_secret in your app settings. Also you can't track any work of your app users (although maybe there is some usage statistics in your app settings/info pages). In this case any user can look into your app code and copy client_secret, unless you compile Python to some .exe/.dll/.so and encrypt you client secret there. If anyone got client_secret he can pretend (fake) to be your application meaning that if you app contacts other users somehow then he can try to authorize other people by showing your app interface while having some other fraudulent code underneath, basically your app is not that secure or trusted anymore. Also local code can be easily modified so you shouldn't trust your application to do exactly your code. Also in order to authorize users like was done in previous steps 5)-7) in case of local app you have to start Python HTTP Server to be able to retrieve redirected results of step 5).
Below is a second solution valid only if your application is a part of LinkedIn Developer Enterprise Products paid subscription, also then you need to Enable Client Credentials Flow in your application settings, next steps uses so-called 2-Legged OAuth 2.0 Authentication:
Login to your account in the browser.
Create new application by this link.
If you already have application you may use it by selecting it here and changing its options if needed.
In application credentials copy ClientID and ClientSecret, you'll need them later.
Create AccessToken by next Python code (put correct client id and client secret), you should run next code only on your server side or on computers of only trusted people, because code uses ClientSecret of your application which is a secret thing and shouldn't be showed to public:
# Needs: python -m pip install requests
import requests
access_token = requests.post(
'https://www.linkedin.com/oauth/v2/accessToken',
params = {
'grant_type': 'client_credentials',
'client_id': 'REPLACE_WITH_YOUR_CLIENT_ID',
'client_secret': 'REPLACE_WITH_YOUR_CLIENT_SECRET',
},
).json()['access_token']
print(access_token)
Copy access_token from previous response, it expires after 30 minutes after issue so you need to use previous script often to gain new access token.
Now you can do any API requests that you need using this token, like in code below (access_token is taken from previous steps):
import requests
print(requests.get(
'https://api.linkedin.com/v2/jobs',
params = {
# Any API params go here
},
headers = {
'Authorization': 'Bearer ' + access_token,
# Any other needed HTTP headers go here
},
).json())
More details can be read here or here.
I have a Flask app in Google App Engine Standard Environment Python, and I also have a Cloud Function with an HTTP trigger which accepts a JSON body including the URL of a file. The CF downloads the file at that URL then saves it to a GCS bucket. The GAE service account has Cloud Function Invoker permissions, yet when using urlfetch.fetch() in my GAE code to trigger CF, the App Engine code gets a 403 Forbidden error unless I make the CF trigger callable by anyone.
How do I successfully call/trigger CF's from GAE in Python? I assume the answer is one of these:
Set IAM permissions on GAE service account to {enlighten me here}
Add authentication headers in urlfetch.fetch() like so {different enlightenment}
Make CF triggerable from anywhere, but hard code some secret key so the CF code itself handles authentication.
It's well documented here: Cloud Functions Authentication
In short you have to provide your service account credentials in the authentication header.
To get your credentials use the Google Auth Client library. If you are testing from local you should create a service account JSON and load it to the environment variable GOOGLE_APPLICATION_CREDENTIALS but on App Engine it will work from scratch.
After you have gotten your token, pass it as an auth header like so:
auth_req = google.auth.transport.requests.Request()
auth_token = google.oauth2.id_token.fetch_id_token(auth_req, cloud_function_url)
response = requests.post(cloud_function_url, json=payload, headers={"Authorization" : f"Bearer {auth_token}"})
I want to use google translation api but I have some problems.
My env is Linux ubuntu 18 and python with Atom idle
I was used gcloud to set my configuration and got auth login, auth login token.
export GOOGLE_APPLICATION_CREDENTIALS=//api_key.json
gcloud init
gcloud auth application-default login
gcloud auth application-default print-access-token
so I could use curl and got some test data
curl -X POST -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) -H "Content-Type: application/json; charset=utf-8" --data
"{
'q': 'Hello world',
'q': 'My name is Jeff',
'target': 'de'
}" "https://translation.googleapis.com/language/translate/v2"
{
"data": {
"translations": [
{
"translatedText": "Hallo Welt",
"detectedSourceLanguage": "en"
},
{
"translatedText": "Mein Name ist Jeff",
"detectedSourceLanguage": "en"
}
]
}
}
When I run test code in Atom idle, my project number is wrong.
It is my past project.
Even I run test code in bash python, it is same situation
I dont know what is wrong, I just guess some problem in python env.
raised error
raise exceptions.from_http_response(response)
google.api_core.exceptions.Forbidden: 403 POST
https://translation.googleapis.com/language/translate/v2: Cloud Translation
API has not been used in project [wrong number] before or it is disabled.
Enable it by visiting
https://console.developers.google.com/apis/api/translate.googleapis.com
/overview?project=[wrong number] then retry. If you enabled this API
recently, wait a few minutes for the action to propagate to our systems and
retry.
This error message is usually thrown when the application is not being authenticated correctly due to several reasons such as missing files, invalid credential paths, incorrect environment variables assignations, among other causes. Since the Client Libraries need to pull the credentials data from the environment variable or the client object, it is required to ensure you are pointing to the correct authentication files. Keep in mind this issue might not occur when using CURL command because you were passing the access-token directly.
Based on this, I recommend you to confirm that you are using the JSON file credentials of your current project, as well as follow the Obtaining and providing service account credentials manually guide in order to explicitly specify your service account file directly into your code; In this way, you will be able to set it permanently and verify if you are passing the service credentials correctly. Additionally, you can take a look on Using Client Libraries guide that contains the step-by-step process required to use the Translation API with Python.
Passing the path to the service account key in code example:
def explicit():
from google.cloud import storage
# Explicitly use service account credentials by specifying the private key
# file.
storage_client = storage.Client.from_service_account_json('service_account.json')
# Make an authenticated API request
buckets = list(storage_client.list_buckets())
print(buckets)
I need to access the Jenkins JSON API from a Python script. The problem is that our Jenkins installation is secured so to log in users have to select a certificate. Sadly, in Jenkins Remote Access Documentation they don't mention a thing about certificates and I tried using the API Token without success.
How can I get to authenticate from a Python script to use their JSON API?
Thanks in advance!
You have to authenticate to the JSON API using HTTP Basic Auth.
To make scripted clients (such as wget) invoke operations that require authorization (such as scheduling a build), use HTTP BASIC authentication to specify the user name and the API token. This is often more convenient than emulating the form-based authentication
https://wiki.jenkins-ci.org/display/JENKINS/Authenticating+scripted+clients
Here is a sample of using Basic Auth with Python.
http://docs.python-requests.org/en/master/user/authentication/
Keep in mind if you are using a Self Signed certificate on an internal Jenkin Server you'll need to turn off certificate validation OR get the certificate from the server and add it to the HTTP request
http://docs.python-requests.org/en/master/user/advanced/
I finally found out how to authenticate to Jenkins using certs and wget. I had to convert my pfx certificates into pem ones with cert and keys in separate files For more info about that come here. In the end this is the command I used.
wget --certificate=/home/B/cert.pem --private-key=/home/B/key.pem --no-check-certificate --output-document=jenkins.json https:<URL>
I'm not completely sure it covers your certificate use case, but since it took me some time to find out, I still want to share this snipped that retrieves the email address for a given user name in Python without special Jenkins libraries. It uses an API token and "supports" (actually ignores) https:
def _get_email_adress(user):
request = urllib.request.Request("https://jenkins_server/user/"+ user +"/api/json")
#according to https://stackoverflow.com/a/28052583/4609258 the following is ugly
context = ssl._create_unverified_context()
base64string = base64.b64encode(bytes('%s:%s' % ('my user name', 'my API token'),'ascii'))
request.add_header("Authorization", "Basic %s" % base64string.decode('utf-8'))
with urllib.request.urlopen(request, context=context) as url:
user_data = json.loads(url.read().decode())
for property in user_data['property']:
if property["_class"]=="hudson.tasks.Mailer$UserProperty":
return property["address"];