We Keep Coding

How to get MinIO metrics (/minio/v2/metrics/cluster)?

Bases on the MinIO documentation at page 14 following API is mentioned /minio/v2/metrics/cluster and /minio/v2/metrics/node.
But looks like these API available only for "Prometheus". And I have no clue to get those using python.
Of course if I try send request to http://<host>:<port>/minio/v2/metrics/cluster I get 403 error:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AllAccessDisabled</Code><Message>All access to this bucket has been disabled.</Message><Resource>/minio/v2/metrics/cluster</Resource><RequestId></RequestId><HostId><uuid-here></HostId></Error>
I've tried to generate presigned URL (like for GET object from bucket) but even with signature result is same as above.
Maybe someone know to reach that API(s)?
And how to do proper authentication for these endpoint without "Prometheus"?

Prometheus just requests an HTTP resource, which presents metrics in plain text format. You can grab the same URL that prometheus would use and then parse it yourself.
If you look at this document, it describes how to configure authentication for the metrics endpoint.
Configure authentication type for Prometheus metrics
MinIO supports two authentication modes for Prometheus either jwt or
public, by default MinIO runs in jwt mode. To allow public access
without authentication for prometheus metrics set environment as
follows.
export MINIO_PROMETHEUS_AUTH_TYPE="public" minio server ~/test
If you're running it in jwt mode (the default), the following
sections tells you how to generate a configuration that will show you
the appropriate bearer token to use for authentication:
3.1 Authenticated Prometheus config
If MinIO is configured to expose metrics without authentication, you
don't need to use mc to generate prometheus config. You can skip
reading further and move to 3.2 section.
The Prometheus endpoint in MinIO requires authentication by default.
Prometheus supports a bearer token approach to authenticate
prometheus scrape requests, override the default Prometheus config
with the one generated using mc. To generate a Prometheus config for
an alias, use mc as follows mc admin prometheus generate .
The command will generate the scrape_configs section of the
prometheus.yml as follows:
scrape_configs:
- job_name: minio-job
bearer_token: <secret>
metrics_path: /minio/v2/metrics/cluster
scheme: http
static_configs:
- targets: ['localhost:9000']
You can just use the bearer token and path from that configuration to
access your metrics. The bearer token should go into an
Authorization header (Authorization: Bearer <token>).

Solution:
import jwt # pyJWT
import requests
import datetime
expired_at = (datetime.datetime.now() + datetime.timedelta(days=1)).timestamp()
token = jwt.encode(
payload={
"exp": expired_at,
"sub": "<AccessKey>",
"iss": "prometheus",
},
key="<SecretKey>",
algorithm="HS512"
)
headers = {
"Authorization": f"Bearer {token.decode()}"
}
response = requests.get(
url="<schema>://<host>:<port>/minio/prometheus/metrics",
headers=headers
)
How mc utility generates bearer token I found at https://github.com/minio/mc/blob/master/cmd/admin-prometheus-generate.go
URL /minio/prometheus/metrics was found during call to utility mc admin prometheus generate test-minio

Google Admin Directory API - Send a query via apiclient

I am retrieving a ChromeOS device MAC address via the Google Admin Directory API using the device's Serial Number as reference, and am making my calls through
apiclient.
service = discovery.build('admin', 'directory_v1', developerKey=settings.API_KEY)
Here are the calls available for ChromeOS devices; my issue is that I require a Device ID in order to execute the following:
service.chromeosdevices().get(customerId=settings.CID, deviceId=obtained_id, projection=None).execute()
I can send a GET query via the following format:
https://www.googleapis.com/admin/directory/v1/customer/my_customer/devices/chromeos?projection=full&query=id:" + serial + "&orderBy=status&sortOrder=ascending&maxResults=10", "GET")
... but I'm trying to avoid using OAuth2 and just use my API key. Passing the key in a GET request doesn't work either, as it still returns a "Login Required" notice.
How do I squeeze the above query into an apiclient-friendly format? The only option I found via the above calls was to request every device we have (via list), then sift through the mountain of data for the matching Serial number, which seems silly and excessive.
I did notice I could call apiclient.http.HttpRequests, but I couldn't find a way to pass the API key through it either. There's new_batch_http_request, but I can't discern from the docs how to simply pass a URL to it.
Thank you!

Got it!
You can't use just a key for Directory API queries, you need a Service account.
I'm using google-auth (see here) since oauth2client is deprecated.
You also need to:
Delegate the necessary permissions for your service account (mine has the role of Viewer and has scope access to https://www.googleapis.com/auth/admin.directory.device.chromeos.readonly)
Delegate API access to it separately in the Admin Console (Security -> Advanced Settings -> Authentication)
Get your json client secret key and place it with your app (don't include it in your VCS)
Obtain your credentials like this:
credentials = service_account.Credentials.from_service_account_file(
settings.CLIENT_KEY,
scopes=settings.SCOPES,
subject=settings.ADMIN_USER)
where ADMIN_USER is the email address of an authorized Domain admin.
Then you send a GET request like so:
authed_session = AuthorizedSession(credentials)
response = authed_session.get(request_id_url)
This returns a Requests object you can read via response.content.
Hope it helps someone else!

Connect to Mojio REST api (Python, OAuth2)

I'm trying to connect to Mojio REST api authenticated with OAuth2 with Python server code. Here's the 'manual' from Mojio: https://docs.moj.io/#/document/view/doc_oauth
I'm doing the second option there 'Authorization Code'. I don't understand what is the 'redirect_uri' parametr for. What should I pass there when I'm developing on localhost?
Thanks

You might want to consider using the "Resource Owner Password Credentials Grant" vs normal UI based oAuth workflows. This allows you to capture the username and password in your application and authenticate with those credentials instead of a UI (which can be challenging if you are on a server). Of course this depends on your specific requirements.
To login, via "Resource Owner" flow...
HTTP POST
Uri:
https://accounts.moj.io/oauth2/token
Body:
grant_type=password&username=USERNAME&password=PASSWORD&redirect_uri=REDIRECT_URI&client_id=CLIENTID&client_secret=CLIENTSECRET&scope=SCOPE
Content-Type:
application/x-www-form-urlencoded
This token you get back will be short lived, I would recommend getting a refresh token (longer lifetime):
To get a refresh token:
HTTP POST
Uri:
https://accounts.moj.io/oauth2/token
Body:
grant_type=refresh_token&refresh_token=API_TOKEN_YOU_GOT_FROM_LOGIN&redirect_uri=REDIRECT_URI&client_id=CLIENTID&client_secret=CLIENTSECRET
Content-Type:
application/x-www-form-urlencoded

How to authenticate in Jenkins while remotely accessing its JSON API?

I need to access the Jenkins JSON API from a Python script. The problem is that our Jenkins installation is secured so to log in users have to select a certificate. Sadly, in Jenkins Remote Access Documentation they don't mention a thing about certificates and I tried using the API Token without success.
How can I get to authenticate from a Python script to use their JSON API?
Thanks in advance!

You have to authenticate to the JSON API using HTTP Basic Auth.
To make scripted clients (such as wget) invoke operations that require authorization (such as scheduling a build), use HTTP BASIC authentication to specify the user name and the API token. This is often more convenient than emulating the form-based authentication
https://wiki.jenkins-ci.org/display/JENKINS/Authenticating+scripted+clients
Here is a sample of using Basic Auth with Python.
http://docs.python-requests.org/en/master/user/authentication/
Keep in mind if you are using a Self Signed certificate on an internal Jenkin Server you'll need to turn off certificate validation OR get the certificate from the server and add it to the HTTP request
http://docs.python-requests.org/en/master/user/advanced/

I finally found out how to authenticate to Jenkins using certs and wget. I had to convert my pfx certificates into pem ones with cert and keys in separate files For more info about that come here. In the end this is the command I used.
wget --certificate=/home/B/cert.pem --private-key=/home/B/key.pem --no-check-certificate --output-document=jenkins.json https:<URL>

I'm not completely sure it covers your certificate use case, but since it took me some time to find out, I still want to share this snipped that retrieves the email address for a given user name in Python without special Jenkins libraries. It uses an API token and "supports" (actually ignores) https:
def _get_email_adress(user):
request = urllib.request.Request("https://jenkins_server/user/"+ user +"/api/json")
#according to https://stackoverflow.com/a/28052583/4609258 the following is ugly
context = ssl._create_unverified_context()
base64string = base64.b64encode(bytes('%s:%s' % ('my user name', 'my API token'),'ascii'))
request.add_header("Authorization", "Basic %s" % base64string.decode('utf-8'))
with urllib.request.urlopen(request, context=context) as url:
user_data = json.loads(url.read().decode())
for property in user_data['property']:
if property["_class"]=="hudson.tasks.Mailer$UserProperty":
return property["address"];

Basic authentication with jira-python

I'm new to Python, new to the jira-python library, and new to network programming, though I do have quite a bit of experience with application and integration programming and database queries (though it's been a while).
Using Python 2.7 and requests 1.0.3
I'm trying to use this library - http://jira-python.readthedocs.org/en/latest/ to query Jira 5.1 using Python. I successfully connected using an unauthenticated query, though I had to make a change to a line in client.py, changing
I changed
self._session = requests.session(verify=verify, hooks={'args': self._add_content_type})
to
self._session = requests.session()
I didn't know what I was doing exactly but before the change I got an error and after the change I got a successful list of project names returned.
Then I tried basic authentication so I can take advantage of my Jira permissions and do reporting. That failed initially too. And I made the same change to
def _create_http_basic_session
in client.py , but now I just get another error. So problem not solved. Now I get a different error:
HTTP Status 415 - Unsupported Media Type
type Status report
message Unsupported Media Type
description The server refused this request because the request entity is in
a format not` `supported by the requested resource for the requested method
(Unsupported Media Type).
So then I decided to do a super simple test just using the requests module, which I believe is being used by the jira-python module and this code seemed to log me in. I got a good response:
import requests
r = requests.get(the_url, auth=(my username , password))
print r.text
Any suggestions?

Here's how I use the jira module with authentication in a Python script:
from jira.client import JIRA
import logging
# Defines a function for connecting to Jira
def connect_jira(log, jira_server, jira_user, jira_password):
'''
Connect to JIRA. Return None on error
'''
try:
log.info("Connecting to JIRA: %s" % jira_server)
jira_options = {'server': jira_server}
jira = JIRA(options=jira_options, basic_auth=(jira_user, jira_password))
# ^--- Note the tuple
return jira
except Exception,e:
log.error("Failed to connect to JIRA: %s" % e)
return None
# create logger
log = logging.getLogger(__name__)
# NOTE: You put your login details in the function call connect_jira(..) below!
# create a connection object, jc
jc = connect_jira(log, "https://myjira.mydom.com", "myusername", "mypassword")
# print names of all projects
projects = jc.projects()
for v in projects:
print v

Below Python script connects to Jira and does basic authentication and lists all projects.
from jira.client import JIRA
options = {'server': 'Jira-URL'}
jira = JIRA(options, basic_auth=('username', 'password'))
projects = jira.projects()
for v in projects:
print v
It prints a list of all the project's available within your instance of Jira.

Problem:
As of June 2019, Atlassian Cloud users who are using a REST endpoint in Jira or Confluence Cloud with basic or cookie-based authentication will need to update their app or integration processes to use an API token, OAuth, or Atlassian Connect.
After June 5th, 2019 attempts to authenticate via basic auth with an Atlassian account password will return an invalid credentials error.
Reference: Deprecation of basic authentication with passwords for Jira and Confluence APIs
Solution to the Above-mentioned Problem:
You can use an API token to authenticate a script or other process with an Atlassian cloud product. You generate the token from your Atlassian account, then copy and paste it to the script.
If you use two-step verification to authenticate, your script will need to use a REST API token to authenticate.
Steps to Create an API Token from your Atlassian Account:
Log in to https://id.atlassian.com/manage/api-tokens
Click Create API token.
From the dialog that appears, enter a memorable and concise Label for your token and click Create.
Click Copy to clipboard, then paste the token to your script.
Reference: API tokens
Python 3.8 Code Reference
from jira.client import JIRA
jira_client = JIRA(options={'server': JIRA_URL}, basic_auth=(JIRA_USERNAME, JIRA_TOKEN))
issue = jira_client.issue('PLAT-8742')
print(issue.fields.summary)

Don't change the library, instead put your credentials inside the ~/.netrc file.
If you put them there you will also be able to test your calls using curl or wget.
I am not sure anymore about compatibility with Jira 5.x, only 7.x and 6.4 are currently tested. If you setup an instance for testing I could modify the integration tests to run against it, too.
My lucky guess is that you broke it with that change.

As of 2019 Atlassian has deprecated authorizing with passwords.
You can easily replace the password with an API Token created here.
Here's a minimalistic example:
pip install jira
from jira import JIRA
jira = JIRA("YOUR-JIRA-URL", basic_auth=("YOUR-EMAIL", "YOUR-API-TOKEN"))
issue = jira.issue("YOUR-ISSUE-KEY (e.g. ABC-13)")
print(issue.fields.summary)
I recommend storing your API Token as an environment variable and accessing it with os.environ[key].