We Keep Coding

Azure POST login.microsoftonline.com/oauty2/token fails on AADSTS900144

I need to automate connecting to logs analytics on Azure. Before I can do that, I need to get an access token.
With different docs and https://www.youtube.com/watch?v=ujzrq8Fg9Gc, I am trying to set that up.
TRY1
My first try was using SoapUI to send a POST request to:
https://login.microsoftonline.com/MY TENANT ID/oauth2/token
?grant_type=client_credentials
&client_id=MY CLIENT ID
&redirect_uri=MY URL
&resource=https%3A%2F%2Fwestus2.api.loganalytics.io
&client_secret=MY CLIENT SECRET
With header:
Content-Type: application/x-www-form-urlencoded
I always get this response:
HTTP/1.1 400 Bad Request
Cache-Control: no-cache, no-store
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
x-ms-request-id: SOMETHING
P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"
Set-Cookie: fpc=SOMETHING; expires=Mon, 05-Aug-2019 13:14:50 GMT; path=/; secure; HttpOnly
Set-Cookie: x-ms-gateway-slice=prod; path=/; secure; HttpOnly
Set-Cookie: stsservicecookie=ests; path=/; secure; HttpOnly
Date: Sat, 06 Jul 2019 13:14:49 GMT
Content-Length: 437
{
"error":"invalid_request",
"error_description":"AADSTS900144: The request body must contain
the following parameter: 'grant_type'.\r\n
Trace ID: SOMETHING\r\n
Correlation ID: SOMETHING\r\n
Timestamp: 2019-07-06 13:14:50Z",
"error_codes":[900144],
"timestamp":"2019-07-06 13:14:50Z",
"trace_id":"SOMETHING",
"correlation_id":"SOMETHING"
}
TRY 2
I wrote it in Python using import requests, like so:
import os,sys
import requests
Azure_client_id = 'MY CLIENT ID'
Azure_redirect_uri = 'MY URL'
Azure_client_secret = 'CLIENT SECRET'
Azure_tenant_id = 'TENANT ID'
Azure_resource = 'https://westus2.api.loganalytics.io'
###############################################################################
token_url = 'https://login.microsoftonline.com/' + Azure_tenant_id + '/oauth2/token'
token_headers = {
'Content-type': 'application/x-www-form-urlencoded',
}
token_params = {
'grant_type': 'client_credentials',
'client_id': Azure_client_id,
'redirect_uri': Azure_redirect_uri,
'resource': Azure_resource,
'client_secret': Azure_client_secret,
}
token_response = requests.post(token_url, headers=token_headers, params=token_params)
# This is to see what was sent
print(token_response.url + "\n\n")
# get the response and print it
token_result = ''
for chunk in token_response.iter_content(chunk_size=128):
token_result = token_result + str(chunk)
print(token_result.replace("\\n","\n"))
The URL that was sent is this (formatted for readability):
https://login.microsoftonline.com/MY TENANT ID/oauth2/token
?grant_type=client_credentials
&client_id=MY CLIENT ID
&redirect_uri=https%3A%2F%2FMY URL
&resource=https%3A%2F%2Fwestus2.api.loganalytics.io
&client_secret=MY SECRET URL ENCODED
The response I get is this (reformatted for readability):
b'{"error":"invalid_request",
"error_description":"AADSTS900144: The request body must contain
the following parameter: \'grant_type'b"'.\\r\
Trace ID: SOMETHING\\r\
Correlation ID: SOMETHING\\r\
Timestamp: 2019-"b'07-06 13:49:59Z",
"error_codes":[900144],
"timestamp":"2019-07-06 13:49:59Z",
"trace_id":"SOMETHING",
"co'b'rrelation_id":"SOMETHING"}''
At least I get the same error (!). Since my requests clearly include a "grant_type" parameter, my guess is that there is either something wrong with the encoding (which is done by SoapUI and Python's requests), something wrong with my URL, or I am not using the proper IDs.
Is there a way in Azure to validate that my client secret is valid? Once created, it cannot be read anymore. And someone else created this key so I cannot assume what he gave me is ok.
Any comment, hints, pointing out blatant mistakes on my part are appreciated.

Change
token_response = requests.post(token_url, headers=token_headers, params=token_params)
to
token_response = requests.post(token_url, data=token_params)
You don't need to specify the Content-type header, it's inferred from your payload (dictionary, so x-www-form-urlencoded), also data is what you want (payload), not params (URL parameters).
Your request should look like this on the wire -
POST /TENANT_ID/oauth2/token HTTP/1.1
Host: localhost:9000
User-Agent: python-requests/2.22.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Content-Length: 151
Content-Type: application/x-www-form-urlencoded
grant_type=client_credentials
&client_id=MY+CLIENT+ID
&redirect_uri=MY+URL
&resource=https%3A%2F%2Fwestus2.api.loganalytics.io
&client_secret=CLIENT+SECRET
Everything is in the body where it should be for x-www-form-urlencoded.
More on x-www-form-urlencoded here -
https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST#Example

Python. Cannot make a request to simple site api. Flask and requests

I am trying to create simple API for my site. I created the route with flask:
#api.route('/api/rate&message_id=<message_id>&performer=<performer_login>', methods=['POST'])
def api_rate_msg(message_id, performer_login):
print("RATE API ", message_id, ' ', performer_id)
return 400
print(...) function don't execute...
I use flask-socketio to communicate between client and server.
I send json from client and process it with:
#socket.on('rate')
def handle_rate(data):
print(data)
payload = {'message_id':data['message_id'], 'performer':data['performer']}
r = requests.post('/api/rate', params=payload)
print (r.status_code)
Note, that data variable is sending from client and is correct(I've checked it).
print(r.status_code) don't exec too...
Where I'm wrong? Please, sorry for my bad english :(
This api function must increase rate of message, which stored in mongodb, if interesting.

Don't put &message_id=<message_id>&performer=<performer_login> in your route string. Instead, get these arguments from request.args.
Try it:
from flask import request
...
#api.route('/api/rate', methods=['POST'])
def api_rate_msg():
print(request.args)
return ''
I've tested it with httpie:
$ http -v POST :5000/api/rate message_id==123 performer_login==foo
POST /api/rate?message_id=123&performer_login=foo HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 0
Host: localhost:5000
User-Agent: HTTPie/0.9.8
HTTP/1.0 200 OK
Content-Length: 0
Content-Type: text/html; charset=utf-8
Date: Sun, 02 Apr 2017 13:54:40 GMT
Server: Werkzeug/0.11.11 Python/2.7.13
And from flask's log:
ImmutableMultiDict([('message_id', u'123'), ('performer_login', u'foo')])
127.0.0.1 - - [02/Apr/2017 22:54:40] "POST /api/rate?message_id=123&performer_login=foo HTTP/1.1" 200 -

Remove the below part from your api route
&message_id=<message_id>&performer=<performer_login
This is not required in POST request. It helps in GET requests. API call in request is not matching the route definition and therefore you have the current problem

httplib - http not accepting content length

Problem
When I switched Macbooks, all of the sudden I am getting an HTTP 411: Length Required (I wasn't getting this using a different Mac) trying to use a POST request with httplib. I cannot seem to find a work around for this.
Code Portion 1: from a supporting class; retrieves data and other things,
class Data(object):
def __init__(self, value):
self.company_id = None
self.host = settings.CONSUMER_URL
self.body = None
self.headers = {"clienttype": "Cloud-web", "Content-Type": "application/json", "ErrorLogging": value}
def login(self):
'''Login and store auth token'''
path = "/Security/Login"
body = self.get_login_info()
status_code, resp = self.submit_request("POST", path, json.dumps(body))
self.info = json.loads(resp)
company_id = self.get_company_id(self.info)
self.set_token(self.info["token"])
return company_id
def submit_request(self, method, path, body=None, header=None):
'''Submit requests for API tests'''
conn = httplib.HTTPSConnection(self.host)
conn.set_debuglevel(1)
conn.request(method, path, body, self.headers)
resp = conn.getresponse()
return resp.status, resp.read()
Code Portion 2: my unittests,
# logging in
cls.api = data.Data(False) # initializing the data class from Code Portion 1
cls.company_id = cls.api.login()
...
# POST Client/Register
def test_client_null_body(self):
'''Null body object - 501'''
status, resp = self.api.submit_request('POST', '/Client/register')
if status != 500:
log.log_warning('POST /Client/register: %s, %s' % (str(status), str(resp)))
self.assertEqual(status, 500)
Code Portion 3: example of the data I send from a settings file,
API_ACCOUNT = {
"userName": "account#account.com",
"password": "password",
"companyId": 107
}
From Logging
WARNING:root: POST /Client/register: 411, <!DOCTYPE HTML PUBLIC "-//W3C//DTD
HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Length Required</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Length Required</h2>
<hr><p>HTTP Error 411. The request must be chunked or have a content length.</p>
</BODY></HTML>
Additional Info: I was using a 2008 Macbook Pro without issue. Switched to a 2013 Macbook Pro and this keeps occurring.
I took a look at this post:
Python httplib and POST and it seems that at the time httplib did not automatically generate the content length.
Now https://docs.python.org/2/library/httplib.html:
If one is not provided in headers, a Content-Length header is added automatically for all methods if the length of the body can be determined, either from the length of the str representation, or from the reported size of the file on disk.
when using conn.set_debuglevel(1) we see that httplib is sending a header
reply: 'HTTP/1.1 411 Length Required\r\n'
header: Content-Type: text/html; charset=us-ascii
header: Server: Microsoft-HTTPAPI/2.0
header: Date: Thu, 26 May 2016 17:08:46 GMT
header: Connection: close
header: Content-Length: 344
Edit
Unittest Failure:
======================================================================
FAIL: test_client_null_body (__main__.NegApi)
Null body object - 501
----------------------------------------------------------------------
Traceback (most recent call last):
File "API_neg.py", line 52, in test_client_null_body
self.assertEqual(status, 500)
AssertionError: 411 != 500
.send: 'POST /Client/register HTTP/1.1\r\nHost: my.host\r\nAccept-Encoding: identity\r\nAuthorizationToken: uhkGGpJ4aQxm8BKOCH5dt3bMcwsHGCHs1p+OJvtf9mHKa/8pTEnKyYeJr+boBr8oUuvWvZLr1Fd+Og2xJP3xVw==\r\nErrorLogging: False\r\nContent-Type: application/json\r\nclienttype: Cloud-web\r\n\r\n'
reply: 'HTTP/1.1 411 Length Required\r\n'
header: Content-Type: text/html; charset=us-ascii
header: Server: Microsoft-HTTPAPI/2.0
header: Date: Thu, 26 May 2016 17:08:27 GMT
header: Connection: close
header: Content-Length: 344
Any ideas as to why this was working on a previous Mac and is currently not working here? It's the same code, same operating systems. Let me know if I can provide any more information.
Edit 2
The issue seemed to be with OSX 10.10.4, after upgrading to 10.10.5 all is well. I still would like to get some insight on why I was having this issue.
The only change from 10.10.4 to 10.10.5, that seems close, would have been the python update from 2.7.6 to 2.7.10 which includes this bug fix: http://bugs.python.org/issue22417

Unable to update github file using CRUD API (using github-flask/python)

I am not able to update a file using github-flask.
It gives me a 404 error (https://developer.github.com/v3/troubleshooting/#why-am-i-getting-a-404-error-on-a-repository-that-exists) but I have verified that my auth token is good and has repo scope. I verified so by
Anshu-MacBook-Pro:~ anshup$ curl -H "Authorization: token mytoken" https://api.github.com/users/anshprat -I
HTTP/1.1 200 OK
Server: GitHub.com
Date: Fri, 11 Jul 2014 07:21:34 GMT
Content-Type: application/json; charset=utf-8
Status: 200 OK
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4991
X-RateLimit-Reset: 1405065424
Cache-Control: private, max-age=60, s-maxage=60
Last-Modified: Fri, 11 Jul 2014 06:25:54 GMT
ETag: "etag"
X-OAuth-Scopes: repo
My code is:
def git_push(file_path,file_path_s,commit_message,raw_file_data):
u_file_data = base64.b64encode(raw_file_data,salt())
#print u_file_data
params = {
'path':file_path_s
'message':commit_message,
'content':u_file_data,
'sha':file_sha #Its a global variable
}
#print raw_file_data,u_file_data,params
params_json = json.dumps(params)
return github.post(file_path,params_json) #params fails as well
The flask frame dumps shows:
resource 'repos/anshprat/git-test/contents/play1.txt'
status_code '404'
self <flask_github.GitHub object at 0x2733b90>
response <Response [404]>
kwargs {'headers': {'Content-Type': 'application/x-www-form-urlencoded'}, 'data': '"{\\"content\\": \\"dXNlcjE6cGFzc3dvcmQxCnVzZXIyOnBhc3N3b3JkMgp1c2VyMzpNQlpvRDJLOWJQQS9vCg==\\", \\"sha\\": \\"ce4979fccba7ef259910c355ca6e1993d7ea436c\\", \\"message\\": \\"modifying user user3\\"}"' }
method 'POST'
If I use params in the post, the double quotes are not escaped.
POST also fails for me on curl, so I think I am doing something very basic wrong out here?
curl -H "Authorization: token mytoken" --data '{"content": "dXNlcjE6cGFzc3dvcmQxCnVzZXIyOnBhc3N3b3JkMgp1c2VyMzpEQXR2VUNweUJrbjdFCg==", "message": "modifying user user3", "sha": "ce4979fccba7ef259910c355ca6e1993d7ea436c"}' https://api.github.com/repos/anshprat/git-test/contents/play1.txt
{
"message": "Not Found",
"documentation_url": "https://developer.github.com/v3"
}
github.get for the same file_path works.
The API is listed at
https://developer.github.com/v3/repos/contents/#update-a-file
My app should be doing a code commit and then deploy it to some other non github related work.

I had to change
def post(self, resource, data, **kwargs):
"""Shortcut for ``request('POST', resource)``.
Use this to make POST request since it will also encode ``data`` to
'application/x-www-form-urlencoded' format."""
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
to
def post(self, resource, data, **kwargs):
"""Shortcut for ``request('POST', resource)``.
Use this to make POST request since it will also encode ``data`` to
'application/x-www-form-urlencoded' format."""
headers = {}
headers['Content-type'] = 'application/json'
headers['X-HTTP-Method-Override'] = 'PUT'
for it work correctly.

HTTP Authentication headers with no "realm"

The following code works but gives no output even though there is a file instance lying behind response
import urllib2
from ntlm import HTTPNtlmAuthHandler
user = 'id'
password = "Password"
url = "http://abc.def.ghij:3080"
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, url, user, password)
# create the NTLM authentication handler
auth_NTLM = HTTPNtlmAuthHandler.HTTPNtlmAuthHandler(passman)
# create and install the opener
opener = urllib2.build_opener(auth_NTLM)
urllib2.install_opener(opener)
# retrieve the result
response = urllib2.urlopen("http://abc.def.ghij:3080")
print response.read()
print response.headers
header response:
Cache-Control: private
Content-Length: 0
Location: /index.epx
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.19
Persistent-Auth: true
X-Powered-By: ASP.NET
Date: Thu, 04 Jul 2013 15:30:03 GMT
Connection: close
but print response.read doesnt give any content :O

WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
The problem is that your webserver is requesting NTLM authentication, so it wont accept BasicAuth. Use NTML authentication while sending request or change your webserver config to allow Basic authentication.
Dont use BasicAuth because it sends user/pwd in plaintext over the network. Minimum safe is DigestAuth or NTLM or GSSNegotiate auth.