We Keep Coding

'Incorrect Padding' error while using google.oauth2 id_token.verify_oauth2_token() in Django

I am trying to verify a user's submitted google authentication JWT from a React frontend to my Django backend. I have my frontend POST a request to the backend with the JWT, and then my backend extracts it from the request and processes it through google's recommend token authentication found here. My problem is that each time I process a JWT received from my frontend through id_token.verify_ouath2_token(), an error about 'incorrect padding' is raised ultimately stemming from the base64 encoding of the token.
When testing this manually by copy-pasting the token I have extracted from the POST request, however, it works perfectly fine and does not raise an error. So somewhere in my handling of the post request something goes wrong, but I'm not sure what. I have tried decoding the POST request in utf-8 and ascii, and simply leaving the JWT in the bytes form that I extracted it as. All of these throw the same error, and I'm left pretty confused.
Here is my code:
views.py
def login(request):
if request.method == "POST":
print(request.body.decode("ascii")) # For decoding. By copy/pasting this output into id_token.verify_oauth2_token instead of the token variable, the function returns successfully
try:
token = request.body.decode("ascii")
# Specify the CLIENT_ID of the app that accesses the backend:
idinfo = id_token.verify_oauth2_token(token, requests.Request(), "MY CLIENT ID")
# ID token is valid. Get the user's Google Account ID from the decoded token.
userid = idinfo['sub']
return(HttpResponse(userid))
except ValueError as e:
# Invalid token
print(e)
return(HttpResponse("Failure"))
else:
pass
and the traceback
File "C:\Users\E\Google Drive\Projects\Django Project\Website\BackEnd\test\Main\views.py", line 80, in login
idinfo = id_token.verify_oauth2_token(token, requests.Request(), "MY CLIENT ID")
File "C:\Users\E\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\oauth2\id_token.py", line 147, in verify_oauth2_token
id_token, request, audience=audience, certs_url=_GOOGLE_OAUTH2_CERTS_URL
File "C:\Users\E\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\oauth2\id_token.py", line 126, in verify_token
return jwt.decode(id_token, certs=certs, audience=audience)
File "C:\Users\E\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\auth\jwt.py", line 230, in decode
header, payload, signed_section, signature = _unverified_decode(token)
File "C:\Users\E\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\auth\jwt.py", line 148, in _unverified_decode
signature = _helpers.padded_urlsafe_b64decode(signature)
File "C:\Users\E\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\auth\_helpers.py", line 215, in padded_urlsafe_b64decode
return base64.urlsafe_b64decode(padded)
File "C:\Users\E\AppData\Local\Programs\Python\Python36-32\lib\base64.py", line 133, in urlsafe_b64decode
return b64decode(s)
File "C:\Users\E\AppData\Local\Programs\Python\Python36-32\lib\base64.py", line 87, in b64decode
return binascii.a2b_base64(s)
binascii.Error: Incorrect padding

After some experimentation I figured it out. It looks like request.body.decode("ascii") ended up producing the JWT encased in double quotes, and the double quotes were what was causing the problem. After stripping them from the string it works perfectly.
Final code in views.py
#csrf_exempt
def login(request):
if request.method == "POST":
try:
token = request.body.decode("ascii").strip("\"")
# Specify the CLIENT_ID of the app that accesses the backend:
idinfo = id_token.verify_oauth2_token(token, requests.Request(), "MY CLIENT ID")

How to obtain csrf token for a python POST request to Django server

Without the csrf token I am getting a 403 Response, so it looks like I need it. I am currently trying to do this:
import requests
import sys
URL = 'http://127.0.0.1:8000/toasterinfo'
client = requests.session()
# Retrieve the CSRF token first
client.get(URL) # sets cookie
csrftoken = client.cookies['csrf']
r = client.post(URL, data={'number': 12524,
'type': 'issue', 'action': 'show', "csrfmiddlewaretoken": csrftoken}, headers=dict(Referer=URL))
print(r)
Yet I am getting an error:
Traceback (most recent call last):
File "django_client.py", line 10, in <module>
csrftoken = client.cookies['csrftoken']
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/cookies.py", line 327, in __getitem__
return self._find_no_duplicates(name)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/cookies.py", line 398, in _find_no_duplicates
raise KeyError('name=%r, domain=%r, path=%r' % (name, domain, path))
KeyError: "name='csrftoken', domain=None, path=None"
What am I doing wrong?

The CSRF Token is a client to server security feature to protect your users from a cross-site request forgery.
You are doing a server to server request so the CSRF is useless. You should consider removing the requirement for the CSRF and using a proper authentification method for your server to server request.
You can remove the CSRF token verification used the following decorator on your view.
#csrf_exempt
See https://docs.djangoproject.com/en/dev/_modules/django/views/decorators/csrf/#csrf_exempt
A token based authentification method with an HTTPS connexion would probably fit your case.

It's not the safest way but you could add a decorator to that view to exclude the CSRF verification and your POST won't fail.
from django.views.decorators.csrf import requires_csrf_token
#requires_csrf_token
def your_view(request):
# your code...
This solution only applies if you are aware of the risks removing the CSRF verification.

Splunk Python Connection Lost

I'm using python to execute a splunk search query and return the results. I connect with the following string:
service = client.connect(
host=HOST,
port=PORT,
username=USERNAME,
password=PASSWORD
)
The variables have been tested to work, and it connects to splunk, but sometimes, when I run these lines of code:
print "Installed App Names \n"
for app in service.apps:
print app.name
It returns this error:
Request Failed: Session is not logged in
About 50% of the time, the code works, and it executes. Is this inconsistency in code results do to the service = lines of code not actually connecting to the splunk server? Can these connections time out?

connect can take an autologin=True argument to allow the bindings to try to re-connect when authentication fails, instead of raising that error immediately.

Probably you should get the token and session id of splunk using your python code. Please find the below code if this could help you.
import json,os,sys,requests
BASE_URL = "https://SPLUNKLB / SPLUNK WEB URL"
def getToken():
# body for token request
payload = {'username': "",'password': ""}
TOKEN_URL = "/services/auth/login?output_mode=json"
# post token request
res = requests.post(BASE_URL+TOKEN_URL, data=payload, verify=False)
if (res.status_code == 200):
# Get token out of response
resJson = json.loads(res.content)
return resJson.get('sessionKey')
else:
print res.status_code, res.content

FIWARE OAuth2 Authentication in Python

I am trying to authenticate user using FIWARE.
It returns a 404. Thus fails at Step 1 itself. What is the access token url ? Any other pointers to check
I have tried variations with 'oauth/access_token', 'oauth/token' 'oauth2/token' 'oauth2/access_token' . All of them dont seem to work.
My Code is Below:
import oauth2 as oauth
# OAuth secret into your project's settings.
consumer = oauth2.Consumer(settings.FIWARE_CLIENT_ID,settings.FIWARE_CLIENT_SECRET)
client = oauth2.Client(consumer)
access_token_url = 'https://account.lab.fiware.org/oauth2/access_token'
# This is the slightly different URL used to authenticate/authorize.
authenticate_url = 'https://account.lab.fiware.org/oauth2/authorize'
def fiware_login(request):
# Step 1. Get a request token from FIWARE.
resp, content = client.request(access_token_url, "GET")
print resp
if resp['status'] != '200':
print content
raise Exception("Invalid response from FIWARE.")
# Step 2. Redirect the user to the authentication URL.
url = "%s?access_token=%s" % (authenticate_url,
resp['access_token'])
return HttpResponseRedirect(url)

Correct endpoint is "/oauth2/token".
Maybe you should use POST method instead of GET.
For more information see https://github.com/ging/fi-ware-idm/wiki/Using-the-FI-LAB-instance

pywikipedia bot with https and http authentication

I'm having trouble getting my bot to login to a MediaWiki install on the intranet. I believe it is due to the http authentication protecting the wiki.
Facts:
The wiki root is: https://local.example.com/mywiki/
When visiting the wiki with a web browser, a popup comes up asking for enterprise credentials (I assume this is basic access authentication)
This is what I have in my user-config.py:
mylang = 'en'
family = 'mywiki'
usernames['mywiki']['en'] = u'Bot'
authenticate['local.example.com'] = ('user', 'pass')
This is what I have in mywiki_family.py:
# -*- coding: utf-8 -*-
import family, config
# The Wikimedia family that is known as mywiki
class Family(family.Family):
def __init__(self):
family.Family.__init__(self)
self.name = 'mywiki'
self.langs = { 'en' : 'local.example.com'}
def scriptpath(self, code):
return '/mywiki'
def version(self, code):
return '1.13.5'
def isPublic(self):
return False
def hostname(self, code):
return 'local.example.com'
def protocol(self, code):
return 'https'
def path(self, code):
return '/mywiki/index.php'
When I execute login.py -v -v, I get this:
urllib2.urlopen(urllib2.Request('https://local.example.com/w/index.php?title=Special:Userlogin&useskin=monobook&action=submit', wpSkipCookieCheck=1&wpPassword=XXXX&wpDomain=&wpRemember=1&wpLoginattempt=Aanmelden%20%26%20Inschrijven&wpName=Bot, {'Content-type': 'application/x-www-form-urlencoded', 'User-agent': 'PythonWikipediaBot/1.0'})):
(Redundant traceback info here)
urllib2.HTTPError: HTTP Error 401: Unauthorized
(I'm not sure why it has 'local.example.com/w' instead of '/mywiki'.)
I thought it might be trying to authenticate to example.com instead of example.com/wiki, so I changed the authenticate line to:
authenticate['local.example.com/mywiki'] = ('user', 'pass')
But then I get an HTTP 401.2 error back from IIS:
You do not have permission to view this directory or page using the credentials that you supplied because your Web browser is sending a WWW-Authenticate header field that the Web server is not configured to accept.
Any help on how to get this working would be appreciated.
Update After fixing my family file, it now says:
Getting information for site mywiki:en
('http error', 401, 'Unauthorized', )
WARNING: Could not open 'https://local.example.com/mywiki/index.php?title=Non-existing_page&action=edit&useskin=monobook'. Maybe the server or your connection is down. Retrying in 1 minutes...
I looked at the HTTP headers on a plan urllib2.ulropen call and it's using WWW-Authenticate: Negotiate WWW-Authenticate: NTLM. I'm guessing urllib2 and thus pywikipedia don't support this?
Update Added a tasty bounty for help in getting this to work. I can authenticate using python-ntlm. How do I integrate this into pywikipedia?

Well the fact that login.py tries accessing '\w' instead of your path shows that there is a family configuration issue.
Your code is indented strangely: is scriptpath a member of the new Family class? as in:
class Family(family.Family):
def __init__(self):
family.Family.__init__(self)
self.name = 'mywiki'
self.langs = { 'en' : 'local.example.com'}
def scriptpath(self, code):
return '/mywiki'
def version(self, code):
return '1.13.5'
def isPublic(self):
return False
def hostname(self, code):
return 'local.example.com'
def protocol(self, code):
return 'https'
?
I believe that something is wrong with your family file. A good way to check is to do in a python console:
import wikipedia
site = wikipedia.getSite('en', 'mywiki')
print site.login_address()
as long as the relative address is wrong, showing '/w' instead of '/mywiki', it means that the family file is still not configured correctly, and that the bot won't work :)
Update: how to integrate ntlm in pywikipedia?
I just had a look at the basic example here. I would integrate the code before that line in login.py:
response = urllib2.urlopen(urllib2.Request(self.site.protocol() + '://' + self.site.hostname() + address, data, headers))
You want to write something of the like:
from ntlm import HTTPNtlmAuthHandler
user = 'DOMAIN\User'
password = "Password"
url = self.site.protocol() + '://' + self.site.hostname()
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, url, user, password)
# create the NTLM authentication handler
auth_NTLM = HTTPNtlmAuthHandler.HTTPNtlmAuthHandler(passman)
# create and install the opener
opener = urllib2.build_opener(auth_NTLM)
urllib2.install_opener(opener)
response = urllib2.urlopen(urllib2.Request(self.site.protocol() + '://' + self.site.hostname() + address, data, headers))
I would test this and integrate it directly into pywikipedia codebase if only I had an available ntlm setup...
Whatever happens, please do not vanish with your solution: we're interested, at pywikipedia, by your solution :)

I am guessing the problem you have is that the server expects basic authentication and you are not handling that in your client. Michael Foord wrote a good article about handling basic authentication in Python.
You did not provide enough information for me to be sure about this, so if that does not work, please provide some additional information, like network dump of you connection attempt.