We Keep Coding

from_db_value() missing 1 required positional argument: 'context'

Trying to create google login for django using googleapiclient and oauth2client.
I am able to open home page, and getting redirected to google login successfully. Once after sign-in, it is redirecting to home page where I'm receiving this error.
Reference link/tutorial
views.py
import httplib2
from googleapiclient.discovery import build
from django.http import HttpResponseBadRequest
from django.http import HttpResponseRedirect
from .models import CredentialsModel
from gfglogin import settings
from oauth2client.contrib import xsrfutil
from oauth2client.client import flow_from_clientsecrets
from oauth2client.contrib.django_util.storage import DjangoORMStorage
from django.shortcuts import render
from httplib2 import Http
def home(request):
print("*****home*****")
status = True
if not request.user.is_authenticated:
return HttpResponseRedirect('admin')
storage = DjangoORMStorage(CredentialsModel, 'id', request.user, 'credential')
credential = storage.get()
print(f"credential: {credential}")
print(f"storage: {storage}")
try:
access_token = credential.access_token
resp, cont = Http().request("https://www.googleapis.com/auth/gmail.readonly",
headers={'Host': 'www.googleapis.com',
'Authorization': access_token})
except:
status = False
print('Not Found')
return render(request, 'index.html', {'status': status})
################################
# GMAIL API IMPLEMENTATION #
################################
# CLIENT_SECRETS, name of a file containing the OAuth 2.0 information for this
# application, including client_id and client_secret, which are found
# on the API Access tab on the Google APIs
# Console <http://code.google.com/apis/console>
FLOW = flow_from_clientsecrets(
settings.GOOGLE_OAUTH2_CLIENT_SECRETS_JSON,
scope='https://www.googleapis.com/auth/gmail.readonly',
redirect_uri='http://127.0.0.1:8080/oauth2callback',
prompt='consent')
def gmail_authenticate(request):
print("*****gmail_authenticate*****")
storage = DjangoORMStorage(CredentialsModel, 'id', request.user, 'credential')
credential = storage.get()
print(f"credential: {credential}")
if credential is None or credential.invalid:
FLOW.params['state'] = xsrfutil.generate_token(settings.SECRET_KEY,
request.user)
authorize_url = FLOW.step1_get_authorize_url()
return HttpResponseRedirect(authorize_url)
else:
http = httplib2.Http()
http = credential.authorize(http)
service = build('gmail', 'v1', http=http)
print('access_token = ', credential.access_token)
status = True
return render(request, 'index.html', {'status': status})
def auth_return(request):
print("*****auth_return*****")
get_state = bytes(request.GET.get('state'), 'utf8')
if not xsrfutil.validate_token(settings.SECRET_KEY, get_state,
request.user):
return HttpResponseBadRequest()
credential = FLOW.step2_exchange(request.GET.get('code'))
print(f"credential: {credential}")
storage = DjangoORMStorage(CredentialsModel, 'id', request.user, 'credential')
storage.put(credential)
print(f"storage: {storage}")
print("access_token: %s" % credential.access_token)
return HttpResponseRedirect("/")
Error:
System check identified no issues (0 silenced).
June 06, 2020 - 14:38:49
Django version 3.0.7, using settings 'gfglogin.settings'
Starting development server at http://127.0.0.1:8080/
Quit the server with CTRL-BREAK.
[06/Jun/2020 14:39:00] "GET /admin/ HTTP/1.1" 200 3042
*****home*****
credential: None
storage: <oauth2client.contrib.django_util.storage.DjangoORMStorage object at 0x0000014A630EABE0>
Not Found
[06/Jun/2020 14:39:05] "GET / HTTP/1.1" 200 327
*****gmail_authenticate*****
credential: None
[06/Jun/2020 14:39:07] "GET /gmailAuthenticate HTTP/1.1" 302 0
*****auth_return*****
credential: <oauth2client.client.OAuth2Credentials object at 0x0000014A6322BA00>
storage: <oauth2client.contrib.django_util.storage.DjangoORMStorage object at 0x0000014A6322B550>
access_token: ya29.a0AfH6SMBLyCWC3cV4iiMk0jWUJaw8ruUFoBqFTkM5LT2acAc6FelcoADU3tn67RslO-24dKEFqrdp4tcLFVuEIMvmn7cHKeb8XeZ9YNQozRoRSTU6hs-jMA9bHP10epw1ImbBaY8SUgQUtF75mRRniR0aELEmzTVKGe8
[06/Jun/2020 14:40:26] "GET /oauth2callback?state=VaASAVe2IusAHCsQfc7EfToxNTkxNDM0NTQ3&code=4/0gFCWPMx4qTUrv4wUWpSKbA8Z_rTaZb-YGGDRrK4NsK2UiW6f4MpA_g1Pr5RpyGcRxbCtWlH6qHvKIJvbpG_L9c&scope=https://www.googleapis.com/auth/gmail.readonly HTTP/1.1" 302 0
*****home*****
Internal Server Error: /
Traceback (most recent call last):
File "D:\PERSONAL DATA\env\lib\site-packages\django\core\handlers\exception.py", line 34, in inner
response = get_response(request)
File "D:\PERSONAL DATA\env\lib\site-packages\django\core\handlers\base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "D:\PERSONAL DATA\env\lib\site-packages\django\core\handlers\base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "D:\PERSONAL DATA\raw_login\google-oauth-mail\gfgauth\views.py", line 23, in home
credential = storage.get()
File "D:\PERSONAL DATA\env\lib\site-packages\oauth2client\client.py", line 407, in get
return self.locked_get()
File "D:\PERSONAL DATA\env\lib\site-packages\oauth2client\contrib\django_util\storage.py", line 58, in locked_get
if len(entities) > 0:
File "D:\PERSONAL DATA\env\lib\site-packages\django\db\models\query.py", line 258, in __len__
self._fetch_all()
File "D:\PERSONAL DATA\env\lib\site-packages\django\db\models\query.py", line 1261, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "D:\PERSONAL DATA\env\lib\site-packages\django\db\models\query.py", line 74, in __iter__
for row in compiler.results_iter(results):
File "D:\PERSONAL DATA\env\lib\site-packages\django\db\models\sql\compiler.py", line 1096, in apply_converters
value = converter(value, expression, connection)
TypeError: from_db_value() missing 1 required positional argument: 'context'
One of SO post, suggest to replace context argument with *args, **kwargs. I'm not able to see from_db_value method in source code.
versions:
django == 3.0.7
python == 3.8
Can anyone please respond to this question and help?
I have also seen that oauth2client is depreciated. Is there any other library with clear example that I can follow? or Can anyone post example code?
In github issue, someone says
You can't use oauth2client's storage with google-auth. google-auth
credentials are relatively straight forward to persist, see:
https://github.com/GoogleCloudPlatform/google-auth-library-python-oauthlib/blob/master/google_auth_oauthlib/tool/main.py#L80
But, as new to this, I'm not able to understand implement this. Please help!

The problem seems to be with oauth2client, which is no longer updated or supported. From the README:
Note: oauth2client is now deprecated. No more features will be added to the libraries and the core team is turning down support. We recommend you use google-auth and oauthlib. For more details on the deprecation, see oauth2client deprecation.
So I'd suggest you take a look at those two packages and integrate those instead.
The takeaway here is that you should be more careful about the tutorials you choose. This one uses Django 2.0, which hasn't been supported for over a year. Always look for tutorials which match either the latest release or the latest LTS release, or you're asking for trouble.

Python HTTPS request SSLError CERTIFICATE_VERIFY_FAILED

PYTHON
import requests
url = "https://REDACTED/pb/s/api/auth/login"
r = requests.post(
url,
data = {
'username': 'username',
'password': 'password'
}
)
NIM
import httpclient, json
let client = newHttpClient()
client.headers = newHttpHeaders({ "Content-Type": "application/json" })
let body = %*{
"username": "username",
"password": "password"
}
let resp = client.request("https://REDACTED.com/pb/s/api/auth/login", httpMethod = httpPOST, body = $body)
echo resp.body
I'm calling an API to get some data. Running the python code I get the traceback below. However, the nim code works perfectly so there must be something wrong with the python code or setup.
I'm running Python version 2.7.15.
requests lib version 2.19.1
Traceback (most recent call last):
File "C:/Python27/testht.py", line 21, in <module>
"Referer": "https://REDACTED.com/pb/a/"
File "C:\Python27\lib\site-packages\requests\api.py", line 112, in post
return request('post', url, data=data, json=json, **kwargs)
File "C:\Python27\lib\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 512, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 622, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests\adapters.py", line 511, in send
raise SSLError(e, request=request)
SSLError: HTTPSConnectionPool(host='REDACTED.com', port=443): Max retries exceeded with url: /pb/s/api/auth/login (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),))

The requests module will verify the cert it gets from the server, much like a browser would. Rather than being able to click through and say "add exception" like you would in your browser, requests will raise that exception.
There's a way around it though: try adding verify=False to your post call.

However, the nim code works perfectly so there must be something wrong with the python code or setup.
Actually, your Python code or setup is less to blame but instead the nim code or better the defaults on the httpclient library. In the documentation for nim can be seen that httpclient.request uses a SSL context returned by getDefaultSSL by default which according to this code creates a context which does not verify the certificate:
proc getDefaultSSL(): SSLContext =
result = defaultSslContext
when defined(ssl):
if result == nil:
defaultSSLContext = newContext(verifyMode = CVerifyNone)
Your Python code instead attempts to properly verify the certificate since the requests library does this by default. And it fails to verify the certificate because something is wrong - either with your setup or the server.
It is unclear who has issued the certificate for your site but if it is not in your default CA store you can use the verify argument of requests to specify the issuer CA. See this documentation for details.
If the site you are trying to access works with the browser but fails with your program it might be that it uses a special CA which was added as trusted to the browser (like a company certificate). Browsers and Python use different trust stores so this added certificate needs to be added to Python or at least to your program as trusted too. It might also be that the setup of the server has problems. Browsers can sometimes work around problems like a missing intermediate certificate but Python doesn't. In case of a public accessible site you could use SSLLabs to check what's wrong.

Travelport Galileo python SoapClient

I need to develop python soapclient for Travelport Galileo uAPI.
This is 30-day trial credentials for Travelport Universal API
Universal API User ID: Universal API/uAPI2514620686-0edbb8e4
Universal API Password: D54HWfck9nRZNPbXmpzCGwc95
Branch Code for Galileo (1G): P7004130
URLs: https://emea.universal-api.pp.travelport.com/B2BGateway/connect/uAPI/
This is quote from documentation galileo
HTTP Header
The HTTP header includes:
SOAP endpoints, which vary by:
Geographical region.
Requested service. In the preceding example, the HotelService is used for the endpoint; however, the service name is modified based on the request transaction.
gzip compression, which is optional, but strongly recommended. To accept gzip compression in the response, specify “Accept-Encoding: gzip,deflate” in the header.
Authorization, which follows the standard basic authorization pattern.
The text that follows “Authorization: Basic” can be encoded using Base 64. This functionality is supported by most programming languages.
The syntax of the authorization credentials must include the prefix "Universal API/" before the User Name and Password assigned by Travelport.
POST https://americas.universal-api.pp.travelport.com/
B2BGateway/connect/uAPI/HotelService HTTP/2.0
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: ""
Authorization: Basic UniversalAPI/UserName:Password
Content-Length: length
This is i my python code
import urllib2
import base64
import suds
class HTTPSudsPreprocessor(urllib2.BaseHandler):
def http_request(self, req):
message = \
"""
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:air="http://www.travelport.com/schema/air_v16_0" xmlns:com="http://www.travelport.com/schema/common_v13_0" -->
<soapenv:header>
<soapenv:body>
<air:availabilitysearchreq xmlns:air="http://www.travelport.com/schema/air_v16_0" xmlns:com="http://www.travelport.com/schema/common_v13_0" authorizedby="Test" targetbranch="P7004130">
<air:searchairleg>
<air:searchorigin>
<com:airport code="LHR">
</com:airport></air:searchorigin>
<air:searchdestination>
<com:airport code="JFK">
</com:airport></air:searchdestination>
<air:searchdeptime preferredtime="2011-11-08">
</air:searchdeptime></air:searchairleg>
</air:availabilitysearchreq>
</soapenv:body>
"""
auth = base64.b64encode('Universal API/uAPI2514620686-0edbb8e4:D54HWfck9nRZNPbXmpzCGwc95')
req.add_header('Content-Type', 'text/xml; charset=utf-8')
req.add_header('Accept', 'gzip,deflate')
req.add_header('Cache-Control','no-cache')
req.add_header('Pragma', 'no-cache')
req.add_header('SOAPAction', '')
req.add_header('Authorization', 'Basic %s'%(auth))
return req
https_request = http_request
URL = "https://emea.universal-api.pp.travelport.com/B2BGateway/connect/uAPI/"
https = suds.transport.https.HttpTransport()
opener = urllib2.build_opener(HTTPSudsPreprocessor)
https.urlopener = opener
suds.client.Client(URL, transport = https)
But it is not working.
Traceback (most recent call last):
File "soap.py", line 42, in <module>
suds.client.Client(URL, transport = https)
File "/usr/local/lib/python2.7/site-packages/suds/client.py", line 112, in __init__
self.wsdl = reader.open(url)
File "/usr/local/lib/python2.7/site-packages/suds/reader.py", line 152, in open
d = self.fn(url, self.options)
File "/usr/local/lib/python2.7/site-packages/suds/wsdl.py", line 136, in __init__
d = reader.open(url)
File "/usr/local/lib/python2.7/site-packages/suds/reader.py", line 79, in open
d = self.download(url)
File "/usr/local/lib/python2.7/site-packages/suds/reader.py", line 95, in download
fp = self.options.transport.open(Request(url))
File "/usr/local/lib/python2.7/site-packages/suds/transport/http.py", line 64, in open
raise TransportError(str(e), e.code, e.fp)
suds.transport.TransportError: HTTP Error 500: Dynamic backend host not specified
I'm trying to solve this problem for the past 2 weeks, so if you can, please advise me solution.

I think you can try to download WSDL files in ZIP archive from this url https://support.travelport.com/webhelp/uAPI/uAPI.htm#Getting_Started/Universal_API_Schemas_and_WSDLs.htm
So you will be able to generate your client classes using those WSDL files, because there is no WSDL endpoint on the https://emea.universal-api.pp.travelport.com/B2BGateway/connect/uAPI/
(like ?wsdl or /.wsdl)

How to send request to endpoint with Boto

I am trying to list items in a S3 container with the following code.
import boto.s3
from boto.s3.connection import OrdinaryCallingFormat
conn = boto.connect_s3(calling_format=OrdinaryCallingFormat())
mybucket = conn.get_bucket('Container001')
for key in mybucket.list():
print key.name.encode('utf-8')
Then I get the following error.
Traceback (most recent call last):
File "test.py", line 5, in <module>
mybucket = conn.get_bucket('Container001')
File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 370, in get_bucket
bucket.get_all_keys(headers, maxkeys=0)
File "/usr/lib/python2.7/dist-packages/boto/s3/bucket.py", line 358, in get_all_keys
'', headers, **params)
File "/usr/lib/python2.7/dist-packages/boto/s3/bucket.py", line 325, in _get_all
response.status, response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 301 Moved Permanently
<?xml version="1.0" encoding="UTF-8"?>
PermanentRedirectThe bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.99EBDB9DE3B6E3AF
Container001
<HostId>5El9MLfgHZmZ1UNw8tjUDAl+XltYelHu6d/JUNQsG3OaM70LFlpRchEJi9oepeMy</HostId><Endpoint>Container001.s3.amazonaws.com</Endpoint></Error>
I tried to search for how to send requests to the specified end point, but couldn't find useful information.
How do I avoid this error?

As #garnaat mentioned and #Rico answered in another question connect_to_region works with OrdinaryCallingFormat:
conn = boto.s3.connect_to_region(
region_name = '<your region>',
aws_access_key_id = '<access key>',
aws_secret_access_key = '<secret key>',
calling_format = boto.s3.connection.OrdinaryCallingFormat()
)
bucket = conn.get_bucket('<bucket name>')

in terminal run
nano ~/.boto
if there is some configs try to comment or rename file and connect again. (it helps me)
http://boto.cloudhackers.com/en/latest/boto_config_tut.html
there is boto config file directories. take a look one by one and clean them all, it will work by default configs. also configs may be in .bash_profile, .bash_source...
I guess you must allow only KEY-SECRET
also try to use
calling_format = boto.s3.connection.OrdinaryCallingFormat()

Requesting Token via Django Piston Throws TypeError Exception

When trying to authenticate via OAuth in Django Piston, the following exception is thrown:
Environment:
Request Method: GET
Request URL: http://localhost:8000/api/oauth/request_token/?oauth_nonce=32921052&oauth_timestamp=1291331173&oauth_consumer_key=ghof7av2vu8hal2hek&oauth_signature_method=HMAC-SHA1&oauth_version=1.0&oauth_signature=
Python Version:
Traceback:
File "/Users/derek/.virtualenvs/optimal-rest/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
100. response = callback(request, *callback_args, **callback_kwargs)
File "/Users/derek/.virtualenvs/optimal-rest/lib/python2.7/site-packages/piston/authentication.py" in oauth_request_token
130. token = oauth_server.fetch_request_token(oauth_request)
File "/Users/derek/.virtualenvs/optimal-rest/lib/python2.7/site-packages/piston/oauth.py" in fetch_request_token
302. self._check_signature(oauth_request, consumer, None)
File "/Users/derek/.virtualenvs/optimal-rest/lib/python2.7/site-packages/piston/oauth.py" in _check_signature
393. valid_sig = signature_method.check_signature(oauth_request, consumer, token, signature)
File "/Users/derek/.virtualenvs/optimal-rest/lib/python2.7/site-packages/piston/oauth.py" in check_signature
482. built = self.build_signature(oauth_request, consumer, token)
File "/Users/derek/.virtualenvs/optimal-rest/lib/python2.7/site-packages/piston/oauth.py" in build_signature
513. hashed = hmac.new(key, raw, sha)
File "/usr/local/Cellar/python/2.7/lib/python2.7/hmac.py" in new
133. return HMAC(key, msg, digestmod)
File "/usr/local/Cellar/python/2.7/lib/python2.7/hmac.py" in __init__
72. self.outer.update(key.translate(trans_5C))
Exception Type: TypeError at /api/oauth/request_token/?oauth_nonce=32921052&oauth_timestamp=1291331173&oauth_consumer_key=ghof7av2vu8hal2hek&oauth_signature_method=HMAC-SHA1&oauth_version=1.0&oauth_signature=
Exception Value: character mapping must return integer, None or unicode
Can't tell if it's a bug in Piston or if I can't use the oauth2 lib.
Consumer Code:
import os
import cgi
import oauth2 as oauth
# settings for the local test consumer
CONSUMER_SERVER = os.environ.get("CONSUMER_SERVER") or 'localhost'
CONSUMER_PORT = os.environ.get("CONSUMER_PORT") or '8000'
print CONSUMER_SERVER , CONSUMER_PORT
# fake urls for the test server (matches ones in server.py)
REQUEST_TOKEN_URL = 'http://%s:%s/api/oauth/request_token/' % (CONSUMER_SERVER, CONSUMER_PORT)
ACCESS_TOKEN_URL = 'http://%s:%s/api/oauth/access_token/' % (CONSUMER_SERVER, CONSUMER_PORT)
AUTHORIZE_URL = 'http://%s:%s/api/oauth/authorize/' % (CONSUMER_SERVER, CONSUMER_PORT)
# key and secret granted by the service provider for this consumer application - same as the MockOAuthDataStore
CONSUMER_KEY = 'ghof7av2vu8hal2hek'
CONSUMER_SECRET = 'ohhey'
consumer = oauth.Consumer(CONSUMER_KEY, CONSUMER_SECRET)
client = oauth.Client(consumer)
# Step 1: Get a request token. This is a temporary token that is used for
# having the user authorize an access token and to sign the request to obtain
# said access token.
resp, content = client.request(REQUEST_TOKEN_URL, "GET")
if resp['status'] != '200':
raise Exception("Invalid response %s." % resp['status'])
Referencing https://github.com/clemesha/django-piston-oauth-example for the consumer code.

This is a piston problem that comes from an encoding problem of the key/secret of the consumer.
The solution is to force the encoding of the key/secret returned from the database to ASCII.
In the store.py file of Piston, modify the lookup_consumer so it look like this:
def lookup_consumer(self, key):
try:
self.consumer = Consumer.objects.get(key=key)
self.consumer.key = self.consumer.key.encode('ascii')
self.consumer.secret = self.consumer.secret.encode('ascii')
return self.consumer
except Consumer.DoesNotExist:
return None
Here is my fork of django-piston fixing this problem.

This problem also occurs inside Piston's "oauth.py" module's "build_signature()" method if a unicode key value is passed in. I discovered this issue while using the clemesha/django-piston-oauth-example client code mentioned above because it kept failing after the prompt for the "PIN Code".
The underlying problem is documented as issue #169 here:
https://bitbucket.org/jespern/django-piston/issue/169/oauth-request-token-error-with-hmac
The re-encoding solution described above is also applicable in this case:
def build_signature(self, oauth_request, consumer, token):
"""Builds the base signature string."""
key, raw = self.build_signature_base_string(oauth_request, consumer,
token)
#BUG: character mapping must return integer, None or unicode
#FIX:
key = key.encode('ascii')
# HMAC object.
try:
import hashlib # 2.5
hashed = hmac.new(key, raw, hashlib.sha1)
except:
import sha # Deprecated
hashed = hmac.new(key, raw, sha)
# Calculate the digest base 64.
return binascii.b2a_base64(hashed.digest())[:-1]