Resolve external API response json with Flask and Graphene - python

Last week I posted a kind of vague question as I was trying to join data from an external Rest API with a local SQLAlchemy schema. Unsurprisingly, I didn't get a lot of responses and after some experimentation with parsing the response json into a temporary table I've decided to move away from this approach and add a new resolver for the external API that can be called separately from the resolver for SQLAlchemy. This is what I have so far for the querying the external API:
class Red(graphene.ObjectType):
redName = graphene.String()
accessLevel = graphene.String()
numBytes = graphene.Int()
class Query(graphene.ObjectType):
red = graphene.Field(Red, redName = graphene.String(required=True))
def resolve_red(self, info, **kwargs):
dataset_Red = <custom library for querying API>.dataset(kwargs['redName'])
dataset_Red.get()
resp = dataset_Red.properties
return json.dumps(resp)
GraphiQL recognizes my new resolver and will run the query with no errors but it returns no data. For example, if this is my query:
query{red(redName:"<dataset_name>") {
accessLevel
numBytes
}
}
I get the following response:
{
"data": {
"red": {
"accessLevel": null,
"numBytes": null
}
}
}
What did I miss? I'm thinking there's an issue with class definition. Can someone show me what I did wrong?

I think you should return Red type from resolver resolve_red instead of json dumped string:
class Red(graphene.ObjectType):
redName = graphene.String()
accessLevel = graphene.String()
numBytes = graphene.Int()
class Query(graphene.ObjectType):
red = graphene.Field(Red, redName = graphene.String(required=True))
def resolve_red(self, info, **kwargs):
dataset_Red = <custom library for querying API>.dataset(kwargs['redName'])
dataset_Red.get()
resp = dataset_Red.properties
return Red(
redName=resp.get('redName'),
accessLevel=resp.get('accessLevel'),
NumBytes=resp.get('numBytes')
)
Or assuming fields in response are the same as Red attributes:
class Red(graphene.ObjectType):
redName = graphene.String()
accessLevel = graphene.String()
numBytes = graphene.Int()
class Query(graphene.ObjectType):
red = graphene.Field(Red, redName = graphene.String(required=True))
def resolve_red(self, info, **kwargs):
dataset_Red = <custom library for querying API>.dataset(kwargs['redName'])
dataset_Red.get()
resp = dataset_Red.properties
return Red(**resp)

Sound I found a solution that seems to work. The issue was the response type, I had to serialize and then deserialize it for flask (obvious right?).
I added this two functions:
def _json_object_hook(d):
return namedtuple('X', d.keys())(*d.values())
def json2obj(data):
return json.loads(data, object_hook=_json_object_hook)
And then changed my resolver to call json2obj on the response:
def resolve_red(self, info, **kwargs):
dataset_Red = <custom library for querying API>.dataset(kwargs['redName'])
dataset_Red.get()
resp = dataset_Red.properties
return json2obj(json.dumps(resp))
I found this solution in another stackoverflow post but now I can't find the original. If anyone else finds it, please add a link in the comments and I'll add it to this answer.

Related

Django: AuthStateMissing at /oauth/complete/google-oauth2/

I am using social-auth-app-django for GoogleOauth2 authentication. It works fine for all users but in case of django admin it gives me following error:
AuthStateMissing at /oauth/complete/google-oauth2/
Session value state missing.
I have tried all answers posted on stackoverflow but the error still persists. This is the result it returns.
The state value seems to be present there but either it gets null or overridden somehow.
This is my GoogleOAuth2 class, created by overriding social-auth-app-django's GoogleOAuth2 class. Though there is not much difference except for pipeline from base class. It works fine for non-admin user login.
class GoogleOAuth2(GoogleOAuth2):
"""Google OAuth2 authentication backend"""
name = 'google-oauth2'
REDIRECT_STATE = False
AUTHORIZATION_URL = 'https://accounts.google.com/o/oauth2/auth'
ACCESS_TOKEN_URL = 'https://accounts.google.com/o/oauth2/token'
ACCESS_TOKEN_METHOD = 'POST'
REVOKE_TOKEN_URL = 'https://accounts.google.com/o/oauth2/revoke'
REVOKE_TOKEN_METHOD = 'GET'
# The order of the default scope is important
DEFAULT_SCOPE = ['openid', 'email', 'profile']
EXTRA_DATA = [
('refresh_token', 'refresh_token', True),
('expires_in', 'expires'),
('token_type', 'token_type', True)
]
def pipeline(self, pipeline, pipeline_index=0, *args, **kwargs):
out = self.run_pipeline(pipeline, pipeline_index, *args, **kwargs)
user_ip = get_request_ip_address(self.strategy.request)
if not isinstance(out, dict):
return out
user = out.get('user')
if user:
user.social_user = out.get('social')
user.is_new = out.get('is_new')
if user.is_new:
logger.info(f'Register attempt', extra={"email": user.email, "remote_ip": user_ip, "status": "success", "user_id": user.pk, "oauth_backend": "google"})
else:
logger.info(f'Login attempt', extra={"email": user.email, "remote_ip": user_ip, "status": "success", "user_id": user.pk, "oauth_backend": "google"})
return user
I have tried following solutions, setting these values in settings.py file:
SOCIAL_AUTH_REDIRECT_IS_HTTPS = True
SESSION_COOKIE_SAMESITE = None
SESSION_COOKIE_HTTPONLY = False

How to use postman to upload DBRef data in json?

I'm trying to post data to mongodb using postman but I don't know the proper convention for uploading the reference to a image file in the fs.files bucket. Basically, the file is already in the database, I'm just trying to post a new user with the reference to the image.
Here is my model:
class Users(db.Document):
_id = db.StringField()
name = db.StringField()
picture = db.FileField()
email = db.StringField()
password = db.StringField()
meta = {'collection': 'Users'}
In postman, I try to post data like so:
{
"_id" : "1",
"name" : "John Doe",
"picture": [{"$id": "5e6a...f9q102"}], #This is the reference id for the image already in the database, in fs.files
"password" : "<hashed pw>",
"email" : "example#example.com"
}
I'm using flask restful api so in the python script, the post function is defined like so:
def post(self):
body = request.get_json()
print (body)
user = Users()
user = Users(**body).save()
return 'Successful Upload', 200
I get the error when I try with the above convention:
mongoengine.errors.ValidationError: ValidationError (Users:None) ('list' object has no attribute
'grid_id': ['picture'])
How do I post a new user in postman? Your help is appreciated
You need to change a bit your code
Add these imports:
from mongoengine.fields import ImageGridFsProxy
from mongoengine import ReferenceField, DynamicDocument
from bson.dbref import DBRef
from bson import ObjectId
Modify your class picture field definition + add extra class fs
class Fs(DynamicDocument):
#Add 'db_alias':'default' to meta
meta = {'collection': 'fs.files'}
class Users(Document):
...
picture = ReferenceField('Fs', dbref=True)
...
Now, you need to create new instance for DBRef this way:
def post(self):
body = request.get_json()
body["picture"] = DBRef('fs.files', ObjectId(body["picture"]))
#mongoengine assumes `ObjectId('xxx')` already exists in `fs.files`.
#If you want to check, run below code:
#if Fs.objects(_id=body["picture"].id).first() is None:
# return 'Picture ' + str(body["picture"].id) + ' not found', 400
user = Users(**body).save()
return 'Successful Upload', 200
At the end, if you need to read picture content:
image = ImageGridFsProxy(grid_id=ObjectId('xxx'))
f = open("image.png", "wb")
f.write(image.read())
f.close()
It was a Validation error. The database was accepting JSON in a particular format than what I was posting. And the way I was processing the post request was also incorrect. This is the format it expected:
{
...,
"picture" = {"$ref": "fs.files",
"$id": ObjectId("5e6a...f9q102")},
...
}
Postman cannot accept the above format, instead, it accepted this:
{
"_id" : "1",
"name" : "John Doe",
"picture": {"$ref": "fs.files", "$id": {"$oid": "5e6a...f9q102"}},
"password" : "<hashed pw>",
"email" : "example#example.com"
}
To make this work I changed the model to look this so in my flask app:
class Users(db.Document):
_id = db.StringField()
name = db.StringField()
picture = db.ReferenceField('fs.files') #I changed this to a reference field because it holds the reference for the file and not the actual file in the database
upload_picture = db.FileField() #I added this field so I can still upload pics via flask and via this document
email = db.StringField()
password = db.StringField()
meta = {'collection': 'Users'}
Then I had to add this import and change the code so that it would read the input as JSON and transfer the reference value of picture to ObjectId(id) so that it matches the format the database was expecting.
from bson.json_util import loads
def post(self):
body = str(request.get_json())
x = body.replace("'", '"') #replace single quotes as double quotes to match JSON format
data = loads(x)
officer = Officers(**data).save()
return 'Successful Upload', 200
Then voila, it works!

Why do I get NameError for the column that is defined in SQLAlchemy model

I am developing REST APIs with Flask. One of the tables is modeled as follows:
class AudioSessionModel(db.Model):
__tablename__ = 'audio_session'
id = db.Column('audio_session_id', db.Integer, primary_key = True)
cs_id = db.Column(db.Integer)
session_id = db.Column(db.Integer)
facility = db.Column(db.Integer)
description = db.Column(db.String(400))
def __init__(self, cs_id, session_id, facility):
self.cs_id = cs_id
self.session_id = session_id
self.facility = facility
Business logics are defined in a DAO class:
class AudioSessionDAO(object):
def update(self, data):
audio = AudioSessionModel.query.filter(cs_id == data['CSID'], session_id == data['Session'])
audio.description = data['Desc']
db.session.commit()
return audio
This upate function is called in my endpoint for PUT request:
#api.route('/OperatorAccessment')
class OperatorAssessment(Resource):
#api.expect(assessment)
def put(self):
as_dao = AudioSessionDAO()
as_dao.update(request.json)
The model assessment looks like this:
assessment = api.model('Operator Assessment', {
'CSID': fields.Integer(required=True, description='Central Station ID'),
'Session': fields.Integer(required=True, description='Session ID'),
'Desc': fields.String(description='Description')
})
When I test the PUT request with the following json in request body:
{
"CSID": 1,
"Session": 1,
"Desc": "Siren"
}
I got the following error:
File "C:\Users\xxx_app\model\dao.py", line 63, in update
audio = AudioSessionModel.query.filter(cs_id == data['CSID'], session_id == data['Session'])
NameError: name 'cs_id' is not defined
Apparently, cs_id is defined. Why am I still getting this error?
You have to use the attributes of the class, i.e.
AudioSessionModel.query.filter(
AudioSessionModel.cs_id == data['CSID'],
AudioSessionModel.session_id == data['Session'])
Or filter_by with keyword arguments using just =:
AudioSessionModel.query.filter_by(
cs_id=data['CSID'],
session_id=data['Session'])
See What's the difference between filter and filter_by in SQLAlchemy?

Uploading files in Google App Engine. How to get the file to the upload Handler Class

I have a form in google app engine where I want to upload an image and all my text at the same time. Do I have to seperate this into two seperate pages and actions?
Here is my upload handler:
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
def upload(self, reseller_id, imgfile):
upload_files = imgfile
blob_info = upload_files[0]
key = blob_info.key()
r = Reseller.get_by_id(reseller_id)
r.blob_key_logo = str(key)
r.put();
Here is my creation of a new reseller object:
class NewReseller(BaseHandler):
def get(self):
if self.user:
self.render("new_reseller.html")
else:
self.redirect("/display_resellers")
def post(self):
name = self.request.get('name')
website = self.request.get('website')
information = self.request.get('information')
address = self.request.get('address')
city = self.request.get('city')
state = self.request.get('state')
zipcode = self.request.get('zipcode')
email = self.request.get('email')
phone = self.request.get('phone')
r = Reseller( name = name,
website = website,
information = information,
address = address,
city = city,
state = state,
zipcode = zipcode,
email = email,
phone = phone)
r.put()
theresellerid = r.key().id()
#And then Upload the image
u = UploadHandler()
logo_img = u.get_uploads('logo_img')
u.upload(theid, logo_img)
self.redirect('/display_resellers')
I think my problem here is this line:
logo_img = u.get_uploads('logo_img')
it pops out the error message
for key, value in self.request.params.items():
AttributeError: 'NoneType' object has no attribute 'params'
Somehow I need this NewReseller class to inherit the .getuploads from BlobstoreUploadHandler so I can do:
logo_img = self.get_uploads('logo_img')
Or there is probably a better way because this seems a little messy.
So my question is how to upload files and data in one form on just one page. I could do it with two seperate pages. One for adding the reseller and one for adding the image but that seems over complicated.
I tried to follow some steps and clues from this question:
Upload files in Google App Engine
******Edit***** Working Implementation Below:
class EditReseller(BaseHandler, blobstore_handlers.BlobstoreUploadHandler):
def get(self, reseller_id):
if self.user:
reseller = Reseller.get_by_id(int(reseller_id))
upload_url = blobstore.create_upload_url('/upload')
image = True
if reseller.blob_key_logo is None:
image = False
self.render('edit_reseller.html', r=reseller, reseller_id=reseller_id, upload_url=upload_url, image=image)
else:
self.redirect('/admin')
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
reseller_id = self.request.get('reseller_id')
upload_files = self.get_uploads('logo_img')
if upload_files:
blob_info = upload_files[0]
key = blob_info.key()
r = Reseller.get_by_id(int(reseller_id))
r.blob_key_logo = str(key)
r.put();
name = self.request.get('name')
website = self.request.get('website')
information = self.request.get('information')
address = self.request.get('address')
city = self.request.get('city')
state = self.request.get('state')
zipcode = self.request.get('zipcode')
email = self.request.get('email')
phone = self.request.get('phone')
if name and website and information and email and phone and address and city and state and zipcode:
r = Reseller.get_by_id(int(reseller_id))
r.name = name
r.website = website
r.information = information
r.address = address
r.city = city
r.state = state
r.zipcode = zipcode
r.email = email
r.phone = phone
r.put()
else:
error = "Looks like your missing some critical info"
self.render("edit_reseller.html", name=name, website=website, information=information, address=address, city=city, zipcode=zipcode, email=email, phone=phone, error=error)
self.redirect("/edit_reseller/" + reseller_id)
You just need to put the logic of the UploadHandler inside the Reseller(BaseHandler) and make Reseller inherit from blobstore_handlers.BlobstoreUploadHandler.
The call to get_uploads fails, as the NewReseller Class does not inherit from BlobstoreUploadHandler. The BlobstoreUploadHandler class takes over the upload operation so you do not need to create a post method, just add the corresponding logic from post ( name = self.request.get('name'), r = Reseller(), r.put(), etc. ) and add it to the upload method.
You should not call or create a new a handler instance by hand (unless you know what you are doing), as it would be missing the things that make it work.
The complete app sample at the official docs, might also be helpful.

upload picture through external website to gravatar profile

I was wondering if it is possible to create an upload function to upload picture through my own site to the gravatar site?
Yes, this is possible. See http://en.gravatar.com/site/implement/xmlrpc/ , specifically the grav.saveData or grav.SaveUrl calls.
Yes it's possible!
import base64
from xmlrpclib import ServerProxy, Fault
from hashlib import md5
class GravatarXMLRPC(object):
API_URI = 'https://secure.gravatar.com/xmlrpc?user={0}'
def __init__(self, request, password=''):
self.request = request
self.password = password
self.email = sanitize_email(request.user.email)
self.email_hash = md5_hash(self.email)
self._server = ServerProxy(
self.API_URI.format(self.email_hash))
def saveData(self, image):
""" Save binary image data as a userimage for this account """
params = { 'data': base64_encode(image.read()), 'rating': 0, }
return self._call('saveData', params)
#return self.useUserimage(image)
def _call(self, method, params={}):
""" Call a method from the API, gets 'grav.' prepended to it. """
args = { 'password': self.password, }
args.update(params)
try:
return getattr(self._server, 'grav.' + method, None)(args)
except Fault as error:
error_msg = "Server error: {1} (error code: {0})"
print error_msg.format(error.faultCode, error.faultString)
def base64_encode(obj):
return base64.b64encode(obj)
def sanitize_email(email):
return email.lower().strip()
def md5_hash(string):
return md5(string.encode('utf-8')).hexdigest()
Just call the class in your view :)

Categories