I'm trying to document an already existing python API with Swagger. I wrote the swagger.yaml with every route documented with the help of their editor. Now i would like to deploy the documentation using connexion.
(brief example of the swagger.yaml file)
swagger: "2.0"
info:
description: "This is a description of the routes of the API"
version: "1.0.0"
title: "API"
basePath: "/api"
paths:
/home:
get:
tags:
- "API"
summary: "Home of the application"
operationId: home
responses:
200:
description: "Success"
schema:
type: object
properties:
user_id:
type: string
username:
type: string
403:
description: "Limit of api connections overrun"
I changed the Flask app by a connexion.app during the launch of the server, and was able to specify the .yaml file. But when i'm trying to launch it, it crashes instantly:
File "/usr/local/lib/python2.7/dist-packages/connexion/utils.py", line 74, in get_function_from_name
raise ValueError("Empty function name")
exceptions.ValueError: Empty function name
From my understanding connexion will base it's manual testing feature from the object operationId in every route that needs to point on the function handling the request.
Problem: every route of the API are defined as nested function.
def add_routes(app, oauth):
#app.route('/api/home', methods=['GET'])
#oauth.require_oauth()
def home():
user = request.oauth.user
return jsonify(
user_id=user.user_id,
username=user.username
)
I know nested functions in python are not actually functions at all: not callable, just present in the language in order for us programmers to organize our code.
I think that would be the issue with connexion, it is just not capable of finding these functions and map them for the manual testing feature, but i'm not sure how to fix this. Do you see something that would allow connexion to map the function without having to refactor the entire API in order not to have nested functions ?
Thanks a lot for any help.
My guess is that you haven't defined your handler function. You need to provide a module+function that matches the operationId in the spec.
For example:
I have a function called get_user() in a file app.py , I need to set the operationId to app.get_user.
operationId: app.get_user
Hope that helps!
Related
We have a python web server app based on connexion and Flask libraries.
For audit purposes we print every request, including request body, to the logs. Those logs are much more extensive than what Flask prints by default.
However, there are parameters like passwords or keys that I don't want to print their values to the logs.
I want to add custom attribute to several parameters' definition in swagger.yml so on web app start up I will get parameters definitions from connexion and save all the parameters that have this custom attribute, so later on I will hide values of those parameters in the logs.
My desired definition:
paths:
/demo_add_user:
post:
summary: "add user"
operationId: api.api.demo_add_user
parameters:
- name: "new_user"
in: body
required: true
description: "Use to add"
schema:
$ref: "#/definitions/NewUser"
- name: "password"
in: body
description: "user password"
required: false
type: string
x-hidden: true
responses:
Obviously, I would like connexion to ignore this attribute since this is specific to our implementation.
Any way to add custom attribute like x-hidden to parameter definition?
The solution is to use OpenApi extensions by adding x-attr like attribute.
The details are in next link:
https://swagger.io/specification/#specificationExtensions
Just extending #D'artanian's answer :
using nodejs, for example it may be :
swagger :
/liveness:
get:
operationId: apiHealthLiveness
x-myCustomParam : "test"
summary: Liveliness test for service
description: deterimines if a service is still alive
responses:
"200":
...
And using req object, access it :
const myCustomParamValue = req.swagger.operation["x-myCustomParam"];
In the python standard environment quickstart, the endpoints method test_api_key returns a 503 Service Unavailable. The error occurs in the API Explorer when run with dev_appser.py and when the API is deployed. The code for it is:
import endpoints
from protorpc import message_types
from protorpc import messages
from protorpc import remote
class TestResponse(messages.Message):
content = messages.StringField(1)
#endpoints.api(name='practice', version='v1', description='My Practice API')
class PracticeApi(remote.Service):
#endpoints.method(
message_types.VoidMessage,
TestResponse,
path='test/getApiKey',
http_method='GET',
name='test_api_key')
def test_api_key(self, request):
return TestResponse(content=request.get_unrecognized_field_info('key'))
api = endpoints.api_server([PracticeApi])
I don't have a good understanding of .get_unrecognized_field_info('key') so I am not sure what the issue is? Thanks.
Firstly, I recommend reading Google Protocol RPC Library Overview, since it's Google Cloud Endpoints uses it extensively.
#endpoints.method allows you to configure a specific method in your API. Configuration options are documented in Google Cloud Platform documentation Creating an API with Cloud Endpoints Frameworks for App Engine, in the section, Defining an API method (#endpoints.method).
If you're restricting access to the test/getApiKey/test_api_key method, then you must configure the method with the api_key_required=True option. Restricting API Access with API Keys (Frameworks) discusses that further, but your method annotation should be:
#endpoints.method(
message_types.VoidMessage,
TestResponse,
path='test/getApiKey',
http_method='GET',
name='test_api_key',
api_key_required=True
)
Notice your method accepts a request parameter representing the HTTP request (i.e. client using your API):
def test_api_key(self, request):
However, the request parameter is actually Google Protocol RPC Message (Proto RPC) Message object and as such is very well defined. If additional fields exist in the ProtoRPC request parameter, beyond what is formally defined, they are still stored with the request object but must be retrieved using the following method:
def get_unrecognized_field_info(self, key, value_default=None,
variant_default=None):
"""Get the value and variant of an unknown field in this message.
Args:
key: The name or number of the field to retrieve.
value_default: Value to be returned if the key isn't found.
variant_default: Value to be returned as variant if the key isn't
found.
Returns:
(value, variant), where value and variant are whatever was passed
to set_unrecognized_field.
"""
Message class code on GitHub is quite well documented. .
No arguments will appear in the body of a request because you've configured the method with to be called with HTTP GET:
http_method='GET'
...you're correctly using the value message_types.VoidMessage.
In terms of your error, 503 is just a generic server error, can you provide any information from the StackDriver logs? They will point you to the exact line and error in your code.
There were three things that were creating the 503 error.
Firstly, I needed to make the method or entire Api require an Api Key. In this case I just applied it to the entire Api:
#endpoints.api(name='practice', version='v1', api_key_required=True)
class PracticeApi(remote.Service):
Secondly, after I generated the Api Key in the cloud console I needed to put the Key into the openapi.json file before deploying it.
Lastly, I was still getting a validation error:
ValidationError: Expected type <type 'unicode'> for field content, found (u'My Api Key', Variant(STRING, 9)) (type <type 'tuple'>)
The get_unrecognized_field_info() function returns a tuple of (value, variant). A tuple was not expected by the response so I updated the method to only show value:
def test_api_key(self, request):
return TestResponse(content=request.get_unrecognized_field_info('key')[0])
editied original question:
Im trying to make a google appengine app which uses the g+ avatar and human name...
...
So it seems that i need the google-api-python-client library in my app.
...
to enable access to the profile scope so i can look up 'me' and grab the users name and avatar and chuck them in a couple of properties in my user objects (with a button to reload the values again or something).
So has anyone does this? Or has a working example (or even a pointer to which of the ways to authorise my app for scope=[profile])?
Discoveries:
I dont need the google-api-python-client library for this. The simple approach was to do the g+ access in pure js on the client and then lookup and push the results to my appengine app. It isnt as secure as doing via the backend, but it is only for displayname and icon (which can be set manually anyway).
I did need to make some other tweaks to make it work though...
following this workflow:
https://developers.google.com/+/web/signin/javascript-flow
Important things to note:
step1 should also state that you MUST fill out "APIs & auth" -> "Consent screen" field "PRODUCT NAME" and "EMAIL ADDRESS" or you get weird errors
You (might) have to do this before you generate the credential (or delete and recreate it)
(credit to answer: Error: invalid_client no application name)
set meta google-signin-scope to "profile" (or maybe "email")
remove the meta header for google-signin-requestvisibleactions (otherwise i got a frame sameorigin error)
obviously the button line from step4 needs to go after the body tag in your document
skip step2, the code from step2 is also included in step4
also on the workflow page, the 'working example' button on that page does not work (dont try it)
Once i did that I could put the following in the successful callback code and do a lookup:
gapi.client.load('plus','v1', function(){
var request = gapi.client.plus.people.get({ 'userId': 'me' });
request.execute(function(resp) {
console.log('Retrieved profile for:' + resp.displayName);
console.log(resp);
console.log(resp.result);
console.log(resp.result.displayName);
console.log(resp.result.image);
});
});
you can see here full example on how to use client library
https://code.google.com/p/google-api-python-client/source/browse/samples/plus/plus.py
i see a snippet of the code stating
try:
person = service.people().get(userId='me').execute()
print 'Got your ID: %s' % person['displayName']
https://developers.google.com/+/api/latest/people#resource
so basically person['image']['url'] will be your path to user's avatar.
full folder: https://code.google.com/p/google-api-python-client/source/browse/samples/plus/
I am working on bugzilla xml-rpc by using "Bugzilla XMLRPC access module" developed in python.
How I can attach/download bugzilla file by using this module ?
According to guideline of API get_attachments_by_bug($bug_id) retrieves and returns the attachments.
But this function didn't worked for me, I got following error message.
<type 'exceptions.AttributeError'>: 'Bugzilla4' object has no attribute 'get_attachments_by_bug'
Any help would be appreciated.
FYI:
I am in contact with supplier of python-bugzilla tool and here I got response from them.
"Not all bugzilla XMLRPC APIs are wrapped by python-bugzilla, that's one of them.
The 'bugzilla' command line tool that python-bugzilla ships has commands for
attaching files and downloading attachments, take a look at the code there for
guidance."
I've figured out the way how to download/upload attachment by using "Bugzilla XMLRPC access module"
you need to pass the id of attached file as parameter to the following function
Download:
downloaded_file = bz.download_attachment(attachment_id)
file_name = str(downloaded_file.name)
Upload:
kwards = {
'contenttype':'application/octet-stream',
# 'filename': file_path #there could be more parameters if needed
}
#attachfile method will return the id of attached file
bz.attachfile(bug_id, file_path, file_name, **kwards)
However attached file got corrupted due to some xmp-rpc API's internal methods described here, here and here, that's another issue :)
I have a thorny problem that I can't seem to get to grips with. I am
currently writing unit tests for a django custom auth-backend. On our
system we actually have two backends: one the built-in django backend
and the custom backend that sends out requests to a Java based API
that returns user info in the form of XML. Now, I am writing unit
tests so I don't want to be sending requests outside the system like
that, I'm not trying to test the Java API, so my question is how can I
get around this and mock the side-effects in the most robust way.
The function I am testing is something like this, where the url
settings value is just the base url for the Java server that
authenticates the username and password data and returns the xml, and the service value is
just some magic for building the url query, its unimportant for
us:
#staticmethod
def get_info_from_api_with_un_pw(username, password, service=12345):
url = settings.AUTHENTICATE_URL_VIA_PASSWORD
if AUTH_FIELD == "username":
params = {"nick": username, "password": password}
elif AUTH_FIELD == "email":
params = {"email": username, "password": password}
params["service"] = service
encoded_params = urlencode([(k, smart_str(v, "latin1")) for k, v in params.items()])
try:
# get the user's data from the api
xml = urlopen(url + encoded_params).read()
userinfo = dict((e.tag, smart_unicode(e.text, strings_only=True))
for e in ET.fromstring(xml).getchildren())
if "nil" in userinfo:
return userinfo
else:
return None
So, we get the xml, parse it into a dict and if the key nil is present
then we can return the dict and carry on happy and authenticated.
Clearly, one solution is just to find a way to somehow override or
monkeypatch the logic in the xml variable, I found this answer:
How can one mock/stub python module like urllib
I tried to implement something like that, but the details there are
very sketchy and I couldn't seem to get that working.
I also captured the xml response and put it in a local file in the
test folder with the intention of finding a way to use that as a mock
response that is passed into the url parameter of the test function,
something like this will override the url:
#override_settings(AUTHENTICATE_URL_VIA_PASSWORD=(os.path.join(os.path.dirname(__file__), "{0}".format("response.xml"))))
def test_get_user_info_username(self):
self.backend = RemoteAuthBackend()
self.backend.get_info_from_api_with_un_pw("user", "pass")
But that also needs to take account of the url building logic that the
function defines, (i.e. "url + encoded_params"). Again, I could rename
the response file to be the same as the concatenated url but this is becoming
less like a good unit-test for the function and more of a "cheat", the whole
thing is just getting more and more brittle all the time with these solutions, and its really just a fixture anyway, which is also something I want to avoid if
at all possible.
I also wondered if there might be a way to serve the xml on the django development server and then point the function at that? It seems like a saner solution, but much googling gave me no clues if such a thing would be possible or advisable and even then I don't think that would be a test to run outside of the development environment.
So, ideally, I need to be able to somehow mock a "server" to
take the place of the Java API in the function call, or somehow serve
up some xml payload that the function can open as its url, or
monkeypatch the function from the test itself, or...
Does the mock library have the appropriate tools to do such things?
http://www.voidspace.org.uk/python/mock
So, there are two points to this question 1) I would like to solve my
particular problem in a clean way, and more importantly 2) what are
the best practices for cleanly writing Django unit-tests when you are
dependent on data, cookies, etc. for user authentication from a remote
API that is outside of your domain?
The mock library should work if used properly. I prefer the minimock library and I wrote a small base unit testcase (minimocktest) that helps with this.
If you want to integrate this testcase with Django to test urllib you can do it as follows:
from minimocktest import MockTestCase
from django.test import TestCase
from django.test.client import Client
class DjangoTestCase(TestCase, MockTestCase):
'''
A TestCase class that combines minimocktest and django.test.TestCase
'''
def _pre_setup(self):
MockTestCase.setUp(self)
TestCase._pre_setup(self)
# optional: shortcut client handle for quick testing
self.client = Client()
def _post_teardown(self):
TestCase._post_teardown(self)
MockTestCase.tearDown(self)
Now you can use this testcase instead of using the Django test case directly:
class MySimpleTestCase(DjangoTestCase):
def setUp(self):
self.file = StringIO.StringIO('MiniMockTest')
self.file.close = self.Mock('file_close_function')
def test_urldump_dumpsContentProperly(self):
self.mock('urllib2.urlopen', returns=self.file)
self.assertEquals(urldump('http://pykler.github.com'), 'MiniMockTest')
self.assertSameTrace('\n'.join([
"Called urllib2.urlopen('http://pykler.github.com')",
"Called file_close_function()",
]))
urllib2.urlopen('anything')
self.mock('urllib2.urlopen', returns=self.file, tracker=None)
urllib2.urlopen('this is not tracked')
self.assertTrace("Called urllib2.urlopen('anything')")
self.assertTrace("Called urllib2.urlopen('this is mocked but not tracked')", includes=False)
self.assertSameTrace('\n'.join([
"Called urllib2.urlopen('http://pykler.github.com')",
"Called file_close_function()",
"Called urllib2.urlopen('anything')",
]))
Here's the basics of the solution that I ended up with for the record. I used the Mock library itself rather than Mockito in the end, but the idea is the same:
from mock import patch
#override_settings(AUTHENTICATE_LOGIN_FIELD="username")
#patch("mymodule.auth_backend.urlopen")
def test_get_user_info_username(self, urlopen_override):
response = "file://" + os.path.join(os.path.dirname(__file__), "{0}".format("response.xml"))
# mock patch replaces API call
urlopen_override.return_value = urlopen(response)
# call the patched object
userinfo = RemoteAuthBackend.get_info_from_api_with_un_pw("user", "pass")
assert_equal(type(userinfo), dict)
assert_equal(userinfo["nick"], "user")
assert_equal(userinfo["pass"], "pass")