I'm building a Django application that needs to interact with a 3rd party RESTful API, making various GETs, PUTs, etc to that resource. What I'm looking for is a good way to represent that API within Django.
The most obvious, but perhaps less elegant solution seems to be creating a model that has various methods mapping to webservice queries. On the other hand, it seems that using something like a custom DB backend would provide more flexibility and be better integrated into Django's ORM.
Caveat: This is the first real project I've done with Django, so it's possible I'm missing something obvious here.
The requests library makes it easy to write a REST API consumer. There is also a Python library called slumber, which is built on top of requests, for the explicit purpose of consuming REST APIs. How well that will work for you likely depends on how RESTful the API really is.
Make the REST calls using the builtin urllib (a bit clunky but functional) and wrap the interface into a class, with a method for each remote call. Your class can then translate to and from native python types. That is what I'd do anyway!
Related
I have the problem that for a project I need to work with a framework (Python), that has a poor documentation. I know what it does since it is the back end of a running application. I also know that no framework is good if the documentation is bad and that I should prob. code it myself. But, I have a time constraint. Therefore my question is: Is there a cooking recipe on how to understand a poorly documented framework?
What I tried until now is checking some functions and identify the organizational units in the framework but I am lacking a system to do it more effectively.
If I were you, with time constaraints, and bound to use a specific framework. I'll go in the following manner:
List down the use cases I desire to implement using the framework
Identify the APIs provided by the framework that helps me implement the use cases
Prototype the usecases based on the available documentation and reading
The prototyping is not implementing the entire use case, but to identify the building blocks around the case and implementing them. e.g., If my usecase is to fetch the Students, along with their courses, and if I were using Hibernate to implement, I would prototype the database accesss, validating how easily am I able to access the database using Hibernate, or how easily I am able to get the relational data by means of joining/aggregation etc.
The prototyping will help me figure out the possible limitations/bugs in the framework. If the limitations are more of show-stoppers, I will implement the supporting APIs myself; or I can take a call to scrap out the entire framework and write one for myself; whichever makes more sense.
You may also use python debugging library: pdb. After importing it with import pdb you may set traces in the body of functions and classes pdb.set_trace(). Then it will stop the execution of the program in the line and you may look at existing variables and processes.
I am building my first Django web application and I need a bit of advice on code layout.
I need to integrate it with several other applications that are exposed through RESTful APIs and additionally Django's internal data. I need to develop the component that will pull data from various sources, django's DB, format it consistently for output and return it for rendering by the template.
I am thinking of the best way to write this component. There are a couple of ways to proceed and I wanted to solicit some feedback from more experienced web developers on things I may have missed.
Way #1
Develop a completely standalone objects for interacting with other applications via their APIs, etc... This would not have anything related with django and test independently. Then in django, import this module in the views that need it, run object methods to get required data, etc...
If I need to access any of this functionality via a GET request (like through JavaScript), I can have a dedicated view that imports the module and returns json.
Way #2
Develop this completely as django view(s) expose as a series of GET/POST calls that would all call each other to get the work done. This would be directly tied in the application.
Way #3
Start with Way #1, but instead of creating a view, package it as a django app. Develop unit tests on the app as well as the individual objects.
I think that way 1 or 3 would be very much encapsulated and controlled.
Way 2 would be more complicated, but facilitate higher component re-use.
What is better from a performance standpoint? If I roll with Way #1 or 3, would an instance of the object be instantiated for each request?
If so this approach may be a bit too heavy for this. If I proceed with this, can they be singletons?
Anyway, I hope this makes sense.
thanks in advance.
Definitely go for "way #1". Keeping an independent layer for your service(s) API will help a lot later when you have to enhance that layer for reliability or extending the API.
Stay away from singletons, they're just global variables with a new name. Use an appropriate life cycle for your interface objects. The obvious "instantiate for each request" is not the worst idea, and it's easier to optimize that if needed (by caching and/or memoization) than it's to unroll global vars everywhere.
Keep in mind that web applications are supposed to be several processes on a "shared nothing" design; the only shared resources must be external to the app: database, queue managers, cache storage.
Finally, try to avoid using this API directly from the view functions/CBV. Either use them from your models, or write a layer conceptually similar to the way models are used from the views. No need of an ORM-like api, but keep any 'business process' away from the views, which should be concerned only with the presentation and high level operations. Think "thick model, thin views" with your APIs as a new kind of models.
I'm trying to pull similar data in from several third party APIs, all of which have slightly varying schemas, and convert them all into a unified schema to store in a DB and expose through a unified API. This is actually a re-write of a system that already does this, minus storing in the DB, but which is hard to test and not very elegant. I figured I'd turn to the community for some wisdom.
Here are some thoughts/what I'd like to achieve.
An easy way to specify schema mappings from the external APIs schema to the internal schema. I realize that some nuances in the data might be lost by converting to a unified schema, but that's life. This schema mapping might not be easy to do and perhaps overkill from the academic papers I've found on the matter.
An alternative solution would be to allow third parties to develop the interfaces to the external APIs. The code quality of these third parties may or may not be known, but could be established via thorough tests.
Therefore the system should be easy to test, I'm thinking by mocking the external API calls to have reproducible data and ensure that parsing and conversion is being done correctly.
One of the external API interfaces crashing should not bring down the rest of them.
Some sort of schema validation/way to detect if the external API schemas have changed without warning
This will end up being integrated into a Django project, so it could be written as a Django app, which would likely make unit and integration testing easier. On the other hand, I would like to keep it as decoupled as possible from Django. Although the API interfaces would have to know what format to convert to, could this be specified at runtime?
Am I missing anything in the wishlist? Unrealistic? Headed down the wrong path? Would love to get some feedback.
I'm not sure if there are libraries/OS project which already do some of this. The less wheels I have to reinvent the better. Would any part of this be valuable as an OS project?
In the previous version I spawned a bunch of threads that would handle individual requests. Although I've never used it, I've been told I should look at gevent as a way to handle this.
For your second bullet point you should check out Temboo. Temboo normalizes access to over 100 APIs, meaning that you can talk to them all using a common syntax in the language of your choice. In this case you would use the Temboo Python SDK - available here.
(Full disclosure: I work at Temboo)
Has anyone used the Cheetah Templating Engine with the Tornado Web Framework before? Do they work well together and are you able to use Cheetah's caching components within the Tornado Framework?
I've been looking for a good, python only solution for this. I originally looking at Twisted for the Web Framework but it is not able to make use of the caching aspects of the Cheetah engine making any benefits from it limited.
I have used Cheetah and Tornado both, but never together. Cheetah's caching is not all that super (as far as time saving). From what I remember, and this may be wrong, but it's caching is just a way of preventing the lookup in the lookup table from taking place. But you still have to actually provide the lookup table to allow for expired cache items to be updated.
Also, why wouldn't Twisted be able to take the same advantage of the Cheetah template caching?
Take a look at this link:
http://twistedmatrix.com/pipermail/twisted-web/2004-July/000552.html
It is a bit dated, but may be helpful.
I'm trying to develop an application that would be perfect for GWT, however I am using this app as a learning example for Django. Is there some precedence for this type of application in Django?
Pyjamas is sort of like GWT which is written with Python. From there you can make it work with your django code.
Lots of people have done this by writing their UI in GWT and having it issue ajax calls back to their python backend. There are basically two ways to go about it. First, you can simply use JSON to communicate between the frontend and the backend. That's the approach you will find here (http://palantar.blogspot.com/2006/06/agad-tutorial-ish-sort-of-post.html). Second, some people want to use GWT's RPC system to talk to python backends. This is a little more involved, but some people have created tools (for example, http://code.google.com/p/python-gwt-rpc/).
To be honest, most successful projects just use JSON to communicate between GWT and the python server. GWT's RPC is pretty advanced in that it is able to serialize arbitrary java object graphs to and from the client. It's a tricky problem to get right and I'm pretty doubtful that any of the python tools have it right.