I'm running an RESTful API with Python (Flask).
I want to be able to track:
which requests have been made
when did those requests happen
how long did it take to send the response
I want to use Google Analytics for this, because of it's nice dashboard and extended functionalities.
My question
How can I implement Google Analytics into a REST API?
OR does anyone know another tool/library that can be implemented?
This is what I found at the moment:
a tracking app that uses MongoDB
the Google data API - but this is for reading GA data, not tracking the API?
There are actually two ways to send server-side data to Google Analytics. The standard method is the GIF Image Request API, which is the same API that ga.js uses on the client-side. Google has started developing a new REST API known as the Measurement Protocol, but this is only in developer preview.
Server-Side GA
There are a few issues to work through when trying to send server-side data to GA.
Like #mehaase pointed out above, the gif API takes the ip address from the request, so all of your server-side requests will appear as users coming from the location of your servers. The measurement protocol doesn't let you change the request's ip either. I'll assume the publicly available gif API in this answer.
Another issue is that the gif endpoint requires a client-side cookie. You can fake this cookie on every request but this will cause each event to look like a new visitor. That's fine as long as you keep the server-side API and website in separate Google Analytics profiles.
Also beware that Google can take up to an hour to show your events once you've sent them. This can make debugging a bit painful, so be patient.
Here's the breakdown of what each variable in the GA cookie means, and a good node.js example of sending server-side data to GA.
Other Event Tracking Options
Even though GA is excellent for tracking website metrics, it's not built for tracking server-side events. A category of analytics known as event tracking is the perfect application for restful API usage tracking.
The API generally looks like this:
analytics.track('API Response', {
method : 'POST',
endpoint: '/comments'
duration: 124
status : 500
});
And lets you see reports on the frequencies and distributions of each event and event property You can answer questions like: how many /comments API calls happened today? How many were 200s? How many had a response higher than 200ms? etc.
Here are some event tracking tools that can help you do this:
Mixpanel
KissMetrics
Keen.IO
I'm the co-founder of Segment.io, a company that provides a simple API for client-side, server-side and mobile analytics. We let you send data from python, php, ruby, node, java, .net, javascript, and iOS, and we'll forward it to Google Analytics, Mixpanel, KissMetrics, Keen.IO, or any of our other supported services without you having to learn their API.
And finally, here's an article from our analytics academy that explains why event tracking is useful.
I know this is very old post. I came across google analytics support in Python
https://developers.google.com/api-client-library/python/apis/analytics/v3
Thought this is right place to document as well (y)
Related
This is what I want to achieve:
Ask the user to authorize the collection of their data on a Google Analytics 4 property (or Universal Analytics but I would rather not)
Programmatically retrieve and store the data every n-hours
I was able to do (1) client-side by asking for authorization with google's OAUTH2 and making a call to Reporting API v4 https://developers.google.com/analytics/devguides/reporting/core/v4 using gapi on the front-end.
However, I'm not sure how to do it on a schedule without user interaction. I've searched Google's API docs and I believe there's a way to do it in python https://developers.google.com/analytics/devguides/reporting/core/v4/quickstart/service-py but I am currently limited to Node and the browser. I guess I could make a server in python that does the data fetching and connects with the Node application, but that's yet another layer of complications that I'm trying to avoid. Is there a way to do everything in Node?
GCP APIs are all documented in a way which allows everyone to generate client libraries in a variety of languages, including node.js. The documentation for the node.js client for Analytics Reporting is here.
For the question of how to schedule this on GCP, I would recommend you to use Cloud Scheduler. This will hit an endpoint running on Cloud Run, which will do the actual work. Alternatively, if you already have a service running somewhere else, you can simply add the required endpoints there and point Cloud Scheduler to it.
The overall design which I would suggest you goes something like this:
Build a site which takes the user through the OAUTH2 login process,
requesting the relevant Google Analytics Reporting API scopes
required to make the request.
Store the obtained credentials in their user database.(preferably
Firestore in Datastore mode)
Set up a Cloud Run service (or anything else), with two endpoints
Iteration endpoint: Iterate through the list of users and add tasks
to Cloud Tasks to hit the download endpoint for each one.
Download endpoint: Takes a user ID (e.g. as a query parameter) and
performs the download for this user. You will need to load the
credentials for the user from the database and use this to access the
reporting API.
Store the downloaded data in the desired location, e.g. Cloud
Storage, Firestore, Cloud SQL, etc.
Set up Cloud Scheduler to hit the iteration endpoint at the desired
frequency.
For the GCP services mentioned above, basically everything other than Analytics, you may use the "cloud" clients for node.js, which are available here
Note : The question you have asked is a very broad question and this answer is just a suggestion. You may think about other designs whichever works best for you.
I'm developing an application with Dialogflow and Actions on Google which requires to send daily birthday notification in Google Assistant.
I'm following this documentation to send daily updates. But I'm having trouble because of the language barrier.
I'm developing backed in Python and using fulfillment to serve the request, but that documentation is in Node.js. So I need help to send daily updates.
I've done up to Console setup. Please guide me for further process.(Which JSON req. should I sent?)
If you look down that documentation page a bit here and click the JSON tab, you'll see the specific format you need to configure the subscription to your birthday updates.
Since you're not using a client library, you'll need to write the Python code yourself to respond with the appropriate JSON.
Hi I would like to do a machine learning hello world, e.g. the Iris data set. I would then like to create a web API (RESTful service) to allow a HTTP Post request to send over (petal_length, sepal_length) and get back the prediction on the type of Iris.
I intend to use Scikit learn but what should I use to make a web API? what do people normally use to make a ML Web API?
Django, I know about this for making websites but I am not sure if it is used for web APIs. http://www.django-rest-framework.org/
Flask, I have also heard about this one but never used it before. http://flask.pocoo.org/
I would like to write unit tests and integration tests. It will also need to use authentication to ensure users have access. Currently my Web API for authenticating users is an ASP.Net Core Web Api and I use JWT.
Flask is just fine for ML API purposes only. It took me very little time to install and configure Flask API and write first API functions calling 3rd party ML API and reformulate its feedback matching to needs. I have added internal ML classifiers modification and usage API functions later.
Instructions:
http://flask.pocoo.org/docs/0.12/deploying/
I have multiple api which we have provided to android developers.
Like :
1) Creating Business card API
2) Creating Contacts API
So these api working fine when app is online. So our requirement is to handle to create business card and contacts when app is offline.
We are following steps but not sure:-
1) Android developer store the business card when app offline and send this data to server using separate offline business card api when app comes online.
2) Same we do for creating contacts offline using offline contact api.
My problem is I want do in one api call to send all data to server and do operation.
Is this approach will right?? Also please suggest what is the best approach to handle offline data. Also how to handle syncing data when app would come online??
Please let me know if I could provide more information.
I'm confused as to how you're approaching this. My understanding is that when the app is offline you want to "queue up" any API requests that are sent.
Your process seems fine however without knowing the terms around the app being "offline" it's hard to understand if this best.
Assuming you're meaning the server(s) holding the application are offline you're correct you want a process in the android app that will store the request until the application becomes online. However, this can be dangerous for end users. They should be receiving a message on the application being offline and to "try again later" as it were. The fear being they submit a request for x new contacts to be queued and then re-submit not realizing the application was offline.
I would suggest you have the android app built to either notify the user of the app being down or provide some very visible notification that requests are queued locally on their phone until the application becomes available and for them to view/modify/delete said locally cached requests until the application becomes available. When the API becomes available a notification can be set for users to release their queue on their device.
I would like to display on my GAE website a number of daily users of my android application (tracked with Google Analytics), both are under the same account. I wanted to use Analytics API for this. I am reading Hello Analytics API tutorial and looks like I need OAuth to authorize my app to get the data from Analytics. It looks quite complex, is there any easy way (like getting some token and using it)?
Looks like I need service account (for server to server applications), but I don't understand how to modify starter application (taken there) to use it.
Since you only want to access data that you own, using a service account is a good idea.
How ever the starter application you linked is for working with a normal OAuth2 and its not the same as the code for a service account.
I haven't done this in python myself before, but i found an example that should get you started. You will need to change it slightly because the scopes are for Google+ not Google analytics. create the service in python on GAE
If you have any problems changing it to work with Google Analtyics let me know i should be able to help with that. Even if i cant test the python code myself.
There is much simpler way to pass authorization - see App Identity doc. See also Java example how to access Analytics data from the GAE.
Another useful link is here.