Python 3.6 Post Request Domain API - python

This is a bit of a newbie Q.
I'm using Python 3.6
I am trying to use the domain realestate api to write a scraper that collects data of houses/apartments for sale in my area, but i am having trouble getting the post request to work. I have registered and retrieved my client_id and secret_id for the authentication. The post request returns a status_code of 400
response = requests.post('https://auth.domain.com.au/v1/connect/token',
data = {'client_id':client_id,
"client_secret":client_secret,
"grant_type":"client_credentials",
"scope":"api_agencies_read api_listings_read",
"Content-Type":"application/json"})
token=response.json()
access_token=token["access_token"]
search_parameters = {
"listingType": "Sale",
"locations": [
{
"state": "NSW",
"suburb": "Balgowlah",
"postcode": 2093,
"includeSurroundingSuburbs": True
}
]
}
url = "https://api.domain.com.au/v1/listings/residential/_search"
auth = {"Authorization":"Bearer "+access_token}
request = requests.post(url, data=search_parameters, headers=auth)
details=request.json()
I know my authentication is correct, because i can use the Live API on the website to test the same request (i had to select the client, secret id and the project to allow direct access), and i get a valid access token from the code above.
access_token:
{'access_token': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
'expires_in': 43200,
'token_type': 'Bearer'}
request.json():
{'errors': {'parameters': ['Undefined error.']},
'message': 'The request is invalid.'}
I've been able to implement the notebook from this post. So i can be sure my client and secret ids are connected to the domain project.

#furas had the solution:
look at the example closer :)
The example uses "Content-Type":"text/json" but you use "application/json" instead of "text/json"

Related

how to get access tokens from refresh token? does refresh token expire?

I'm trying to create a python script which takes a (.csv with access tokens and a file) as input and uploads that file to multiple google drives whose access tokens are in that csv
but after sometime access tokens get expired and I have to get them again...just saw there's something called refresh and it refreshes access token
Is it possible to do this from python script, please explain.
Do refresh token expire?
import json
import requests
import pandas as pd
headers = {}
para = {
"name": "update",
}
files = {
'data': ('metadata', json.dumps(para), 'application/json; charset=UTF-8'),
'file': open("./update.txt", "rb")
}
tokens = pd.read_csv('tokens.csv')
for i in tokens.token:
headers={"Authorization": i}
r = requests.post(
"https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart",
headers=headers,
files=files
)
print(r.text)
In order to be able to get a new access_token programmatically using a refresh_token, you must have set access_type to offline when redirecting the user to Google's OAuth 2.0 server.
If you did that, you can get a new access_token if you do the following POST request to https://oauth2.googleapis.com/token:
POST /token HTTP/1.1
Host: oauth2.googleapis.com
Content-Type: application/x-www-form-urlencoded
client_id=your_client_id&
client_secret=your_client_secret&
refresh_token=refresh_token&
grant_type=refresh_token
The corresponding response would be something like:
{
"access_token": "1/fFAGRNJru1FTz70BzhT3Zg",
"expires_in": 3920,
"scope": "https://www.googleapis.com/auth/drive",
"token_type": "Bearer"
}
Note:
You can find code snippets for several languages in the reference I provide below, including Python, but considering you are not using the Python library, I think the HTTP/REST snippet I provided might be more useful in your situation.
Reference:
Refreshing an access token (offline access)

How do I format my GET request to the Spotify Web API in Python?

I'm new to working with APIs. I'm trying to create a script that will output the tracks of a public playlist from Spotify. I have hit a roadblock at an early stage: I'm unsure how exactly to format my GET request to retrieve the tracks of the playlist in question and where I put the OAuth access token. My current attempt returns a 401 status code.
What I've done so far:
I have created an App on the Spotify Developers website dashboard.
I have saved client_id and client_secret from the dashboard into a script ("spotifysecrets.py").
I have run the following code in my main script:
import requests
from spotifysecrets import client_id as cid, client_secret as cs
AUTH_URL = "https://accounts.spotify.com/api/token"
token_req = requests.post(AUTH_URL, {
"grant_type": "client_credentials",
"client_id": cid,
"client_secret": cs
})
access_token = token_req.json()["access_token"]
Everything appears to have worked fine so far. The problem occurs in the next step.
Having obtained the token, I am getting a 401 error when trying to make my tracks request. The code I have written is as follows:
pl_id = "…" # I have elided the actual playlist ID for the purposes of this question
tracks = "https://api.spotify.com/v1/playlists/{}/tracks".format(pl_id)
songs_req = requests.get(tracks, {
"Authorization": "Basic {}".format(access_token)
})
Edit
I have tried using
songs_req = requests.get(tracks, headers={
"Authorization": "Basic {}".format(access_token)
})
instead but this gives a 400 error.
I looked into the api and I believe you have a syntax error:
songs_req = requests.get(tracks, {
"Authorization": "Basic {}".format(access_token)
})
should be:
songs_req = requests.get(tracks, {
"Authorization": "Bearer {}".format(access_token)
})

get end user tokens for eBay restful in Python

currently I am using eBay Trading API with Python. Thanks to: https://github.com/timotheus/ebaysdk-python
I used https://github.com/luke-dixon/django-ebay-accounts to get tokens for user.
Now, I would like to use Restful API (https://developer.ebay.com/docs#Acc). I don't think I can use tokens I have already. So, I managed thanks to Getting an Ebay OAuth Token get one. But I think I missing something, because during the process I cannot include info for user (name/password), so, for example https://api.ebay.com/sell/fulfillment/v1/order?limit=10 returns:
{
"errors": [{
"errorId": 1100,
"domain": "ACCESS",
"category": "REQUEST",
"message": "Access denied",
"longMessage": "Insufficient permissions to fulfill the request."
}]
}
Any idea how can I get a token for the user?
Just snippet of code to make things more clear:
AppSettings = {
'app_id': EBAY_PRODUCTION_APPID,
'app_secret': EBAY_PRODUCTION_CERTID,
'dev_id': EBAY_PRODUCTION_DEVID,
'ruName': EBAY_PRODUCTION_RU_NAME
}
authHeaderData = AppSettings['app_id'] + ':' + AppSettings['app_secret']
encodedAuthHeader = base64.b64encode(authHeaderData.encode())
headers = {
"Content-Type": "application/x-www-form-urlencoded",
"Authorization": "Basic ".encode() + encodedAuthHeader
}
body = {
"grant_type": "client_credentials",
"redirect_uri": settings.EBAY_PRODUCTION_RU_NAME,
"scope": "https://api.ebay.com/oauth/api_scope"
}
data = urllib.parse.urlencode(body)
tokenURL = "https://api.ebay.com/identity/v1/oauth2/token"
response = requests.post(tokenURL, headers=headers, data=body)
authDict = response.json()
So the request to run I need is:
r = requests.get("https://api.ebay.com/sell/fulfillment/v1/order?limit=10",
headers={"Authorization": "{}".format(authDict['access_token']),
"Content-Type": "application/json",
"X-EBAY-C-MARKETPLACE-ID": "EBAY_UK",
"Accept": "application/json"
})
According to this, I believe you are supposed to use the following authorization header:
headers['Authorization'] = "Bearer " + USER_ACCESS_TOKEN
Where the USER_ACCESS_TOKEN is the massive token generated on this page.
It looks something like this:
'v^1.1#i^1#p^3#f^0#I^3#r^0#t^ ...
...
...
...
... bfxr8BJtphi2M/oo2xpYo2hiMWxmZt4fVzS7qe2tMXUSAAA='
The Authorization you are using is for requests that aren't linked to a specific user account (search results, meta data for items, etc.). To make requests that are for specific users (eg. orders or inventory updates), you have to get their permission via their USER_ACCESS_TOKEN.
If you need help getting the USER_ACCESS_TOKEN let me know and I'll update.
Note that I have been trying to do the same thing that you are doing for about 6 hours now, and still haven't figured it out, so I am not confident in this answer.
Hope this helps. If you do figure it out, you should post an answer so others can too (ie myself xD).
eBay definitely wins the gold metal for worst api docs in the history of api docs...
The API Explore # developer.ebay.com has description of HTTP Headers for each RestFul API. E.G. Fulfillment API - getOrdres:
HTTP Headers
Authorization:Bearer <OAUTH_token>
Accept:application/json
Content-Type:application/json
Sample code:
import requests,json
headers = {
"Authorization": "Bearer Type_Your_Token_here_or_Paste_IF_too_long",
'Accept':'application/json',
'Content-Type':'application/json'
}
EndPoint = "https://api.ebay.com/sell/fulfillment/v1/order?filter=orderfulfillmentstatus:%7BNOT_STARTED|IN_PROGRESS%7D"
response = requests.get(EndPoint,headers = headers)
https://newbedev.com/ebay-oauth-token-and-refresh-tokens has introduced eBay OAuth token much better than eBay.
By the way, "grant_type": "client_credentials" is only valid for clients who can on have one scope. https://api.ebay.com/oauth/api_scope.
A shortcut to get your code run: the refresh token is actually the token you have for standard API, which is valid for 18 months. With a refresh token, you can get token without getting the annoying "authorization code" via user consent.
In short, please use refresh token to get user access token for the restful API.
Hope the above helps.

Microsoft Graph API: Authorization_IdentityNotFound

I'm following the Get access without a user guide to write a Python script that will call Microsoft Graph.
This script will be scheduled from cron so it cannot get admin consent (therefore authorize using Client Credentials). I am able to successfully obtain a token using this call:
request_url = "https://login.microsoftonline.com/mytenant.onmicrosoft.com/oauth2/v2.0/token"
data = {
'Host' : 'login.microsoftonline.com',
'Content-Type' : 'application/x-www-form-urlencoded',
'client_id' : 'my-client-id-1234',
'scope' : 'https://graph.microsoft.com/.default',
'client_secret' : client_secret,
'grant_type' : 'client_credentials'
}
response = requests.post(url = request_url, data = data)
I then try to get a user listing with this call, using the valid token:
request_url = "https://graph.microsoft.com/v1.0/users"
headers = {
'Authorization' : 'Bearer ' + token,
'Host' : 'graph.microsoft.com'
}
response = requests.get(url = request_url, headers = headers)
The problem is that I get an Authorization_IdentityNotFound error:
<Response [401]>
{
"error": {
"code": "Authorization_IdentityNotFound",
"message": "The identity of the calling application could not be established.",
"innerError": {
"request-id": "2257f532-abc4-4465-b19f-f33541787e76",
"date": "2018-03-27T19:11:07"
}
}
}
These are the permissions I've selected:
Any idea how to fix this error?
For others running into this issue, I was also getting this error until found out the documentation omits a very important caveat:
For client credentials, if the app belongs to a work or school (organization) context then for https://login.microsoftonline.com/common/oauth2/token replace common with a tenantId or domain name
See
Authorization_IdentityNotFound on Microsoft Graph API request
First things first, you can go ahead an remove all those Delegated Permission scopes. If you're using the Client Credentials Grant, you will only be using Application Permission scopes.
Second, you need to execute the Admin Consent flow before you can use Client Credentials. This is done by having a Global Admin from the tenant authenticate and accept your scope request:
https://login.microsoftonline.com/common/adminconsent?client_id=[APPLICATION ID]&redirect_uri=[REDIRECT URI]
You can read more about Admin Consent here: v2 Endpoint and Admin Consent

Facebook Messenger with Flask

I'm trying to get the FB messenger API working using Python's Flask, adapting the following instructions: https://developers.facebook.com/docs/messenger-platform/quickstart
So far, things have been going pretty well. I have verified my callback and am able to receive the messages I send using Messenger on my page, as in the logs in my heroku server indicate the appropriate packets of data are being received by my server. Right now I'm struggling a bit to send responses to the client messenging my app. In particular, I am not sure how to perform the following segment from the tutorial in Flask:
var token = "<page_access_token>";
function sendTextMessage(sender, text) {
messageData = {
text:text
}
request({
url: 'https://graph.facebook.com/v2.6/me/messages',
qs: {access_token:token},
method: 'POST',
json: {
recipient: {id:sender},
message: messageData,
}
}, function(error, response, body) {
if (error) {
console.log('Error sending message: ', error);
} else if (response.body.error) {
console.log('Error: ', response.body.error);
}
});
}
So far, I have this bit in my server-side Flask module:
#app.route('/', methods=["GET", "POST"])
def chatbot_response():
data = json.loads(req_data)
sender_id = data["entry"][0]["messaging"][0]["sender"]["id"]
url = "https://graph.facebook.com/v2.6/me/messages"
qs_value = {"access_token": TOKEN_OMITTED}
json_response = {"recipient": {"id": sender_id}, "message": "this is a test response message"}
response = ("my response text", 200, {"url": url, "qs": qs_value, "method": "POST", "json": json_response})
return response
However, running this, I find that while I can process what someone send my Page, it does not send a response back (i.e. nothing shows up in the messenger chat box). I'm new to Flask so any help would be greatly appreciated in doing the equivalent of the Javascript bit above in Flask.
Thanks!
This is the code that works for me:
data = json.loads(request.data)['entry'][0]['messaging']
for m in data:
resp_id = m['sender']['id']
resp_mess = {
'recipient': {
'id': resp_id,
},
'message': {
'text': m['message']['text'],
}
}
fb_response = requests.post(FB_MESSAGES_ENDPOINT,
params={"access_token": FB_TOKEN},
data=json.dumps(resp_mess),
headers = {'content-type': 'application/json'})
key differences:
message needs a text key for the actual response message, and you need to add the application/json content-type header.
Without the content-type header you get the The parameter recipient is required error response, and without the text key under message you get the param message must be non-empty error response.
This is the Flask example using fbmq library that works for me:
echo example :
from flask import Flask, request
from fbmq import Page
page = fbmq.Page(PAGE_ACCESS_TOKEN)
#app.route('/webhook', methods=['POST'])
def webhook():
page.handle_webhook(request.get_data(as_text=True))
return "ok"
#page.handle_message
def message_handler(event):
page.send(event.sender_id, event.message_text)
In that scenario in your tutorial, the node.js application is sending an HTTP POST request back to Facebook's servers, which then forwards the content on to the client.
So far, sounds like your Flask app is only receiving (AKA serving) HTTP requests. The reason is that that's what the Flask library is all about, and it's the only thing that Flask does.
To send an HTTP request back to Facebook, you can use any Python HTTP client library you like. There is one called urllib in the standard library, but it's a bit clunky to use... try the Requests library.
Since your request handler is delegating to an outgoing HTTP call, you need to look at the response to this sub-request also, to make sure everything went as planned.
Your handler may end up looking something like
import json
import os
from flask import app, request
# confusingly similar name, keep these straight in your head
import requests
FB_MESSAGES_ENDPOINT = "https://graph.facebook.com/v2.6/me/messages"
# good practice: don't keep secrets in files, one day you'll accidentally
# commit it and push it to github and then you'll be sad. in bash:
# $ export FB_ACCESS_TOKEN=my-secret-fb-token
FB_TOKEN = os.environ['FB_ACCESS_TOKEN']
#app.route('/', method="POST")
def chatbot_response():
data = request.json() # flasks's request object
sender_id = data["entry"][0]["messaging"][0]["sender"]["id"]
send_back_to_fb = {
"recipient": {
"id": sender_id,
},
"message": "this is a test response message"
}
# the big change: use another library to send an HTTP request back to FB
fb_response = requests.post(FB_MESSAGES_ENDPOINT,
params={"access_token": FB_TOKEN},
data=json.dumps(send_back_to_fb))
# handle the response to the subrequest you made
if not fb_response.ok:
# log some useful info for yourself, for debugging
print 'jeepers. %s: %s' % (fb_response.status_code, fb_response.text)
# always return 200 to Facebook's original POST request so they know you
# handled their request
return "OK", 200
When doing responses in Flask, you have to be careful. Simply doing a return statement won't return anything to the requester.
In your case, you might want to look at jsonify(). It will take a Python dictionary and return it to your browser as a JSON object.
from flask import jsonify
return jsonify({"url": url, "qs": qs_value, "method": "POST", "json": json_response})
If you want more control over the responses, like setting codes, take a look at make_response()

Categories