tl;dr:
I am trying to set headers in a Python HTTP request for the first time and am getting a 404. I would appreciate any help (see bottom).
I have been experimenting with the Twitter API and have not been having much luck. Eventually I am trying to get all of the media (photos) a user has posted (20 or 50 or whatever per fetch)
In my experience with other APIs, this process would go as follows: Get The userID, Make a get request to some endpoint using that userId, get a JSON feed response.
It seems to be much more complicated in Twitter.
For instance, I do not see any URLs where I can attach an access token or client ID. Instead, in their documentation they show a place where I can retrieve my
Comsumer Key, Consumer Secret, Access Token, and, Access Token Secret
If I enter my request URI and "query", it generates an oAuth Signature, which in this case consists of a
Signature base string, Authorization header and cURL command
This is where things get confusing. It says
Important: This will only be valid for a few minutes. Also remember the cURL command will actually execute the request.
So:
Question 1: right away I am wondering how I can use these credentials to retrieve media over an entire day or a weekend if they become invalid only a few minutes later?
Question 2: Using their "exploring API" console, I can test this query (where I am trying to get the user ID for the Ford" twitter account I use GET https://api.twitter.com/1.1/users/lookup.json?screen_name=hyundai
Typing that into the browser alone returns a 404
{"errors": [{"message": "Bad Authentication data","code": 215}]}
But using their little console APP I can pick "O Auth 1 authentication" (using a twitter app I made) and I get the JSON response I want. Examining the request object in the console shows:
GET /1.1/users/lookup.json?screen_name=hyundai HTTP/1.1
Authorization:
OAuth oauth_consumer_key="555SECRET555",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1421370510",oauth_nonce="1869828628",oauth_version="1.0",oauth_token="333DONTHACKMEPLEASE333",oauth_signature="444SECRET444"
Host:
api.twitter.com
X-Target-URI:
https://api.twitter.com
Connection:
Keep-Alive
tl;dr Version:
So, I thought this was the headers object I would need to send from Python (3) to make the identical request. So here is that code:
import urllib.request
header = {
"Authorization" : "OAuth",
"oauth_consumer_key" :"555SECRET555",
"oauth_signature_method": "HMAC-SHA1",
"oauth_timestamp" : "1421362844",
"oauth_nonce":"1201915763",
"oauth_version": "1.0",
"oauth_token":"333CHANGINGTHIS33333",
"oauth_signature":"222CHANGEDTHIS222",
"Host": "api.twitter.com",
"X-Target-URI": "https://api.twitter.com",
"Connection": "Keep-Alive"
}
endpoint = 'https://api.twitter.com/1.1/users/lookup.json?screen_name=hyundai'
q = urllib.request.Request(endpoint, headers=header)
a = urllib.request.urlopen(q)
print(a.read().decode('utf-8'))
But I get a bad, 404 response.
Have I formatted my headers wrong here or is there another way to do this?
If you capture the network traffic from your request (use http not https), you will see that the headers you send are not the same as the header that are expected. This is why you are getting a 404 response.
What you want is something like
header = {
"Authorization": 'OAuth oauth_consumer_key="555SECRET555",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1421362844",oauth_nonce="1201915763",oauth_version="1.0",oauth_token="333CHANGINGTHIS33333",oauth_signature="222CHANGEDTHIS222"',
"Host": "api.twitter.com",
"X-Target-URI": "https://api.twitter.com",
"Connection": "Keep-Alive"
}
Of course, you could always use an OAuth library, such as RAuth or similar, see
Python: OAuth Library for a discussion
Related
I want to create container in splunk phantom using phantom rest api. I am using splunk phantom community version. This is the body I am passing using python post method
payload={
"description": "this is Useful description of this container.",
"label":"events",
"name":"xOkta event 110",
"sensitivity": "red",
"severity":"medium",
"source_data_identifier": "4",
"status": "new",
"container_type": "default",
"run_automation": "False",
"due_time": "2020-06-10T19:29:23.759Z",
}
and this is the code :
requests.post(url, auth=(username, password),json=payload, verify=False)
GET method is working. I am just unable to create container using phantom rest api(I am getting http status code 400 when post called). Any help will be appreciated.
That's an aged question, but to help others, you never told what is the URL endpoint you are trying, but assuming it was right (/rest/container), then the 400 returned means that the endpoint rejected your payload for some reason, but when it does, it outputs the error message in a JSON response (message key).
I would suggest to enable debug mode OR to try the same call via cURL or Postman to validate.
So, in a nutshell:
Make sure you are using the right protocol (HTTP/HTTPS).
Make sure you are sending the POST to /rest/container endpoint with the payload in JSON format.
Make sure the payload has the label, which is mandatory, and it does exist.
I am trying to use AWS DynamoDB in a Flutter app, and given the lack of an official AWS SDK for Dart I am forced to use the low level HTTP REST API.
The method for signing an AWS HTTP request is quite tedious, but using an AWS supplied sample as a guide, I was able to convert the Python to Dart pretty much line-for-line relatively easily. The end result was both sets of code producing the same auth signatures.
My issue came when I actually went to sent the request. The Python works as expected but sending a POST with Dart's HTTP package gives the error
The request signature we calculated does not match the signature you
provided. Check your AWS Secret Access Key and signing method. Consult
the service documentation for details.
I'll spare you the actual code for generating the auth signature, as the issue can be replicated simply by sending the same request hard-coded. See the Python and Dart code below.
Note: A valid response will return
Signature expired: 20190307T214900Z is now earlier than
20190307T215809Z (20190307T221309Z - 15 min.)
as the request signature uses current date and is only valid for 15 mins.
*****PYTHON CODE*****
import requests
headers = {'Content-Type':'application/json',
'X-Amz-Date':'20190307T214900Z',
'X-Amz-Target':'DynamoDB_20120810.GetItem',
'Authorization':'AWS4-HMAC-SHA256 Credential=AKIAJFZWA7QQAQT474EQ/20190307/ap-southeast-2/dynamodb/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=297c5a03c59db6da45bfe2fda6017f89a0a1b2ab6da2bb6e0d838ca40be84320'}
endpoint = 'https://dynamodb.ap-southeast-2.amazonaws.com/'
request_parameters = '{"TableName": "player-exports","Key": {"exportId": {"S": "HG1T"}}}'
r = requests.post(endpoint, data=request_parameters, headers=headers)
print('Response status: %d\n' % r.status_code)
print('Response body: %s\n' % r.text)
*****DART CODE*****
import 'package:http/http.dart' as http;
void main(List<String> arguments) async {
var headers = {'Content-Type':'application/json',
'X-Amz-Date':'20190307T214900Z',
'X-Amz-Target':'DynamoDB_20120810.GetItem',
'Authorization':'AWS4-HMAC-SHA256 Credential=AKIAJFZWA7QQAQT474EQ/20190307/ap-southeast-2/dynamodb/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=297c5a03c59db6da45bfe2fda6017f89a0a1b2ab6da2bb6e0d838ca40be84320'};
var endpoint = 'https://dynamodb.ap-southeast-2.amazonaws.com/';
var request_parameters = '{"TableName": "player-exports","Key": {"exportId": {"S": "HG1T"}}}';
http.post(endpoint, body: request_parameters, headers: headers).then((response) {
print("Response status: ${response.statusCode}");
print("Response body: ${response.body}");
});
}
The endpoint, headers and body are literally copy and pasted between the two sets of code.
Is there some nuance to how Dart HTTP works that I am missing here? Is there some map/string/json conversion of the headers or request_paramaters happening?
One thing I did note is that in the AWS provided example it states
For DynamoDB, the request can include any headers, but MUST include
"host", "x-amz-date", "x-amz-target", "content-type", and
"Authorization". Except for the authorization header, the headers must
be included in the canonical_headers and signed_headers values, as
noted earlier. Order here is not significant. Python note: The 'host'
header is added automatically by the Python 'requests' library.
But
a) When I add 'Host':'dynamodb.ap-southeast-2.amazonaws.com' to the headers in the Dart code I get the same result
and
b) If I look at r.request.headers after the Python requests returns, I can see that it has added a few new headers (Content-Length etc) automatically, but "Host" isn't one of them.
Any ideas why the seemingly same HTTP request works for Python Requests but not Dart HTTP?
Ok this is resolved now. My issue was in part a massive user-error. I was using a new IDE and when I generated the hardcoded example I provided I was actually still executing the previous file. Stupid, stupid, stupid.
But...
I was able to sort out the actual issue that caused me raise the question in the first place. I found that if you set the content type to "application/json" in the headers, the dart HTTP package automatically appends "; charset=utf-8". Because this value is part of the auth signature, when AWS encodes the values from the header to compare to the user-generated signature, they don't match.
The fix is simply to ensure that when you are setting the header content-type, make sure that you manually set it to "application/json; charset=utf-8" and not "application/json".
Found a bit more discussion about this "bug" after the fact here.
I'm trying to use Streamlabs API. Streamlabs API uses Oauth2 for creating apps. So first I send whoever's using my app to an authorization link containing my app's client id and the scopes I want to use.
(Something like this: streamlabs.com/api/v1.0/authorize?client_id=CLIENT-ID-HERE&redirect_uri=REDIRECT-URI&response_type=code&scope=SOME+SCOPES+HERE)
Once I've done that I receive a code at the redirect uri specified. I then use that code to get the access token for permanent access to the connected user's account. I then receive the access token from a POST request that works perfectly... Now I run into the problem. When getting the temporary code before the access token I specified the scopes: "donations.read +donations.create+alerts.write+alerts.create".
When authorizing, the app asks for permission to the different scopes. The scope in focus is "alerts.write" so that I can send test alerts using POST requests. But this doesn't work for some reason. To send a test alert I have to send a POST request to this url: "https://streamlabs.com/api/alerts/send_test_alert"
I've tried doing that in two different ways.
1:
import requests
url = "https://streamlabs.com/api/alerts/send_test_alert"
data = {
"access_token":"UserAccessTokenHere",
"type":"donation"
}
response = requests.post(url=url, data=data)
print(response.text)
2:
import requests
url = "https://streamlabs.com/api/alerts/send_test_alert?access_token=UserAccessTokenHere&type=donation"
response = requests.post(url=url)
print(response.text)
If I do print(response) it prints "Response [405]".
But if I do print(response.text) I get a long HTML document for this page: Error response page
Any ideas what's going wrong with my Python requests? send_test_alert documentation here: Link
I've contacted support and looks like you've made the same error as me.
You're not actually sending a request to the right URL.
You are a sending a request to: "https://streamlabs.com/api/alerts/send_test_alert"
You should be using the URL: "https://streamlabs.com/api/v1.0/alerts/send_test_alert"
I have been trying to use the Django-REST authentication to validate the user name /password given in a desktop app.
On the server side, I have installed the following DJANGO-REST-FRAMEWORK-JWT package found here:
https://github.com/GetBlimp/django-rest-framework-jwt
I have gone through the example and when I run the following on the command line get a token as a response:
curl -X POST -d "username=luca&password=letmein123" http://localhost:8000/api-token-auth/
And I get:
{"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6InBhbmthaiIsInVzZXJfaWQiOjIsImVtYWlsIjoiIiwiZXhwIjoxNDc5MTE5NzQ2fQ.RA085m-YSnGFheykCCxSVxI_9rW9AC9kEaOkUB5Gm0A"}
I tried something like:
import requests
resp = requests.post('http://127.0.0.1:8000/api-token-auth/', data={}, auth=('luca', 'letmein123'))
However, this always returns response code 400 with Bad request
My question is how can I do that from my desktop python app. I basically want to call the same API with the username and passord and be able to process the response and access protected APIs.
The auth parameter of requests.request is by default meant for Basic/Digest/Custom HTTP Auth, analogous to the -u, --user <user:password> parameter of curl.
You could define your own custom Authentication class to achieve the desired result, but the basic way to achieve the same result as your original curl request is:
resp = requests.post(
'http://localhost:8000/api-token-auth/',
data={"username": "luca", "password": "letmein123"})
The data dictionary can alternatively be supplied as json by using the json parameter if you prefer (the request would be different, but also supported by Django REST framework JWT).
You can then use the token (extracted with token = resp.json()['token']) to access the restricted urls as following:
requests.post(
'http://localhost:8000/some-personal-function/',
json={"foo": 42},
headers={'Authorization': 'JWT ' + token})
By the way looking at response.text might help finding the reason, which would, in the case of your 400 response, contain the following:
'{"username":["This field is required."],"password":["This field is required."]}'
This error is so common that I found several hits on this error. However, couldn't find a solution to my problem.
I am trying to consume a REST API (not one of the publicly available ones) using requests module in Python. However, the specifications for the API consumption indicates that I call a POST method with URI, Content-type = 'application/xml' and body "<authCommand><userID> </userID><password> </password></authCommand>".
I tried this in the Python using requests:
r1 = requests.post('https://abc.360.net/rest/auth/1/login.xml',data= None, auth=('sdm#company.com','*********'), headers = {'content-type':'application/xml'})
When I run this, I receive a bad request error.
Is this the correct approach? Why am I receiving this error?
auth argument is to set Authorization header (basic http authentication by default). The API expects credentials in the body:
r = request.post(url, data=xml, headers={'Content-Type': 'application/xml'})
where xml=b"<authCommand><userID>sdm#company.com</authCommand>...".