I used python 3.6 and I want to post video stream to aws kinesis with API.
I used python aws client to create stream and GetDataEndPoint but when I want to post my data with my custom request (PutMedia doesn't include in python client actually), I get an error Unable to determine service/operation name to be authorized.
I've follow the api doc of aws kinesis video media PutMedia and GetMedia.
So I start by getting endpoint with GetDataEndPoint with client method:
response = client.get_data_endpoint( # aws client method
StreamName=STREAM_NAME,
APIName='PUT_MEDIA'
)
end_point = response['DataEndpoint'] # https://s-EXAMPLE.kinesisvideo.eu-west-1.amazonaws.com
and I post my data at this url:
headers = {
"x-amzn-stream-arn": STREAM_ARN,
"x-amzn-fragment-timecode-type": "ABSOLUTE",
"x-amzn-producer-start-timestamp": start_tmstp
}
# Sign header...
response = requests.post(end_point, data=data, headers=headers) # 403 - Unable to determine service/operation name to be authorized
So I don't understand why I get this error... I've found this troubleshooting on aws doc. But they say we must specify ApiName parameter. What I do...
This error might occur if the endpoint is not properly specified. When you are getting the endpoint, be sure to include the following parameter in the GetDataEndpoint call, depending on the API to be called:
I'm also wondering if the GetMedia method is actually implemented in client as they say here because when I debug this method, client don't call GetDataEndPoint and so make request at https://kinesisvideo.region.amazonaws.com insteed of https://ID_EXAMPLE.kinesisvideo.region.amazonaws.com. So method get error Unable to determine service/operation name to be authorized as explained in troubleshooting
The error you're getting is because you're probably providing the endpoint without the "action" that in your case would be putMedia.
Try to append /putMedia to your endpoint and don't forget to specify the "content-type": "application/json" header.
Btw you have also to generate the v4 signatures for your request. You can use a lib or follow this python guide to do it.
Related
regarding Authentification on DHL-SOAP API with zeep
i tried authenticating with the code provided by derAndre. but for me it didn't worked.
i am just trying to get the "getVersion" operation but it failed because of the Authentification.
But if i use it, i get an 401.
Server returned response (401) with invalid XML: Invalid XML content received (Space required after the Public Identifier, line 1, column 50).
Content: b'\n\n401 Unauthorized\n\nUnauthorized\nThis server could not verify that you\nare authorized to access the document\nrequested. Either you supplied the wrong\ncredentials (e.g., bad password), or your\nbrowser doesn\'t understand how to supply\nthe credentials required.\n\n'
this is the code, and i'm not getting the hang of it.
session = Session()
session.auth = HTTPBasicAuth(settings.DHL_SOAP_API_USER,settings.DHL_SOAP_API_PW)
client = Client(settings.DHL_WSDL, transport=Transport(session=session))
# Build Authentification header for API-Endpoint using zeep xsd
header = xsd.Element(
'{http://test.python-zeep.org}Authentification',
xsd.ComplexType([
xsd.Element(
'{http://test.python-zeep.org}user',
xsd.String()),
xsd.Element(
'{http://test.python-zeep.org}signature',
xsd.String()),
])
)
header_value = header(user=settings.DHL_SOAP_API_USER, signature=settings.DHL_SOAP_API_PW)
result = client.service.getVersion('majorRelease:?', 'minorRelease:?', _soapheaders=[header_value])
I was facing similar problem and found a solution by following zeep debugger
Note that, my purpose was to test data via sandbox endpoint: https://cig.dhl.de/services/sandbox/soap
However, the debugger said, it was sending request to production endpoint: https://cig.dhl.de/services/production/soap
The problem was in the wsdl file that I stored geschaeftskundenversand-api-3.2.2.wsdl locally.
If you notice, by the end of the file production address has been set to production by default.
<soap:address location="https://cig.dhl.de/services/production/soap"/>
Just set it to -
<soap:address location="https://cig.dhl.de/services/sandbox/soap"/>
Everything else are okay.
One just needs to make sure that the endpoint is okay. Because credentials are bit different for these two endpoints.
I am trying to use AWS DynamoDB in a Flutter app, and given the lack of an official AWS SDK for Dart I am forced to use the low level HTTP REST API.
The method for signing an AWS HTTP request is quite tedious, but using an AWS supplied sample as a guide, I was able to convert the Python to Dart pretty much line-for-line relatively easily. The end result was both sets of code producing the same auth signatures.
My issue came when I actually went to sent the request. The Python works as expected but sending a POST with Dart's HTTP package gives the error
The request signature we calculated does not match the signature you
provided. Check your AWS Secret Access Key and signing method. Consult
the service documentation for details.
I'll spare you the actual code for generating the auth signature, as the issue can be replicated simply by sending the same request hard-coded. See the Python and Dart code below.
Note: A valid response will return
Signature expired: 20190307T214900Z is now earlier than
20190307T215809Z (20190307T221309Z - 15 min.)
as the request signature uses current date and is only valid for 15 mins.
*****PYTHON CODE*****
import requests
headers = {'Content-Type':'application/json',
'X-Amz-Date':'20190307T214900Z',
'X-Amz-Target':'DynamoDB_20120810.GetItem',
'Authorization':'AWS4-HMAC-SHA256 Credential=AKIAJFZWA7QQAQT474EQ/20190307/ap-southeast-2/dynamodb/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=297c5a03c59db6da45bfe2fda6017f89a0a1b2ab6da2bb6e0d838ca40be84320'}
endpoint = 'https://dynamodb.ap-southeast-2.amazonaws.com/'
request_parameters = '{"TableName": "player-exports","Key": {"exportId": {"S": "HG1T"}}}'
r = requests.post(endpoint, data=request_parameters, headers=headers)
print('Response status: %d\n' % r.status_code)
print('Response body: %s\n' % r.text)
*****DART CODE*****
import 'package:http/http.dart' as http;
void main(List<String> arguments) async {
var headers = {'Content-Type':'application/json',
'X-Amz-Date':'20190307T214900Z',
'X-Amz-Target':'DynamoDB_20120810.GetItem',
'Authorization':'AWS4-HMAC-SHA256 Credential=AKIAJFZWA7QQAQT474EQ/20190307/ap-southeast-2/dynamodb/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=297c5a03c59db6da45bfe2fda6017f89a0a1b2ab6da2bb6e0d838ca40be84320'};
var endpoint = 'https://dynamodb.ap-southeast-2.amazonaws.com/';
var request_parameters = '{"TableName": "player-exports","Key": {"exportId": {"S": "HG1T"}}}';
http.post(endpoint, body: request_parameters, headers: headers).then((response) {
print("Response status: ${response.statusCode}");
print("Response body: ${response.body}");
});
}
The endpoint, headers and body are literally copy and pasted between the two sets of code.
Is there some nuance to how Dart HTTP works that I am missing here? Is there some map/string/json conversion of the headers or request_paramaters happening?
One thing I did note is that in the AWS provided example it states
For DynamoDB, the request can include any headers, but MUST include
"host", "x-amz-date", "x-amz-target", "content-type", and
"Authorization". Except for the authorization header, the headers must
be included in the canonical_headers and signed_headers values, as
noted earlier. Order here is not significant. Python note: The 'host'
header is added automatically by the Python 'requests' library.
But
a) When I add 'Host':'dynamodb.ap-southeast-2.amazonaws.com' to the headers in the Dart code I get the same result
and
b) If I look at r.request.headers after the Python requests returns, I can see that it has added a few new headers (Content-Length etc) automatically, but "Host" isn't one of them.
Any ideas why the seemingly same HTTP request works for Python Requests but not Dart HTTP?
Ok this is resolved now. My issue was in part a massive user-error. I was using a new IDE and when I generated the hardcoded example I provided I was actually still executing the previous file. Stupid, stupid, stupid.
But...
I was able to sort out the actual issue that caused me raise the question in the first place. I found that if you set the content type to "application/json" in the headers, the dart HTTP package automatically appends "; charset=utf-8". Because this value is part of the auth signature, when AWS encodes the values from the header to compare to the user-generated signature, they don't match.
The fix is simply to ensure that when you are setting the header content-type, make sure that you manually set it to "application/json; charset=utf-8" and not "application/json".
Found a bit more discussion about this "bug" after the fact here.
I'm trying to connect our application to Streamlabs' API so we can post donation alerts. To do that, I need to get an access token for the user whose channel we want to alert on. It's a normal OAuth2 thing. We hit the /authorize endpoint and get a code back, which we're then supposed to able to use to get an access token from the /token endpoint (https://dev.streamlabs.com/v1.0/reference#token-1).
But when we send the POST request for the token, we get an error saying the request is missing the "grant_type" parameter.
We're using the normal requests library. I've tried changing the format of the request to requests.post. I've tried altering the data by wrapping it in urlencode and json.dumps. Still no luck.
streamlabs_client_id = config('STREAMLABS_CLIENT_ID')
streamlabs_client_secret = config('STREAMLABS_CLIENT_SECRET')
streamlabs_redirect_uri = config('STREAMLABS_REDIRECT_URI')
grant_type = 'authorization_code'
querydict = {
"grant_type":"authorization_code",
"client_id":streamlabs_client_id,
"client_secret":streamlabs_client_secret,
"redirect_uri":streamlabs_redirect_uri,
"code":code
}
url = "https://streamlabs.com/api/v1.0/token"
streamlabs_response = requests.request("POST", url, data=querydict)
This is the json I get back every time:
{"error":"invalid_request","error_description":"The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed. Check the \"grant_type\" parameter."}
Any ideas what I'm doing wrong with the data?
tl;dr:
I am trying to set headers in a Python HTTP request for the first time and am getting a 404. I would appreciate any help (see bottom).
I have been experimenting with the Twitter API and have not been having much luck. Eventually I am trying to get all of the media (photos) a user has posted (20 or 50 or whatever per fetch)
In my experience with other APIs, this process would go as follows: Get The userID, Make a get request to some endpoint using that userId, get a JSON feed response.
It seems to be much more complicated in Twitter.
For instance, I do not see any URLs where I can attach an access token or client ID. Instead, in their documentation they show a place where I can retrieve my
Comsumer Key, Consumer Secret, Access Token, and, Access Token Secret
If I enter my request URI and "query", it generates an oAuth Signature, which in this case consists of a
Signature base string, Authorization header and cURL command
This is where things get confusing. It says
Important: This will only be valid for a few minutes. Also remember the cURL command will actually execute the request.
So:
Question 1: right away I am wondering how I can use these credentials to retrieve media over an entire day or a weekend if they become invalid only a few minutes later?
Question 2: Using their "exploring API" console, I can test this query (where I am trying to get the user ID for the Ford" twitter account I use GET https://api.twitter.com/1.1/users/lookup.json?screen_name=hyundai
Typing that into the browser alone returns a 404
{"errors": [{"message": "Bad Authentication data","code": 215}]}
But using their little console APP I can pick "O Auth 1 authentication" (using a twitter app I made) and I get the JSON response I want. Examining the request object in the console shows:
GET /1.1/users/lookup.json?screen_name=hyundai HTTP/1.1
Authorization:
OAuth oauth_consumer_key="555SECRET555",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1421370510",oauth_nonce="1869828628",oauth_version="1.0",oauth_token="333DONTHACKMEPLEASE333",oauth_signature="444SECRET444"
Host:
api.twitter.com
X-Target-URI:
https://api.twitter.com
Connection:
Keep-Alive
tl;dr Version:
So, I thought this was the headers object I would need to send from Python (3) to make the identical request. So here is that code:
import urllib.request
header = {
"Authorization" : "OAuth",
"oauth_consumer_key" :"555SECRET555",
"oauth_signature_method": "HMAC-SHA1",
"oauth_timestamp" : "1421362844",
"oauth_nonce":"1201915763",
"oauth_version": "1.0",
"oauth_token":"333CHANGINGTHIS33333",
"oauth_signature":"222CHANGEDTHIS222",
"Host": "api.twitter.com",
"X-Target-URI": "https://api.twitter.com",
"Connection": "Keep-Alive"
}
endpoint = 'https://api.twitter.com/1.1/users/lookup.json?screen_name=hyundai'
q = urllib.request.Request(endpoint, headers=header)
a = urllib.request.urlopen(q)
print(a.read().decode('utf-8'))
But I get a bad, 404 response.
Have I formatted my headers wrong here or is there another way to do this?
If you capture the network traffic from your request (use http not https), you will see that the headers you send are not the same as the header that are expected. This is why you are getting a 404 response.
What you want is something like
header = {
"Authorization": 'OAuth oauth_consumer_key="555SECRET555",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1421362844",oauth_nonce="1201915763",oauth_version="1.0",oauth_token="333CHANGINGTHIS33333",oauth_signature="222CHANGEDTHIS222"',
"Host": "api.twitter.com",
"X-Target-URI": "https://api.twitter.com",
"Connection": "Keep-Alive"
}
Of course, you could always use an OAuth library, such as RAuth or similar, see
Python: OAuth Library for a discussion
This error is so common that I found several hits on this error. However, couldn't find a solution to my problem.
I am trying to consume a REST API (not one of the publicly available ones) using requests module in Python. However, the specifications for the API consumption indicates that I call a POST method with URI, Content-type = 'application/xml' and body "<authCommand><userID> </userID><password> </password></authCommand>".
I tried this in the Python using requests:
r1 = requests.post('https://abc.360.net/rest/auth/1/login.xml',data= None, auth=('sdm#company.com','*********'), headers = {'content-type':'application/xml'})
When I run this, I receive a bad request error.
Is this the correct approach? Why am I receiving this error?
auth argument is to set Authorization header (basic http authentication by default). The API expects credentials in the body:
r = request.post(url, data=xml, headers={'Content-Type': 'application/xml'})
where xml=b"<authCommand><userID>sdm#company.com</authCommand>...".