Sending data in a POST message to a RESTful web service - python

I need to send some JSON data in a POST message to a RESTful webservice.
Which python module should I be using for this? Is there some sample code I can refer to?

Which bit are you having trouble with? The JSON, or the POST?
For JSON, the json module has been included in Python since version 2.5. Just do json.dumps(my_data) to convert a data variable to JSON.
For the POST, there are various modules in the standard library, but the best bet is to install the third-party requests library.

Requests is probably the best library for the job. It certainly beats urllib and urllib2. You can get it and see an example at http://pypi.python.org/pypi/requests
or you can just install it with "pip install requests"
There's a few more examples using the Github API with both the requests library and others at https://github.com/issackelly/Consuming-Web-APIs-with-Python-Talk

here is what I've used for post and get requests
import httplib
connection = httplib.HTTPConnection('192.168.38.38:6543')
body_content = 'abcd123456xyz'
connection.request('POST', '/foo/bar/baa.html', body_content)
postResult = connection.getresponse()
connection.request('GET', '/foo/bar/baa.html')
response = connection.getresponse()
getResult = response.read()
It does same as this sequence of CLI commands:
curl -X POST -d "abcd123456xyz" 192.168.38.38:6543/foo/bar/baa.html
curl 192.168.38.38:6543/foo/bar/baa.html

Related

access ADFS/OIDC protected webapi with curl or python

I would like to access a webapi by a script(bash or python), which is protected by mod_openidc/apache2 and an self-hosted ADFS.
For the authentication, a certificate from a smartcard or locally stored certificate is required.
I already tried several approaches with python or curl, but got no nearly working script.
approach at python:
from oauthlib.oauth2 import BackendApplicationClient
from requests_oauthlib import OAuth2Session
client_id="abcdef-abcd-abcd-abcd-abcdefghijk"
client = BackendApplicationClient(client_id=client_id)
#client = BackendApplicationClient()
oauth = OAuth2Session(client=client)
protected_url="https://protectedurl/page/"
oauth.fetch_token(token_url='https://sts.myserver.net/adfs/oauth2/token/', include_client_id=True, cert=('/home/user/cert.pem', '/home/user/server.key'))
which lead to: "oauthlib.oauth2.rfc6749.errors.InvalidClientError: (invalid_client) MSIS9627: Received invalid OAuth client credentials request. Client credentials are missing or found empty"
curl:
curl --cert /home/user/cert.pem --key /home/user/server.key
https://sts.example.net/adfs/oauth2/authorize/?response_type=code&scope=openid%20email%20profile%20allatclaims&client_id=XXX&state=XXXredirect_uri=https%3A%2F%2Fexample.net%2Fpage%2Fredirect_uri&nonceXXX
Which gives the sts page in html
So I think I dont have some small bug, but a wrong approach
Since it works in a browser, I dont suggest a issue on server side
Any approaches and examples are warmly welcome

Call API in python to run Jenkins Job - problem with authentication

I have got a jenkins job which i can run with making a post request:
curl -u albert405:{mytoken} http://172.31.32.33:8080/job/URL_Job_Trigger/build?token=ozSVoEQfLg
Could you tell me how to place this authentication (albert405:{mytoken}) into my python script:
import requests
url = 'http://172.31.32.33:8080/job/URL_Job_Trigger/build?token=ozSVoEQfLg'
x = requests.post(url)
print(x.text)
I have managed to solve it by this :
http://YOUR_JENKINS_USER_ID:YOUR_API_TOKEN#YOUR_JENKINS_URL/job/YOUR_JENKINS_JOB/build
import requests
build = requests.post("http://YOUR_JENKINS_USER_ID:YOUR_API_TOKEN#YOUR_JENKINS_URL/job/YOUR_JENKINS_JOB/build?token=TokenName")
Where is your authentication code? I see none of it.
Jenkins uses Basic Auth, which is indicated here https://wiki.jenkins.io/display/JENKINS/Remote+access+API.
In order to send auth parameters with requests it's a simple:
res =requests.post(url, auth=("albert405", "password"))
Which is the first documentation you get when googling basic auth requests: https://2.python-requests.org/en/master/user/authentication/

Simple not-authorized request to a Github API using Tornado httpclient returns Forbidden

I am trying to make a HTTP Request to a JSON API like https://api.github.com using tornado.httpclient and I found that it always responses with FORBIDDEN 403.
Simplifying, I make a request using the CLI with:
$ python -m tornado.httpclient https://api.github.com
getting a tornado.httpclient.HTTPError: HTTP 403: Forbidden.
In other hand, if I try to request this URL via browser or a simple $ curl https://api.github.com, the response is 200 OK and the proper JSON file.
What is causing this? Should I set some specific Headers on the tornado.httpclient request? What's the difference with a curl request?
You have to put user agent in the request, see Github API for more details:
All API requests MUST include a valid User-Agent header. Requests with
no User-Agent header will be rejected. We request that you use your
GitHub username, or the name of your application, for the User-Agent
header value. This allows us to contact you if there are problems
It might be an issue with their robots.txt. Maybe tornado.httpclient modifies the User-Agent in a way that makes it appear to be a web crawler? I'm not that familiar with it.
I faced similar issue & problem was with configurable-http-proxy so I killed its process & restarted jupyterhub
ps aux | grep configurable-http-proxy
if there are any pid's from above command, kill them with
kill -9 <PID>
and restart ``

Python mongolab REST api

I am trying to access the mongolab REST api through python. Is the correct way to do this via pythons urllib2? I have tried the following:
import urllib2
p = urllib2.urlopen("https://api.mongolab.com/api/1/databases/mydb/collections/mycollection?apiKey=XXXXXXXXXXXXXXXX")
But this gives me an error:
urllib2.URLError: <urlopen error unknown url type: https>
What is the correct way of doing this? After connecting, how do I go on to POST a document to my collection? If someone could post up a code example, I would be very grateful. Thanks all for the help!
EDIT:
I've recompiled python with ssl support. How do I POST insert a document to a collection using mongolab REST API? Here is the code I have atm:
import urllib
import urllib2
url = "https://api.mongolab.com/api/1/databases/mydb/collections/mycollection?apiKey=XXXXXXXXXXXXXXXX"
data = {"x" : "1"}
request = urllib2.Request(url, data)
p = urllib2.urlopen(request)
Now, when I run this, I get the error
urllib2.HTTPError: HTTP Error 415: Unsupported Media Type
How do I insert documents using HTTP POST? Thanks!
That error is raised if you version of python does not include ssl support. What version are you using? Did you compile it yourself?
That said, when you get a version including ssl, using requests is a lot easier than urllib2, especially when POSTing data.

How to use urllib2 when users only have a API token?

how would i tranfoms this curl command:
curl -v -u 82xxxxxxxxxxxx63e6:api_token -X GET https://www.toggl.com/api/v6/time_entries.json
into urlib2?
I found this tutorial: http://www.voidspace.org.uk/python/articles/authentication.shtml
but they use a password and username. I can only use an API token.
Thank you.
see also this question:
Urllib2 raises 403 error while the same request in curl works fine
urllib2 is known to suck big time when it comes to authentication. To save some kitten lifes you should use http://docs.python-requests.org/en/latest/index.html

Categories