I need to write a python that accesses an internal to organization URL. I have an auth token.
How should my python look
At the moment I have this
import json
import requests
from pprint import pprint
path='/Users/Documents/sample_2.dat'
for url in open(path):
print url[1:-2]
headers = {'Content-type': 'application/json'}
response = requests.get(url[1:-2], headers=headers)
field_value = response.json()
print field_value["externals"]
sample_2.dat has 2 urls 1 below other
Example:
"http://xxx.abc.com/mfc/abc/v1/ext_info?id=1841261718,3421035156,B0185LBO7I,B0082SIL3K,B000PS8P3Q,B00G441OMY,0793522048,B00B12D2WY,3637015080,B00TNOUNVU&fields=ex.title,ex.url&fieldgroups=default"
"http://xxx.abc.com/mfc/abc/v1/ext_info?id=0553153617,B003W0CI6Y,B000R08E7Y,B001O2SAAU,B00B1MP3MG,B00QRHJBPU,B00007B4DC,0852597088,B0000003H4,1937715213&fields=ex.title,ex.url&fieldgroups=default"
Perhaps this might be useful, which can be found in the documentations
For GET requests that might require basic authentication, you can
include the auth paramter as follows:
response = requests.get('https://api.github.com/user', auth=('user','pass'))
As you can see, it is as simple as adding the auth parameter inside your get request.
Related
So i was following the guide for getting a OAUTH token from https://developer.spotify.com/documentation/general/guides/authorization/client-credentials/
I tried doing a python requests library of the above equivalent code but i got response 400 from the server. May I know where i am going wrong?
import requests
import json
import clientidandsecret
headers ={"Authorization": "Basic " + clientidandsecret.C_ID +":" +clientidandsecret.C_SECRET}
form = {"form":{"grant_type" :"client_credentials"}, "json":"true"}
result = requests.post("https://accounts.spotify.com/api/token", headers=headers, data=form)
print(result)
Your variable form is a Dict, if you want to use the parameter data in requests it needs to be a string, this will fix it:
import json
...
result = requests.post(url, headers=headers, data=json.dumps(form))
Or even better:
result = requests.post(url, headers=headers, json=form)
I am trying to login to http://127.0.0.1/dvwa/login.php, with Python requests.post method.
Currently I am doing as follows:
import requests
payload = {'username':'admin','password':'password'}
response = requests.post('http://127.0.0.1/dvwa/login.php', data=payload)
However it does not seem to be working. I should be getting a 301 status code from the response object, but I am only receiving 200 codes. I've also taken the cookies from my browser and set them in the requests object; however, this does not work, and also defeats the purpose of what I am trying to do.
I've also tried the following with no luck:
from requests.auth import HTTPBasicAuth
import requests
response = requests.get("http://127.0.0.1/dvwa/login.php",auth=HTTPBasicAuth('admin','password'))
and
from requests.auth import HTTPBasicAuth
import requests
cookies = {'PHPSESSID':'07761e3f52ae72fa7d0e2c57569c32a7'}
response = requests.get("http://127.0.0.1/dvwa/login.php",auth=HTTPBasicAuth('admin','password'),cookies=cookies)
None of the above methods give the result I require/want, which is simply logging in.
By default, requests will follow redirects. response.status_code will be the status code of the ultimate location. If you want to check if you've been redirected, look at response.history.
import requests
response = requests.get("http://google.com/") #301 redirects to 'www.google.com'
response.status_code
#200
response.history
#[<Respone [301]>]
response.url
#'http://www.google.com/'
Additionally, a good way to have requests keep track of your session/cookies is by using requests.Session
import requests
with requests.Session() as sesh:
sesh.post(the_url, data=payload)
#do more stuff in session
I appreciate your answer, however I found my answer question. It is as follows in case anyone else has the same issue.
instead of:
import requests
response = requests.post('http://127.0.0.1/dvwa/login.php',data={'username':'admin','password':'password'})
You also need the login token stored in the payload, as follows:
import requests
response = requests.post('http://127.0.0.1/dvwa/login.php',data={'username':'admin','password':'password','Login':'Login'})
It then logs me in correctly.
Trying to use method = plaintext for an oauth. I'm having a tough time finding any examples, or previous questions on plain text.
For those who don't know what it is but would like to help, this document provides a nice overview.
import requests
from requests_oauthlib import OAuth1
from rauth import OAuth1Session, OAuth1Service
myheaders = {'Authorization': 'OAuth ,oauth_consumer_key="5C82CC6BC7C6472154FBC9CAB24A29A2",oauth_signature_method="PLAINTEXT", oauth_signature="F9D6B42C41A618C273AB501F2F2613F1"'}
url = 'https://secure.tmsandbox.co.nz/Oauth/RequestToken?scope=MyTradeMeRead,MyTradeMeWrite '
r = requests.get(url, params=myheaders)
print(r)
This gives me < Response [400]>
Any ideas why?
(keys given work but are dummy)
When printing content this way:
>>>print (r.content)
The oauth_consumer_key parameter is required.
you have some syntax errors, your myheaders dictionary is not well formatted, fix it this way:
import requests
from requests_oauthlib import OAuth1
from rauth import OAuth1Session, OAuth1Service
myheaders = {'Authorization':'OAuth',
'oauth_consumer_key':'5C82CC6BC7C6472154FBC9CAB24A29A2',
'oauth_signature_method': 'PLAINTEXT',
'oauth_signature': 'F9D6B42C41A618C273AB501F2F2613F1'}
url = 'https://secure.tmsandbox.co.nz/Oauth/RequestToken?scope=MyTradeMeRead,MyTradeMeWrite '
r = requests.get(url, params=myheaders)
print(r.status_code)
print(r.content)
>>401
>>Invalid PLAINTEXT signature.
It seems you have another error that I can't figure out
rocksteady's solution worked
He did originally refer to dictionaries. But the following code to send the JSON string also worked wonders using requests:
import requests
headers = {
'Authorization': app_token
}
url = api_url + "/b2api/v1/b2_get_upload_url"
content = json.dumps({'bucketId': bucket_id})
r = requests.post(url, data = content, headers = headers)
I'm working with an API that requires me to send JSON as a POST request to get results. Problem is that Python 3 won't allow me to do this.
The following Python 2 code works fine, in fact it's the official sample:
request = urllib2.Request(
api_url +'/b2api/v1/b2_get_upload_url',
json.dumps({ 'bucketId' : bucket_id }),
headers = { 'Authorization': account_authorization_token }
)
response = urllib2.urlopen(request)
However, using this code in Python 3 only makes it complain about data being invalid:
import json
from urllib.request import Request, urlopen
from urllib.parse import urlencode
# -! Irrelevant code has been cut out !-
headers = {
'Authorization': app_token
}
url = api_url + "/b2api/v1/b2_get_upload_url"
# Tested both with encode and without
content = json.dumps({'bucketId': bucket_id}).encode('utf-8')
request = Request(
url=url,
data=content,
headers=headers
)
response = urlopen(req)
I've tried doing urlencode(), like you're supposed to. But this returns a 400 status code from the web server, because it's expecting pure JSON. Even if the pure JSON data is invalid, I need to somehow force Python into sending it.
EDIT: As requested, here are the errors I get. Since this is a flask application, here's a screenshot of the debugger:
Screenshot
Adding .encode('utf-8') gives me an "Expected string or buffer" error
EDIT 2: Screenshot of the debugger with .encode('utf-8') added
Since I have a similar application running, but the client still was missing, I tried it myself.
The server which is running is from the following exercise:
Miguel Grinberg - designing a restful API using Flask
That's why it uses authentication.
But the interesting part: Using requests you can leave the dictionary as it is.
Look at this:
username = 'miguel'
password = 'python'
import requests
content = {"title":"Read a book"}
request = requests.get("http://127.0.0.1:5000/api/v1.0/projects", auth=(username, password), params=content)
print request.text
It seems to work :)
Update 1:
POST requests are done using requests.post(...)
This here describes it well : python requests
Update 2:
In order to complete the answer:
requests.post("http://127.0.0.1:5000/api/v1.0/projects", json=content)
sends the json-string.
json is a valid parameter of the request and internally uses json.dumps()...
Is there any standard way of getting JSON data from RESTful service using Python?
I need to use kerberos for authentication.
some snippet would help.
I would give the requests library a try for this. Essentially just a much easier to use wrapper around the standard library modules (i.e. urllib2, httplib2, etc.) you would use for the same thing. For example, to fetch json data from a url that requires basic authentication would look like this:
import requests
response = requests.get('http://thedataishere.com',
auth=('user', 'password'))
data = response.json()
For kerberos authentication the requests project has the reqests-kerberos library which provides a kerberos authentication class that you can use with requests:
import requests
from requests_kerberos import HTTPKerberosAuth
response = requests.get('http://thedataishere.com',
auth=HTTPKerberosAuth())
data = response.json()
Something like this should work unless I'm missing the point:
import json
import urllib2
json.load(urllib2.urlopen("url"))
You basically need to make a HTTP request to the service, and then parse the body of the response. I like to use httplib2 for it:
import httplib2 as http
import json
try:
from urlparse import urlparse
except ImportError:
from urllib.parse import urlparse
headers = {
'Accept': 'application/json',
'Content-Type': 'application/json; charset=UTF-8'
}
uri = 'http://yourservice.com'
path = '/path/to/resource/'
target = urlparse(uri+path)
method = 'GET'
body = ''
h = http.Http()
# If you need authentication some example:
if auth:
h.add_credentials(auth.user, auth.password)
response, content = h.request(
target.geturl(),
method,
body,
headers)
# assume that content is a json reply
# parse content with the json module
data = json.loads(content)
If you desire to use Python 3, you can use the following:
import json
import urllib.request
req = urllib.request.Request('url')
with urllib.request.urlopen(req) as response:
result = json.loads(response.readall().decode('utf-8'))
Well first of all I think rolling out your own solution for this all you need is urllib2 or httplib2 . Anyways in case you do require a generic REST client check this out .
https://github.com/scastillo/siesta
However i think the feature set of the library will not work for most web services because they shall probably using oauth etc .. . Also I don't like the fact that it is written over httplib which is a pain as compared to httplib2 still should work for you if you don't have to handle a lot of redirections etc ..