This code produces a POST request:
urllib2.urlopen("http://somedomain.com/", data)
I would like to produce a GET request - any ideas on how to do that?
Thanks for the help!
Try:
urllib2.urlopen("http://somedomain.com/?" + data)
[edited]
If you want to send xml/json/etc data in the body, use something like:
urllib2.urlopen("http://somedomain.com/?" + parameters, data)
This will use the POST method, but any "GET" parameters will also be available to your application.
Alternatively, you also use requests that has a more explicit API:
Make a GET request:
r = requests.get('https://github.com/timeline.json')
Make a POST request:
r = requests.post("http://httpbin.org/post")
Related
does anyone have any idea, why the output of this script, where i use requests.post to login is code 404, Not found, and the same script, where I use only requests.get has code 200 OK? What should I change?
import requests
URL = 'https://www.stratfor.com/login'
session = requests.Session()
page = session.post(URL)
print(page.status_code, page.reason)
Thank you.
it seem to be worked with get request and should returned 405 but it depends on the server
One good way to note the right page to log in is to log the network calls.
After looking at the calls, a request is sent to
URL = https://www.stratfor.com/api/v3/user/login
The API endpoint actually expects a payload like this:
payload = {username: "YOU_USER", password: "YOUR_PASS"}
Try something like this:
r = requests.post(URL,json=payload)
You might need to pass more headers, which you can poke the network call log for. Although, it seems like that user and password are passed as raw strings here? If so, that's definitely not safe.
I'm trying to write API client for Jira with Python requests lib according reference:
https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples/
Request to be generated:
http://localhost:8080/rest/api/2/search?jql=assignee=charlie&startAt=2&maxResults=2
As I know, parameters to GET request should be passed as dictionary like:
params = {'assignee':'charlie', 'startAt':'2'}
But all main parameters are nested in jql parameter, so I assume there is should be a nested dict like:
params = {'jql': {'assignee': 'charlie'}}
But that's doesn't work - as a result I've got request to
/rest/api/2/search?jql=assignee
As expect /rest/api/2/search?jql=assignee=charlie
using
r = requests.get(url, params=params)
How to manage such request?
UPD:
To be more clear, I'd like to wrap request in a method with kwargs, like:
search_query(assignee='charlie', startAt=1, etc...)
And then generate a query using this params, but maybe there are any other ideas.
You are missing couple of key parameters, mainly if you are pushing data via requests, the data go into the data argument. Also the moment you push JSON data, you need to set the headers correctly as well. The last thing is authentication. Have you tried to post it in this manner?
import json
requests.post(url=url, headers={"Content-Type": "application/json"},
auth=('username', 'password'), # your username and password
data=json.dumps(params)
)
Also by the JIRA documentation you've provided (https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples/) if you want to push query as data, the url you want is /rest/api/2/search.
I am trying to send HTTP GET request using urllib/urllib2 with some data.
If we set some value of data param in urllib2.urlopen(url, data) the request object is changed to send POST request instead of GET.
Is there any way to achieve this? Standard or Hack?
Code snippet,
import requests
import urllib
query = urllib.urlencode({'query':'["=", ["fact", "role"], "storage"]'})
# using request object
print 'Output 1.'
response = requests.get("http://localhost:8082/v3/nodes", data=query)
print response.json()
print
# using urllib object
print 'Output 2.'
resp = urllib.urlopen('http://localhost:8082/v3/nodes', data=query)
print resp.read()
Output:
Output 1.
[{u'deactivated': None, u'facts_timestamp': u'2016-02-04T14:06:07.269Z', u'name': u'node_xx_11', u'report_timestamp': None, u'catalog_timestamp': u'2016-02-04T14:06:16.958Z'}, {u'deactivated': None, u'facts_timestamp': u'2016-02-04T14:06:05.865Z', u'name': u'node_xx_12', u'report_timestamp': None, u'catalog_timestamp': u'2016-02-04T14:06:13.614Z'}]
Output 2.
The POST method is not allowed for /v3/nodes
For the
References I have gone through,
https://docs.python.org/2/library/urllib2.html#urllib2.urlopen
https://docs.python.org/2/library/urllib2.html#urllib2.Request.add_data
This is not the road block for me, as I am able to use requests module for sending the data with GET request type. Curiosity is the reason of this post.
The data parameter of urlopen is used to set the body of the request. GET requests cannot contain a body, as they should only be used to return a resource, which should only be defined by it's URL.
If you need to pass parameters, you can append them to the url, in your case :
from urllib.request import urlopen
urlopen('http://localhost:8082/v3/nodes?{}'.format(query))
The data parameter is for POST only and you cannot send a body in a GET request, so if you want to specify parameters you have to pass them through the URL.
One easy way to build such an URL is through the help of urllib.urlencode. Take a look at the documentation for this function.
I am using python mechanize lib and I am trying to use http PUT method on some url - but I cant find any option for this. I see only GET and POST methods...
If the PUT method is not working maybe some1 can tell me a better lib for doing this?
One possible solution:
class PutRequest(mechanize.Request):
"Extend the mechanize Request class to allow a http PUT"
def get_method(self):
return "PUT"
You can then use this when making a request like this:
browser.open(PutRequest(url,data=your_encoded_params,headers=your_headers))
NOTE: I arrived at this solution by digging into the mechanize code packages to find out where mechanize was setting the HTTP method. I noticed that when we call mechanize.Request, we are using the Request class in _request.py which in turn is extending the Request class in _urllib2_fork.py. The http method is actually set in get_method of the Request class in _urllib2_fork.py. Turns out get_method in _urllib2_fork.py was allowing only GET and POST methods. To get past this limitation, I ended up writing my own put and delete classes that extended mechanize. Request but over-rode get_method() only.
Use Requests:
>>> import requests
>>> result = requests.put("http://httpbin.org/put", data='hello')
>>> result.text
Per documentation:
requests.put(url, data=None, **kwargs)
Sends a PUT request. Returns Response object.
Parameters:
url – URL for the new Request object.
data – (optional) Dictionary or bytes to send in the body of the Request.
**kwargs – Optional arguments that request takes.
Via Mechanize:
import mechanize
import json
class PutRequest(mechanize.Request):
def get_method(self):
return 'PUT'
browser = mechanize.Browser()
browser.open(
PutRequest('http://example.com/',
data=json.dumps({'locale': 'en'}),
headers={'Content-Type': 'application/json'}))
See also http://qxf2.com/blog/python-mechanize-the-missing-manual/ (probably outdated).
Requests does it in a nicer way as Key Zhu said.
There's a lot of stuff out there on urllib2 and POST calls, but I'm stuck on a problem.
I'm trying to do a simple POST call to a service:
url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe',
'age' : '10'})
content = urllib2.urlopen(url=url, data=data).read()
print content
I can see the server logs and it says that I'm doing GET calls, when I'm sending the data
argument to urlopen.
The library is raising an 404 error (not found), which is correct for a GET call, POST calls are processed well (I'm also trying with a POST within a HTML form).
Do it in stages, and modify the object, like this:
# make a string with the request type in it:
method = "POST"
# create a handler. you can specify different handlers here (file uploads etc)
# but we go for the default
handler = urllib2.HTTPHandler()
# create an openerdirector instance
opener = urllib2.build_opener(handler)
# build a request
data = urllib.urlencode(dictionary_of_POST_fields_or_None)
request = urllib2.Request(url, data=data)
# add any other information you want
request.add_header("Content-Type",'application/json')
# overload the get method function with a small anonymous function...
request.get_method = lambda: method
# try it; don't forget to catch the result
try:
connection = opener.open(request)
except urllib2.HTTPError,e:
connection = e
# check. Substitute with appropriate HTTP code.
if connection.code == 200:
data = connection.read()
else:
# handle the error case. connection.read() will still contain data
# if any was returned, but it probably won't be of any use
This way allows you to extend to making PUT, DELETE, HEAD and OPTIONS requests too, simply by substituting the value of method or even wrapping it up in a function. Depending on what you're trying to do, you may also need a different HTTP handler, e.g. for multi file upload.
This may have been answered before: Python URLLib / URLLib2 POST.
Your server is likely performing a 302 redirect from http://myserver/post_service to http://myserver/post_service/. When the 302 redirect is performed, the request changes from POST to GET (see Issue 1401). Try changing url to http://myserver/post_service/.
Have a read of the urllib Missing Manual. Pulled from there is the following simple example of a POST request.
url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe', 'age' : '10'})
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
print response.read()
As suggested by #Michael Kent do consider requests, it's great.
EDIT: This said, I do not know why passing data to urlopen() does not result in a POST request; It should. I suspect your server is redirecting, or misbehaving.
The requests module may ease your pain.
url = 'http://myserver/post_service'
data = dict(name='joe', age='10')
r = requests.post(url, data=data, allow_redirects=True)
print r.content
it should be sending a POST if you provide a data parameter (like you are doing):
from the docs:
"the HTTP request will be a POST instead of a GET when the data parameter is provided"
so.. add some debug output to see what's up from the client side.
you can modify your code to this and try again:
import urllib
import urllib2
url = 'http://myserver/post_service'
opener = urllib2.build_opener(urllib2.HTTPHandler(debuglevel=1))
data = urllib.urlencode({'name' : 'joe',
'age' : '10'})
content = opener.open(url, data=data).read()
Try this instead:
url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe',
'age' : '10'})
req = urllib2.Request(url=url,data=data)
content = urllib2.urlopen(req).read()
print content
url="https://myserver/post_service"
data["name"] = "joe"
data["age"] = "20"
data_encoded = urllib2.urlencode(data)
print urllib2.urlopen(url + "?" + data_encoded).read()
May be this can help