try:
request_url = http://sandbox.api.hmhco.com/v1/documents
with open("C:\\Users\\Animesh\\Downloads\\assignment_4.pdf", "rb") as f:
files = {
"file": f
}
r = requests.post(request_url, files=files)
r.raise_for_status()
return r.json()
except exceptions.RequestException as e:
print e
sys.exit(-1)
This gives a 500 status error. I used logging to get what is being sent:
send:
POST /v1/documents HTTP/1.1
Host: sandbox.api.hmhco.com
Vnd-HMH-Api-Key: some_api_key
Accept-Encoding: gzip, deflate
Content-Length: 56926
Accept: application/json
User-Agent: python-requests/2.9.1
Connection: keep-alive
Content-Type: multipart/form-data
Authorization: some_access_token
--e11c78c0aeeb4cafa9837388ab386660
Content-Disposition: form-data; name="file"; filename="assignment_4.pdf"
%PDF-1.5\n%\xc7\xec\x8f\xa2\n5.......afa9837388ab386660--'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Access-Control-Allow-Headers: Authorization, Content-Type, Vnd-HMH-Api-Key
header: Access-Control-Allow-Methods: POST, GET, OPTIONS, DELETE, PUT, PATCH
header: Access-Control-Allow-Origin: *
header: Access-Control-Max-Age: 3600
header: Date: Tue, 29 Dec 2015 05:21:58 GMT
header: Server: nginx/1.4.6 (Ubuntu)
header: X-Application-Context: palantir:8070
header: Content-Length: 0
header: Connection: keep-alive
500 Server Error: Internal Server Error for url: http://sandbox.api.hmhco.com/v1/documents
But the corresponding cURL command works:
curl -v -X POST -H "Vnd-HMH-Api-Key:some_api_key" -H "accepts:application/json" -H "Content-Type:application/json" -H "Authorization:some_access_token" "http://sandbox.api.hmhco.com/v1/documents" -F file=#"C:\Users\Animesh\Downloads\assignment_4.pdf"
The verbose out gives:
> POST /v1/documents HTTP/1.1
> User-Agent: curl/7.37.0
> Host: sandbox.api.hmhco.com
> Accept: */*
> Vnd-HMH-Api-Key:some_api_key
> accepts:application/json
> Authorization:some_access_token
> Content-Length: 56982
> Expect: 100-continue
> Content-Type:multipart/form-data; boundary=------------------------20b5cc23cfa6d1d4
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Access-Control-Allow-Headers: Authorization, Content-Type, Vnd-HMH-Api-Key
< Access-Control-Allow-Headers: accept, api_key, x-mashery-debug, Authorization, authCurrentDateTime, Content-Type
< Access-Control-Allow-Methods: POST, GET, OPTIONS, DELETE, PUT, PATCH
< Access-Control-Allow-Methods: POST, GET, OPTIONS, PUT, DELETE
< Access-Control-Allow-Origin: *
< Access-Control-Max-Age: 3600
< Access-Control-Request-Headers: accept, api_key, x-mashery-debug, Authorization, authCurrentDateTime, Content-Type
< Cache-Control: max-age=0, private, must-revalidate
< Content-Type: application/json;charset=UTF-8
< Date: Tue, 29 Dec 2015 05:28:40 GMT
< ETag: "e421273e09891349ef4808ca5677345c"
< P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
* Server nginx/1.4.6 (Ubuntu) is not blacklisted
< Server: nginx/1.4.6 (Ubuntu)
< Status: 200 OK
< User-Info: {"iss":"https://identity.api.hmhco.com","aud":"http://www.hmhco.com","iat":1451323840,"sub":"cn\u003dSauron Baraddur,uid\u003dsauron,uniqueIdentifier\u003dab1e436e-8177-4139-8e2d-52a6f7a8be27,dc\u0
3d1","http://www.imsglobal.org/imspurl/lis/v1/vocab/person":["Instructor"],"client_id":"ef5f7a03-58e8-48d7-a38a-abbd2696bdb6.hmhco.com","exp":1451409840}
< X-Application-Context: palantir:8070
< X-Rack-Cache: invalidate, pass
< X-Request-Id: a0078f7f5daf804affb98de6947170ca
< X-Runtime: 0.066617
< X-UA-Compatible: IE=Edge,chrome=1
< Content-Length: 452
< Connection: keep-alive
<
{"content_type":"application/pdf","created_at":"2015-12-29T00:28:40-05:00","file_tmp":"1451366920-13680-2938/assignment_4.pdf","original_filename":"assignment_4.pdf","secure_token":"7f395da4-5ccf-41a6-86a3-16a5
af0de79","size":56774,"updated_at":"2015-12-29T00:2 ......
This gives response code 200 and response body as a json:
{
"content_type": "application/pdf",
"created_at": "2015-12-28T14:13:25-05:00",
"file_tmp": "1451330005-13680-5506/assignment_4.pdf",
"original_filename": "assignment_4.pdf",
"secure_token": "9546cf79-0b00-4454-b577-6bdc8ddd7315",
"size": 56774,
"updated_at": "2015-12-28T14:13:25-05:00", ......
Change line which sends post request to:
r = requests.post(request_url, files=files, headers={'content-type': 'application/json'})
Because in sample above curl making post with 'content-type': 'application/json' and everything goes well and your python requests isn't. Setting content-type in python code should fix your issue.
Related
I get the below messages for every test step, which is bit annoying. I need to process the console logs in a different way.
send: b'PUT /api/v2/superadmin_personal/item/14278b98-4430-4d2e-8301-1e30501da3b3 HTTP/1.1\r\nHost: abc.lab.com:8080\r\nUser-Agent: python-requests/2.27.1\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nAuthorization: Bearer 2c0717a7-b477-4e02-b1b5-df2a2757db70\r\nContent-Length: 137\r\nContent-Type: application/json\r\n\r\n'
send: b'{"endTime": "1646987482101", "status": "PASSED", "issue": null, "launchUuid": "f380b026-d7c9-4596-b80a-dcaec6fa82f2", "attributes": null}'
reply: 'HTTP/1.1 200 OK\r\n'
header: Cache-Control: no-cache, no-store, max-age=0, must-revalidate
header: Content-Type: application/json
header: Date: Fri, 11 Mar 2022 08:30:58 GMT
header: Expires: 0
header: Pragma: no-cache
header: X-Content-Type-Options: nosniff
header: X-Frame-Options: DENY
header: X-Xss-Protection: 1; mode=block
header: Content-Length: 93
You can set rp.http.logging=false in the reportportal.prop file or as a JVM parameter.
There is a common switch for all HTTP requests/responses Python sends:
from http.client import HTTPConnection
HTTPConnection.debuglevel = 0
Unfortunately Python uses just print to log HTTP (as here), ignoring his own logging framework. That's really silly, but here where Python is. Therefore there is no any straight way to configure what you want log and what you would like to skip. You can just turn on or off console printing for all HTTP requests.
I'm new with rest APIs and I'm trying to set up an OAuth handshake and I need help with requesting the request token. I'm using the requests_oauthlib module in Python. Here is the sample code and it is returning Response [400].
consumer_key, consumer_secret, and request_url are all loaded in properly. I got my code to work using a different Auth module. Can someone explain what http headers are and how they are used in a GET request?
from requests_oauthlib import OAuth1
from variables import *
oauth = OAuth1(consumer_key, client_secret = consumer_secret)
request_token = requests.get(request_url, auth=oauth, params={'oauth_callback':'oob', 'format':'json'})
print request_token
request : your computer sends a http message to another computer usually on port 443 or 80
response : the server listens for any connection requests, and responds if it understands the message.
For example telnet stackoverflow.com 80 you can type in
GET /questions/52350391/can-someone-explain-get-requests-specifically-the-http-header HTTP/2
Host: stackoverflow.com
User-Agent: curl/7.54.0
Accept: */*
And then press enter twice to conclude the request header, at which point the server responds:
➜ mysite telnet stackoverflow.com 80
Trying 151.101.1.69...
Connected to stackoverflow.com.
Escape character is '^]'.
GET /questions/52350391/can-someone-explain-get-requests-specifically-the-http- header HTTP/2
Host: stackoverflow.com
User-Agent: curl/7.54.0
Accept: */*
HTTP/1.1 301 Moved Permanently
Content-Type: text/html; charset=utf-8
Location: https://stackoverflow.com/questions/52350391/can-someone-explain-get-requests-specifically-the-http-
X-Request-Guid: xxx
Content-Security-Policy: upgrade-insecure-requests
Accept-Ranges: bytes
Age: 0
Content-Length: 217
Accept-Ranges: bytes
Date: Sun, 16 Sep 2018 03:29:16 GMT
Via: 1.1 varnish
Age: 0
Connection: close
X-Served-By: cache-ord1744-ORD
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1537068557.736123,VS0,VE25
Vary: Fastly-SSL
X-DNS-Prefetch-Control: off
Set-Cookie: prov=xxx; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>
Connection closed by foreign host.
The telnet session then prints out the response from the server and closes the connection. The response will include several pieces, the response headers, and the response body.
Your example might look something like:
GET /some/oauth/api?oauth_callback=oob&format=json
Host: someplace.com
Authorization: Bearer asdfasdfasdfasdf
HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: no-store
Pragma: no-cache
{
"access_token":"sdfasasdfasdf",
"token_type":"bearer",
"expires_in":3600,
"refresh_token":"asdfasdfasdfasdf",
"scope":"create"
}
also check out :
curl -Lv https://stackoverflow.com/questions/52350391/can-someone-explain-get-requests-specifically-the-http-header | head -n 100
related:
https://www.oauth.com/oauth2-servers/access-tokens/access-token-response/
HTTP Request\Response Header Grammer
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
https://www.oauth.com/oauth2-servers/authorization/the-authorization-request/
I'm trying to debug why i'm not able to POST an xml file with requests .post() to an API. The following wget command works fine:
wget -vv --no-check-certificate --post-file dynobjadd.xml \
"https://1.1.1.1/api/?type=user-id&action=set&key=$MYSUPERSECRETKEY=&file-name=dynobjadd.xml&client=wget" \
--no-http-keep-alive -O response.out
successful wget output:
...
URI encoding = ‘UTF-8’
--2017-01-05 13:21:11-- https://1.1.1.1/api/?type=user-id&action=set&key=$MYSUPERSECRETKEY=&file-name=dynobjadd.xml&client=wget
Certificates loaded: 165
Connecting to 1.1.1.1:443... connected.
...
---request begin---
POST https://1.1.1.1/api/?type=user-id&action=set&key=$MYSUPERSECRETKEY=&file-name=dynobjadd.xml&client=wget HTTP/1.1
User-Agent: Wget/1.18 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: 1.1.1.1
Connection: Close
Content-Type: application/x-www-form-urlencoded
Content-Length: 175
---request end---
[writing BODY file dynobjadd.xml ... done]
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 200 OK
Server:
Date: Thu, 05 Jan 2017 20:21:12 GMT
Content-Type: application/xml; charset=UTF-8
Content-Length: 255
Connection: close
ETag: "437cf-12b-56e39c36"
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
X-FRAME-OPTIONS: SAMEORIGIN
Set-Cookie: PHPSESSID=123123; path=/; secure; HttpOnly
---response end---
200 OK
python code i am trying:
xml = open('dynobjadd.xml').read()
url = 'https://1.1.1.1/api/?type=user-id&action=set&key=$MYSUPERSECRETKEY=&file-name=dynobjadd.xml&client=requests'
r = requests.post(url, data=xml, verify=False )
r.content() output:
<response status = 'error' code = '400'><result><msg>No file uploaded</msg></result></response>
You should use argument 'files'.
From http://docs.python-requests.org/en/master/user/quickstart/#post-a-multipart-encoded-file :
>>> url = 'http://httpbin.org/post'
>>> files = {'file': open('report.xls', 'rb')}
>>> r = requests.post(url, files=files)
>>> r.text
{
...
"files": {
"file": "<censored...binary...data>"
},
...
}
By default httplib debug send, headers and reply information returns as logger.info,
Instead can how do i display send, headers and replay as part of Debug information?
import requests
import logging
import httplib
httplib.HTTPConnection.debuglevel = 1
logging.basicConfig() # you need to initialize logging, otherwise you will not see anything from requests
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True
requests.get('http://httpbin.org/headers')
It prints
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP Connection (1):
httpbin.org
send: 'GET /headers HTTP/1.1\r\nHost: httpbin.org\r\nConnection: keep-alive\r\nA
ccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.8.
1\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Server: nginx
header: Date: Mon, 14 Dec 2015 12:50:44 GMT
header: Content-Type: application/json
header: Content-Length: 156
header: Connection: keep-alive
header: Access-Control-Allow-Origin: *
header: Access-Control-Allow-Credentials: true
DEBUG:requests.packages.urllib3.connectionpool:"GET /headers HTTP/1.1" 200 156
<Response [200]>
Thanks #Eli
I could achieve using this post http://stefaanlippens.net/redirect_python_print
import logging
import sys
import requests
import httplib
# HTTP stream handler
class WritableObject:
def __init__(self):
self.content = []
def write(self, string):
self.content.append(string)
# A writable object
http_log = WritableObject()
# Redirection
sys.stdout = http_log
# Enable
httplib.HTTPConnection.debuglevel = 2
# get operation
requests.get('http://httpbin.org/headers')
# Remember to reset sys.stdout!
sys.stdout = sys.__stdout__
debug_info = ''.join(http_log.content).replace('\\r', '').decode('string_escape').replace('\'', '')
# Remove empty lines
debug_info = "\n".join([ll.rstrip() for ll in debug_info.splitlines() if ll.strip()])
It prints like
C:\Users\vkosuri\Dropbox\robot\lab>python New-Redirect_Stdout.py
send: GET /headers HTTP/1.1
Host: httpbin.org
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.8.1
reply: HTTP/1.1 200 OK
header: Server: nginx
header: Date: Tue, 15 Dec 2015 09:36:36 GMT
header: Content-Type: application/json
header: Content-Length: 156
header: Connection: keep-alive
header: Access-Control-Allow-Origin: *
header: Access-Control-Allow-Credentials: true
Thanks
Malli
some_logger.set_level() does not do what you think it does. It doesn't set the level of the logs being emitted by a logger. It sets the minimum level of log emitted by the logger that your handler will care about and acknowledge. To do what you're asking, I can only think of one real, reasonable way:
Capture the logs as they're coming in and re-log them. You can capture them with the idea described here, and use that in a subclass of requests. This would without a doubt be complicated. So, this is probably a good time to start asking yourself, "what am I really trying to achieve and is there another way to go about it?"
I've hit something truly strange with a pyCurl script hitting a local Django-Tastypie REST webserver.
Issuing HTTP PUT requests to the server succeeds when I use everything but pycurl (including curl), and fails with error 400 in pycurl.
After much googling and experimentation, I'm stumped. What's wrong here?
Curl call that works:
curl --verbose -X PUT -H 'Content-Type: application/json' -d '{"first_name": "Gaius","id": 1,"last_name": "Balthazar","login": "gbalthazar"}' http://localhost:8000/api/person/1/
PyCurl call that DOESN'T work (error 400):
import pycurl
import StringIO
curl = pycurl.Curl()
url = 'http://localhost:8000/api/person/1/'
curl.setopt(pycurl.URL,url)
curl.setopt(pycurl.VERBOSE, 1)
body = '{"first_name": "Gaius","id": 1,"last_name": "Baltar","login": "gbaltar"}'
curl.setopt(pycurl.READFUNCTION, StringIO.StringIO(body).read)
curl.setopt(pycurl.UPLOAD, 1)
curl.setopt(pycurl.HTTPHEADER,['Content-Type: application/json','Expect:'])
curl.setopt(curl.TIMEOUT, 5)
curl.perform()
(I've tried removing the Expects header as well, I see the header set to 100-Continue in the pycurl call, but same result.)
Unfortunately this project really does need pycurl's low-level access to HTTP timing stats to measure performance, so I can't do it with another HTTP/REST library.
Output of Curl Call:
* About to connect() to localhost port 8000 (#0)
* Trying 127.0.0.1...
* connected
* Connected to localhost (127.0.0.1) port 8000 (#0)
> PUT /api/person/1/ HTTP/1.1
> User-Agent: curl/7.27.0
> Host: localhost:8000
> Accept: */*
> Content-Type: application/json
> Content-Length: 78
>
* upload completely sent off: 78 out of 78 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Date: Thu, 05 Jun 2014 23:45:26 GMT
< Server: WSGIServer/0.1 Python/2.7.3
< Vary: Accept
< X-Frame-Options: SAMEORIGIN
< Content-Type: application/json
<
* Closing connection #0
{"first_name": "Gaius", "id": 1, "last_name": "Balthazar", "login": "gbalthazar", "pk": "1", "resource_uri": "/api/person/1/"}
Output of PyCurl Verbose Call:
* About to connect() to localhost port 8000 (#0)
* Trying 127.0.0.1...
* connected
* Connected to localhost (127.0.0.1) port 8000 (#0)
> PUT /api/person/1/ HTTP/1.1
User-Agent: PycURL/7.27.0
Host: localhost:8000
Accept: */*
Transfer-Encoding: chunked
Content-Type: application/json
* HTTP 1.0, assume close after body
< HTTP/1.0 400 BAD REQUEST
< Date: Thu, 05 Jun 2014 23:44:25 GMT
< Server: WSGIServer/0.1 Python/2.7.3
< X-Frame-Options: SAMEORIGIN
< Content-Type: application/json
<
* Closing connection #0
{"error": ""}
What am I missing here?
Found the answer:
It needs length of the request body to handle correctly
For POST:
curl.setopt(pycurl.POSTFIELDSIZE, len(body))
For PUT:
curl.setopt(pycurl.INFILESIZE, len(body))
(Yes, it's a different option for different HTTP calls... that's libcurl for you)
Not completely sure what triggers this behaviour, but the above fixes it and the tests work now.
EDIT: Adding verbose pycurl output from this:
* About to connect() to localhost port 8000 (#0)
* Trying 127.0.0.1...
* connected
* Connected to localhost (127.0.0.1) port 8000 (#0)
> PUT /api/person/1/ HTTP/1.1
User-Agent: PycURL/7.27.0
Host: localhost:8000
Accept: */*
Content-Type: application/json
Content-Length: 72
* We are completely uploaded and fine
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Date: Fri, 06 Jun 2014 17:41:38 GMT
< Server: WSGIServer/0.1 Python/2.7.3
< Vary: Accept
< X-Frame-Options: SAMEORIGIN
< Content-Type: application/json
<
* Closing connection #0
{"first_name": "Gaius", "id": 1, "last_name": "Baltar", "login": "gbaltar", "pk": "1", "resource_uri": "/api/person/1/"}