I'm trying to upload an image using requests on python.
This is what I send using browser
POST /upload-photo/{res1}/{res2}/{res3}/ HTTP/1.1
Host: tgt.tgdot.com
Connection: keep-alive
Content-Length: 280487
Authorization: Basic {value}=
Accept: */*
Origin: http://tgt.tgdot.com
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryA8sGeB48ZZCvG127
Referer: http://tgt.tgdot.com/{res1}/
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8,es;q=0.6
Cookie: fttoken={cookie_value}
This is my code
with open(os.getcwd()+"/images/thee1.JPG", "rb") as image_file:
encoded_image = base64.b64encode(image_file.read())
headers = {"Content-Type":"multipart/form-data", "Authorization":"Basic " + authvalue}
cookie = {cookiename: token.value}
r = requests.post(url, headers =headers, cookies = cookie, params=encoded_image)
print r.request.headers
print r.status_code
print r.text
I keep getting 414 Request-URI Too Large
I'm not sure what's missing here. I would really appreciate help
You are encoding the whole image into the request parameters, effectively extending the URL by the length of the image.
If you already encoded the image data, use the data parameter:
r = requests.post(url, headers=headers, cookies=cookie, data=encoded_image)
Note that requests can encode multipart/form-data POST bodies directly, there is no need for you to encode it yourself. Use the files parameter in that case, passing in a dictionary or sequence of tuples. See the POST Multiple Multipart-Encoded Files section of the documentation.
The library can also handle a username and password pair to handle the Authorization header; simply pass in a (username, password) tuple for the auth keyword argument.
Encoding an image to Base64 is not sufficient however. Your content-type header and your POST payload are not matching. You'd instead post the file with a field name:
with open(os.getcwd()+"/images/thee1.JPG", "rb") as image_file:
files = {'field_name': image_file}
cookie = {cookiename: token.value}
r = requests.post(url, cookies = cookie, files=files, auth=(username, password)
Related
I am trying to fill a form like that and submit it automaticly. To do that, I sniffed the packets while logging in.
POST /?pg=ogrgiris HTTP/1.1
Host: xxx.xxx.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded
Origin: http://xxx.xxx.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Safari/605.1.15
Referer: http://xxx.xxx.com/?pg=ogrgiris
Upgrade-Insecure-Requests: 1
DNT: 1
Content-Length: 60
Connection: close
seviye=700&ilkodu=34&kurumkodu=317381&ogrencino=40&isim=ahm
I repeated that packet by burp suite and saw works porperly. the response was the html of the member page.
Now I tried to do that on python. The code is below:
import requests
url = 'http://xxx.xxx.com/?pg=ogrgiris'
headers = {'Host':'xxx.xxx.com',
'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Encoding':'gzip, deflate',
'Content-Type':'application/x-www-form-urlencoded',
'Referer':'http://xxx.xxx.com/?pg=ogrgiris',
'Content-Lenght':'60','Connection':'close'}
credentials = {'seviye': '700','ilkodu': '34','kurumkodu': '317381','ogrecino': '40','isim': 'ahm'}
r = requests.post(url,headers=headers, data=credentials)
print(r.content)
the problem is, that code prints the html of the login page even I send all of the credentials enough to log in. How can I get the member page? thanks.
If the POST request displays a page with the content you want, then the problem is only that you are sending data as JSON, not in "form" data format (application/x-www-form-urlencoded).
If a session is created at the request base and you have to make another request for the requested data, then you have to deal with cookies.
Problem with data format:
r = requests.post(url, headers=headers, data=credentials)
Kwarg json = creates a request body as follows:
{"ogrecino": "40", "ilkodu": "34", "isim": "ahm", "kurumkodu": "317381", "seviye": "700"}
While data= creates a request body like this:
seviye=700&ilkodu=34&kurumkodu=317381&ogrencino=40&isim=ahm
You can try https://httpbin.org:
from requests import post
msg = {"a": 1, "b": True}
print(post("https://httpbin.org/post", data=msg).json()) # Data as Form data, look at key `form`, it's object in JSON because it's Form data format
print(post("https://httpbin.org/post", json=msg).json()) # Data as json, look at key `data`, it's string
If your goal is to replicate the sample request, you are missing a lot of the headers; this in particular is very important Content-Type: application/x-www-form-urlencoded because it will tell your HTTP client how to format/encode the payload.
Check the documentation for requests so see how these form posts can work.
I am on the hook to write a Python script to interact with a remote web server with http. Here is the server (name: username; password: passw0rd), basically I will need to upload an image to the remote server, and printout its analysis output.
I have almost zero knowledge on Python network programming and really have no idea how this can be worked out. Could anyone shed some lights on where should I start to write such a script? I can find the following http post request from chrome, but just have no idea how to proceed further:
POST /post HTTP/1.1
Host: 34.65.71.65
Connection: keep-alive
Content-Length: 3185
Cache-Control: max-age=0
Authorization: Basic dXNlcm5hbWU6cGFzc3cwcmQ=
Origin: http://34.65.71.65
Upgrade-Insecure-Requests: 1
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryUPXn3eOKoasOQMwW
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Referer: http://34.65.71.65/post
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7
This is the Python script I am writing right now:
import requests
# defining the api-endpoint
API_ENDPOINT = "http://34.65.71.65/post"
# your API key here
username = "username"
pwd = "passw0rd"
path = "./kite.png"
image_path = path
# Read the image into a byte array
image_data = open(image_path, "rb").read()
# data to be sent to api
data = image_data
# sending post request and saving response as response object
# r = requests.post(url = API_ENDPOINT, auth=(username, pwd), data = data)
r = requests.post(url = API_ENDPOINT, auth=(username, pwd), data = data)
# extracting response text
pastebin_url = r.text
print("The pastebin URL is:%s"%pastebin_url)
but somehow it triggers the following issue:
requests.exceptions.ConnectionError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))
And here is another trial:
import requests
# defining the api-endpoint
API_ENDPOINT = "http://34.65.71.65/post"
# your API key here
username = "username"
pwd = "passw0rd"
path = "./kite.png"
with open(path, 'rb') as file:
body = {'foo': 'bar'}
body_file = {'file_field': file}
response = requests.post(API_ENDPOINT, auth=(username, pwd), data=body, files=body_file)
print(response.content) # Prints result
Making an HTTP request in Python is very easy thanks to the requests API. Uploading a file requires you to read it first and then upload it in the body of a POST request.
The broken PIPE error often occurs when the server closes the connection before the client could send all the data. Which is often due to an inconsistency between the content size announced in the headers and the real content size. To resolve this, you should read the file as 'r' or 'rb' (if it is binary) and use requests API files kwargs to send the file.
import requests
with open(file.name, 'rb') as file:
body = {'foo': 'bar'}
body_file = {'file_field': file}
response = requests.post('your.url.example', data=body, files=body_file)
print(response.content) # Prints result
I'm automatizing the configuration of network devices. I have to login each device, change network address and passwords.
Using: Requests Python Module.
The first process I need to do, is login. It's a simple form. By inspecting the website I will analyze the Headers and the parameters necessary:
General:
Request URL:http://192.168.7.2:8000/api/login
Request Method:POST
Status Code:200 OK
Remote Address:192.168.7.2:8000
Referrer Policy:no-referrer-when-downgrade
Response Headers:
Transfer-Encoding:chunked
Request Headers:
POST /api/login HTTP/1.1
Host: 192.168.7.2:8000
Connection: keep-alive
Content-Length: 37
Accept: application/json, text/plain, */*
Origin: http://192.168.7.2:8000
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36
Content-Type: application/json;charset=UTF-8
Referer: http://192.168.7.2:8000/login.html
Accept-Encoding: gzip, deflate
Accept-Language: pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7
Request Payload:
{"user":"user","password":"password"}
Following the documentation I got in this code:
import requests
import json
url = "http://192.168.7.2:8000/api/login"
payload = {"user":"admin","password":"admin"}
r = requests.post(url,data=payload)
#r.encoding = "UTF-8"
print r.text
What does happen? Well, running the script I receive this output: {"errno":"20"} the same error if I login with user/password wrong. That's is not possible, because using this user/password through of web interface, the authentication is successful. I think that something is wrong in the code.
Changing the payload format: Same error {"errno":"20"}
payload = (("user","admin"),("password","admin"))
r = requests.post(url,data=payload)
#r.encoding = "UTF-8"
print r.text
JSON-Enconded: Reports a different error: {"errno":"0","key":"fc2d253fb9b25d5880dd4843f439bc7b"}
import requests
import json
url = "http://192.168.7.2:8000/api/login"
payload = {"user":"admin","password":"admin"}
r = requests.post(url,data=json.dumps(payload))
#r.encoding = "UTF-8"
print r.text
Passing json directly: Reports {"errno":"0","key":"6f7ed0ae5f14833f689fcb34c2bf5b77"}
import requests
import json
url = "http://192.168.7.2:8000/api/login"
payload = {"user":"admin","password":"admin"}
r = requests.post(url,json=payload)
#r.encoding = "UTF-8"
print r.text
Using a different url:
import requests
import json
url = "http://192.168.7.2:8000/login.html"
payload = {"user":"admin","password":"admin"}
r = requests.post(url,data=payload)
#r.encoding = "UTF-8"
print r.text
It's reports UnicodeEncodeError: 'ascii' codec can't encode characters in position 3426-3427: ordinal not in range(128)
Printing r.encoding -> ISO-8859-1 is the output.
Set r.encoding="utf-8" and print r.text Output: UnicodeEncodeError: 'ascii' codec can't encode character u'\xed' in position 3426: ordinal not in range(128)
Anyone has experience with this lib?
You may need to pass the username and password in the headers.
import requests
import json
url = "http://192.168.7.2:8000/api/login"
headers = {"user":"admin","password":"admin"}
r = requests.post(url, headers=headers)
print r.text.encode('utf-8')
POST /search HTTP/1.1
Host: chatango.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: cookies_enabled.chatango.com=yes; fph.chatango.com=http; id.chatango.com=programmable; auth.chatango.com={MY AUTH KEY - I already have this}
Connection: keep-alive
Referer: http://st.chatango.com/flash/sellers_external.swf
Content-Type: application/x-www-form-urlencoded
Content-Length: 27
s=B&ama=99&ami=13&t=25&f=20
I'd really like to know how to send this via python, I haven't found anything except sending that data part, I really don't understand how I'm supposed to send the cookie data as I have it stored into a variable which I got through an API, which obtains it through sockets.
You can add new headers in the request() method:
HTTPConnection.request(method, url[, body[, headers]])
See request documentation.
To add a cookie, just add the Cookie header.
Here is a POST example from the Python site:
import httplib, urllib
params = urllib.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
headers = {"Content-type": "application/x-www-form-urlencoded",
"Accept": "text/plain"}
conn = httplib.HTTPConnection("musi-cal.mojam.com:80")
conn.request("POST", "/cgi-bin/query", params, headers)
response = conn.getresponse()
print response.status, response.reason
data = response.read()
conn.close()
I'm trying to connect to the twiiter API through a POST request as the docs say but I always get a 403 forbidden error.
This is my code. I'm using urlib2 in python 2.7:
def auth_API():
url = 'https://api.twitter.com/oauth2/token'
header = {}
values = {}
header['User-Agent'] = 'Mozilla/6.0 (Windows NT 6.2; WOW64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1'
header['Authorization'] = 'Basic ' + B64BEARERTOKENCREDS
header['Content-Type'] = 'application/x-www-form-urlencoded;charset=UTF-8'
header['Accept-Encoding'] = 'gzip'
values['grant_type'] = 'client_credentials'
data = urllib.urlencode(values)
req = urllib2.Request(url, data, header)
try:
response = urllib2.urlopen(req)
response.read()
except urllib2.HTTPError as e:
print e
Checking the docs I found an example request wich is the same as mine:
Twitter example:
POST /oauth2/token HTTP/1.1
Host: api.twitter.com
User-Agent: My Twitter App v1.0.23
Authorization: Basic NnB1[...]9JM29jYTNFOA==
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 29
Accept-Encoding: gzip
grant_type=client_credentials
My request:
POST /oauth2/token HTTP/1.1
Content-Length: 29
Accept-Encoding: gzip
Connection: close
User-Agent: Mozilla/6.0 (Windows NT 6.2; WOW64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1
Host: api.twitter.com
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Authorization: Basic NnB1[...]YTNFOA==
grant_type=client_credentials
Any idea with what could be wrong with this?
Regards.
PS: I know that there are some third party libs for this but I want to make it by myself.
I solved my problem, the error was with base64.encodestring() which adds an \n at the end of the string messing up the request.
Using base64.b64encode() instead worked fine.
Regards