I'm automatizing the configuration of network devices. I have to login each device, change network address and passwords.
Using: Requests Python Module.
The first process I need to do, is login. It's a simple form. By inspecting the website I will analyze the Headers and the parameters necessary:
General:
Request URL:http://192.168.7.2:8000/api/login
Request Method:POST
Status Code:200 OK
Remote Address:192.168.7.2:8000
Referrer Policy:no-referrer-when-downgrade
Response Headers:
Transfer-Encoding:chunked
Request Headers:
POST /api/login HTTP/1.1
Host: 192.168.7.2:8000
Connection: keep-alive
Content-Length: 37
Accept: application/json, text/plain, */*
Origin: http://192.168.7.2:8000
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36
Content-Type: application/json;charset=UTF-8
Referer: http://192.168.7.2:8000/login.html
Accept-Encoding: gzip, deflate
Accept-Language: pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7
Request Payload:
{"user":"user","password":"password"}
Following the documentation I got in this code:
import requests
import json
url = "http://192.168.7.2:8000/api/login"
payload = {"user":"admin","password":"admin"}
r = requests.post(url,data=payload)
#r.encoding = "UTF-8"
print r.text
What does happen? Well, running the script I receive this output: {"errno":"20"} the same error if I login with user/password wrong. That's is not possible, because using this user/password through of web interface, the authentication is successful. I think that something is wrong in the code.
Changing the payload format: Same error {"errno":"20"}
payload = (("user","admin"),("password","admin"))
r = requests.post(url,data=payload)
#r.encoding = "UTF-8"
print r.text
JSON-Enconded: Reports a different error: {"errno":"0","key":"fc2d253fb9b25d5880dd4843f439bc7b"}
import requests
import json
url = "http://192.168.7.2:8000/api/login"
payload = {"user":"admin","password":"admin"}
r = requests.post(url,data=json.dumps(payload))
#r.encoding = "UTF-8"
print r.text
Passing json directly: Reports {"errno":"0","key":"6f7ed0ae5f14833f689fcb34c2bf5b77"}
import requests
import json
url = "http://192.168.7.2:8000/api/login"
payload = {"user":"admin","password":"admin"}
r = requests.post(url,json=payload)
#r.encoding = "UTF-8"
print r.text
Using a different url:
import requests
import json
url = "http://192.168.7.2:8000/login.html"
payload = {"user":"admin","password":"admin"}
r = requests.post(url,data=payload)
#r.encoding = "UTF-8"
print r.text
It's reports UnicodeEncodeError: 'ascii' codec can't encode characters in position 3426-3427: ordinal not in range(128)
Printing r.encoding -> ISO-8859-1 is the output.
Set r.encoding="utf-8" and print r.text Output: UnicodeEncodeError: 'ascii' codec can't encode character u'\xed' in position 3426: ordinal not in range(128)
Anyone has experience with this lib?
You may need to pass the username and password in the headers.
import requests
import json
url = "http://192.168.7.2:8000/api/login"
headers = {"user":"admin","password":"admin"}
r = requests.post(url, headers=headers)
print r.text.encode('utf-8')
Related
I am trying to fill a form like that and submit it automaticly. To do that, I sniffed the packets while logging in.
POST /?pg=ogrgiris HTTP/1.1
Host: xxx.xxx.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded
Origin: http://xxx.xxx.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Safari/605.1.15
Referer: http://xxx.xxx.com/?pg=ogrgiris
Upgrade-Insecure-Requests: 1
DNT: 1
Content-Length: 60
Connection: close
seviye=700&ilkodu=34&kurumkodu=317381&ogrencino=40&isim=ahm
I repeated that packet by burp suite and saw works porperly. the response was the html of the member page.
Now I tried to do that on python. The code is below:
import requests
url = 'http://xxx.xxx.com/?pg=ogrgiris'
headers = {'Host':'xxx.xxx.com',
'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Encoding':'gzip, deflate',
'Content-Type':'application/x-www-form-urlencoded',
'Referer':'http://xxx.xxx.com/?pg=ogrgiris',
'Content-Lenght':'60','Connection':'close'}
credentials = {'seviye': '700','ilkodu': '34','kurumkodu': '317381','ogrecino': '40','isim': 'ahm'}
r = requests.post(url,headers=headers, data=credentials)
print(r.content)
the problem is, that code prints the html of the login page even I send all of the credentials enough to log in. How can I get the member page? thanks.
If the POST request displays a page with the content you want, then the problem is only that you are sending data as JSON, not in "form" data format (application/x-www-form-urlencoded).
If a session is created at the request base and you have to make another request for the requested data, then you have to deal with cookies.
Problem with data format:
r = requests.post(url, headers=headers, data=credentials)
Kwarg json = creates a request body as follows:
{"ogrecino": "40", "ilkodu": "34", "isim": "ahm", "kurumkodu": "317381", "seviye": "700"}
While data= creates a request body like this:
seviye=700&ilkodu=34&kurumkodu=317381&ogrencino=40&isim=ahm
You can try https://httpbin.org:
from requests import post
msg = {"a": 1, "b": True}
print(post("https://httpbin.org/post", data=msg).json()) # Data as Form data, look at key `form`, it's object in JSON because it's Form data format
print(post("https://httpbin.org/post", json=msg).json()) # Data as json, look at key `data`, it's string
If your goal is to replicate the sample request, you are missing a lot of the headers; this in particular is very important Content-Type: application/x-www-form-urlencoded because it will tell your HTTP client how to format/encode the payload.
Check the documentation for requests so see how these form posts can work.
I am on the hook to write a Python script to interact with a remote web server with http. Here is the server (name: username; password: passw0rd), basically I will need to upload an image to the remote server, and printout its analysis output.
I have almost zero knowledge on Python network programming and really have no idea how this can be worked out. Could anyone shed some lights on where should I start to write such a script? I can find the following http post request from chrome, but just have no idea how to proceed further:
POST /post HTTP/1.1
Host: 34.65.71.65
Connection: keep-alive
Content-Length: 3185
Cache-Control: max-age=0
Authorization: Basic dXNlcm5hbWU6cGFzc3cwcmQ=
Origin: http://34.65.71.65
Upgrade-Insecure-Requests: 1
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryUPXn3eOKoasOQMwW
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Referer: http://34.65.71.65/post
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7
This is the Python script I am writing right now:
import requests
# defining the api-endpoint
API_ENDPOINT = "http://34.65.71.65/post"
# your API key here
username = "username"
pwd = "passw0rd"
path = "./kite.png"
image_path = path
# Read the image into a byte array
image_data = open(image_path, "rb").read()
# data to be sent to api
data = image_data
# sending post request and saving response as response object
# r = requests.post(url = API_ENDPOINT, auth=(username, pwd), data = data)
r = requests.post(url = API_ENDPOINT, auth=(username, pwd), data = data)
# extracting response text
pastebin_url = r.text
print("The pastebin URL is:%s"%pastebin_url)
but somehow it triggers the following issue:
requests.exceptions.ConnectionError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))
And here is another trial:
import requests
# defining the api-endpoint
API_ENDPOINT = "http://34.65.71.65/post"
# your API key here
username = "username"
pwd = "passw0rd"
path = "./kite.png"
with open(path, 'rb') as file:
body = {'foo': 'bar'}
body_file = {'file_field': file}
response = requests.post(API_ENDPOINT, auth=(username, pwd), data=body, files=body_file)
print(response.content) # Prints result
Making an HTTP request in Python is very easy thanks to the requests API. Uploading a file requires you to read it first and then upload it in the body of a POST request.
The broken PIPE error often occurs when the server closes the connection before the client could send all the data. Which is often due to an inconsistency between the content size announced in the headers and the real content size. To resolve this, you should read the file as 'r' or 'rb' (if it is binary) and use requests API files kwargs to send the file.
import requests
with open(file.name, 'rb') as file:
body = {'foo': 'bar'}
body_file = {'file_field': file}
response = requests.post('your.url.example', data=body, files=body_file)
print(response.content) # Prints result
My goal is to access a website that uses HTTP authentication using python. I can open the website from my web browser and the header tells me that I should use HTTBDigestAuth:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.5
Authorization: Digest username="user", realm="CG Downloads", nonce="cTN0pKxqBQA=e46ad250f42f73e9076ebc97c417f0d38bac094a", uri="/fileadmin/teaching/2017/WS/adip/exercises/adip-uebung-00-.pdf", algorithm=MD5, response="5a57ddbcd1b20444100a91b1967a2782", qop=auth, nc=00000001, cnonce="5a6b041b4113bb9a"
Connection: keep-alive
Cookie: fe_typo_user=76b7e7e25372f782d94e91b51b854568
Host: cg.cs.uni-bonn.de
Referer: http://cg.cs.uni-bonn.de/de/lehre/ws-2017/vorlesung-algorithmisches-denken-und-imperative-programmierung/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0
However when I try to go the the page using requests and HTTPDigestAuth I get "401 Unauthorized" as a response.
import logging
import requests
from requests.auth import HTTPDigestAuth
try:
import httplib
except ImportError:
import http.client as httplib
httplib.HTTPConnection.debuglevel = 1
logging.basicConfig(level=logging.DEBUG)
url = 'http://cg.cs.uni-bonn.de/fileadmin/teaching/2017/WS/adip/exercises/adip-uebung-00-.pdf'
response = requests.get(url, auth=HTTPDigestAuth('user', 'pass'),
timeout=10)
print(response.status_code)
print(response.headers)
print(r.text)
Am I using the wrong authorization method or is my code wrong? I appreciate any help you can give me.
EDIT:
I am trying to access sites on cg.cs.uni-bonn.de, for example http://cg.cs.uni-bonn.de/fileadmin/teaching/2017/WS/adip/exercises/adip-uebung-00-.pdf
You just need to pass values to auth=HTTPDigestAuth('user', 'pass') like this:
user = 'admin' # change to your username
pass = '123456' # change to your password
...
response = requests.get(url, auth=HTTPDigestAuth(user, pass),
timeout=10)
...
I'm trying to upload an image using requests on python.
This is what I send using browser
POST /upload-photo/{res1}/{res2}/{res3}/ HTTP/1.1
Host: tgt.tgdot.com
Connection: keep-alive
Content-Length: 280487
Authorization: Basic {value}=
Accept: */*
Origin: http://tgt.tgdot.com
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryA8sGeB48ZZCvG127
Referer: http://tgt.tgdot.com/{res1}/
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8,es;q=0.6
Cookie: fttoken={cookie_value}
This is my code
with open(os.getcwd()+"/images/thee1.JPG", "rb") as image_file:
encoded_image = base64.b64encode(image_file.read())
headers = {"Content-Type":"multipart/form-data", "Authorization":"Basic " + authvalue}
cookie = {cookiename: token.value}
r = requests.post(url, headers =headers, cookies = cookie, params=encoded_image)
print r.request.headers
print r.status_code
print r.text
I keep getting 414 Request-URI Too Large
I'm not sure what's missing here. I would really appreciate help
You are encoding the whole image into the request parameters, effectively extending the URL by the length of the image.
If you already encoded the image data, use the data parameter:
r = requests.post(url, headers=headers, cookies=cookie, data=encoded_image)
Note that requests can encode multipart/form-data POST bodies directly, there is no need for you to encode it yourself. Use the files parameter in that case, passing in a dictionary or sequence of tuples. See the POST Multiple Multipart-Encoded Files section of the documentation.
The library can also handle a username and password pair to handle the Authorization header; simply pass in a (username, password) tuple for the auth keyword argument.
Encoding an image to Base64 is not sufficient however. Your content-type header and your POST payload are not matching. You'd instead post the file with a field name:
with open(os.getcwd()+"/images/thee1.JPG", "rb") as image_file:
files = {'field_name': image_file}
cookie = {cookiename: token.value}
r = requests.post(url, cookies = cookie, files=files, auth=(username, password)
I'm trying to connect to the twiiter API through a POST request as the docs say but I always get a 403 forbidden error.
This is my code. I'm using urlib2 in python 2.7:
def auth_API():
url = 'https://api.twitter.com/oauth2/token'
header = {}
values = {}
header['User-Agent'] = 'Mozilla/6.0 (Windows NT 6.2; WOW64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1'
header['Authorization'] = 'Basic ' + B64BEARERTOKENCREDS
header['Content-Type'] = 'application/x-www-form-urlencoded;charset=UTF-8'
header['Accept-Encoding'] = 'gzip'
values['grant_type'] = 'client_credentials'
data = urllib.urlencode(values)
req = urllib2.Request(url, data, header)
try:
response = urllib2.urlopen(req)
response.read()
except urllib2.HTTPError as e:
print e
Checking the docs I found an example request wich is the same as mine:
Twitter example:
POST /oauth2/token HTTP/1.1
Host: api.twitter.com
User-Agent: My Twitter App v1.0.23
Authorization: Basic NnB1[...]9JM29jYTNFOA==
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 29
Accept-Encoding: gzip
grant_type=client_credentials
My request:
POST /oauth2/token HTTP/1.1
Content-Length: 29
Accept-Encoding: gzip
Connection: close
User-Agent: Mozilla/6.0 (Windows NT 6.2; WOW64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1
Host: api.twitter.com
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Authorization: Basic NnB1[...]YTNFOA==
grant_type=client_credentials
Any idea with what could be wrong with this?
Regards.
PS: I know that there are some third party libs for this but I want to make it by myself.
I solved my problem, the error was with base64.encodestring() which adds an \n at the end of the string messing up the request.
Using base64.b64encode() instead worked fine.
Regards