I used the following URL to search for Piano with distance of 20 miles.
When I ran it with BurpSuite, the request appeared as:
POST /MTNA/FindATeacherSearch.aspx?cak=31022421-c202-431a-b5ef-c6d10c90e6be HTTP/1.1
Host: www.mtna.org
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: https://www.mtna.org/MTNA/FindATeacherSearch.aspx?cak=31022421-c202-431a-b5ef-c6d10c90e6be
X-Requested-With: XMLHttpRequest
X-MicrosoftAjax: Delta=true
Cache-Control: no-cache
Content-Type: application/x-www-form-urlencoded
Content-Length: 4107
Connection: close
Cookie: ASP.NET_SessionId=cxqktlukl4zprfps4wfyex44; Asi.Web.Browser.CookiesEnabled=true; AnonymousCartId=00000000-0000-0000-0000-000000000000; _ga=GA1.2.1639735034.1545046479; _gid=GA1.2.6620465.1545046479
ctl01%24ScriptManager1=ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ListerPanel%7Cctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24SubmitButton&__WPPS=s&__CTRLKEY=&__SHIFTKEY=&ctl01_ScriptManager1_TSM=%3B%3BAjaxControlToolkit%2C%20Version%3D4.1.50508%2C%20Culture%3Dneutral%2C%20PublicKeyToken%3D28f01b0e84b6d53e%3Aen-US%3A0c8c847b-b611-49a7-8e75-2196aa6e72fa%3Aea597d4b%3Ab25378d2%3BTelerik.Web.UI%2C%20Version%3D2014.1.403.45%2C%20Culture%3Dneutral%2C%20PublicKeyToken%3D121fae78165ba3d4%3Aen-US%3A68d9452f-f268-45b2-8db7-8c3bbf305b8d%3A16e4e7cd%3Af7645509%3A24ee1bba%3Ae330518b%3A2003d0b8%3Ac128760b%3A1e771326%3A88144a7a%3Ac8618e41%3A1a73651d%3A16d8629e%3A874f8ea2%3Af46195d3%3A19620875%3Acda80b3%3Aed16cbdc%3A92fe8ea0%3Afa31b949%3A490a9d4e%3Abd8f85e4%3BAjaxControlToolkit%2C%20Version%3D4.1.50508.0%2C%20Culture%3Dneutral%2C%20PublicKeyToken%3D28f01b0e84b6d53e%3Aen-US%3A0c8c847b-b611-49a7-8e75-2196aa6e72fa%3A782b16ab&PageInstanceKey=0428325d-7c92-4972-a40f-a36c36bad719&TemplateUserMessagesID=ctl01_TemplateUserMessages_ctl00_Messages&PageIsDirty=false&IsControlPostBackctl01%24HeaderLogo%24HeaderLogoSpan=1&IsControlPostBackctl01%24SocialNetworking%24SocialNetworking=1&IsControlPostBackctl01%24SearchField=1&__EVENTTARGET=ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24SubmitButton&__EVENTARGUMENT=&NavMenuClientID=ctl01_Primary_NavMenu&IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewContentHtml%24ciNewContentHtml=1&IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewShowConsumerAddressCommon%24ciNewShowConsumerAddressCommon=1&IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries=1&IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewContentHtml2%24ciNewContentHtml2=1&IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciScriptforadtracking%24ciScriptforadtracking=1&IsControlPostBackctl01%24TemplateBody%24ContentPage1=1&IsControlPostBackctl01%24TemplateBody%24ContentPageFooter1=1&IsControlPostBackctl01%24FooterCommunications%24FooterCommunications=1&IsControlPostBackctl01%24FooterCommunications%24NewContentHtml=1&IsControlPostBackctl01%24FooterCopyright%24FooterCopyright=1&__VIEWSTATE=%2FwEPaA8FDzhkNjYzZjU1MjZmNTFlYhgBBR5fX0NvbnRyb2xzUmVxdWlyZVBvc3RCYWNrS2V5X18WCAUYY3RsMDEkTG9naW5TdGF0dXMxJGN0bDAxBRhjdGwwMSRMb2dpblN0YXR1czEkY3RsMDMFFWN0bDAxJFByaW1hcnkkTmF2TWVudQVXY3RsMDEkVGVtcGxhdGVCb2R5JFdlYlBhcnRNYW5hZ2VyMSRnd3BjaVRlYWNoZXJRdWVyaWVzJGNpVGVhY2hlclF1ZXJpZXMkUXVlcnlMaXN0RGlhbG9nBQtjdGwwMSRjdGwxNgUTY3RsMDEkR2VuZXJpY1dpbmRvdwUTY3RsMDEkT2JqZWN0QnJvd3NlcgUZY3RsMDEkT2JqZWN0QnJvd3NlckRpYWxvZ5YeffipvvNAXInmJ3MH5Z%2FoW9I1&__VIEWSTATEGENERATOR=C56D694D&ctl01%24lastClickedElementId=&ctl01%24SearchField%24SearchTerms=Keyword%20Search&ctl01_Primary_NavMenu_ClientState=&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24mHiddenCacheQueryId=&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24mHiddenQueryCached=False&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24ctl01=277d8abd-5f3f-4b6d-8991-a624daea0d07.FS1.FL7&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24ctl04=277d8abd-5f3f-4b6d-8991-a624daea0d07.FS1.FL9&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24Input0%24DropDown1=Piano&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24Input1%24DropDown1=20&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24HiddenKeyField1=&ctl01_TemplateBody_WebPartManager1_gwpciTeacherQueries_ciTeacherQueries_QueryListDialog_ClientState=&ctl01_GenericWindow_ClientState=&ctl01_ObjectBrowser_ClientState=&ctl01_ObjectBrowserDialog_ClientState=&ctl01_ctl16_ClientState=&__ASYNCPOST=true
In the response I get 63 results.
I converted the POST request to Python:
import requests
session = requests.Session()
paramsGet = {"cak":"31022421-c202-431a-b5ef-c6d10c90e6be"}
paramsPost = {"ctl01_ScriptManager1_TSM":";;AjaxControlToolkit, Version=4.1.50508, Culture=neutral, PublicKeyToken=28f01b0e84b6d53e:en-US:0c8c847b-b611-49a7-8e75-2196aa6e72fa:ea597d4b:b25378d2;Telerik.Web.UI, Version=2014.1.403.45, Culture=neutral, PublicKeyToken=121fae78165ba3d4:en-US:68d9452f-f268-45b2-8db7-8c3bbf305b8d:16e4e7cd:f7645509:24ee1bba:e330518b:2003d0b8:c128760b:1e771326:88144a7a:c8618e41:1a73651d:16d8629e:874f8ea2:f46195d3:19620875:cda80b3:ed16cbdc:92fe8ea0:fa31b949:490a9d4e:bd8f85e4;AjaxControlToolkit, Version=4.1.50508.0, Culture=neutral, PublicKeyToken=28f01b0e84b6d53e:en-US:0c8c847b-b611-49a7-8e75-2196aa6e72fa:782b16ab","__VIEWSTATEGENERATOR":"C56D694D","__EVENTARGUMENT":"","IsControlPostBackctl01%24HeaderLogo%24HeaderLogoSpan":"1","__VIEWSTATE":"/wEPaA8FDzhkNjYzZWY4NmM2Y2Y4MBgBBR5fX0NvbnRyb2xzUmVxdWlyZVBvc3RCYWNrS2V5X18WCAUYY3RsMDEkTG9naW5TdGF0dXMxJGN0bDAxBRhjdGwwMSRMb2dpblN0YXR1czEkY3RsMDMFFWN0bDAxJFByaW1hcnkkTmF2TWVudQVXY3RsMDEkVGVtcGxhdGVCb2R5JFdlYlBhcnRNYW5hZ2VyMSRnd3BjaVRlYWNoZXJRdWVyaWVzJGNpVGVhY2hlclF1ZXJpZXMkUXVlcnlMaXN0RGlhbG9nBQtjdGwwMSRjdGwxNgUTY3RsMDEkR2VuZXJpY1dpbmRvdwUTY3RsMDEkT2JqZWN0QnJvd3NlcgUZY3RsMDEkT2JqZWN0QnJvd3NlckRpYWxvZ5t6EvKfQQS4e3kXIpgfVSNxCTNO","__WPPS":"s","IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries":"1","__SHIFTKEY":"","IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciScriptforadtracking%24ciScriptforadtracking":"1","NavMenuClientID":"ctl01_Primary_NavMenu","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24mHiddenCacheQueryId":"","ctl01%24SearchField%24SearchTerms":"Keyword Search","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24HiddenKeyField1":"","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24mHiddenQueryCached":"False","IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewContentHtml2%24ciNewContentHtml2":"1","ctl01_GenericWindow_ClientState":"","ctl01_Primary_NavMenu_ClientState":"","PageInstanceKey":"1f89a5d5-ce61-4147-9eb5-32cafa6174a6","ctl01_ObjectBrowserDialog_ClientState":"","__CTRLKEY":"","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24ctl04":"277d8abd-5f3f-4b6d-8991-a624daea0d07.FS1.FL9","ctl01%24ScriptManager1":"ctl01\x24TemplateBody\x24WebPartManager1\x24gwpciTeacherQueries\x24ciTeacherQueries\x24ListerPanel|ctl01\x24TemplateBody\x24WebPartManager1\x24gwpciTeacherQueries\x24ciTeacherQueries\x24ResultsGrid\x24Sheet0\x24SubmitButton","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24Input1%24DropDown1":"20","IsControlPostBackctl01%24SocialNetworking%24SocialNetworking":"1","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24Input0%24DropDown1":"Piano","PageIsDirty":"false","__ASYNCPOST":"true","IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewShowConsumerAddressCommon%24ciNewShowConsumerAddressCommon":"1","IsControlPostBackctl01%24TemplateBody%24ContentPageFooter1":"1","__EVENTTARGET":"ctl01\x24TemplateBody\x24WebPartManager1\x24gwpciTeacherQueries\x24ciTeacherQueries\x24ResultsGrid\x24Sheet0\x24SubmitButton","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24ctl01":"277d8abd-5f3f-4b6d-8991-a624daea0d07.FS1.FL7","ctl01_ctl16_ClientState":"","IsControlPostBackctl01%24SearchField":"1","ctl01_TemplateBody_WebPartManager1_gwpciTeacherQueries_ciTeacherQueries_QueryListDialog_ClientState":"","IsControlPostBackctl01%24FooterCommunications%24FooterCommunications":"1","ctl01%24lastClickedElementId":"","TemplateUserMessagesID":"ctl01_TemplateUserMessages_ctl00_Messages","IsControlPostBackctl01%24TemplateBody%24ContentPage1":"1","IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewContentHtml%24ciNewContentHtml":"1","ctl01_ObjectBrowser_ClientState":"","IsControlPostBackctl01%24FooterCommunications%24NewContentHtml":"1","IsControlPostBackctl01%24FooterCopyright%24FooterCopyright":"1"}
headers = {"X-MicrosoftAjax":"Delta=true","Accept":"*/*","X-Requested-With":"XMLHttpRequest","Cache-Control":"no-cache","User-Agent":"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0","Referer":"https://www.mtna.org/MTNA/FindATeacherSearch.aspx?cak=31022421-c202-431a-b5ef-c6d10c90e6be","Connection":"close","Accept-Encoding":"gzip, deflate","Accept-Language":"en-US,en;q=0.5","Content-Type":"application/x-www-form-urlencoded; charset=utf-8"}
cookies = {"AnonymousCartId":"00000000-0000-0000-0000-000000000000","ASP.NET_SessionId":"cxqktlukl4zprfps4wfyex44","_gat":"1","Asi.Web.Browser.CookiesEnabled":"true","_ga":"GA1.2.1639735034.1545046479","_gid":"GA1.2.6620465.1545046479"}
response = session.post("https://www.mtna.org/MTNA/FindATeacherSearch.aspx", data=paramsPost, params=paramsGet, headers=headers, cookies=cookies)
print("Status code: %i" % response.status_code)
print("Response body: %s" % response.content)
But the results leave me on the same page.
Any idea why ?
What cause its different ?
Related
I am using Laravel 8 and Sanctum and attempting to make an API call from Python into the Laravel app.
I don't seem to be sending the request variables correctly from python as they are not being picked up as $request->variable (see "auction_item_id":null in Laravel log below) however they do show up in the request (see below Laravel log).
Is this a problem with the python request header?
Python:
import requests
import re
import json
############## LOGIN ####################
url = 'http://127.0.0.1:8082/api/apilogin'
data = {
'email': 'user#email.com',
'password': 'password'}
r = requests.post(url, data=data)
user = json.loads(r.text)
############## MAKE REST API CALL using login token from prior call ####################
url2 = 'http://127.0.0.1:8082/api/beginlivebiddingforitemAPI'
token = "Bearer " + user["token"]
headers = {"Authorization": token, 'Accept': 'application/json'}
data = {
'auction_id' : 103,
'auction_item_id' : 1224
}
rr = requests.get(url2, data=data, headers=headers)
AuctionBidsController:
public function beginlivebiddingforitem (Request $request){
Log::info("AuctionBidsController#beginlivebiddingforitem", ['auction_item_id' => $request->auction_item_id]);
Api.php:
Route::post('/apilogin', 'UsersController#APIlogin');
Route::middleware(['auth:sanctum'])->group(function () {
Route::get('/beginlivebiddingforitemAPI', 'AuctionBidsController#beginlivebiddingforitem');
});
Laravel log:
[2020-11-06 16:46:44] local.INFO: AuctionBidsController#beginlivebiddingforitem {"auction_item_id":null,"request":{"Illuminate\\Http\\Request":"GET /api/beginlivebiddingforitemAPI HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate
Authorization: Bearer 100|3EjawNPRinapHXxZvDirzyKM73AzeuJC8OTP6xOZ
Connection: keep-alive
Content-Length: 35
Content-Type: application/x-www-form-urlencoded
Host: 127.0.0.1:8082
User-Agent: python-requests/2.23.0
auction_id=103&auction_item_id=1224"}}
^^^^^^^^^^^^^^^
THEY ARE HERE
** But look at "auction_item_id":null **
Here is what a normal request looks like inside laravel:
[2020-11-06 18:34:56] local.INFO: AuctionBidsController#beginlivebiddingforitem {"auction_item_id":"1223","request":{"Illuminate\\Http\\Request":"GET /beginlivebiddingforitem?auction_id=103&auction_item_id=1223 HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Cookie: laravel_cookie_consent=1; _ga=GA1.1.1339345323.1594720189; PGADMIN_INT_KEY=129e52c8-dddd-41a0-bc43-53e91dec5402; pga4_session=1c3a240a-46a1-4a9e-8e25-d489bb8ddddddddNltIXmby1M+34KepOYCsGAsY=; PGADMIN_LANGUAGE=en; _gid=GA1.1.1579327561.1604514920; _gat_gtag_UA_170463940_1=1; XSRF-TOKEN=eyJpdiI6ImIxT00yWW54REE2emVQSEpZZzZRRmc9PSIsInZhbHVlIjoiWmlCZGhia1pxeVJ0TmU4NGRBV2xMUURsS21RT0cyWGZUYddddddddddddddddddddddU9oUHg3cGQrdWZhd2gybnRCZ0xUajcwdEhuREh5OE5FcWkzV2tWb3lKOTRwUlljWld4TE9KL3BiMFp0M1dzeklmY2VhaEUiLCJtYWMiOiI0OWZhYmE3ZWM0ZjFhNTU5OWM1ZWE1MmY0MmIyMWQ3YTU1YzE3MmI0NWM1YjY0NTE0MzdhYTVmOWNmODIzMWVlIn0%3D; icollect_session=eyJpdiI6Ik5xSlAzR3NaZUR4QUhxV0RBRFBvd2c9PSIsInZhbHVlIjoianJLbFdVUDJCL2FKZEo5NTFUMElVUFJMRm9BdddddddddddddddHdYL3Arc3Avc1JTcUE5RmNPZ1J2MUVxblFUbmREcHJJeUZJaHZNVzJHTVNhR2p6Snp6M2JkQXJObE9BQU91Si9RbVIzRXMiLCJtYWMiOiI5MzcwMjc5ZWQ0MDE5ZDk2NTVmYmJjNGQ5NWYzMGY2YTVkMDI5YWQ0YTlkNGU1YTUyZTIyYzhkODU4NjNiNDA2In0%3D
Dnt: 1
Host: 127.0.0.1:8082
Referer: http://127.0.0.1:8082/bidliveauctioneer/103
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.80 Safari/537.36 Edg/86.0.622.43
X-Csrf-Token: wrdZoCDCVeqddddzVxAxByw183bFMuddddP4QBKy
X-Requested-With: XMLHttpRequest
X-Socket-Id: 9357.547174
Cookie: laravel_cookie_consent=1; _ga=; PGADMIN_INT_KEY=; pga4_session=; PGADMIN_LANGUAGE=; _gid=; _gat_gtag_UA_170463940_1=; XSRF-TOKEN=wrdZoCDCVeqddddzVxAxByw183bFMuddddP4QBKy; icollect_session=Qx2QddddNvlIg3ogQ596BdddducrFk4Bmi0nLA89
**note that variables occur right after the GET
GET /beginlivebiddingforitem?auction_id=103&auction_item_id=1223 **
I was using data = data and it's params = data on requests.get(url2, params=data, headers=headers).
Source: https://requests.readthedocs.io/en/master/user/quickstart
I'm trying to log in into one web application with python but very attempt ends with 500 error and the html body shows the error: [HttpAntiForgeryException]. I tried to apply a few solutions from the other questions here but nothing helped. So now, I'm sucked at first request which response Is giving me 500.
import requests
from bs4 import BeautifulSoup
url = "http://localhost:52053/Account/Login"
username = "test#test.sk"
user_password = "pass"
session = requests.Session()
response = session.get(url)
soup = BeautifulSoup(response.content, features="html.parser")
#print(soup)
states = ["__RequestVerificationToken", "Email", "RememberMe"]
login_data = {"username": username, "password": user_password, "Login": "submit"}
headers = {"Host": "localhost:52053",
"Content-Type": "application/x-www-form-urlencoded",
"Connection": "close",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0)",
"Cookie": str(session.cookies.get_dict())}
for state in states: # search for existing aspnet states and get its values
result = soup.find('input', {'name': state})
if not (result is None): # when existent (some may not be needed!)
if state == "Email":
login_data.update({state: login_data["username"]})
else:
login_data.update({state: result['value']})
post_request = session.post(url, headers=headers, data=login_data)
Successful login attempt looks like this.
POST /Account/Login HTTP/1.1
Host: localhost:52053
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: sk,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded
Content-Length: 193
Origin: http://localhost:52053
Connection: close
Referer: http://localhost:52053/Account/Login
Cookie: __RequestVerificationToken=j9yFGpTFSlH5_aQt0k-Gvz10I16TVXbDk31NKPm1HkcWsksUfKXkjL567yFplCS_VovTR7lVuEgNjwgp-EO3RjNj4gQOvNUXnPkjymZx_jA1
Upgrade-Insecure-Requests: 1
__RequestVerificationToken=LjHuOdKSCr1A7KRDNie4GUnCZ3qRwUCdHyLlPYT40DsEB-GNUvEKxe5nvZWf5gZ4ZflwI43xGWPyYu8GI15wroEg9WRRVtSzZ9-KY9Mu_JA1&Email=test%40test.sk&Password=pass&RememberMe=false
Following response is:
HTTP/1.1 302 Found
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Expires: -1
Location: /
Server: Microsoft-IIS/10.0
X-AspNetMvc-Version: 5.2
X-AspNet-Version: 4.0.30319
Set-Cookie: .AspNet.ExternalCookie=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
Set-Cookie: .AspNet.ApplicationCookie=KCLm03FHj8v_6rIpTzBTm7EzEtzpKmIz1Z9_z29wycUSqUVyKbGEmptXUwG41MqNOMR7Vbeq2u576ijazupNLffLP-Ua0n60aLmnVSDsLsdTqYT7jjqyGPw1Ppp8AnIDs3sdefmksazX2UvKTxzxRBufFCoxtCJx51mWtBv7v0JzUeC1hnfu1AIJ7GH_8T59KD3iv0hRSHDqlWHlkWzyN1Xt0m5ixC14e4eC2YxEm3_acy96atB2Jv5u0HREPzssLmywuzj6sLa9cHCllTG2gMVWvHA3IDhCWu7Ojf8BO02Eml3pPM5QTJ-sq540fcj9QyELayUOwBZWffSgsJeq8mlt3FupQcJ-JTJxDzAsDc4Cmk-BcvYSfpAJq4SdR-Y4mTN_6vu-wwAOLZPSgh-5K7guWmZ3VfRitZHXd_rvTEmMiVrgHFTEQAkUYu4zTSupxRplTtKb1VSDs0Nc1uEos2z0_aw-nBbRBrTPpvmqGok
Auth flow continues with this request. I'm not trying to sent this request yet (I put it here just for better imagination):
GET / HTTP/1.1
Host: localhost:52053
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: sk,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://localhost:52053/Account/Login
Connection: close
Cookie: __RequestVerificationToken=j9yFGpTFSlH5_aQt0k-Gvz10I16TVXbDk31NKPm1HkcWsksUfKXkjL567yFplCS_VovTR7lVuEgNjwgp-EO3RjNj4gQOvNUXnPkjymZx_jA1;
.AspNet.ApplicationCookie=gvv113IJhtdaOhdc0Rz2N--5Ob18W6gS64J3wtOJggRTqE70h-8HyBGQAmLvSM2qCV2e-dXR2Uto-BktD6NmNz6dJtxckIYasPOfqodDNZX33YJxNEDg7a64LPi1bNnmrnvQcOHAceQNqZDykXrhFm55dqoo1oZnJHfZQnltwqAdg7DGO31PZpzu-GAZh2_gzuxd_saJdS09ZZQrc9h7WiU2ONqeya87pSAN7ZyHQ_XvsU5cUwDGq7FWLpzlIeeZWkay6iWVmCSwNEofpdVsb880P3XZnFKEj2SW2PfazdNLfgy86YNjkoD6_3Vb1BLirRoSP0XIQMcs2F_CzgXkxD5GvDray8TPYqcQJ4L2fikReUJHadx9fFnslF2BFcnKYC8D-Xusrda_5r-CQoQ4SzAe2Cqn0h1NYHxS1wsxt35neC5RuQ3geadAEEghjrSSVhSl8jCfACtQtcBeNL2x_m6I9L3XJCjMpzJjtP6up3E
Upgrade-Insecure-Requests: 1
Next response is just kind of 200 - you are in.
So my problem is that the response from the first request is failing. Is someone able to see some mistake or did I forgot something?
Failed response from the first request call:
HTTP/1.1 500 Internal Server Error
Cache-Control: private
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/10.0
X-AspNet-Version: 4.0.30319
X-SourceFiles: =?UTF-8?B?QzpcVXNlcnNcUENBZG1pbmlzdHJhdG9yXERlc2t0b3BccGVuIHRlc3RpbmdcU2VjdXJpdHlXb3Jrc2hvcC1EVldBLW1hc3RlclxkdndhLXRyYWluaW5nXGR2d2EtdHJhaW5pbmdcQWNjb3VudFxMb2dpbg==?=
If I try to pint request headers and login_data, result is:
print(post_request.request.headers)
{'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0)', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'close', 'Host': 'localhost:52053', 'Content-Type': 'application/x-www-form-urlencoded', 'Cookie': "{'__RequestVerificationToken': 'yg-7mFRyZiONwsZ2dIVkIIW5tB7gSL_sazgphg-VuW2OpNNRRkxmLH-9SZJXiN9whUC_BYTo8RgsiDrVjcYtLEf9anW56rVwZ2RQPzxHA481'}", 'Content-Length': '249'}
print(login_data)
{'username': 'test#test.sk', 'password': 'pass', 'Login': 'submit', '__RequestVerificationToken': '14OuwaRqldlGKi93C91zf6QD_ouOorHBDe63s4KgfP3gbt85V0QMy2X5OMwWAo1TUrD8zJ-zoZbXLPpgDI_wrxVZv3ceYNos_e5_elFhVt01', 'Email': 'test#test.sk', 'RememberMe': 'true', 'Password': 'pass'}
I just find the solution out.
Requests could handle all headers by itself (and my headers was, for some reason, causing errors), it was redirecting my request to 200 response so I didn't instantly saw, that it is actually working and catching 302 response.
I found out by printing:
print(post_request.history)
Which gave me <[302]>
Now, when I know, there is a redirection, I just have to allow_redirects=False and now I'm able to catch my set-cookie header
Full code, witch is getting an expected responses is:
import requests
from bs4 import BeautifulSoup
url = "http://localhost:52053/Account/Login"
username = "test#test.sk"
user_password = "pass"
session = requests.Session()
response = session.get(url)
soup = BeautifulSoup(response.content, features="html.parser")
#print(soup)
states = ["__RequestVerificationToken", "Email", "RememberMe"]
login_data = {"username": username, "password": user_password, "Login": "submit"}
for state in states: # search for existing aspnet states and get its values
result = soup.find('input', {'name': state})
if not (result is None): # when existent (some may not be needed!)
if state == "Email":
login_data.update({state: login_data["username"]})
else:
login_data.update({state: result['value']})
post_request = session.post(url, data=login_data, allow_redirects=False)
print(login_data)
#the code below is testing, if the HttpAntiForgeryException is in code
if "HttpAntiForgeryException" not in post_request.text:
print(post_request.headers)
else:
print("antiforgery")
After I simulate to log in, when I try to post the original website, it returns 302. When I open the original website in Chrome, it returns 415.
I tried several ways:
session.post(url,headers = headers,data = data)
requests.post(url,headers = headers,data = data)
urllib.request.urlopen.read(url).decode()
import requets
import json
header = {'Host': 'sty.js118114.com:8080',
'Connection': 'keep-alive',
'Content-Length': '8188',
'Accept': '*/*',
'X-Requested-With': 'XMLHttpRequest',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36',
'Content-Type': 'text/plain;charset=UTF-8;application/xml',
'Origin': 'http://sty.js118114.com:8080',
'Referer':
'http://sty.js118114.com:8080/Report/report/movecar_list.html',
'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'zh-CN,zh;q=0.9',
'Cookie': cookie_name + '=' + cookie_value
}
data = {"calling_no":"","begin_time":"","end_time":"","called_car_no":""}
res = requests.post(target,data = json.dumps(data),headers = header)
print(res.content.decode())
I expect the content must be the json version or html version so that I can use re model or xpath to get the infomation I want.(without any redirects
Lastly, I provide the necessary infomation about the problem:
Chrome Network
General
Request URL: http://sty.js118114.com:8080/Report/movecar/list/1/10
Request Method: POST
Status Code: 200 OK
Remote Address: 127.0.0.1:8888
Referrer Policy: no-referrer-when-downgrade
Response Headers
Content-Length: 8150
Content-Type: application/json;charset=UTF-8
Date: Thu, 22 Aug 2019 00:47:51 GMT
Server: Apache-Coyote/1.1
Request Headers
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9
Content-Length: 66
Content-Type: text/plain;charset=UTF-8;
Cookie: JSESSIONID=0A474B00017BFFD89A515B336F482905
Host: sty.js118114.com:8080
Origin: http://sty.js118114.com:8080
Proxy-Connection: keep-alive
Referer: http://sty.js118114.com:8080/Report/report/movecar_list.html
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36
X-Requested-With: XMLHttpRequest
Request Payload
{calling_no: "", begin_time: "", end_time: "", called_car_no: ""}
begin_time: ""
called_car_no: ""
calling_no: ""
end_time: ""
Fiddler Inspectors Raw
POST http://sty.js118114.com:8080/Report/movecar/list/1/10 HTTP/1.1
Host: sty.js118114.com:8080
Connection: keep-alive
Content-Length: 66
Accept: */*
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36
Content-Type: text/plain;charset=UTF-8;
Origin: http://sty.js118114.com:8080
Referer: http://sty.js118114.com:8080/Report/report/movecar_list.html
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9
Cookie: JSESSIONID=0A474B00017BFFD89A515B336F482905
{"calling_no":"","begin_time":"","end_time":"","called_car_no":""}
Response Raw
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json;charset=UTF-8
Date: Thu, 22 Aug 2019 00:27:59 GMT
Content-Length: 8150
I write a code who has to click a button on website, by sending a POST request. I use sockets to this. Here is my code:
import sys
import socket
import time
import socks
red = "\033[1;31m"
green = "\033[1;32m"
yellow = "\033[1;33m"
blue = "\033[1;34m"
defcol = "\033[0m"
s = socks.socksocket()
s.set_proxy(socks.SOCKS5, "209.33.198.133", 9999)
ipaddr = socket.gethostbyname('mclista.pl')
#ipaddr = 'mclista.pl'
s.connect(("149.202.218.96", 80))
header = ("""
POST /json/daj_diax/ HTTP/1.1
Host: www.mclista.pl
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: pl,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: https://www.google.com
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Content-Length: 68
Connection: keep-alive
Cookie: _ga=GA1.2.942145996.1539109230; _gid=GA1.2.1111512675.1541185659; csrf_mclista=65d10295bd238ea3c81f19b3e02d073c; ci_session=a%3A5%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%22d1e9deec40aa713329f98ddd11ce042f%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A10%3A%225.184.63.6%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A78%3A%22Mozilla%2F5.0+%28Windows+NT+10.0%3B+Win64%3B+x64%3B+rv%3A63.0%29+Gecko%2F20100101+Firefox%2F63.0%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1541242717%3Bs%3A9%3A%22user_data%22%3Bs%3A0%3A%22%22%3B%7D913ae0005d8fb07d1b6dc486b7fc9297; _gat=1
""")
print(yellow + header)
request = header
s.sendall(request.encode('utf-8'))
response = s.recv(4096)
time.sleep(2)
print('\n\n\n\n\n\n\n' + blue + str(response) + '\n')
s.close()
My question is, why on send i getting it response:
HTTP/1.1 400 Bad Request
Date: Sat, 03 Nov 2018 11:20:02 GMT
Server: Apache
Content-Length: 226
Connection: close
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>
Anyone know, how to fix it? Header is good, i copy it from firefox network tools. I need to send POST request, not GET. Code written in python 3.7
On i send it post from firefox, i getting normal response:
status: "ok"
Now i change a header to:
header = ("""POST /json/daj_diax/ HTTP/1.1 Host: www.mclista.pl User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0 Accept: application/json, text/javascript, */*; q=0.01 Accept-Language: pl,en-US;q=0.7,en;q=0.3 Accept-Encoding: gzip, deflate Referer: http://www.mclista.pl/42204 Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Requested-With: XMLHttpRequest Content-Length: 68 Connection: keep-alive Cookie: _ga=GA1.2.942145996.1539109230; _gid=GA1.2.1111512675.1541185659; csrf_mclista=8cb69323a6f12ff338c06d4bea489c7f; ci_session=a%3A5%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%22b3df7ad8aef06a7428e497f90361eaef%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A10%3A%225.184.63.6%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A78%3A%22Mozilla%2F5.0+%28Windows+NT+10.0%3B+Win64%3B+x64%3B+rv%3A63.0%29+Gecko%2F20100101+Firefox%2F63.0%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1541265085%3Bs%3A9%3A%22user_data%22%3Bs%3A0%3A%22%22%3B%7D14f5f5334fa1876215c215f1f81889c6; _gat=1""")
and to:
header = ("""POST /json/daj_diax/ HTTP/1.1
Host: www.mclista.pl
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: pl,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://www.mclista.pl/42204
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Content-Length: 68
Connection: keep-alive
Cookie: _ga=GA1.2.942145996.1539109230; _gid=GA1.2.1111512675.1541185659; csrf_mclista=8cb69323a6f12ff338c06d4bea489c7f; ci_session=a%3A5%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%22b3df7ad8aef06a7428e497f90361eaef%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A10%3A%225.184.63.6%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A78%3A%22Mozilla%2F5.0+%28Windows+NT+10.0%3B+Win64%3B+x64%3B+rv%3A63.0%29+Gecko%2F20100101+Firefox%2F63.0%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1541265085%3Bs%3A9%3A%22user_data%22%3Bs%3A0%3A%22%22%3B%7D14f5f5334fa1876215c215f1f81889c6; _gat=1""")
doesn't working now too. On first header the same response, on secound header no response, recive sending a empty byte (b'')
You may need to use CRLF \r\n:
header = ("POST /json/daj_diax/ HTTP/1.1\r\n"
"Host: www.mclista.pl\r\n"
"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0\r\n"
"Accept: application/json, text/javascript, */*; q=0.01\r\n"
"Accept-Language: pl,en-US;q=0.7,en;q=0.3\r\n"
"Accept-Encoding: gzip, deflate\r\n"
"Referer: http://www.mclista.pl/42204\r\n"
"Content-Type: application/x-www-form-urlencoded; charset=UTF-8\r\n"
"X-Requested-With: XMLHttpRequest\r\n"
"Content-Length: 68\r\n"
"Connection: keep-alive\r\n"
"Cookie: _ga=GA1.2.942145996.1539109230; _gid=GA1.2.1111512675.1541185659; csrf_mclista=8cb69323a6f12ff338c06d4bea489c7f; ci_session=a%3A5%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%22b3df7ad8aef06a7428e497f90361eaef%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A10%3A%225.184.63.6%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A78%3A%22Mozilla%2F5.0+%28Windows+NT+10.0%3B+Win64%3B+x64%3B+rv%3A63.0%29+Gecko%2F20100101+Firefox%2F63.0%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1541265085%3Bs%3A9%3A%22user_data%22%3Bs%3A0%3A%22%22%3B%7D14f5f5334fa1876215c215f1f81889c6; _gat=1\r\n\r\n")
I tried to use python requests package to connect over SSL and i used below code.
resp = requests.get(addr , auth=HttpNtlmAuth(userFile[unum],passFile[pnum]) , headers = {'User-Agent':'Mozilla/4.01 (compatible; MSIE 6.0; Windows NT 5.1)'} , verify = False)
but every time i got below error.
("bad handshake: SysCallError(-1, 'Unexpected EOF')",)
UPDATE 1:
I dump host certificate and save that to c.cert and used below code.
resp = requests.get(addr , auth=HttpNtlmAuth(userFile[unum],passFile[pnum]) , headers = {'User-Agent':'Mozilla/4.01 (compatible; MSIE 6.0; Windows NT 5.1)'} , cert = '/home/user/Desktop/c.cert' , verify = '/home/user/Desktop/c.cert')
and new error.
[('PEM routines', 'PEM_read_bio', 'no start line'), ('SSL routines', 'SSL_CTX_use_PrivateKey_file', 'PEM lib')]
UPDATE 2 :
Here is what Burp Suite captured.
(First , session creation):
GET /PATH HTTP/1.1
Host: HOST
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Cookie: ASPSESSIONIDCQCTCTCQ=EPEHGDDAKOINODAJJKCELEOK
DNT: 1
Connection: close
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
(Second , when use credentials):
GET /PATH HTTP/1.1
Host: HOST
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Cookie: ASPSESSIONIDCQCTCTCQ=EPEHGDDAKOINODAJJKCELEOK
DNT: 1
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
Authorization: NTLM TlRMTVNTUAABAAAAB4IIAAAAAAAAAAAAAAAAAAAAAAA=
Connection: close
(Third , credentials was wrong)
GET /PATH HTTP/1.1
Host: HOST
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Cookie: ASPSESSIONIDCQCTCTCQ=EPEHGDDAKOINODAJJKCELEOK
DNT: 1
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
Authorization: NTLM TlRMTVNTUAADAAAAGAAYAGAAAACIAIgAeAAAAAAAAABAAAAACgAKAEAAAAAWABYASgAAAAAAAAAAAAAABYIIAGEAZABtAGkAbgBXAE8AUgBLAFMAVABBAFQASQBPAE4Ao8+kG6lMZcLmQys5IUkpayq+W1VFZPkjUQjxBhzugt88vU6RR3wahQEBAAAAAAAAgPonWPz30gGr58MfeUgvgQAAAAACABIAVwBFAEIAUwBFAFIAVgBFAFIAAQASAFcARQBCAFMARQBSAFYARQBSAAQAEgBXAEUAQgBTAEUAUgBWAEUAUgADABIAVwBFAEIAUwBFAFIAVgBFAFIAAAAAAA==
Connection: close
when using URL without HTTPS it gives me page with below custom error
Bad Request (Invalid Hostname)
Maybe you need to create a SSL context, and pass it to requests.
import ssl
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2) #if you need TLS v1.2
and then, pass this context var to the requests method. If an optional keyword argument, name context. So, the call will be like this:
resp = requests.get(addr, ...., context=context)