I tried to use python requests package to connect over SSL and i used below code.
resp = requests.get(addr , auth=HttpNtlmAuth(userFile[unum],passFile[pnum]) , headers = {'User-Agent':'Mozilla/4.01 (compatible; MSIE 6.0; Windows NT 5.1)'} , verify = False)
but every time i got below error.
("bad handshake: SysCallError(-1, 'Unexpected EOF')",)
UPDATE 1:
I dump host certificate and save that to c.cert and used below code.
resp = requests.get(addr , auth=HttpNtlmAuth(userFile[unum],passFile[pnum]) , headers = {'User-Agent':'Mozilla/4.01 (compatible; MSIE 6.0; Windows NT 5.1)'} , cert = '/home/user/Desktop/c.cert' , verify = '/home/user/Desktop/c.cert')
and new error.
[('PEM routines', 'PEM_read_bio', 'no start line'), ('SSL routines', 'SSL_CTX_use_PrivateKey_file', 'PEM lib')]
UPDATE 2 :
Here is what Burp Suite captured.
(First , session creation):
GET /PATH HTTP/1.1
Host: HOST
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Cookie: ASPSESSIONIDCQCTCTCQ=EPEHGDDAKOINODAJJKCELEOK
DNT: 1
Connection: close
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
(Second , when use credentials):
GET /PATH HTTP/1.1
Host: HOST
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Cookie: ASPSESSIONIDCQCTCTCQ=EPEHGDDAKOINODAJJKCELEOK
DNT: 1
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
Authorization: NTLM TlRMTVNTUAABAAAAB4IIAAAAAAAAAAAAAAAAAAAAAAA=
Connection: close
(Third , credentials was wrong)
GET /PATH HTTP/1.1
Host: HOST
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Cookie: ASPSESSIONIDCQCTCTCQ=EPEHGDDAKOINODAJJKCELEOK
DNT: 1
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
Authorization: NTLM TlRMTVNTUAADAAAAGAAYAGAAAACIAIgAeAAAAAAAAABAAAAACgAKAEAAAAAWABYASgAAAAAAAAAAAAAABYIIAGEAZABtAGkAbgBXAE8AUgBLAFMAVABBAFQASQBPAE4Ao8+kG6lMZcLmQys5IUkpayq+W1VFZPkjUQjxBhzugt88vU6RR3wahQEBAAAAAAAAgPonWPz30gGr58MfeUgvgQAAAAACABIAVwBFAEIAUwBFAFIAVgBFAFIAAQASAFcARQBCAFMARQBSAFYARQBSAAQAEgBXAEUAQgBTAEUAUgBWAEUAUgADABIAVwBFAEIAUwBFAFIAVgBFAFIAAAAAAA==
Connection: close
when using URL without HTTPS it gives me page with below custom error
Bad Request (Invalid Hostname)
Maybe you need to create a SSL context, and pass it to requests.
import ssl
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2) #if you need TLS v1.2
and then, pass this context var to the requests method. If an optional keyword argument, name context. So, the call will be like this:
resp = requests.get(addr, ...., context=context)
Related
I need a python script that posts a given Authorization Bearer header to a specific ipand port.
This is what I have so far.
#!/usr/bin/python
import urllib3
import certifi
http = urllib3.PoolManager(ca_certs=certifi.where())
url = 'http://172.10.10.2:3000'
req = http.request('POST', url, fields={'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbWQiOiJscyR7SUZTfS1sYSR7SUZTfS90bXAvbmMifQ.EziCTtJn1PpPXvemJllF36w7ADNkhKiktZ5sv9ADR3o'})
print(req.data.decode('utf-8'))
I currently get an error when running this stating that the Required authorization token is not found
The bearer code is created manually and then input into the script, if there was a way to import that from the site I create it on that would be helpful.
What I need the output to be in this -
GET / HTTP/1.1
Host: 172.10.10.2:3000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: close
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbWQiOiJpcCR7SUZTfWEifQ.RkoZinBcg2_5HRGgv1AtErhscIVBRv2hUcGIF08ZlCM
I used the following URL to search for Piano with distance of 20 miles.
When I ran it with BurpSuite, the request appeared as:
POST /MTNA/FindATeacherSearch.aspx?cak=31022421-c202-431a-b5ef-c6d10c90e6be HTTP/1.1
Host: www.mtna.org
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: https://www.mtna.org/MTNA/FindATeacherSearch.aspx?cak=31022421-c202-431a-b5ef-c6d10c90e6be
X-Requested-With: XMLHttpRequest
X-MicrosoftAjax: Delta=true
Cache-Control: no-cache
Content-Type: application/x-www-form-urlencoded
Content-Length: 4107
Connection: close
Cookie: ASP.NET_SessionId=cxqktlukl4zprfps4wfyex44; Asi.Web.Browser.CookiesEnabled=true; AnonymousCartId=00000000-0000-0000-0000-000000000000; _ga=GA1.2.1639735034.1545046479; _gid=GA1.2.6620465.1545046479
ctl01%24ScriptManager1=ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ListerPanel%7Cctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24SubmitButton&__WPPS=s&__CTRLKEY=&__SHIFTKEY=&ctl01_ScriptManager1_TSM=%3B%3BAjaxControlToolkit%2C%20Version%3D4.1.50508%2C%20Culture%3Dneutral%2C%20PublicKeyToken%3D28f01b0e84b6d53e%3Aen-US%3A0c8c847b-b611-49a7-8e75-2196aa6e72fa%3Aea597d4b%3Ab25378d2%3BTelerik.Web.UI%2C%20Version%3D2014.1.403.45%2C%20Culture%3Dneutral%2C%20PublicKeyToken%3D121fae78165ba3d4%3Aen-US%3A68d9452f-f268-45b2-8db7-8c3bbf305b8d%3A16e4e7cd%3Af7645509%3A24ee1bba%3Ae330518b%3A2003d0b8%3Ac128760b%3A1e771326%3A88144a7a%3Ac8618e41%3A1a73651d%3A16d8629e%3A874f8ea2%3Af46195d3%3A19620875%3Acda80b3%3Aed16cbdc%3A92fe8ea0%3Afa31b949%3A490a9d4e%3Abd8f85e4%3BAjaxControlToolkit%2C%20Version%3D4.1.50508.0%2C%20Culture%3Dneutral%2C%20PublicKeyToken%3D28f01b0e84b6d53e%3Aen-US%3A0c8c847b-b611-49a7-8e75-2196aa6e72fa%3A782b16ab&PageInstanceKey=0428325d-7c92-4972-a40f-a36c36bad719&TemplateUserMessagesID=ctl01_TemplateUserMessages_ctl00_Messages&PageIsDirty=false&IsControlPostBackctl01%24HeaderLogo%24HeaderLogoSpan=1&IsControlPostBackctl01%24SocialNetworking%24SocialNetworking=1&IsControlPostBackctl01%24SearchField=1&__EVENTTARGET=ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24SubmitButton&__EVENTARGUMENT=&NavMenuClientID=ctl01_Primary_NavMenu&IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewContentHtml%24ciNewContentHtml=1&IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewShowConsumerAddressCommon%24ciNewShowConsumerAddressCommon=1&IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries=1&IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewContentHtml2%24ciNewContentHtml2=1&IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciScriptforadtracking%24ciScriptforadtracking=1&IsControlPostBackctl01%24TemplateBody%24ContentPage1=1&IsControlPostBackctl01%24TemplateBody%24ContentPageFooter1=1&IsControlPostBackctl01%24FooterCommunications%24FooterCommunications=1&IsControlPostBackctl01%24FooterCommunications%24NewContentHtml=1&IsControlPostBackctl01%24FooterCopyright%24FooterCopyright=1&__VIEWSTATE=%2FwEPaA8FDzhkNjYzZjU1MjZmNTFlYhgBBR5fX0NvbnRyb2xzUmVxdWlyZVBvc3RCYWNrS2V5X18WCAUYY3RsMDEkTG9naW5TdGF0dXMxJGN0bDAxBRhjdGwwMSRMb2dpblN0YXR1czEkY3RsMDMFFWN0bDAxJFByaW1hcnkkTmF2TWVudQVXY3RsMDEkVGVtcGxhdGVCb2R5JFdlYlBhcnRNYW5hZ2VyMSRnd3BjaVRlYWNoZXJRdWVyaWVzJGNpVGVhY2hlclF1ZXJpZXMkUXVlcnlMaXN0RGlhbG9nBQtjdGwwMSRjdGwxNgUTY3RsMDEkR2VuZXJpY1dpbmRvdwUTY3RsMDEkT2JqZWN0QnJvd3NlcgUZY3RsMDEkT2JqZWN0QnJvd3NlckRpYWxvZ5YeffipvvNAXInmJ3MH5Z%2FoW9I1&__VIEWSTATEGENERATOR=C56D694D&ctl01%24lastClickedElementId=&ctl01%24SearchField%24SearchTerms=Keyword%20Search&ctl01_Primary_NavMenu_ClientState=&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24mHiddenCacheQueryId=&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24mHiddenQueryCached=False&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24ctl01=277d8abd-5f3f-4b6d-8991-a624daea0d07.FS1.FL7&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24ctl04=277d8abd-5f3f-4b6d-8991-a624daea0d07.FS1.FL9&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24Input0%24DropDown1=Piano&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24Input1%24DropDown1=20&ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24HiddenKeyField1=&ctl01_TemplateBody_WebPartManager1_gwpciTeacherQueries_ciTeacherQueries_QueryListDialog_ClientState=&ctl01_GenericWindow_ClientState=&ctl01_ObjectBrowser_ClientState=&ctl01_ObjectBrowserDialog_ClientState=&ctl01_ctl16_ClientState=&__ASYNCPOST=true
In the response I get 63 results.
I converted the POST request to Python:
import requests
session = requests.Session()
paramsGet = {"cak":"31022421-c202-431a-b5ef-c6d10c90e6be"}
paramsPost = {"ctl01_ScriptManager1_TSM":";;AjaxControlToolkit, Version=4.1.50508, Culture=neutral, PublicKeyToken=28f01b0e84b6d53e:en-US:0c8c847b-b611-49a7-8e75-2196aa6e72fa:ea597d4b:b25378d2;Telerik.Web.UI, Version=2014.1.403.45, Culture=neutral, PublicKeyToken=121fae78165ba3d4:en-US:68d9452f-f268-45b2-8db7-8c3bbf305b8d:16e4e7cd:f7645509:24ee1bba:e330518b:2003d0b8:c128760b:1e771326:88144a7a:c8618e41:1a73651d:16d8629e:874f8ea2:f46195d3:19620875:cda80b3:ed16cbdc:92fe8ea0:fa31b949:490a9d4e:bd8f85e4;AjaxControlToolkit, Version=4.1.50508.0, Culture=neutral, PublicKeyToken=28f01b0e84b6d53e:en-US:0c8c847b-b611-49a7-8e75-2196aa6e72fa:782b16ab","__VIEWSTATEGENERATOR":"C56D694D","__EVENTARGUMENT":"","IsControlPostBackctl01%24HeaderLogo%24HeaderLogoSpan":"1","__VIEWSTATE":"/wEPaA8FDzhkNjYzZWY4NmM2Y2Y4MBgBBR5fX0NvbnRyb2xzUmVxdWlyZVBvc3RCYWNrS2V5X18WCAUYY3RsMDEkTG9naW5TdGF0dXMxJGN0bDAxBRhjdGwwMSRMb2dpblN0YXR1czEkY3RsMDMFFWN0bDAxJFByaW1hcnkkTmF2TWVudQVXY3RsMDEkVGVtcGxhdGVCb2R5JFdlYlBhcnRNYW5hZ2VyMSRnd3BjaVRlYWNoZXJRdWVyaWVzJGNpVGVhY2hlclF1ZXJpZXMkUXVlcnlMaXN0RGlhbG9nBQtjdGwwMSRjdGwxNgUTY3RsMDEkR2VuZXJpY1dpbmRvdwUTY3RsMDEkT2JqZWN0QnJvd3NlcgUZY3RsMDEkT2JqZWN0QnJvd3NlckRpYWxvZ5t6EvKfQQS4e3kXIpgfVSNxCTNO","__WPPS":"s","IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries":"1","__SHIFTKEY":"","IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciScriptforadtracking%24ciScriptforadtracking":"1","NavMenuClientID":"ctl01_Primary_NavMenu","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24mHiddenCacheQueryId":"","ctl01%24SearchField%24SearchTerms":"Keyword Search","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24HiddenKeyField1":"","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24mHiddenQueryCached":"False","IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewContentHtml2%24ciNewContentHtml2":"1","ctl01_GenericWindow_ClientState":"","ctl01_Primary_NavMenu_ClientState":"","PageInstanceKey":"1f89a5d5-ce61-4147-9eb5-32cafa6174a6","ctl01_ObjectBrowserDialog_ClientState":"","__CTRLKEY":"","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24ctl04":"277d8abd-5f3f-4b6d-8991-a624daea0d07.FS1.FL9","ctl01%24ScriptManager1":"ctl01\x24TemplateBody\x24WebPartManager1\x24gwpciTeacherQueries\x24ciTeacherQueries\x24ListerPanel|ctl01\x24TemplateBody\x24WebPartManager1\x24gwpciTeacherQueries\x24ciTeacherQueries\x24ResultsGrid\x24Sheet0\x24SubmitButton","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24Input1%24DropDown1":"20","IsControlPostBackctl01%24SocialNetworking%24SocialNetworking":"1","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24Input0%24DropDown1":"Piano","PageIsDirty":"false","__ASYNCPOST":"true","IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewShowConsumerAddressCommon%24ciNewShowConsumerAddressCommon":"1","IsControlPostBackctl01%24TemplateBody%24ContentPageFooter1":"1","__EVENTTARGET":"ctl01\x24TemplateBody\x24WebPartManager1\x24gwpciTeacherQueries\x24ciTeacherQueries\x24ResultsGrid\x24Sheet0\x24SubmitButton","ctl01%24TemplateBody%24WebPartManager1%24gwpciTeacherQueries%24ciTeacherQueries%24ResultsGrid%24Sheet0%24ctl01":"277d8abd-5f3f-4b6d-8991-a624daea0d07.FS1.FL7","ctl01_ctl16_ClientState":"","IsControlPostBackctl01%24SearchField":"1","ctl01_TemplateBody_WebPartManager1_gwpciTeacherQueries_ciTeacherQueries_QueryListDialog_ClientState":"","IsControlPostBackctl01%24FooterCommunications%24FooterCommunications":"1","ctl01%24lastClickedElementId":"","TemplateUserMessagesID":"ctl01_TemplateUserMessages_ctl00_Messages","IsControlPostBackctl01%24TemplateBody%24ContentPage1":"1","IsControlPostBackctl01%24TemplateBody%24WebPartManager1%24gwpciNewContentHtml%24ciNewContentHtml":"1","ctl01_ObjectBrowser_ClientState":"","IsControlPostBackctl01%24FooterCommunications%24NewContentHtml":"1","IsControlPostBackctl01%24FooterCopyright%24FooterCopyright":"1"}
headers = {"X-MicrosoftAjax":"Delta=true","Accept":"*/*","X-Requested-With":"XMLHttpRequest","Cache-Control":"no-cache","User-Agent":"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0","Referer":"https://www.mtna.org/MTNA/FindATeacherSearch.aspx?cak=31022421-c202-431a-b5ef-c6d10c90e6be","Connection":"close","Accept-Encoding":"gzip, deflate","Accept-Language":"en-US,en;q=0.5","Content-Type":"application/x-www-form-urlencoded; charset=utf-8"}
cookies = {"AnonymousCartId":"00000000-0000-0000-0000-000000000000","ASP.NET_SessionId":"cxqktlukl4zprfps4wfyex44","_gat":"1","Asi.Web.Browser.CookiesEnabled":"true","_ga":"GA1.2.1639735034.1545046479","_gid":"GA1.2.6620465.1545046479"}
response = session.post("https://www.mtna.org/MTNA/FindATeacherSearch.aspx", data=paramsPost, params=paramsGet, headers=headers, cookies=cookies)
print("Status code: %i" % response.status_code)
print("Response body: %s" % response.content)
But the results leave me on the same page.
Any idea why ?
What cause its different ?
I write a code who has to click a button on website, by sending a POST request. I use sockets to this. Here is my code:
import sys
import socket
import time
import socks
red = "\033[1;31m"
green = "\033[1;32m"
yellow = "\033[1;33m"
blue = "\033[1;34m"
defcol = "\033[0m"
s = socks.socksocket()
s.set_proxy(socks.SOCKS5, "209.33.198.133", 9999)
ipaddr = socket.gethostbyname('mclista.pl')
#ipaddr = 'mclista.pl'
s.connect(("149.202.218.96", 80))
header = ("""
POST /json/daj_diax/ HTTP/1.1
Host: www.mclista.pl
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: pl,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: https://www.google.com
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Content-Length: 68
Connection: keep-alive
Cookie: _ga=GA1.2.942145996.1539109230; _gid=GA1.2.1111512675.1541185659; csrf_mclista=65d10295bd238ea3c81f19b3e02d073c; ci_session=a%3A5%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%22d1e9deec40aa713329f98ddd11ce042f%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A10%3A%225.184.63.6%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A78%3A%22Mozilla%2F5.0+%28Windows+NT+10.0%3B+Win64%3B+x64%3B+rv%3A63.0%29+Gecko%2F20100101+Firefox%2F63.0%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1541242717%3Bs%3A9%3A%22user_data%22%3Bs%3A0%3A%22%22%3B%7D913ae0005d8fb07d1b6dc486b7fc9297; _gat=1
""")
print(yellow + header)
request = header
s.sendall(request.encode('utf-8'))
response = s.recv(4096)
time.sleep(2)
print('\n\n\n\n\n\n\n' + blue + str(response) + '\n')
s.close()
My question is, why on send i getting it response:
HTTP/1.1 400 Bad Request
Date: Sat, 03 Nov 2018 11:20:02 GMT
Server: Apache
Content-Length: 226
Connection: close
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>
Anyone know, how to fix it? Header is good, i copy it from firefox network tools. I need to send POST request, not GET. Code written in python 3.7
On i send it post from firefox, i getting normal response:
status: "ok"
Now i change a header to:
header = ("""POST /json/daj_diax/ HTTP/1.1 Host: www.mclista.pl User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0 Accept: application/json, text/javascript, */*; q=0.01 Accept-Language: pl,en-US;q=0.7,en;q=0.3 Accept-Encoding: gzip, deflate Referer: http://www.mclista.pl/42204 Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Requested-With: XMLHttpRequest Content-Length: 68 Connection: keep-alive Cookie: _ga=GA1.2.942145996.1539109230; _gid=GA1.2.1111512675.1541185659; csrf_mclista=8cb69323a6f12ff338c06d4bea489c7f; ci_session=a%3A5%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%22b3df7ad8aef06a7428e497f90361eaef%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A10%3A%225.184.63.6%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A78%3A%22Mozilla%2F5.0+%28Windows+NT+10.0%3B+Win64%3B+x64%3B+rv%3A63.0%29+Gecko%2F20100101+Firefox%2F63.0%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1541265085%3Bs%3A9%3A%22user_data%22%3Bs%3A0%3A%22%22%3B%7D14f5f5334fa1876215c215f1f81889c6; _gat=1""")
and to:
header = ("""POST /json/daj_diax/ HTTP/1.1
Host: www.mclista.pl
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: pl,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://www.mclista.pl/42204
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Content-Length: 68
Connection: keep-alive
Cookie: _ga=GA1.2.942145996.1539109230; _gid=GA1.2.1111512675.1541185659; csrf_mclista=8cb69323a6f12ff338c06d4bea489c7f; ci_session=a%3A5%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%22b3df7ad8aef06a7428e497f90361eaef%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A10%3A%225.184.63.6%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A78%3A%22Mozilla%2F5.0+%28Windows+NT+10.0%3B+Win64%3B+x64%3B+rv%3A63.0%29+Gecko%2F20100101+Firefox%2F63.0%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1541265085%3Bs%3A9%3A%22user_data%22%3Bs%3A0%3A%22%22%3B%7D14f5f5334fa1876215c215f1f81889c6; _gat=1""")
doesn't working now too. On first header the same response, on secound header no response, recive sending a empty byte (b'')
You may need to use CRLF \r\n:
header = ("POST /json/daj_diax/ HTTP/1.1\r\n"
"Host: www.mclista.pl\r\n"
"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0\r\n"
"Accept: application/json, text/javascript, */*; q=0.01\r\n"
"Accept-Language: pl,en-US;q=0.7,en;q=0.3\r\n"
"Accept-Encoding: gzip, deflate\r\n"
"Referer: http://www.mclista.pl/42204\r\n"
"Content-Type: application/x-www-form-urlencoded; charset=UTF-8\r\n"
"X-Requested-With: XMLHttpRequest\r\n"
"Content-Length: 68\r\n"
"Connection: keep-alive\r\n"
"Cookie: _ga=GA1.2.942145996.1539109230; _gid=GA1.2.1111512675.1541185659; csrf_mclista=8cb69323a6f12ff338c06d4bea489c7f; ci_session=a%3A5%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%22b3df7ad8aef06a7428e497f90361eaef%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A10%3A%225.184.63.6%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A78%3A%22Mozilla%2F5.0+%28Windows+NT+10.0%3B+Win64%3B+x64%3B+rv%3A63.0%29+Gecko%2F20100101+Firefox%2F63.0%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1541265085%3Bs%3A9%3A%22user_data%22%3Bs%3A0%3A%22%22%3B%7D14f5f5334fa1876215c215f1f81889c6; _gat=1\r\n\r\n")
I'm trying to put variables into long string but it raises error. I've tried %s notation and now I'm trying to put there .format() notation but it doesn't work either.
Could you tell me what is the problem? Is it because the string is on multiple lines?
num = 5
r=requests.post("http://www.quoka.de/kleinanzeigen/nachhilfe/cat_0_ct_0_page_'{0}'.html".format(str(num)), headers=mLib.firebug_headers_to_dict("""POST /kleinanzeigen/nachhilfe/cat_0_ct_0_page_'{1}'.html HTTP/1.1
Host: www.quoka.de
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://www.quoka.de/kleinanzeigen/nachhilfe/cat_0_ct_0_page_'{2}'.html
Cookie: QSESSID=cs9djh8q8c6mjgsme85s4mf7iq24pqrthag630ar6po9fp078e20; PARTNER=VIEW%02quoka%01COOKIEBEGIN%021438270171; QUUHS=QPV%027%01QABAFS%02A%01ARYSEARCHODER%02%7B%22search1%22%3A%22nachhilfe%22%7D; __utma=195481459.415565446.1438270093.1438270093.1438270093.1; __utmb=195481459.22.8.1438270093; __utmc=195481459; __utmz=195481459.1438270093.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmt=1; __utmt_t2=1; POPUPCHECK=1438356493224; axd=100086901728180087; __gads=ID=92bfc541911a1c81:T=1438270098:S=ALNI_MYuEdhQnu7sWAfK-fyKf1Ej93_9KA; crtg_rta=; OX_sd=5; OX_plg=wmp|pm; rsi_segs=L11278_10123|L11278_10571|L11278_11639|F12351_10001|F12351_0; PURESADSCL=done
Connection: keep-alive
""".format(str(num),str(num-1))))
ERROR:
""".format(str(num),str(num-1))))
IndexError: tuple index out of range
In the second multi-line string, you are giving indexes as - {1} and {2} at -
_page_'{1}'.html
and
_0_page_'{2}'.html
But you are only giving two variables as argument for the format() function.
That is why when format function tries to access variable at {2} (which is the third argument, as its 0 indexed) it gets error.
You should define them as {0} and {1} .
Example -
r=requests.post("http://www.quoka.de/kleinanzeigen/nachhilfe/cat_0_ct_0_page_'{0}'.html".format(str(num)), headers=mLib.firebug_headers_to_dict("""POST /kleinanzeigen/nachhilfe/cat_0_ct_0_page_'{0}'.html HTTP/1.1
Host: www.quoka.de
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://www.quoka.de/kleinanzeigen/nachhilfe/cat_0_ct_0_page_'{1}'.html
Cookie: QSESSID=cs9djh8q8c6mjgsme85s4mf7iq24pqrthag630ar6po9fp078e20; PARTNER=VIEW%02quoka%01COOKIEBEGIN%021438270171; QUUHS=QPV%027%01QABAFS%02A%01ARYSEARCHODER%02%7B%22search1%22%3A%22nachhilfe%22%7D; __utma=195481459.415565446.1438270093.1438270093.1438270093.1; __utmb=195481459.22.8.1438270093; __utmc=195481459; __utmz=195481459.1438270093.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmt=1; __utmt_t2=1; POPUPCHECK=1438356493224; axd=100086901728180087; __gads=ID=92bfc541911a1c81:T=1438270098:S=ALNI_MYuEdhQnu7sWAfK-fyKf1Ej93_9KA; crtg_rta=; OX_sd=5; OX_plg=wmp|pm; rsi_segs=L11278_10123|L11278_10571|L11278_11639|F12351_10001|F12351_0; PURESADSCL=done
Connection: keep-alive
""".format(str(num),str(num-1))))
Your format string starts numbering at 1, not 0. You only provide 2 values for slots 0 and 1, but your template expects parameters numbered 1 and 2.
Note that you have two separate strings here. The first string uses {0}, the second uses {1} and {2}. Renumber those to {0} and {1} instead.
To reiterate, the first string is fine:
"http://www.quoka.de/kleinanzeigen/nachhilfe/cat_0_ct_0_page_'{0}'.html".format(str(num))
but you have to restart the numbering in the second string. In that string you have
POST /kleinanzeigen/nachhilfe/cat_0_ct_0_page_'{1}'.html HTTP/1.1
and
Referer: http://www.quoka.de/kleinanzeigen/nachhilfe/cat_0_ct_0_page_'{2}'.html
Simply use
POST /kleinanzeigen/nachhilfe/cat_0_ct_0_page_'{0}'.html HTTP/1.1
and
Referer: http://www.quoka.de/kleinanzeigen/nachhilfe/cat_0_ct_0_page_'{1}'.html
Note that those '..' single quotes are part of the string you are producing. I just tested the site and the URLs there do not use those quotes. You should probably remove them. You also don't have to call str() on everything, the template takes care of that for you:
num = 5
r = requests.post(
"http://www.quoka.de/kleinanzeigen/nachhilfe/cat_0_ct_0_page_{0}.html".format(num),
headers=mLib.firebug_headers_to_dict("""\
POST /kleinanzeigen/nachhilfe/cat_0_ct_0_page_{0}.html HTTP/1.1
Host: www.quoka.de
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://www.quoka.de/kleinanzeigen/nachhilfe/cat_0_ct_0_page_{1}.html
Cookie: QSESSID=cs9djh8q8c6mjgsme85s4mf7iq24pqrthag630ar6po9fp078e20; PARTNER=VIEW%02quoka%01COOKIEBEGIN%021438270171; QUUHS=QPV%027%01QABAFS%02A%01ARYSEARCHODER%02%7B%22search1%22%3A%22nachhilfe%22%7D; __utma=195481459.415565446.1438270093.1438270093.1438270093.1; __utmb=195481459.22.8.1438270093; __utmc=195481459; __utmz=195481459.1438270093.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmt=1; __utmt_t2=1; POPUPCHECK=1438356493224; axd=100086901728180087; __gads=ID=92bfc541911a1c81:T=1438270098:S=ALNI_MYuEdhQnu7sWAfK-fyKf1Ej93_9KA; crtg_rta=; OX_sd=5; OX_plg=wmp|pm; rsi_segs=L11278_10123|L11278_10571|L11278_11639|F12351_10001|F12351_0; PURESADSCL=done
Connection: keep-alive
""".format(num, num - 1)
))
Furthermore, I strongly doubt you need to include the POST <path> HTTP/1.1line at all. The Host, Accept-Encoding and Connection headers will be taken care of by requests, and if you were to use a Session() object, so will the cookies. Try not to set all those headers explicitly. Setting the headers as a dictionary is perhaps more readable:
session = requests.Session()
url = 'http://www.quoka.de/kleinanzeigen/nachhilfe/cat_0_ct_0_page_{0}.html'
num = 5
r = session.post(
url.format(num),
headers={
'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0',
'Referer': url.format(num - 1),
}
)
Last but not least, are you sure you need to perform a POST here? You are not posting anything, and a GET works just fine too.
i have tried clicking a button through telit modem,these were the commands i sent through the modem as http request,if the button is clicked the text in the Textbox will be deposited in the database.
POST / HTTP/1.1
Host: www.apcogsys.com
Connection: keep-alive
Content-Length: 253
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Origin: http://www.apcogsys.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Content-Type: application/x-www-form-urlencoded
Referer: http://www.apcogsys.com/
Accept-Encoding: gzip,deflate,sdch
__VIEWSTATE=%2FwEPDwULLTE0MDM4MzYxMjNkZMHxueuplKms5LprRiXrYhjNw1RQHyrxnbqdvUd7WuME&__EVENTVALIDATION=%2FwEdAAOXYiZ8Tx8yD9ysXMQzXW6cESCFkFW%2FRuhzY1oLb%2FNUVM34O%2FGfAV4V4n0wgFZHr3cvuT1CKH816XKYM4BbXnMYv8RXVACd0FKbHPtTjQthHg%3D%3D&TextBox1=abcde&Button1=click
Button1=click means that the button has been clicked.i have copied this set of commands from http debugger.But this is not working and the text in the textbox is not being deposited in the database.
All that has to happen is what ever is there in TextBox1=abcde part of the viewstate has to be deposited in the database.