I'm trying to imitate a request
POST /default/latex2image HTTP/2
Host: e1kf0882p7.execute-api.us-east-1.amazonaws.com
Content-Length: 96
Sec-Ch-Ua: " Not A;Brand";v="99", "Chromium";v="104"
Accept: application/json, text/javascript, */*; q=0.01
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Sec-Ch-Ua-Mobile: ?0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.81 Safari/537.36
Sec-Ch-Ua-Platform: "Windows"
Origin: https://latex2image.joeraut.com
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://latex2image.joeraut.com/
Accept-Encoding: gzip, deflate
Accept-Language: es-ES,es;q=0.9
{"latexInput":"\\begin{align*}\n{1}\n\\end{align*}\n",
"outputFormat":"PNG",
"outputScale":"125%"}
When its sent from the original brower, there is no problem.
However when I try to do it in python, the server rejects the request, and I don't know why.
This is what I tried:
pload = {
"latexInput":"{0}",
"outputFormat":"PNG",
"outputScale":"125%"
}
header = {
"Content-Length": "96",
"Sec-Ch-Ua": "Not A;Brand;v=99, Chromium;v=104",
"Accept": "application/json, text/javascript, */*; q=0.01",
"Content-Type": "application/x-www-form-urlencoded; charset=UTF-8",
"Sec-Ch-Ua-Mobile":"?0",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.81 Safari/537.36",
"Sec-Ch-Ua-Platform": "Windows",
"Origin": "https://latex2image.joeraut.com",
"Sec-Fetch-Site": "cross-site",
"Sec-Fetch-Mode": "cors",
"Sec-Fetch-Dest": "empty",
"Referer": "https://latex2image.joeraut.com/",
"Accept-Encoding": "gzip, deflate",
"Accept-Language": "es-ES,es;q=0.9",
}
r = requests.post("http://e1kf0882p7.execute-api.us-east-1.amazonaws.com", data=pload, headers=header)
print(r.text)
print(r.status_code)
And the error it raised:
requests.exceptions.ConnectionError: HTTPConnectionPool(host='e1kf0882p7.execute-api.us-east-1.amazonaws.com',
port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000259A4D20670>: Failed to establish a new connection: [WinError 10061] No se puede establecer una
conexión ya que el equipo de destino denegó expresamente dicha conexión'))
Related
Im trying to send a post request using vba and struggling to convert my python code to vba.
In python I can do this
import requests
headers = {
"authority": "platform.example.com",
"accept": "application/json, text/plain, */*",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36",
"content-type": "application/json",
"sec-fetch-site": "same-site",
"sec-fetch-mode": "cors",
"sec-fetch-dest": "empty",
"accept-language": "en-US,en;q=0.9",
}
data = "{"messages":[{"to":"12345","channel":"sms","content":"Test message"}]}"
r=requests.post("https://example.com/v1/message", headers=headers, data=data)
I have tried this in vba but no luck. Msgbox comes up blank
Sub test()
Set objHTTP = CreateObject("MSXML2.ServerXMLHTTP")
Url = "https://example.com/v1/message"
objHTTP.Open "POST", Url, False
objHTTP.setRequestHeader "authority", "platform.example.com",
objHTTP.setRequestHeader "accept", "application/json, text/plain, */*"
objHTTP.setRequestHeader "user-agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36"
objHTTP.setRequestHeader "content-type", "application/json"
objHTTP.setRequestHeader "sec-fetch-site", "same-site"
objHTTP.setRequestHeader "sec-fetch-mode", "cors"
objHTTP.setRequestHeader "sec-fetch-dest", "empty"
objHTTP.setRequestHeader "accept-language", "en-US,en;q=0.9"
Data = """messages=[{'to':'1234','channel':'sms','content':'Test message'}]"""
objHTTP.send Data
MsgBox objHTTP.ResponseText
End Sub
How can I get the vba working?
Try with double-quotes to replicate the Data that works:
Data = "{""messages"":[{""to"":""1234"",""channel"":""sms"",""content"":""Test message""}]}"
I am scraping a website with the following url and headers:
url : 'https://tennistonic.com/tennis-news/'
headers :
{
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-GB,en-US;q=0.9,en;q=0.8",
"Cache-Control": "no-cache",
"content-length": "0",
"content-type": "text/plain",
"cookie": "IDE=AHWqTUl3YRZ8Od9MzGofphNI-OCOFESmxlN69Ekm4Sbh9tcBDXGJQ1LVwbDd2uX_; DSID=AAO-7r74ByYt6ieW2yasN78hFsOGY6mrhpN5pEOWQ1vGRnAOdolIlKv23JqCRf11OpFUGFdZ-yxB3Ii1VE6UjcK-jny-4mcJ5uO-_BaV3bEFbLvU7rJNBlc",
"origin": "https//tennistonic.com",
"Connection": "keep-alive",
"Pragma": "no-cache",
"Referer": "https://tennistonic.com/",
"Sec-Fetch-Dest": "empty",
"Sec-Fetch-Mode": "cors",
"Sec-Fetch-Site": "cross-site",
"user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.80 Safari/537.36",
"x-client-data": "CI22yQEIprbJAQjBtskBCKmdygEIl6zKAQisx8oBCPXHygEI58jKAQjpyMoBCOLNygEI3NXKAQjB18oBCP2XywEIj5nLARiKwcoB"}
The x client data has a decoded section afterwards which I have left out but also tried with. The full request on dev tools is shown below:
:authority: stats.g.doubleclick.net
:method: POST
:path: /j/collect?t=dc&aip=1&_r=3&v=1&_v=j87&tid=UA-13059318-2&cid=1499412700.1601628730&jid=598376897&gjid=243704922&_gid=1691643639.1604317227&_u=QACAAEAAAAAAAC~&z=1736278164
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-GB,en-US;q=0.9,en;q=0.8
cache-control: no-cache
content-length: 0
content-type: text/plain
cookie: IDE=AHWqTUl3YRZ8Od9MzGofphNI-OCOFESmxlN69Ekm4Sbh9tcBDXGJQ1LVwbDd2uX_; DSID=AAO-7r74ByYt6ieW2yasN78hFsOGY6mrhpN5pEOWQ1vGRnAOdolIlKv23JqCRf11OpFUGFdZ-yxB3Ii1VE6UjcK-jny-4mcJ5uO-_BaV3bEFbLvU7rJNBlc
origin: https://tennistonic.com
pragma: no-cache
referer: https://tennistonic.com/
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: cross-site
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.80 Safari/537.36
x-client-data: CI22yQEIprbJAQjBtskBCKmdygEIl6zKAQisx8oBCPXHygEI58jKAQjpyMoBCOLNygEI3NXKAQjB18oBCP2XywEIj5nLARiKwcoB
Decoded:
message ClientVariations {
// Active client experiment variation IDs.
repeated int32 variation_id = [3300109, 3300134, 3300161, 3313321, 3315223, 3318700, 3318773, 3318887, 3318889, 3319522, 3320540, 3320769, 3329021, 3329167];
// Active client experiment variation IDs that trigger server-side behavior.
repeated int32 trigger_variation_id = [3317898];
}
r = requests.get(url2, headers=headers2)
soup_cont = soup(r.content, 'html.parser')
My soup contents from the response is as follows:
Is this website protected or am I sending wrong requests?
Try using selenium:
from selenium import webdriver
from bs4 import BeautifulSoup
import time
driver = webdriver.Chrome()
driver.get('https://tennistonic.com/tennis-news/')
time.sleep(3)
soup = BeautifulSoup(driver.page_source,'html5lib')
print(soup.prettify())
driver.close()
After I simulate to log in, when I try to post the original website, it returns 302. When I open the original website in Chrome, it returns 415.
I tried several ways:
session.post(url,headers = headers,data = data)
requests.post(url,headers = headers,data = data)
urllib.request.urlopen.read(url).decode()
import requets
import json
header = {'Host': 'sty.js118114.com:8080',
'Connection': 'keep-alive',
'Content-Length': '8188',
'Accept': '*/*',
'X-Requested-With': 'XMLHttpRequest',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36',
'Content-Type': 'text/plain;charset=UTF-8;application/xml',
'Origin': 'http://sty.js118114.com:8080',
'Referer':
'http://sty.js118114.com:8080/Report/report/movecar_list.html',
'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'zh-CN,zh;q=0.9',
'Cookie': cookie_name + '=' + cookie_value
}
data = {"calling_no":"","begin_time":"","end_time":"","called_car_no":""}
res = requests.post(target,data = json.dumps(data),headers = header)
print(res.content.decode())
I expect the content must be the json version or html version so that I can use re model or xpath to get the infomation I want.(without any redirects
Lastly, I provide the necessary infomation about the problem:
Chrome Network
General
Request URL: http://sty.js118114.com:8080/Report/movecar/list/1/10
Request Method: POST
Status Code: 200 OK
Remote Address: 127.0.0.1:8888
Referrer Policy: no-referrer-when-downgrade
Response Headers
Content-Length: 8150
Content-Type: application/json;charset=UTF-8
Date: Thu, 22 Aug 2019 00:47:51 GMT
Server: Apache-Coyote/1.1
Request Headers
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9
Content-Length: 66
Content-Type: text/plain;charset=UTF-8;
Cookie: JSESSIONID=0A474B00017BFFD89A515B336F482905
Host: sty.js118114.com:8080
Origin: http://sty.js118114.com:8080
Proxy-Connection: keep-alive
Referer: http://sty.js118114.com:8080/Report/report/movecar_list.html
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36
X-Requested-With: XMLHttpRequest
Request Payload
{calling_no: "", begin_time: "", end_time: "", called_car_no: ""}
begin_time: ""
called_car_no: ""
calling_no: ""
end_time: ""
Fiddler Inspectors Raw
POST http://sty.js118114.com:8080/Report/movecar/list/1/10 HTTP/1.1
Host: sty.js118114.com:8080
Connection: keep-alive
Content-Length: 66
Accept: */*
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36
Content-Type: text/plain;charset=UTF-8;
Origin: http://sty.js118114.com:8080
Referer: http://sty.js118114.com:8080/Report/report/movecar_list.html
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9
Cookie: JSESSIONID=0A474B00017BFFD89A515B336F482905
{"calling_no":"","begin_time":"","end_time":"","called_car_no":""}
Response Raw
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json;charset=UTF-8
Date: Thu, 22 Aug 2019 00:27:59 GMT
Content-Length: 8150
I have problem on site where email is under obfuscator.
When i run my program i get output:
The email is:{"success":"","code":1,"msg":"ReCAPTCHA"}
But when i want click 'watch email' on computer all is fine and i get:
The email is:{"success":"","code":0,"msg":"xxxx#gmail.com"}
Code in POST:
REQUEST HEADERS
Accept: application/json, text/javascript, */*; q=0.01
Accept-Encoding: gzip, deflate, br
Accept-Language: pl-PL,pl;q=0.9,en-US;q=0.8,en;q=0.7
Connection: keep-alive
Content-Length: 141
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Cookie: SOOOME COOKIES.
Host: https://xxxxxxx.com
Origin: https://xxxxxxx.com
Referer: https://xxxxxxx.com/asas
User-Agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Mobile Safari/537.36
X-Requested-With: XMLHttpRequest
QUERY STRING PARAMETRS
decode:
FORM DATA
hash: YToyOntpOjA7czo0NDoidHh3VFlXck83eFdza1FRUWgydUlvb0MveHRRemNLaCtNa3BuenVJU0VmUT0iO2k6MTtzOjE2OiK3SJ7OlhTa5DgPfA1YqCfRIjt9
type: ademail
And here is my code:
import requests
url = "https://xxxxx.com/_ajax/obfuscator/?decode"
headers = {
'User-Agent':'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Mobile Safari/537.36',
'Accept': 'application/json, text/javascript, */*; q=0.01',
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language': 'pl-PL,pl;q=0.9,en-US;q=0.8,en;q=0.7',
'Connection': 'keep-alive',
'Content-Length': '141',
'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
'Host': 'https://xxxxx.com/',
'Origin': 'https://xxxxx.com/',
'Referer':'https://xxxxx.com/asd',
'X-Requested-With':'XMLHttpRequest' }
data = {'hash':'YToyOntpOjA7czo0NDoiQStHbXkrY2p1dllrUmlXSWdWTjdNbHF2Y3cyak13QU5GeUtaQXZReFcrbz0iO2k6MTtzOjE2OiJ7Byq7O88ydxCtVWgoEETOIjt9',
'type':'adsemail'}
r = requests.post(url, data, headers)
pastebin_url = r.text
print("The email is:%s"%pastebin_url)
I also try do it the same as Webdriver
driver = webdriver.Chrome("C:/Users/User/Desktop/Email/chromedriver.exe")
driver.set_page_load_timeout(5000)
driver.get("https://xxxx.com/asd")
driver.implicitly_wait(3000)
sleep(1)
RODO = "//input[#class='btn btn-confirm']"
driver.find_element_by_xpath(RODO).click()
sleep(7)
email = "//span[#class='click_to_show']"
driver.find_element_by_xpath(email).click()
But i get Recaptcha to do.... ;/
Where is the problem?
I also try:
``` user_agent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.50 Safari/537.36'
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('user-agent={0}'.format(user_agent))
driver = webdriver.Chrome("C:/Users/User/Desktop/Email/chromedriver.exe")
driver.set_page_load_timeout(5000)
But not working, site want captcha ;/
Updated code - I'm using this code to send the request:
headers = {
"Host": "www.roblox.com",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0",
"Accept": "application/json, text/plain, */*",
"Accept-Language": "en-US;q=0.7,en;q=0.3",
"Referer": "https://www.roblox.com/users/12345/profile",
"Content-Type": "application/json;charset=utf-8",
"X-CSRF-TOKEN": "some-xsrf-token",
"Content-Length": "27",
"DNT": "1",
"Connection": "close"
}
data = {"targetUserId":"56789"}
url = "http://www.roblox.com/user/follow"
r = requests.post(url, headers=headers, data=data, cookies={"name":"value"})
Response (using r.text):
{"isValid":false,"data":null,"error":""}
The request itself is valid, I sent it using burp and it worked:
POST /user/follow HTTP/1.1
Host: www.roblox.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0
Accept: application/json, text/plain, */*
Accept-Language: pl,en-US;q=0.7,en;q=0.3
Referer: https://www.roblox.com/users/12345/profile
Content-Type: application/json;charset=utf-8
X-CSRF-TOKEN: Ab1/2cde3fGH
Content-Length: 27
Cookie: some-cookie=;
DNT: 1
Connection: close
{"targetUser":"56789"}
Because it works in Burp but not in Python requests, get a packet sniffer (Wireshark is the simplest IMO) and look to see the difference in the packet sent by Burp that works and the one sent from Python that does not work. I am suspecting that the problem is that the website is HTTPS but you are using http://www.roblox.com . Do try https://www.roblox.com , but I am not sure if it will work.