There are similar questions posted, but I still seem to have a problem. I am expecting to receive a registration email after running this. I receive nothing. Two questions. What is wrong? How would I even know if the data was successfully submitted as opposed to the page just loading normally?
serviceurl = 'https://signup.com/'
payload = {'register-fname': 'Peter', 'register-lname': "Parker", 'register-email': 'xyz#email.com', 'register-password': '9dlD313kF'}
r2 = requests.post(serviceurl, data=payload)
print(r2.status_code)
The url for the POST request is actually https://signup.com/api/users, and it returns 200 (in my browser).
You need to replicate what your browser does. This might include certain request headers.
You will want to use your browser's dev tools/network inspector to gather this information.
The information below it from my Firefox on my computer:
Request headers:
Host: signup.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0
Accept: application/json, text/plain, */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: application/json;charset=utf-8
Content-Length: 107
Origin: https://signup.com
Connection: keep-alive
Referer: https://signup.com/
Cookie: _vspot_session_id=ce1937cf52382239112bd4b98e0f1bce; G_ENABLED_IDPS=google; _ga=GA1.2.712393353.1584425227; _gid=GA1.2.1095477818.1584425227; __utma=160565439.712393353.1584425227.1584425227.1584425227.1; __utmb=160565439.2.10.1584425227; __utmc=160565439; __utmz=160565439.1584425227.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmt=1; __qca=P0-1580853344-1584425227133; _gat=1
Pragma: no-cache
Cache-Control: no-cache
Payload:
{"status":true,"code":null,"email":"TestEmail#hotmail.com","user":{"id":20540206,"email":"TestEmail#hotmail.com","name":"TestName TestSurname","hashedpassword":"4ffdbb1c33d14ed2bd02164755c43b4ad8098be2","salt":"700264767700800.7531319164902858","accesskey":"68dd25c3ae0290be69c0b59877636a5bc5190078","isregistered":true,"activationkey":"f1a6732b237379a8a1e6c5d14e58cf4958bf2cea","isactivated":false,"chgpwd":false,"timezone":"","phonenumber":"","zipcode":"","gender":"N","age":-1,"isdeferred":false,"wasdeferred":false,"deferreddate":null,"registerdate":"2020/03/17 06:09:27 +0000","activationdate":null,"addeddate":"2020/03/17 06:09:27 +0000","admin":false,"democount":0,"demodate":null,"invitationsrequest":null,"isvalid":true,"timesinvalidated":0,"invaliddate":null,"subscribe":0,"premium":false,"contributiondate":null,"contributionamount":0,"premiumenddate":null,"promo":"","register_token":"","premiumstartdate":null,"premiumsubscrlength":0,"initial_reg_type":"","retailmenot":null,"sees":null,"created_at":"2020/03/17 06:09:27 +0000","updated_at":"2020/03/17 06:09:27 +0000","first_name":"TestName","last_name":"TestSurname"},"first_name":"TestName","last_name":"TestSurname","mobile_redirect":false}
There's a lot to replicate. Things like the hashed password, salt, dates, etc would have been generated by JavaScript executed by your browser.
Keep in mind, the website owner might not appreciate a bot creating user accounts.
Related
I am trying to web scrap "https://pricehistoryapp.com/" to obtain the product's highest and lowest prices. I am using python requests library for this.
I have observed that this site obtains the next address to go by using a request named 'getSlugFromUrl' made to server. This is also first xhr request made as search button is pressed. I understood some part of pay load and headers but not able to figure out others:
Header:
:authority: ph.pricetoolkit.com
:method: POST
:path: /api/product/history/getSlugFromUrl
:scheme: https
accept: application/json
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
auth: 8nV5lXoVu/Qk2z5BhZmbPj4rxSdd35ThEbiRgK1kCCz+wU0Guh+6qal03DS3J6HG
cache-control: no-cache
content-length: 738
content-type: application/x-www-form-urlencoded
origin: https://pricehistoryapp.com
pragma: no-cache
referer: https://pricehistoryapp.com/
sec-ch-ua: "Microsoft Edge";v="107", "Chromium";v="107", "Not=A?Brand";v="24"
sec-ch-ua-mobile: ?1
sec-ch-ua-platform: "Android"
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: cross-site
user-agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Mobile Safari/537.36 Edg/107.0.1418.24
Payload:
purl:https://www.flipkart.com/viewsonic-vx-series-24-inch-wqhd-led-backlit-ips-panel-frameless-monitor-vx2480-2k-shd/p/itmedaf0773f47ba?pid=MONG5KEK2GDSGTSY
lid: LSTMONG5KEK2GDSGTSYAQGKIT
marketplace: FLIPKART
store: 6bo/g0i/9no
srno: b_1_5
otracker: hp_omu_Best+of+Electronics_4_3.dealCard.OMU_NOBMPKW1HQ7A_3
iid: 083d5b0d-6840-426e-811b-28b45d6e6ea7.MONG5KEK2GDSGTSY.SEARCH
ssid: d5n99toygg0000001667914476777
For instance from where is auth obtained from in header, or from where lid, iid, ssid obtained for payload. I know the question is really stupid, but please guide me towards a solution. Thanks in advance.
Hi I have a request as so
GET https://dropezy.goadda.in/master/v1/id/products/search?q=buah&storeId=619e02f0f580166d48277fc0&sortBy=relevance HTTP/2.0
accept: application/json
enro-api-key: rhcvwcdd-7aza-zvv7qskyu-4b8jm8kwppkf
cache-control: max-age=31536000
store-id: 619e02f0f580166d48277fc0
user-agent: Mozilla/5.0 (Linux; Android 11; sdk_gphone_x86 Build/RSR1.201013.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/83.0.4103.106 Mobile Safari/537.36
content-type: application/json; charset=utf-8
origin: https://www.dropezy.com
x-requested-with: com.enroco.dropezy
sec-fetch-site: cross-site
sec-fetch-mode: cors
sec-fetch-dest: empty
referer: https://www.dropezy.com/id/search/
accept-encoding: gzip, deflate
accept-language: en-US,en;q=0.9
content-length: 0
When I tried using request in python it returns message not found
This is my current code
url = 'https://dropezy.goadda.in/master/v1/id/products/search?q=buah&storeId=619e02f0f580166d48277fc0&sortBy=relevance HTTP/2.0'
accept='application/json'
jsonData = requests.post(url, accept).json()
Can somebody help? How to insert the other components such as user-agent, enro-api-key into requests?
The two requests aren't the same. The first is a GET request, but the second is a POST.
Use requests.get to make a GET request.
If you need to supply headers in a request, yes, you can do that: https://docs.python-requests.org/en/latest/user/quickstart/#custom-headers.
Generally, the docs for Requests are quite good: https://docs.python-requests.org/en/latest/
I need a python script that posts a given Authorization Bearer header to a specific ipand port.
This is what I have so far.
#!/usr/bin/python
import urllib3
import certifi
http = urllib3.PoolManager(ca_certs=certifi.where())
url = 'http://172.10.10.2:3000'
req = http.request('POST', url, fields={'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbWQiOiJscyR7SUZTfS1sYSR7SUZTfS90bXAvbmMifQ.EziCTtJn1PpPXvemJllF36w7ADNkhKiktZ5sv9ADR3o'})
print(req.data.decode('utf-8'))
I currently get an error when running this stating that the Required authorization token is not found
The bearer code is created manually and then input into the script, if there was a way to import that from the site I create it on that would be helpful.
What I need the output to be in this -
GET / HTTP/1.1
Host: 172.10.10.2:3000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: close
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbWQiOiJpcCR7SUZTfWEifQ.RkoZinBcg2_5HRGgv1AtErhscIVBRv2hUcGIF08ZlCM
(I'm new to posting on stack overflow tell me stuff to improve on ty)
I recently got into python requests and I am a bit new to web stuff related to python so if its something stupid don't flame me. I've been trying a lot of ways to get one of my post events to not return error 400. I messed with headers and data and a bunch of other stuff and I would either get code 404 or 400.
I have tried adding a bunch of headers to my project. Im trying to login to the site "Roblox" making a auto purchase bot. The Csrf Token was not in cookies so I found a loop hole getting the token with sending a request to a logout url (If anyone gets confused). I double checked the Data I inputted aswell and it is indeed correct. I was researching for multiple hours and I couldn't find a way to fix so I came to stack overflow for the first time.
RobloxSignInDeatils = {
"cvalue": "TestWebDevAcc", #Username
"ctype": "Username", #Leave Alone
"password": "Test123123" #Password
}
def GetToken(Session, Headers):
LogoutUrl = "https://api.roblox.com/sign-out/v1" #Url to get CSRF
Request = Session.post(LogoutUrl, Headers) #Sent Post
print("Token: ", Request.headers["X-CSRF-TOKEN"]) #Steal the token
return Request.headers["X-CSRF-TOKEN"] #Return Token
with requests.session() as CurrentSession:
Header = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36"
} #Get the UserAgent Header Only
CurrentSession.get("https://www.roblox.com/login", headers=Header) #Send to the login page
CsrfToken = GetToken(CurrentSession, Header) #Get The Token
Header["X-CSRF-TOKEN"] = CsrfToken #Set the token
SignIn = CurrentSession.post("https://auth.roblox.com/v2/login",
data=RobloxSignInDeatils, headers=Header) # Send Post to Sign in
print(SignIn.status_code) #Returns 400 ?
Printing "print(SignIn.status_code)" just returns 400 nothing else to explain
EDIT: If this helps heres a list of ALL the headers:
Request URL: https://auth.roblox.com/v2/login
Request Method: OPTIONS
Status Code: 200 OK
Remote Address: 128.116.112.44:443
Referrer Policy: no-referrer-when-downgrade
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: X-CSRF-TOKEN,Content-Type,Pragma,Cache-Control,Expires
Access-Control-Allow-Methods: OPTIONS, TRACE, GET, HEAD, POST, DELETE, PATCH
Access-Control-Allow-Origin: https://www.roblox.com
Access-Control-Max-Age: 600
Cache-Control: max-age=120, private
Content-Length: 0
Date: Thu, 14 Feb 2019 04:04:13 GMT
P3P: CP="CAO DSP COR CURa ADMa DEVa OUR IND PHY ONL UNI COM NAV INT DEM PRE"
Roblox-Machine-Id: CHI1-WEB2875
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Access-Control-Request-Headers: content-type,x-csrf-token
Access-Control-Request-Method: POST
Connection: keep-alive
Host: auth.roblox.com
Origin: https://www.roblox.com
Referer: https://www.roblox.com/
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36
Payload:
{cvalue: "TestWebDevAcc", ctype: "Username", password: "Test123123"}
I wrote a small module about 1 year age.
When I tried to add some feature to it these days, I found there is a big change in python's urllib and I was confused about how to build a request.
In the old module FancyURLopener was used as my base class, yet I found it was Deprecated since version 3.3.
So I read the document again, and try to build a request instead of a opener.
However, when I tried to add headers, only one function Request.add_header(key, val) was provided. I have headers copied from fiddler like this:
GET some_url HTTP/1.1
Host: sosu.qidian.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: zh-cn,en-us;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
X-Requested-With: XMLHttpRequest
Referer: anothe_url
Cookie: a lot of data
Connection: keep-alive
So do I have to add them to the request one by one?
Also I found another openner urllib.request.build_opener() which can add a lot of headers in one time. But I could not set method 'get' or 'post'.
I am a newbie on python, any suggestions ?
By far the best way to make http requests in Python is to install and use this module:
http://docs.python-requests.org/en/latest/
You can write code like this:
headers = {
'X-Requested-With': 'XMLHttpRequest',
'Referer': 'anothe_url',
# etc
}
response = requests.get(some_url, headers=headers)
http://docs.python-requests.org/en/latest/user/quickstart/#custom-headers