I am trying to send a file for a candidate in POST request naturalHR API:
I have tried the same request using POSTMAN and it worked fine. But when i try to integrate the API's POST request using python to attach the file I am getting an error that It cv parameter should be a file(its API error response).
Source Code:
from pprint import pprint
import json
import requests
import urllib.request
headers = {
'accept': 'application/json',
'Authorization': api_key,
'Host': 'api02.naturalhr.net',
'Referer': 'https://api02.naturalhr.net/api/documentation',
'Content-type': 'multipart/form-data',
'Sec-Fetch-Site': 'same-origin',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36'
}
payLoad = dict()
payLoad["firstname"] = json_of_vals['firstname']
payLoad["surname"] = json_of_vals['surname']
payLoad["email"] = json_of_vals['email']
payLoad["cv"] = "Path/To/PDF_File"
files = {'file': "outfilename.pdf"}
api_url = "https://api02.naturalhr.net/api/v1/candidate"
res = requests.post(api_url, files=files, headers=headers, data=request_data)
print(res.content)
Please dont mark this as a duplicate to a question here which is already been answered because I have tested it by using files as request's argument like:
res = requests.post(api_url, files=files, headers=headers, data=request_data)
Edited:
The answer which I have tried:
Using Python Requests to send file and JSON in single request
I was adding a header
'accept': 'application/json'
Which should not be there, I tried it using only user-agent and an API-key and it worked perfectly fine as per requirements.
Corrected Code:
from pprint import pprint
import json
import requests
import urllib.request
headers = {
'Authorization': api_key,
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36'
}
payLoad = dict()
payLoad["firstname"] = json_of_vals['firstname']
payLoad["surname"] = json_of_vals['surname']
payLoad["email"] = json_of_vals['email']
files = {'file': "PATH/TO/FILE/FROM/LOCAL/DRIVE"}
api_url = "https://api02.naturalhr.net/api/v1/candidate"
res = requests.post(api_url, headers=headers, data=payLoad, files=files)
print("Status Code is: ", res.status_code)
print("Returned JSON Response is:\n")
pprint(res.text)
Related
I'm trying to return a GET request from an API using HTTPBasicAuth.
I've tested the following in Postman, and received the correct response
URL:"https://someapi.data.io"
username:"username"
password:"password"
And this returns me the data I expect, and all is well.
When I've tried this in python however, I get kicked back a 403 error, alongside a
""error_type":"ACCESS DENIED","message":"Please confirm api-key, api-secret, and permission is correct."
Below is my code:
import requests
from requests.auth import HTTPBasicAuth
URL = 'https://someapi.data.io'
authBasic=HTTPBasicAuth(username='username', password='password')
r = requests.get(URL, auth = authBasic)
print(r)
I honestly can't tell why this isn't working since the same username and password passes in Postman using HTTPBasicAuth
You have not conveyed all the required parameters. And postman is doing this automatically for you.
To be able to use in python requests just specify all the required parameters.
headers = {
'Host': 'sub.example.com',
'User-Agent': 'Chrome v22.2 Linux Ubuntu',
'Accept': '*/*',
'Accept-Encoding': 'gzip, deflate, br',
'Connection': 'keep-alive',
'X-Requested-With': 'XMLHttpRequest'
}
url = 'https://sub.example.com'
response = requests.get(url, headers=headers)
It could be due to the fact that the user-agent is not defined
try the following:
import requests
from requests.auth import HTTPBasicAuth
URL = 'https://someapi.data.io'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36'}
authBasic=HTTPBasicAuth(username='username', password='password')
r = requests.get(URL, auth = authBasic, headers=headers)
print(r)
As table data is generating dynamically by JavaScript that's why I use api calls url
but throwing the above mentioned warning. Anyone's help is appreciated.
Base_URL
My code:
import requests
import pandas as pd
import json
body = { 'tipoEmpresa': '0'}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36',
'x-dtpc': '33$511511524_409h2vHHVRBIAIGILPJNCRGRCECUBIACWCBUEE-0e37',
'X-Requested-With': 'XMLHttpRequest',
'Content-Type': 'application/json'
}
def main(url):
with requests.Session() as req:
req.headers.update(headers)
r = req.post(url, data=json.dumps(body), headers =headers, verify = False)
resp = r.json()['d']
#df = pd.DataFrame(resp)
#print(df)
main('https://www.rad.cvm.gov.br/ENET/frmConsultaExternaCVM.aspx/PopulaComboEmpresas')
Try to add these two lines at the beginning of your script to suppress warning messages:
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
I have implemented a code to download bhav-copies for all the dates in the stock market. After scraping about 2 years, it seems like my IP got blocked.
This code doesn't work for me.
import urllib.request
url = 'https://www1.nseindia.com/content/historical/DERIVATIVES/2014/APR/fo01APR2014bhav.csv.zip'
response = urllib.request.urlopen(url)
It gives the following error :
urllib.error.HTTPError: HTTP Error 403: Forbidden
I would like to know how I can use some proxy to get the data. Any help would be really appreciated.
import urllib.request
proxy_host = '1.2.3.4:8080' # host and port of your proxy
url = 'https://www1.nseindia.com/content/historical/DERIVATIVES/2014/APR/fo01APR2014bhav.csv.zip'
req = urllib.request.Request(url)
req.set_proxy(proxy_host, 'http')
response = urllib.request.urlopen(req)
For more flexibility, you can use a Proxy Handler - https://docs.python.org/3/library/urllib.request.html
proxy_handler = urllib.request.ProxyHandler({'http': '1.2.3.4:3128/'})
proxy_auth_handler = urllib.request.ProxyBasicAuthHandler()
proxy_auth_handler.add_password('realm', 'host', 'username', 'password')
opener = urllib.request.build_opener(proxy_handler, proxy_auth_handler)
This works fine,
import requests
headers = {
'authority': 'www.nseindia.com',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36',
'accept': '*/*',
'sec-fetch-site': 'same-origin',
'sec-fetch-mode': 'cors',
'sec-fetch-dest': 'empty',
'referer': 'https://www.nseindia.com/content/',
'accept-language': 'en-US,en;q=0.9,lb;q=0.8',
}
url = "https://www1.nseindia.com/content/historical/DERIVATIVES/2014/APR/fo01APR2014bhav.csv.zip"
r = requests.get(url,headers=headers)
with open("data.zip","wb") as f:
f.write(r.content)
if you have proxies,
proxy = {"http" : "x.x.x.x:pppp",
"https" :"x.x.x.x:pppp",
}
r = requests.get(url, headers=headers, proxies=proxy)
You don't need to use proxies to download this file. The code below will work like a charm:
import urllib.request
url = 'https://www1.nseindia.com/content/historical/DERIVATIVES/2014/APR/fo01APR2014bhav.csv.zip'
req = urllib.request.Request(url)
# Add referer header to bypass "HTTP Error 403: Forbidden"
req.add_header('Referer', 'https://www.nseindia.com')
res = urllib.request.urlopen(req)
# Save it into file.zip
with open("file.zip", "wb") as f:
f.write(res.read())
In case you want to get free proxies, visit https://free-proxy-list.net/. Then follow the answer of #pyd at https://stackoverflow.com/a/63328368/8009647
In the request headers when logging in, there's a header called "cookie" that changes every time, how would I grab that each time and put it in the headers using python requests?
screenshot of network tab in chrome
Heres my code:
import requests
import time
proxies = {
"http": "http://us.proxiware.com:2000"
}
login_data = {'op':'login-main', 'user':'UpbeatPark', 'passwd':'Testingreddit123', 'api_type':'json'}
comment_data = {'thing_id':'t3_gluktj', 'text':'epical. redditor', 'id':'#form-t3_gluktjbx2', 'r':'gaming','renderstyle':'html'}
s = requests.Session()
s.headers.update({'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/82.0.4085.6 Safari/537.36'})
r = s.get('https://old.reddit.com/', proxies=proxies)
time.sleep(2)
r = s.post('https://old.reddit.com/api/login/UpbeatPark', proxies=proxies, data=login_data)
print(r.text)
here's the output (I know for a fact it is the correct password):
{"json": {"errors": [["WRONG_PASSWORD", "wrong password", "passwd"]]}}
This worked for me:
import requests
login_data = {
"op": "login-main",
"user": "USER",
"passwd": "PASS",
"api_type": "json",
}
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/82.0.4085.6 Safari/537.36",
}
s = requests.Session()
r = s.post("https://old.reddit.com/api/login/USER", headers=headers, data=login_data)
print(r.text)
It seems exactly like the code you are using but without proxy. Can you try to turn it off? The proxy might block cookies.
For some reason python requests does not do rePOST after encountered redirect header
import requests
proxies = {'http': 'http://127.0.0.1:8888',}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36',
'content-type': 'application/x-www-form-urlencoded'}
r = requests.post(url, data, headers=headers, timeout=timeout, proxies=proxies, allow_redirects=True,)
html = r.text
So it means I can't login to any form that is behind redirect. How can I solve this issue? Thank you!