Circumvent Error 414 on HTTP POST using python requests - python

I am using python requests HTTP POST to send a data to a certain third-party website I do not own. But I can't get it to work because I am getting 414 status code.
url = "http://someurl.com"
headers = {
'Content-Type': "application/x-www-form-urlencoded; charset=UTF-8"
}
params = {'input': "Lorem ipsum... very long string"}
result = requests.post(url, params=params, headers=headers)
print(result.status_code)
How can I get this to work?


Looking at the docs, it seems that result = requests.post(url, params=params, headers=headers) should be result = requests.post(url, data=params, headers=headers) (credit to John La Rooy).
IIRC the params flag means the params you see tacked onto the end of your url, while data is the POST data.

Related

{"Error":"Invalid JSON syntax at offset 2"} - receiving this error while trying to get everflow report

I need to get a report from Everflow, a marketing platform.
The idea behind the code is pretty straightforward - I make a POST request:
url = "https://api.eflow.team/v1/affiliates/reporting/daily"
headers = {'x-eflow-api-key': '*MY*API*KEY', 'Content-Type': 'application/json',}
payload = '{ "from": "2020-09-01", "to": "2020-09-10", "timezone_id": 67, "currency_id": "USD"}'
response = requests.request("POST", url, data=payload, headers=headers)
print(response.text)
What I get as a result is:
{"Error":"Invalid JSON syntax at offset 2"}
As far as I understand, the problem might relate to the payload format, but no success with it so far after two days of trying.
Thanks for any help!

Can you try the below code?
import requests
import json
url = "https://api.eflow.team/v1/affiliates/reporting/daily"
headers = {"x-eflow-api-key": "*MY*API*KEY", "Content-type": "application/json"}
payload = {'from': '2020-09-01', 'to': '2020-09-10', 'timezone_id': 67, 'currency_id': 'USD'}
payload = json.dumps(payload)
response = requests.request("POST", url, data=payload, headers=headers)
print(response.text)
But, from requests 2.4.2, the "json" parameter is supported. No need to specify "Content-Type" (in most cases).
requests.post('http://httpbin.org/post', json={'test': 'foo'})

401 with a POST using requests

Using Postman, I can make a POST to an endpoint with a JSON body, and it returns be some results.
No authorisation on the endpoint.
Trying to get this to work with requests fails with a 401.
What am I missing here:
Code:
import json
import requests
url = 'https://wabi-australia-southeast-api.analysis.windows.net/public/reports/querydata?synchronous=true'
header = {"content-type": "application/json"}
body = {"version":"1.0.0","queries":[{"Query":{"Commands":[{"SemanticQueryDataShapeCommand":{"Query":{"Version":2,"From":[{"Name":"d1","Entity":"dimLGA","Type":0},{"Name":"l","Entity":"Linelist","Type":0}],"Select":[{"Column":{"Expression":{"SourceRef":{"Source":"d1"}},"Property":"LGAName"},"Name":"dimLGA.LGAName"},{"Measure":{"Expression":{"SourceRef":{"Source":"l"}},"Property":"Cases"},"Name":"Linelist.Cases"}],"Where":[{"Condition":{"Not":{"Expression":{"Comparison":{"ComparisonKind":0,"Left":{"Column":{"Expression":{"SourceRef":{"Source":"d1"}},"Property":"LGAName"}},"Right":{"Literal":{"Value":"null"}}}}}}},{"Condition":{"In":{"Expressions":[{"Column":{"Expression":{"SourceRef":{"Source":"l"}},"Property":"clin_status_n"}}],"Values":[[{"Literal":{"Value":"'Admitted to ICU'"}}],[{"Literal":{"Value":"'Admitted, not known to be in ICU'"}}],[{"Literal":{"Value":"'Home isolation'"}}],[{"Literal":{"Value":"'Hotel detention'"}}],[{"Literal":{"Value":"'Hospital in the home'"}}],[{"Literal":{"Value":"'Under investigation'"}}]]}}}]},"Binding":{"Primary":{"Groupings":[{"Projections":[0,1]}]},"DataReduction":{"DataVolume":3,"Primary":{"Window":{"Count":500}}},"Version":1}}}]},"CacheKey":"{\"Commands\":[{\"SemanticQueryDataShapeCommand\":{\"Query\":{\"Version\":2,\"From\":[{\"Name\":\"d1\",\"Entity\":\"dimLGA\",\"Type\":0},{\"Name\":\"l\",\"Entity\":\"Linelist\",\"Type\":0}],\"Select\":[{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d1\"}},\"Property\":\"LGAName\"},\"Name\":\"dimLGA.LGAName\"},{\"Measure\":{\"Expression\":{\"SourceRef\":{\"Source\":\"l\"}},\"Property\":\"Cases\"},\"Name\":\"Linelist.Cases\"}],\"Where\":[{\"Condition\":{\"Not\":{\"Expression\":{\"Comparison\":{\"ComparisonKind\":0,\"Left\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d1\"}},\"Property\":\"LGAName\"}},\"Right\":{\"Literal\":{\"Value\":\"null\"}}}}}}},{\"Condition\":{\"In\":{\"Expressions\":[{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"l\"}},\"Property\":\"clin_status_n\"}}],\"Values\":[[{\"Literal\":{\"Value\":\"'Admitted to ICU'\"}}],[{\"Literal\":{\"Value\":\"'Admitted, not known to be in ICU'\"}}],[{\"Literal\":{\"Value\":\"'Home isolation'\"}}],[{\"Literal\":{\"Value\":\"'Hotel detention'\"}}],[{\"Literal\":{\"Value\":\"'Hospital in the home'\"}}],[{\"Literal\":{\"Value\":\"'Under investigation'\"}}]]}}}]},\"Binding\":{\"Primary\":{\"Groupings\":[{\"Projections\":[0,1]}]},\"DataReduction\":{\"DataVolume\":3,\"Primary\":{\"Window\":{\"Count\":500}}},\"Version\":1}}}]}","QueryId":"","ApplicationContext":{"DatasetId":"5b547437-24c9-4b22-92de-900b3b3f4785","Sources":[{"ReportId":"964ef513-8ff4-407c-8068-ade1e7f64ca5"}]}}],"cancelQueries":[],"modelId":1959902}
r = requests.post(url, data=json.dumps(body), headers=header)
print(r.status_code)
print(r.raise_for_status())
This returns a 401:
HTTPError: 401 Client Error: Unauthorized for url:
https://wabi-australia-southeast-api.analysis.windows.net/public/reports/querydata?synchronous=true

Just found out Postman lets you generate a python code snippet for Requests. Amazing.
This solved my problem.
import requests
url = "https://wabi-australia-southeast-api.analysis.windows.net/public/reports/querydata"
payload = "{\"version\":\"1.0.0\",\"queries\":[{\"Query\":{\"Commands\":[{\"SemanticQueryDataShapeCommand\":{\"Query\":{\"Version\":2,\"From\":[{\"Name\":\"d1\",\"Entity\":\"dimLGA\",\"Type\":0},{\"Name\":\"l\",\"Entity\":\"Linelist\",\"Type\":0}],\"Select\":[{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d1\"}},\"Property\":\"LGAName\"},\"Name\":\"dimLGA.LGAName\"},{\"Measure\":{\"Expression\":{\"SourceRef\":{\"Source\":\"l\"}},\"Property\":\"Cases\"},\"Name\":\"Linelist.Cases\"}],\"Where\":[{\"Condition\":{\"Not\":{\"Expression\":{\"Comparison\":{\"ComparisonKind\":0,\"Left\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d1\"}},\"Property\":\"LGAName\"}},\"Right\":{\"Literal\":{\"Value\":\"null\"}}}}}}},{\"Condition\":{\"In\":{\"Expressions\":[{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"l\"}},\"Property\":\"clin_status_n\"}}],\"Values\":[[{\"Literal\":{\"Value\":\"'Admitted to ICU'\"}}],[{\"Literal\":{\"Value\":\"'Admitted, not known to be in ICU'\"}}],[{\"Literal\":{\"Value\":\"'Home isolation'\"}}],[{\"Literal\":{\"Value\":\"'Hotel detention'\"}}],[{\"Literal\":{\"Value\":\"'Hospital in the home'\"}}],[{\"Literal\":{\"Value\":\"'Under investigation'\"}}]]}}}]},\"Binding\":{\"Primary\":{\"Groupings\":[{\"Projections\":[0,1]}]},\"DataReduction\":{\"DataVolume\":3,\"Primary\":{\"Window\":{\"Count\":500}}},\"Version\":1}}}]},\"CacheKey\":\"{\\\"Commands\\\":[{\\\"SemanticQueryDataShapeCommand\\\":{\\\"Query\\\":{\\\"Version\\\":2,\\\"From\\\":[{\\\"Name\\\":\\\"d1\\\",\\\"Entity\\\":\\\"dimLGA\\\",\\\"Type\\\":0},{\\\"Name\\\":\\\"l\\\",\\\"Entity\\\":\\\"Linelist\\\",\\\"Type\\\":0}],\\\"Select\\\":[{\\\"Column\\\":{\\\"Expression\\\":{\\\"SourceRef\\\":{\\\"Source\\\":\\\"d1\\\"}},\\\"Property\\\":\\\"LGAName\\\"},\\\"Name\\\":\\\"dimLGA.LGAName\\\"},{\\\"Measure\\\":{\\\"Expression\\\":{\\\"SourceRef\\\":{\\\"Source\\\":\\\"l\\\"}},\\\"Property\\\":\\\"Cases\\\"},\\\"Name\\\":\\\"Linelist.Cases\\\"}],\\\"Where\\\":[{\\\"Condition\\\":{\\\"Not\\\":{\\\"Expression\\\":{\\\"Comparison\\\":{\\\"ComparisonKind\\\":0,\\\"Left\\\":{\\\"Column\\\":{\\\"Expression\\\":{\\\"SourceRef\\\":{\\\"Source\\\":\\\"d1\\\"}},\\\"Property\\\":\\\"LGAName\\\"}},\\\"Right\\\":{\\\"Literal\\\":{\\\"Value\\\":\\\"null\\\"}}}}}}},{\\\"Condition\\\":{\\\"In\\\":{\\\"Expressions\\\":[{\\\"Column\\\":{\\\"Expression\\\":{\\\"SourceRef\\\":{\\\"Source\\\":\\\"l\\\"}},\\\"Property\\\":\\\"clin_status_n\\\"}}],\\\"Values\\\":[[{\\\"Literal\\\":{\\\"Value\\\":\\\"'Admitted to ICU'\\\"}}],[{\\\"Literal\\\":{\\\"Value\\\":\\\"'Admitted, not known to be in ICU'\\\"}}],[{\\\"Literal\\\":{\\\"Value\\\":\\\"'Home isolation'\\\"}}],[{\\\"Literal\\\":{\\\"Value\\\":\\\"'Hotel detention'\\\"}}],[{\\\"Literal\\\":{\\\"Value\\\":\\\"'Hospital in the home'\\\"}}],[{\\\"Literal\\\":{\\\"Value\\\":\\\"'Under investigation'\\\"}}]]}}}]},\\\"Binding\\\":{\\\"Primary\\\":{\\\"Groupings\\\":[{\\\"Projections\\\":[0,1]}]},\\\"DataReduction\\\":{\\\"DataVolume\\\":3,\\\"Primary\\\":{\\\"Window\\\":{\\\"Count\\\":500}}},\\\"Version\\\":1}}}]}\",\"QueryId\":\"\",\"ApplicationContext\":{\"DatasetId\":\"5b547437-24c9-4b22-92de-900b3b3f4785\",\"Sources\":[{\"ReportId\":\"964ef513-8ff4-407c-8068-ade1e7f64ca5\"}]}}],\"cancelQueries\":[],\"modelId\":1959902}"
headers = {
'Content-Type': 'text/plain'
}
response = requests.request("POST", url, headers=headers, data = payload)
print(response.text.encode('utf8'))

Scraping this site

Im trying to mimic this POST request from this site with this payload:
from this URL: https://surviv.io/stats/gert1
Here is an image of the request im trying to mimic.
Here is my current code in python:
import requests
headers = {'content-type': 'application/json; charset=UTF-8'}
url = 'https://surviv.io/api/user_stats'
payload = {"slug":"gert1","interval":"all","mapIdFilter":"-1"}
r = requests.post(url=url, headers=headers, data=payload)
print(r.content)
This returns:
b'<html>\r\n<head><title>500 Internal Server Error</title></head>\r\n<body bgcolor="white">\r\n<center><h1>500 Internal Server Error</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n'
This is not what I want it return. I want it to return the exact response shown in the response tab of the user_stats requests, which contains the player's stats.
This is what I want it to return:
{"slug":"gert1","username":"GERT","player_icon":"","banned":false,"wins":61,"kills":2830,"games":2034,"kpg":"1.4","modes":[{"teamMode":1,"games":1512,"wins":46,"kills":2230,"winPct":"3.0","mostKills":21,"mostDamage":1872,"kpg":"1.5","avgDamage":169,"avgTimeAlive":92},{"teamMode":2,"games":255,"wins":4,"kills":234,"winPct":"1.6","mostKills":8,"mostDamage":861,"kpg":"0.9","avgDamage":162,"avgTimeAlive":102},{"teamMode":4,"games":267,"wins":11,"kills":366,"winPct":"4.1","mostKills":17,"mostDamage":2225,"kpg":"1.4","avgDamage":246,"avgTimeAlive":125}]}

You should use the json attribute rather than data in the post method. r = requests.post(url=url, headers=headers, json=payload)

Change your code to following your forgot to use json :
import json
import requests
headers = {'content-type': 'application/json; charset=UTF-8'}
url = 'https://surviv.io/api/user_stats'
payload = {"slug":"gert1","interval":"all","mapIdFilter":"-1"}
r = requests.post(url=url, headers=headers, data=json.dumps(payload))
print(r.content)

I have some troubles with a post request

I'm trying to scrape a website but i get a 500 response.
It's a relocation website and i'm trying to get prices for different input parameters (volume, destination, departureCity ...).
import requests
from bs4 import BeautifulSoup
url = "https://www.demenagerfacile.com/devis/create"
payload = "volume_set_with_vcalc=false&id_vc_quote=&mode=MOVE_CLASSIC&fromCity=150+Rue+Saint-Maur%2C+Paris%2C+France&locationDeparture=%7B%22lat%22%3A%2248.8690238%22%2C%22lng%22%3A%222.3745715999999675%22%7D&placeIdDeparture=%22ChIJOTRebuRt5kcRoklIrmMZx6Y%22&countryDep=FR&toCity=Rue+de+Marseille%2C+Lyon%2C+France&locationDestination=%7B%22lat%22%3A%2245.7509119%22%2C%22lng%22%3A%224.83963289999997%22%7D&placeIdDestination=%22Eh5SdWUgZGUgTWFyc2VpbGxlLCBMeW9uLCBGcmFuY2UiLiosChQKEgm7NI_oRer0RxEcVzzVtXmrQxIUChIJl4foalHq9EcR8CG75CqrCAQ%22&countryDest=FR&fixedDate=on&date=2019-07-18&volume=30"
headers = {
"content-type": "application/x-www-form-urlencoded",
"cookie": "PHPSESSID=k3f99mg9urimu9dpc1ot7vudf5; __stripe_mid=e5c971f7-5e78-4e61-92db-a62dda48fd2c; _ga=GA1.2.110393239.1561101566; _gid=GA1.2.1597485519.1561101566; previousServerId=0; _fbp=fb.1.1561101566214.588023040; _hjIncludedInSample=1; hubspotutk=e7e9f524c880e1091d415cfc5cbd4903; __hssrc=1; apzContact=%5B%7B%22buttonIdKey%22%3A%229deacb07173c79d76baf4f58c56037ad%22%2C%22apizeeSessionId%22%3A%22334d6232-930f-41ec-aab9-0753c7b3036a%22%2C%22apiconfSession%22%3A%229deacb07173c79d76baf4f58c56037ad%22%2C%22apiconfOfflineMessageState%22%3Anull%7D%5D; cookiebanner-accepted=1; __stripe_sid=5e2fb6df-ea5c-470a-98f3-44e172955633; __hstc=78993675.e7e9f524c880e1091d415cfc5cbd4903.1561101566944.1561101566944.1561106755827.2; _gat_UA-65262710-1=1; __hssc=78993675.3.1561106755827; apiCCId=334d6232-930f-41ec-aab9-0753c7b3036a-0000; apiKey=d7897a4712a9413c3a363e737954d853; sessionId=230a5a70-9401-11e9-935a-19ca5a0a60c1",
}
response = requests.request("POST", url, data=payload)
print(response)

Requests’ simple API means that all forms of HTTP request are as obvious. For example, this is how you make an HTTP POST request
Replace your code
response = requests.request("POST", url, data=payload)
TO
response = requests.post(url,headers=headers, data=payload)

Hi #mhabak1 I just checked and it seems to be working,
Please add headers while posting the request like below,
response = requests.post(url, data=payload, headers=headers)
Thanks

Tomtom webfleet order api gives 40 general error

I am trying to insert a destination order using action=insertDestinationOrder
I am using POST method with all required parameters, but keep getting
{
"errorCode": 40,
"errorMsg": "general error"
}
I have checked using postman too. But still same.
Below is the request using python requests package.
import requests
url = "https://csv.telematics.tomtom.com/extern"
payload = "orderid=TO0049&country=DE&city=Cologne&latitude=50974519&ordertype=delivery%20order&zip=50735&longitude=6977319&street=Am%20Niehler%20Hafen%20%26%20Stapelkai%2C%2050735%20Cologne%20(Niehl)&account=XXXX&username=XXXX&password=XXXX&apikey=XXXX&lang=en&action=insertDestinationOrder&ordertext=Am%20Niehler%20Hafen%20%26%20Stapelkai%2C%2050735%20Cologne%20(Niehl)&useUTF8=true&outputformat=json"
headers = {
'content-type': "application/x-www-form-urlencoded"
}
response = requests.request("POST", url, data=payload, headers=headers)
print(response.text)
Need some help.

Try adding a ? to the end of the url, or to the start of the payload:
url = "https://csv.telematics.tomtom.com/extern?"
or
payload = "?orderid=TO0049&country=DE&city=Cologne&latitude=50974519&ordertype=delivery%20order&zip=50735&longitude=6977319&street=Am%20Niehler%20Hafen%20%26%20Stapelkai%2C%2050735%20Cologne%20(Niehl)&account=XXXX&username=XXXX&password=XXXX&apikey=XXXX&lang=en&action=insertDestinationOrder&ordertext=Am%20Niehler%20Hafen%20%26%20Stapelkai%2C%2050735%20Cologne%20(Niehl)&useUTF8=true&outputformat=json"

Categories