Normally a request would be formatted in Python as:
varName = {'username': 'password'}
The normal request would be:
username=password
My problem is the request that I am trying to send is:
{"part1":"part2"}
I don't know how to format this for python. I simply need assistance with formatting it. Thank you.
Related
Can someone help me please I am having a difficult time trying to figure out how to send a dictionary in the params of a Get Request using Python's Request Library. Below is what the Param looks like in inspect element. I have tried several different combinations to get this work, and i keep getting an error from the API saying {'error_details': "required selection'sellout_inclusion' is missing.",'data': None}. Any help on this woudl be much appreciated. Thanks.
Param
selections: {sellout_inclusion: ["Include All"]}
To send a get request to the server you can define a dictionary with params:
Example:
import requests
PARAMS = {"selections": {'sellout_inclusion': ["Include All"]}}
res = requests.get('https://www.google.com/', params=PARAMS)
print(res.text)
the problem can be with the API you request data from
you can read this article maybe it will help if my example didn't
https://jakevdp.github.io/PythonDataScienceHandbook/03.04-missing-values.html
I am a new programmer and I'm learning the request module. I'm stuck on the fact that I don't know how to get a specific part of a json response, I think it's called a header? or its the thing inside of a header? I'm not sure. But the API returns simple json code. This is the api
https://mcapi.us/server/status?ip=mc.hypixel.net
for more of a example, lets say it returns this json code from the api
{"status":"success","online":true"}
And I wanted to get the "online" response, how would I do that?
And this is the code im currently working with.
import requests
def main():
ask = input("IP : ")
response = requests.get('https://mcapi.us/server/status?ip=' + ask)
print(response.content)
main()
And to be honest, I don't even know if this is json. I think it is but the api page says its cors? if it isn't I'm sorry.
In your example you have a dictionary with key "online"
You need to parse it first with .json() and then you can get it in form dict[key]
In your case
response = requests.get('https://mcapi.us/server/status?ip=' + ask).json()
print(response["online"])
or in case of actual content
response = requests.get('https://mcapi.us/server/status?ip=' + ask).json()
print(response["content"])
I'm trying to understand what exactly I'm getting back when I make a POST request using the Requests module — is it always JSON? Seems like every response I get appears to be JSON, but I'm not sure.
Where r is my response object, when I do:
print r.apparent_encoding
It always seems to return ascii
And when I try type():
>>>print type(r)
<class 'requests.models.Response'
I pasted the output from print r.text into a JSON validator, and it reported no errors. So should I assume Requests is providing my with JSON objects here?
A response can be anything. If you've posted to a REST endpoint, it will usually respond with JSON. If so, requests will detect that and allow you to decode it via the .json() method.
But it's perfectly possible for you to post to a normal web URL, in effect pretending to be a browser, and unless the server is doing something really clever it will just respond with the standard HTML it would serve to the browser. In that case, doing response.json() will raise a ValueError.
No, the response text for a POST request is totally up to the web service. A good REST API will always respond with JSON, but you will not always get that.
Example
A common pattern in PHP is
<?php
$successful_whatever = false;
if (isset($_POST['whatever'])) {
# put $_POST['whatever'] in a database
$successful_whatever = true;
}
echo $twig->render('gallery.twig',
array('successful_whatever' => $successful_whatever));
?>
As you can see the response text will be a rendered template (HTML). I'm not saying it is good, just that it is common.
I have a site that I want to extract data from. The data retrieval is very straight forward.
It takes the parameters using HTTP POST and returns a JSON object. So, I have a list of queries that I want to do and then repeat at certain intervals to update a database. Is scrapy suitable for this or should I be using something else?
I don't actually need to follow links but I do need to send multiple requests at the same time.
How does looks like the POST request? There are many variations, like simple query parameters (?a=1&b=2), form-like payload (the body contains a=1&b=2), or any other kind of payload (the body contains a string in some format, like json or xml).
In scrapy is fairly straightforward to make POST requests, see: http://doc.scrapy.org/en/latest/topics/request-response.html#request-usage-examples
For example, you may need something like this:
# Warning: take care of the undefined variables and modules!
def start_requests(self):
payload = {"a": 1, "b": 2}
yield Request(url, self.parse_data, method="POST", body=urllib.urlencode(payload))
def parse_data(self, response):
# do stuff with data...
data = json.loads(response.body)
For handling requests and retrieving response, scrapy is more than enough. And to parse JSON, just use the json module in the standard library:
import json
data = ...
json_data = json.loads(data)
Hope this helps!
Based on my understanding of the question, you just want to fetch/scrape data from a web page at certain intervals. Scrapy is generally used for crawling.
If you just want to make http post requests you might consider using the python requests library.
I would like to create a python script that makes a series of web requests to endpoints that will spit out JSON. After receiving the response, I want to validate that it is well-formed JSON, and not some error page. Also, I will need to insert an API key into the request header to get a proper response. what is the best way to go about doing this in python?
Thanks!
To validate the JSON you can use http://python-jsonschema.readthedocs.org/en/latest/
To insert the API key in the header you can use Requests http://docs.python-requests.org/en/latest/user/quickstart/
Validate the json use json.dump(), if your json object incorect it's return the error, handle the error using try catch and answer to your second question , i think you talking about X-AUTH-TOKEN , if your request return json object use this header.
headers = {
"Content-Type": "application/json",
"X-AUTH-TOKEN": your API Token
}
otherwise use
"Content-type":"application/x-www-form-urlencoded"