We Keep Coding

Overwrite response from api with python

I would like to get data from api, but api is presented in pages. So I have to iterate through all of them and save wanted data in variable.
I was trying to attach new page in loop and add data to my response, but only I got was error: "TypeError: must be str, not Response". I wanted to do it in this way:
response = "https://api.dane.gov.pl/resources/17201/data?page=1"
for i in range(2,32):
url = "https://api.dane.gov.pl/resources/17201/data?page="+str(i)
response += requests.get(url)
data = response.text
When I get the data I want to extract and operate on them.

requests.get(url) returns a Response object. At the moment, you are trying to add the Response object to a string.
Try something like this:
response = []
for i in range(2,32):
url = "https://api.dane.gov.pl/resources/17201/data?page="+str(i)
response.append(requests.get(url).text)
When that finishes running, response will be a list full of the response text instead of response objects.

PUT method using python3 and urbllib - headers

So I am trying to just receive the data from this json. I to use POST, GET on any link but the link I am currently trying to read. It needs [PUT]. So I wanted to know if I was calling this url correctly via urllib or am I missing something?
Request
{"DataType":"Word","Params":["1234"], "ID":"22"}
Response {
JSON DATA IN HERE
}
I feel like I am doing the PUT method call wrong since it is wrapped around Request{}.
import urllib.request, json
from pprint import pprint
header = {"DataType":"Word","Params":"[1234"]", "ID":"22"}
req = urllib.request.Request(url = "website/api/json.service", headers =
heaer, method = 'PUT')
with urllib.request.urlopen(req) as url:
data = json.loads(url.read(), decode())
pprint(data)
I am able to print json data as long as its anything but PUT. As soon as I get a site with put on it with the following JSON template I get an Internal Error 500. So I assumed it was was my header.
Thank you in advanced!

Obtain both headers and content from single GET request with the Python requests library

I am trying to, with a single get request, obtain both headers and content.
At the moment, I am able to obtain them individually:
Headers=requests.get('https://coinmarketcap.com', verify=False).headers
and
ParseLM=requests.get('https://coinmarketcap.com', verify=False).content
However, this makes two separate GET requests while I am trying to parse both headers and content from the same request, although separately.

Call requests.get() once, saving the entire result:
response = requests.get('https://coinmarketcap.com', verify=False)
Then you can access individual pieces of the result:
headers = response.headers
content = response.content

POST request via requests (python) not returning data

I have another question about posts.
This post should be almost identical to one referenced on stack overflow using this question 'Using request.post to post multipart form data via python not working', but for some reason I can't get it to work. The website is http://www.camp.bicnirrh.res.in/predict/. I want to post a file that is already in the FASTA format to this website and select the 'SVM' option using requests in python. This is based on what #NorthCat gave me previously, which worked like a charm:
import requests
import urllib
file={'file':(open('Bishop/newdenovo2.txt','r').read())}
url = 'http://www.camp.bicnirrh.res.in/predict/hii.php'
payload = {"algo[]":"svm"}
raw = urllib.urlencode(payload)
response = session.post(url, files=file, data=payload)
print(response.text)
Since it's not working, I assumed the payload was the problem. I've been playing with the payload, but I can't get any of these to work.
payload = {'S1':str(data), 'filename':'', 'algo[]':'svm'} # where I tried just reading the file in, called 'data'
payload = {'svm':'svm'} # not actually in the headers, but I tried this too)
payload = {'S1': '', 'algo[]':'svm', 'B1': 'Submit'}
None of these payloads resulted in data.
Any help is appreciated. Thanks so much!

You need to set the file post variable name to "userfile", i.e.
file={'userfile':(open('Bishop/newdenovo2.txt','r').read())}
Note that the read() is unnecessary, but it doesn't prevent the file upload succeeding. Here is some code that should work for you:
import requests
session = requests.session()
response = session.post('http://www.camp.bicnirrh.res.in/predict/hii.php',
files={'userfile': ('fasta.txt', open('fasta.txt'), 'text/plain')},
data={'algo[]':'svm'})
response.text contains the HTML results, save it to a file and view it in your browser, or parse it with something like Beautiful Soup and extract the results.
In the request I've specified a mime type of "text/plain" for the file. This is not necessary, but it serves as documentation and might help the receiving server.
The content of my fasta.txt file is:
>24.6jsd2.Tut
GGTGTTGATCATGGCTCAGGACAAACGCTGGCGGCGTGCTTAATACATGCAAGTCGAACGGGCTACCTTCGGGTAGCTAGTGGCGGACGGGTGAGTAACACGTAGGTTTTCTGCCCAATAGTGGGGAATAACAGCTCGAAAGAGTTGCTAATACCGCATAAGCTCTCTTGCGTGGGCAGGAGAGGAAACCCCAGGAGCAATTCTGGGGGCTATAGGAGGAGCCTGCGGCGGATTAGCTAGATGGTGGGGTAAAGGCCTACCATGGCGACGATCCGTAGCTGGTCTGAGAGGACGGCCAGCCACACTGGGACTGAGACACGGCCCAGACTCCTACGGGAGGCAGCAGTAAGGAATATTCCACAATGGCCGAAAGCGTGATGGAGCGAAACCGCGTGCGGGAGGAAGCCTTTCGGGGTGTAAACCGCTTTTAGGGGAGATGAAACGCCACCGTAAGGTGGCTAAGACAGTACCCCCTGAATAAGCATCGGCTAACTACGTGCCAGCAGCCGCGGTAATACGTAGGATGCAAGCGTTGTCCGGATTTACTGGGCGTAAAGCGCGCGCAGGCGGCAGGTTAAGTAAGGTGTGAAATCTCCCTGCTCAACGGGGAGGGTGCACTCCAGACTGACCAGCTAGAGGACGGTAGAGGGTGGTGGAATTGCTGGTGTAGCGGTGAAATGCGTAGAGATCAGCAGGAACACCCGTGGCGAAGGCGGCCACCTGGGCCGTACCTGACGCTGAGGCGCGAAGGCTAGGGGAGCGAACGGGATTAGATACCCCGGTAGTCCTAGCAGTAAACGATGTCCACTAGGTGTGGGGGGTTGTTGACCCCTTCCGTGCCGAAGCCAACGCATTAAGTGGACCGCCTGGGGAGTACGGTCGCAAGACTAAAACTCAAAGGAATTGACGGGGACCCGCACAAGCAGCGGAGCGTGTGGTTTAATTCGATGCGACGCGAAGAACCTTACCTGGGCTTGACATGCTATCGCAACACCCTGAAAGGGGTGCCTCCTTCGGGACGGTAGCACAGATGCTGCATGGCTGTCGTCAGCTCGTGTCGTGAGATGTTGGGTTAAGTCCCGCAACGAGCGCAACCCCTGTCCTTAGTTGTATATCTAAGGAGACTGCCGGAGACAAACCGGAGGAAGGTGGGGATGACGTCAAGTCAGCATGGCTCTTACGTCCAGGGCTACACATACGCTACAATGGCCGTTACAGTGAGATGCCACACCGCGAGGTGGAGCAGATCTCCAAAGGCGGCCTCAGTTCAGATTGCACTCTGCAACCCGAGTGCATGAAGTCGGAGTTGCTAGTAACCGCGTGTCAGCATAGCGCGGTGAATATGTTCCCGGGTCTTGTACACACCGCCCGTCACGTCATGGGAGCCGGCAACACTTCGAGTCCGTGAGCTAACCCCCCCTTTCGAGGGTGTGGGAGGCAGCGGCCGAGGGTGGGGCTGGTGACTGGGACGAAGTCGTAACAAGGT

TypeError with urlopen()

I am a bit confused with using Request, urlopen and JSONDecoder().decode().
Currently I have:
hdr = {'User-agent' : 'anything'} # header, User-agent header describes my web browser
I am assuming that the server uses this to determine which browsers are acceptable? Not sure
my url is:
url = 'http://wwww.reddit.com/r/aww.json'
I set a req variable
req = Request(url,hdr) #request to access the url with header
json = urlopen(req).read() # read json page
I tried using urlopen in terminal and I get this error:
TypeError: must be string or buffer, not dict # This has to do with me header?
data = JSONDecoder().decode(json) # translate json data so I can parse through it with regular python functions?
I'm not really sure why I get the TypeError

If you look at the documentation of Request, you can see that the constructor signature is actually Request(url, data=None, headers={}, …). So the second parameter, the one after the URL, is the data you are sending with the request. But if you want to set the headers instead, you will have to specify the headers parameter.
You can do this in two different ways. Either you pass None as the data parameter:
Request(url, None, hdr)
But, well, this requires you to pass the data parameter explicitely and you have to make sure that you pass the default value to not cause any unwanted effects. So instead, you can tell Python to explicitely pass the header parameter instead, without specifying data:
Request(url, headers=hdr)