I want to loop over the content of a JSON file and print it to the console.
I think I did mix up something with lists.
This is what I tried to get all the team_name elements
from urllib2 import urlopen
import json
url = 'http://openligadb-json.heroku.com/api/teams_by_league_saison?league_saison=2012&league_shortcut=bl1'
response = urlopen(url)
json_obj = json.load(response)
for i in json_obj['team']:
print i
And this is my JSON (simplified:)
{
"team": [
{
"team_icon_url": "http://www.openligadb.de/images/teamicons/Hamburger_SV.gif",
"team_id": "100",
"team_name": "Hamburger SV"
},
{
"team_icon_url": "http://www.openligadb.de/images/teamicons/FC_Schalke_04.gif",
"team_id": "9",
"team_name": "FC Schalke 04"
}
]
}
(Full JSON output to be found here: Link)
And of course I get an error, that I should use integer input in [], not string, but I don't get how I could do that.
for i in json_obj['team']:
TypeError: string indices must be integers, not str
Here is the response:
http://openligadb-json.heroku.com/api/teams_by_league_saison?league_saison=2012&league_shortcut=bl1
<addinfourl at 139755086292608 whose fp = <socket._fileobject object at 0x7f1b446d33d0>>
What did I get wrong?
Actually, to query the team_name, just add it in brackets to the last line. Apart from that, it seems to work on Python 2.7.3 on command line.
from urllib2 import urlopen
import json
url = 'http://openligadb-json.heroku.com/api/teams_by_league_saison?league_saison=2012&league_shortcut=bl1'
response = urlopen(url)
json_obj = json.load(response)
for i in json_obj['team']:
print i['team_name']
Try this :
import urllib, urllib2, json
url = 'http://openligadb-json.heroku.com/api/teams_by_league_saison?league_saison=2012&league_shortcut=bl1'
request = urllib2.Request(url)
request.add_header('User-Agent','Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)')
request.add_header('Content-Type','application/json')
response = urllib2.urlopen(request)
json_object = json.load(response)
#print json_object['results']
if json_object['team'] == []:
print 'No Data!'
else:
for rows in json_object['team']:
print 'Team ID:' + rows['team_id']
print 'Team Name:' + rows['team_name']
print 'Team URL:' + rows['team_icon_url']
To decode json, you have to pass the json string. Currently you're trying to pass an object:
>>> response = urlopen(url)
>>> response
<addinfourl at 2146100812 whose fp = <socket._fileobject object at 0x7fe8cc2c>>
You can fetch the data with response.read().
Related
My Json is like this
[
{
"name":"abc"
"emai":"hotmail"
},
{
"name":"def"
"emai":"gmail"
}
]
I have to read one record at time and post request
in 1st post request with data
'{
"name":"abc"
"emai":"hotmail"
}'
in 2nd post request with data
'{
"name":"def"
"emai":"gmail"
}'
i tried with this code,
import json
import requests
from requests.structures import CaseInsensitiveDict
url = "https://testurl.com"
headers = CaseInsensitiveDict()
headers["accept"] = "text/plain"
headers["username"] = "username"
headers["password"] = "password"
headers["Content-Type"] = "application/json"
with open("data.json", encoding='utf-8', errors='ignore') as json_data:
for i in json_data:
data = json.load(i, strict=False)
idata = json.dumps(data) ## converting to string
fData = idata.replace("'", '"') # replacing ' with "
output = "'"+fData+"'" ## placing ' before and after json string which is mandatory in my case
print(fData)
# resp = requests.post(url, headers=headers, data=fData)
# print(f"status_code: : {resp.status_code}, response : {resp.json}")
I'm getting error 'str' object has no attribute error, can you please suggest me what i have to change ?
load all json, then loop through it
import json
import requests
from requests.structures import CaseInsensitiveDict
url = "https://testurl.com"
headers = CaseInsensitiveDict()
headers["accept"] = "text/plain"
headers["username"] = "username"
headers["password"] = "password"
headers["Content-Type"] = "application/json"
with open("data.json", encoding='utf-8', errors='ignore') as json_string:
json_data = json.load(json_string.read(), strict=False)
for i in json_data:
data = i
idata = json.dumps(data) ## converting to string
fData = idata.replace("'", '"') # replacing ' with "
output = "'"+fData+"'" ## placing ' before and after json string which is mandatory in my case
print(fData)
# resp = requests.post(url, headers=headers, data=fData)
# print(f"status_code: : {resp.status_code}, response : {resp.json}")
I guess you want to convert your JSON data to python structure first and then process it, something like:
import json
with open("data.json", encoding='utf-8', errors='ignore') as json_data:
data = json.loads(json_data.read())
for obj in data:
resp = requests.post(url, headers=headers, data=obj)
print(f"status_code: : {resp.status_code}, response : {resp.json}")
I have a string like this below
data = '[{"cId": "y02", "name": "Test 02", "description": "My Test"}]'
I am sending this data in a http request like
import requests
headers = {
'Content-Type': 'application/json',
}
response = requests.post('http://localhost:8000/targets', headers=headers,data=data, auth=('user', 'pass'))
Now what i would like to do is e.g. i have a string in a variable like..
id=str('random string')
and i would like to add it in data like this..
data = '[{"cId": id, "name": "Test 02", "description": "My Test"}]'
But i am not able to add it. I have tried to first convert the entry in json and then adding it into the array. But server sends the exception back. How can i do that?
You could convert it to a list of dicts.Then convert it to str.(With json module):
import json
data_before = '[{"cId": "y02", "name": "Test 02", "description": "My Test"}]'
tmp = json.loads(data_before)
tmp[0]["cId"] = str('random string')
data_after = json.dumps(tmp)
Then use data_after it like before:
import requests
headers = {
'Content-Type': 'application/json',
}
response = requests.post('http://localhost:8000/targets', headers=headers, data=data_after, auth=('user', 'pass'))
Or pass it as json parameter directly in requests.post:
import json
data_before = '[{"cId": "y02", "name": "Test 02", "description": "My Test"}]'
data_after = json.loads(data_before)
data_after[0]["cId"] = str('random string')
Then do:
import requests
headers = {
'Content-Type': 'application/json',
}
response = requests.post('http://localhost:8000/targets', headers=headers, json=data_after, auth=('user', 'pass'))
data in request.post() should be a :
Dictionary, list of tuples, bytes, or file-like object to send in the body of the :class:Request.
If your data is json formatted, you can load the data and convert the string into a dict using json.load("<str in json format>").
I'm trying to parse a website with the requests module:
import requests
some_data = {'a':'',
'b':''}
with requests.Session() as s:
result = s.post('http://website.com',data=some_data)
print(result.text)
The page is responding as below:
{
"arrangetype":"U",
"list": [
{
"product_no":43,
"display_order":4,
"is_selling":"T",
"product_empty":"F",
"fix_position":null,
"is_auto_sort":false
},
{
"product_no":44,
"display_order":6,
"is_selling":"T",
"product_empty":"F",
"fix_position":null,
"is_auto_sort":false
}
],
"length":2
}
I found that instead of parsing full HTML, it would be better to deal with the response as all the data I want is in that response.
What I want to get is a list of the values of product_no, so the expected result is:
[43,44]
How do I do this?
Convert your JSON response to a dictionary with json.loads(), and collect your results in a list comprehension.
Demo:
from json import loads
data = """{
"arrangetype":"U",
"list": [
{
"product_no":43,
"display_order":4,
"is_selling":"T",
"product_empty":"F",
"fix_position":null,
"is_auto_sort":false
},
{
"product_no":44,
"display_order":6,
"is_selling":"T",
"product_empty":"F",
"fix_position":null,
"is_auto_sort":false
}
],
"length":2
}"""
json_dict = loads(data)
print([x['product_no'] for x in json_dict['list']])
# [43, 44]
Full Code:
import requests
from json import loads
some_data = {'a':'',
'b':''}
with requests.Session() as s:
result = s.post('http://website.com',data=some_data)
json_dict = loads(result.text)
print([x["product_no"] for x in json_dict["list"]])
Am working with an API that asks to specify a file (Excel in my case) to upload to the API. Documentation specifies: JSON Property = file, Data Type = FileUpload. My question: What is FileUpload. I tried simply specifying the name of the file (e.g. c:\test\Data.xls) but obviously that does not work.
I am working with an API for FastField Mobile Forms (www.fastfield.com). Documentation shown in screen shot, as is code and result. Somehow, I am not posting the file data to the API correctly.
So this is the Python code I am attempting to run
import requests
import json
from requests.auth import HTTPBasicAuth
import base64
# Get session token, this must be specified in header of subsequent request and returns a JSON object ready for insertion into header
rqstResponse = requests.post('https://manage.fastfieldforms.com/api/authenticate', auth=HTTPBasicAuth('***', '***'))
jsonObj = json.loads(rqstResponse.content)
sessionToken = jsonObj['data']['sessionToken']
headers = {'X-Gatekeeper-SessionToken': sessionToken}
# run this code to get listIds - which are hard coded further down
rqstResponse = requests.get("https://manage.fastfieldforms.com/api/globallists", headers=headers)
print (rqstResponse.content)
del rqstResponse
# Read file and convert to binary string
filePath = r"J:\Properties\PropGIS\proj\20150820140457_TelecoMapping\data\Survey_Feb17\FastField_Test01.xlsx"
with open(filePath, 'r') as f:
filecontents = f.read()
fileDataEncoded = base64.b64encode(filecontents)
# create JSON
payloadDictObj = {}
payloadDictObj['file'] = fileDataEncoded
payloadDictObj['id'] = "03c804cb-b983-4e4c-956b-96ac23da16b2"
#payloadDictObj['listname'] = "Test02"
serializedJsonStr = json.dumps(payloadDictObj)
print serializedJsonStr
# Update Global List
rqstResponse = requests.post("https://manage.fastfieldforms.com/api//globallist", data=serializedJsonStr, headers=headers)
print (rqstResponse.content)
del rqstResponse
# --------------------
# Response
{
"code": 200,
"data": {
"searchResults": [
{
"id": 7793,
"accountId": 43600,
"name": "Test01",
"active": true,
"createdAt": "2017-05-24T06:37:28.49Z",
"updatedAt": "2017-05-24T06:37:28.49Z",
"version": 1,
"listId": "03c804cb-b983-4e4c-956b-96ac23da16b2",
"path": "{ bucket:'fastfield-globallists', key:'43600/ca4b89df75db4ef8b513d15d59f654d8.csv'}"
}
]
}
}
{"id": "03c804cb-b983-4e4c-956b-96ac23da16b2", "file": "UEsDB...qaJXQ=="}
{
"code": 403,
"error": "listname name is required",
"data": {}
}
OK. For what it's worth. This is how I finally got this to work. I now realise that this question was more about my understanding, or lack thereof, of the Python request module.
import requests
import json
from requests.auth import HTTPBasicAuth
import base64
# Get session token, this must be specified in header of subsequent request and returns a JSON object ready for insertion into header
rqstResponse = requests.post('https://manage.fastfieldforms.com/api/authenticate', auth=HTTPBasicAuth('XXX', 'XXX'))
jsonObj = json.loads(rqstResponse.content)
sessionToken = jsonObj['data']['sessionToken']
headers = {'X-Gatekeeper-SessionToken': sessionToken}
def getGloballistsDetails():
# run this code to get listIds - which are hard coded further down
rqstResponse = requests.get("https://manage.fastfieldforms.com/api/globallists", headers=headers)
print (rqstResponse.content)
del rqstResponse
def createGloballist(listname, filePath):
# usage example createGloballist("test01", r"c:\temp\test.xlsx")
files = {'file': open(filePath, 'rb')}
data = {'listname': listname}
rqstResponse = requests.post("https://manage.fastfieldforms.com/api//globallist", files=files, data=data, headers=headers)
print (rqstResponse.content)
del rqstResponse
def updateGloballist(id, filePath):
# usage example createGloballist("f03c7db1-cfea-4486-8350-53381ac048b4", r"c:\temp\test.xlsx")
files = files = {'file': open(filePath, 'rb')}
data = {'id': id}
rqstResponse = requests.post("https://manage.fastfieldforms.com/api//globallist", files=files, data=data, headers=headers)
print (rqstResponse.content)
del rqstResponse
filePath = r"J:\Properties\PropGIS\proj\20150820140457_TelecoMapping\data\Survey_Feb17\FastField_Test01.xlsx"
getGloballistsDetails()
#createGloballist("Test02", filePath)
updateGloballist('f03c7db1-cfea-4486-8350-53381ac048b4', filePath)
In Python 3.4 I'm using Requests and a for loop to combine the body for a multiple item API request fetching JSON content. It yields a HTTP 400 error even if the body variable is recognized as a str class with type(). However if I print and copy the content into a new variable it is successful. What kind of formatting is going on?
import requests,json
list_length = len(namelist) #namelist arranged earlier in code
payload='['
for x in range(0, list_length):
payload += '{"name": "'+ namelist[x] + '"}'
if x<list_length-1:
payload += ', '
payload += ']'
url = 'http://api.turfgame.com/v4/users'
headers = {'Content-Type': 'application/json'}
req = requests.post(url, data=json.dumps(payload),headers=headers)
>>> payload
'[{"name": "sune"}, {"name": "Demon"}, {"name": "kingenin"}]'
You are creating a JSON string, then encode it as a JSON string. This double encoding is not what you'll want here:
>>> payload = '[{"name": "sune"}, {"name": "Demon"}, {"name": "kingenin"}]'
>>> print(json.dumps(payload))
"[{\"name\": \"sune\"}, {\"name\": \"Demon\"}, {\"name\": \"kingenin\"}]"
That's a JSON string, containing a quoted JSON list..
Build a list, and pass that to json.dumps():
payload = [{'name': name} for name in namelist]
url = 'http://api.turfgame.com/v4/users'
headers = {'Content-Type': 'application/json'}
req = requests.post(url, data=json.dumps(payload),headers=headers)
This sends a proper JSON list instead:
>>> payload
[{'name': 'sune'}, {'name': 'Demon'}, {'name': 'kingenin'}]
>>> print(json.dumps(payload))
[{"name": "sune"}, {"name": "Demon"}, {"name": "kingenin"}]
You could also send payload as you built it, without passing it to json.dumps(), but why have a dog and bark yourself at all?
If you were to use requests version 2.4.2 or up, you can have it handle the JSON encoding for you; pass in the Python object into the json keyword argument and it'll even set the correct Content-Type header:
payload = [{'name': name} for name in namelist]
url = 'http://api.turfgame.com/v4/users'
req = requests.post(url, json=payload)