I have the following code to load JSON:
import json
import requests
r = requests.get('http://api.reddit.com/controversial?limit=5')
if r.status_code = 200:
reddit_data = json.loads(r.content)
print reddit_data['data']['children'][1]['data']
else:
print "Errror."
And I got this message.
arsh#arsh:~$ python q.py
Traceback (most recent call last):
File "q.py", line 1, in <module>
import json
File "/home/arsh/json.py", line 5, in <module>
reddit_data = json.loads(r.content)
AttributeError: 'module' object has no attribute 'loads'
You have a different file called json.py in your home directory:
File "/home/arsh/json.py", line 5, in <module>
This file is in the way, you did not import the standard library version. Rename it to something else or delete it. You'll also have to remove the json.pyc file.
Note that requests response objects can already handle JSON responses for you:
import requests
r = requests.get('http://api.reddit.com/controversial?limit=5')
r.raise_for_status()
reddit_data = r.json()
print reddit_data['data']['children'][1]['data']
The Response.json() method handles decoding JSON for you, including detecting the correct characterset to use when decoding.
Related
In python 3.6.8 I am trying to download a 'file' from a URL and process it directly, without creating a local file. I have tried the following code
import io
import requests
url = "https://raw.githubusercontent.com/enzoftware/random/master/README.md"
response = requests.get(url, stream=True)
with io.BytesIO(response.text) as f:
print(f.readlines())
but I get an error
Traceback (most recent call last):
File "tester.py", line 7, in <module>
with io.BytesIO(response.text) as f:
TypeError: a bytes-like object is required, not 'str'
How to do it right?
assuming you just want to read it line by line rather than considering any document (html) structure it may have you can just do
import requests
url = "https://raw.githubusercontent.com/enzoftware/random/master/README.md"
response = requests.get(url, stream=True)
for line in response.text.splitlines():
print (line)
I have an issue with json on Python3.
I try to get the "pairs" from string of URL: https://wex.nz/api/3/info
This is my code:
import urllib.request
import json
url= 'https://wex.nz/api/3/info'
content=urllib.request.urlopen(url)
for line in content:
liste=json.loads(line)
pairliste=liste['pairs']
print(pairliste)
This is my error:
Traceback (most recent call last):
File "/home/lennart/Documents/Test/Test.py", line 8, in <module>
liste=json.loads(line)
File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
Thanks to roganjosh.
This fixed my issue:
import urllib.request
import json
url= 'https://wex.nz/api/3/info'
content = urllib.request.urlopen(url)
data =json.loads(content.read().decode())
print(data)
I am partially able to work with json saved as file:
#! /usr/bin/python3
import json
from pprint import pprint
json_file='a.json'
json_data=open(json_file)
data = json.load(json_data)
json_data.close()
print(data[10])
But I am trying to achieve the same from data directly from web. I am trying with the accepted answer here:
#! /usr/bin/python3
from urllib.request import urlopen
import json
from pprint import pprint
jsonget=urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee")
data = json.load(jsonget)
pprint(data)
which is giving me error:
Traceback (most recent call last):
File "i.py", line 10, in <module>
data = json.load(jsonget)
File "/usr/lib64/python3.5/json/__init__.py", line 268, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.5/json/__init__.py", line 312, in loads
s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
What is going wrong here?
Changing the code as par Charlie's reply to:
jsonget=str(urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee"))
data = json.load(jsonget)
pprint(jsonget)
breaks at json.load:
Traceback (most recent call last):
File "i.py", line 9, in <module>
data = json.load(jsonget)
File "/usr/lib64/python3.5/json/__init__.py", line 265, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
It's actually telling you the answer: you're getting back a byte array, where in Python 3 a string is different because of dealing with unicode. In Python 2.7, it would work. You should be able to fix it by converting your bytes explicitly to a string with
jsonget=str(urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee")_
I am using python 2.7 and urllib2 command for doing this.But I am facing error that urllib2 has no attribute name urlopen.Please help me.thanx,Here is my code.
import urllib2
import re
pat = re.compile('target="_parent">(.*?)</a>')
url = 'http://timesofindia.indiatimes.com/home/headlines'
sock = urllib2.urlopen(url)
li = pat.findall(sock.read())
sock.close()
print li
f=open("headlines.txt", 'a+')
for i in range(len(li)):
f.write(li[i]+"\n")
f.close()
Traceback
Error:Traceback (most recent call last):
File "C:/Users/Training/PycharmProjects/758702_Python_Program/ReadTOI/ReadTOI.py", line 1, in <module>
import urllib2
File "C:\Python27\lib\urllib2.py", line 111, in <module>
from urllib import (unwrap, unquote, splittype, splithost, quote,
File "C:\Users\Training\PycharmProjects\758702_Python_Program\urllib.py", line 4, in <module>
f=urllib.urlopen("http://www.python.org/")
AttributeError: 'module' object has no attribute 'urlopen'
File "C:\Users\Training\PycharmProjects\758702_Python_Program\urllib.py", line 4, in <module>
It looks like you have an other file by the name urllib.py in your working directory. When doing an import urllib python is importing your local file which does not contain urlopen(). Renaming your local file to something else or changing your working directory would solve this
Python 3.3.2 import json & urllib.request
Json
[{"link":"www.google.com","orderid":"100000222"},
{"link":"www.google.com","orderid":"100000222"},
{"link":"www.google.com","orderid":"100000222"}]
print(response.info())
Date: Sun, 20 Oct 2013 07:06:51 GMT
Server: Apache
X-Powered-By: PHP/5.4.12
Content-Length: 145
Connection: close
Content-Type: application/json
Codes
url = "http://www.Link.com"
request = urllib.request.Request(url)
request.add_header('User-Agent','Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)')
request.add_header('Content-Type','application/json')
response = urllib.request.urlopen(request)
decodedRes = response.read().decode('utf-8')
json_object = json.load(decodedRes)
The following are my codes
Error
Traceback (most recent call last):
File "C:\Users\Jonathan\Desktop\python.py", line 57, in <module>
checkLink()
File "C:\Users\Jonathan\Desktop\python.py", line 50, in checkLink
json_object = json.load(decodedRes)
File "C:\Python33\lib\json\__init__.py", line 271, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
>>> .
Any idea how i can fix this issue?
Use json.loads instead of json.load.
json.loads(decodedRes)
json.load accept file-like object.
>>> import json
>>> json.load('{"a": 1}')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\json\__init__.py", line 286, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
>>> json.loads('{"a": 1}')
{u'a': 1}
Alternatively you can pass response object to json.load:
## decodedRes = response.read().decode('utf-8')
json_object = json.load(response)
Try replacing the json.load() with json.loads() . The former needs a file-stream thats the reason you are running into the attribute error.
Following code loads json file into document. Document will have values of dict type.
file_name = "my_file.json"
with open(file_name, 'r') as f:
document = json.loads(f.read())
It's very simple, You just need to import beautiful soup in another way.
Right now you are importing as
from beautifulSoup import BeautifulSoup
Change it to
from bs4 import BeautifulSoup