Python 3.3.2 import json & urllib.request
Json
[{"link":"www.google.com","orderid":"100000222"},
{"link":"www.google.com","orderid":"100000222"},
{"link":"www.google.com","orderid":"100000222"}]
print(response.info())
Date: Sun, 20 Oct 2013 07:06:51 GMT
Server: Apache
X-Powered-By: PHP/5.4.12
Content-Length: 145
Connection: close
Content-Type: application/json
Codes
url = "http://www.Link.com"
request = urllib.request.Request(url)
request.add_header('User-Agent','Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)')
request.add_header('Content-Type','application/json')
response = urllib.request.urlopen(request)
decodedRes = response.read().decode('utf-8')
json_object = json.load(decodedRes)
The following are my codes
Error
Traceback (most recent call last):
File "C:\Users\Jonathan\Desktop\python.py", line 57, in <module>
checkLink()
File "C:\Users\Jonathan\Desktop\python.py", line 50, in checkLink
json_object = json.load(decodedRes)
File "C:\Python33\lib\json\__init__.py", line 271, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
>>> .
Any idea how i can fix this issue?
Use json.loads instead of json.load.
json.loads(decodedRes)
json.load accept file-like object.
>>> import json
>>> json.load('{"a": 1}')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\json\__init__.py", line 286, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
>>> json.loads('{"a": 1}')
{u'a': 1}
Alternatively you can pass response object to json.load:
## decodedRes = response.read().decode('utf-8')
json_object = json.load(response)
Try replacing the json.load() with json.loads() . The former needs a file-stream thats the reason you are running into the attribute error.
Following code loads json file into document. Document will have values of dict type.
file_name = "my_file.json"
with open(file_name, 'r') as f:
document = json.loads(f.read())
It's very simple, You just need to import beautiful soup in another way.
Right now you are importing as
from beautifulSoup import BeautifulSoup
Change it to
from bs4 import BeautifulSoup
Related
In python 3.6.8 I am trying to download a 'file' from a URL and process it directly, without creating a local file. I have tried the following code
import io
import requests
url = "https://raw.githubusercontent.com/enzoftware/random/master/README.md"
response = requests.get(url, stream=True)
with io.BytesIO(response.text) as f:
print(f.readlines())
but I get an error
Traceback (most recent call last):
File "tester.py", line 7, in <module>
with io.BytesIO(response.text) as f:
TypeError: a bytes-like object is required, not 'str'
How to do it right?
assuming you just want to read it line by line rather than considering any document (html) structure it may have you can just do
import requests
url = "https://raw.githubusercontent.com/enzoftware/random/master/README.md"
response = requests.get(url, stream=True)
for line in response.text.splitlines():
print (line)
I have an issue with json on Python3.
I try to get the "pairs" from string of URL: https://wex.nz/api/3/info
This is my code:
import urllib.request
import json
url= 'https://wex.nz/api/3/info'
content=urllib.request.urlopen(url)
for line in content:
liste=json.loads(line)
pairliste=liste['pairs']
print(pairliste)
This is my error:
Traceback (most recent call last):
File "/home/lennart/Documents/Test/Test.py", line 8, in <module>
liste=json.loads(line)
File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
Thanks to roganjosh.
This fixed my issue:
import urllib.request
import json
url= 'https://wex.nz/api/3/info'
content = urllib.request.urlopen(url)
data =json.loads(content.read().decode())
print(data)
I am partially able to work with json saved as file:
#! /usr/bin/python3
import json
from pprint import pprint
json_file='a.json'
json_data=open(json_file)
data = json.load(json_data)
json_data.close()
print(data[10])
But I am trying to achieve the same from data directly from web. I am trying with the accepted answer here:
#! /usr/bin/python3
from urllib.request import urlopen
import json
from pprint import pprint
jsonget=urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee")
data = json.load(jsonget)
pprint(data)
which is giving me error:
Traceback (most recent call last):
File "i.py", line 10, in <module>
data = json.load(jsonget)
File "/usr/lib64/python3.5/json/__init__.py", line 268, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.5/json/__init__.py", line 312, in loads
s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
What is going wrong here?
Changing the code as par Charlie's reply to:
jsonget=str(urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee"))
data = json.load(jsonget)
pprint(jsonget)
breaks at json.load:
Traceback (most recent call last):
File "i.py", line 9, in <module>
data = json.load(jsonget)
File "/usr/lib64/python3.5/json/__init__.py", line 265, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
It's actually telling you the answer: you're getting back a byte array, where in Python 3 a string is different because of dealing with unicode. In Python 2.7, it would work. You should be able to fix it by converting your bytes explicitly to a string with
jsonget=str(urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee")_
I have the following code to load JSON:
import json
import requests
r = requests.get('http://api.reddit.com/controversial?limit=5')
if r.status_code = 200:
reddit_data = json.loads(r.content)
print reddit_data['data']['children'][1]['data']
else:
print "Errror."
And I got this message.
arsh#arsh:~$ python q.py
Traceback (most recent call last):
File "q.py", line 1, in <module>
import json
File "/home/arsh/json.py", line 5, in <module>
reddit_data = json.loads(r.content)
AttributeError: 'module' object has no attribute 'loads'
You have a different file called json.py in your home directory:
File "/home/arsh/json.py", line 5, in <module>
This file is in the way, you did not import the standard library version. Rename it to something else or delete it. You'll also have to remove the json.pyc file.
Note that requests response objects can already handle JSON responses for you:
import requests
r = requests.get('http://api.reddit.com/controversial?limit=5')
r.raise_for_status()
reddit_data = r.json()
print reddit_data['data']['children'][1]['data']
The Response.json() method handles decoding JSON for you, including detecting the correct characterset to use when decoding.
I thought json.load() should be able to read objects exactly like http.client.HTTPResponse, but it seems to be tripping up on its read() being a bytes-like object. (I am using Python 3.3.) To my surprise, I found no resource directly addressing this use though I thought this was a primary use case.
import urllib.request, json
# Y!F url
yf = 'http://d.yimg.com/autoc.finance.yahoo.com/autoc'
# Mock lookup
data = urllib.parse.urlencode({'query': 'Ford', 'callback': 'YAHOO.Finance.SymbolSuggest.ssCallback'})
data = data.encode('utf-8')
request = urllib.request.Request(yf)
request.add_header('User-Agent','Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/536.26.14 (KHTML, like Gecko) Version/6.0.1 Safari/536.26.14')
request.add_header('Content-type','text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8;charset=utf-8')
request.add_header('Accept','text/plain')
mock = urllib.request.urlopen(request, data)
json.load(mock)
This results in the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/__init__.py", line 264, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/__init__.py", line 309, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/decoder.py", line 352, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: can't use a string pattern on a bytes-like object
This has been solved in a previous thread: Python 3, let json object accept bytes or let urlopen output strings
(HT: Matthew Trevor)
Also, what Yahoo! returns here is not just to the JSON object but also a YAHOO.Finance.SymbolSuggest.ssCallback() wrapper. Stripping that fixes things. (Though still sad it's needed.)
This works:
import urllib.request, json, re
# Y!F url
yf = 'http://d.yimg.com/autoc.finance.yahoo.com/autoc'
# Mock lookup
data = urllib.parse.urlencode({'query': 'Ford', 'callback': 'YAHOO.Finance.SymbolSuggest.ssCallback'})
data = data.encode('utf-8')
request = urllib.request.Request(yf)
response = urllib.request.urlopen(request, data)
j = json.loads(re.search(r'{.*}',response.readall().decode('utf-8')).group())