i'm learning BeautifulSoup and i encountered this
from bs4 import BeautifulSoup
import urllib2
url = "https://en.wikipedia.org/wiki/Katy_Perry"
open_url = urllib2.urlopen(url)
read = open_url.read()
print(read)
This prints the html code of the page. But how can we use read() here ? Its a FileIO function and should be used along with the file object. but the variable "open_url" here isn't a file object.
print(type(open_url))
output:
<type 'instance'>
Obviously "open_url" isn't a file object, So what made it possible to bind read() to "open_url" ?
If you print both open_url you will see that fp = socket._fileobject ..
<addinfourl at 139707791457312 whose fp = <socket._fileobject object at 0x7f104303bd50>>
So you see the file object is actually a socket._fileobject which you can access with open_url.fp:
<socket._fileobject object at 0x7f104303bd50>
If you remove the first read call you will see that you can access the socket object and call .read on that directly, that is what happens when you call open_url.read() etc..:
open_url.fp.read()
Usually my webservice built with Bottle return JSON files, which works fine. But, I've an exception that need to call a local function.
Here is what I tried to do:
import json
def getData():
return json.dumps({'data': someData })
def function():
try:
# Fail
except:
print getData()
print type(getData())
json.load(getData())
So it prints:
{"data": "myData"}
<type 'str'>
[...]
AttributeError: 'str' object has no attribute 'read'
So json.dumps gives me a string. How can I use it as JSON ?
json.load loads JSON from a file object.
json.loads loads from a string. This is what you want.
Use json.loads instead of json.load. As per the docs.
This question already has answers here:
How can I parse (read) and use JSON?
(5 answers)
Closed 29 days ago.
In Python I'm getting an error:
Exception: (<type 'exceptions.AttributeError'>,
AttributeError("'str' object has no attribute 'read'",), <traceback object at 0x1543ab8>)
Given python code:
def getEntries (self, sub):
url = 'http://www.reddit.com/'
if (sub != ''):
url += 'r/' + sub
request = urllib2.Request (url +
'.json', None, {'User-Agent' : 'Reddit desktop client by /user/RobinJ1995/'})
response = urllib2.urlopen (request)
jsonStr = response.read()
return json.load(jsonStr)['data']['children']
What does this error mean and what did I do to cause it?
The problem is that for json.load you should pass a file like object with a read function defined. So either you use json.load(response) or json.loads(response.read()).
Ok, this is an old thread but.
I had a same issue, my problem was I used json.load instead of json.loads
This way, json has no problem with loading any kind of dictionary.
Official documentation
json.load - Deserialize fp (a .read()-supporting text file or binary file containing a JSON document) to a Python object using this conversion table.
json.loads - Deserialize s (a str, bytes or bytearray instance containing a JSON document) to a Python object using this conversion table.
You need to open the file first. This doesn't work:
json_file = json.load('test.json')
But this works:
f = open('test.json')
json_file = json.load(f)
If you get a python error like this:
AttributeError: 'str' object has no attribute 'some_method'
You probably poisoned your object accidentally by overwriting your object with a string.
How to reproduce this error in python with a few lines of code:
#!/usr/bin/env python
import json
def foobar(json):
msg = json.loads(json)
foobar('{"batman": "yes"}')
Run it, which prints:
AttributeError: 'str' object has no attribute 'loads'
But change the name of the variablename, and it works fine:
#!/usr/bin/env python
import json
def foobar(jsonstring):
msg = json.loads(jsonstring)
foobar('{"batman": "yes"}')
This error is caused when you tried to run a method within a string. String has a few methods, but not the one you are invoking. So stop trying to invoke a method which String does not define and start looking for where you poisoned your object.
AttributeError("'str' object has no attribute 'read'",)
This means exactly what it says: something tried to find a .read attribute on the object that you gave it, and you gave it an object of type str (i.e., you gave it a string).
The error occurred here:
json.load(jsonStr)['data']['children']
Well, you aren't looking for read anywhere, so it must happen in the json.load function that you called (as indicated by the full traceback). That is because json.load is trying to .read the thing that you gave it, but you gave it jsonStr, which currently names a string (which you created by calling .read on the response).
Solution: don't call .read yourself; the function will do this, and is expecting you to give it the response directly so that it can do so.
You could also have figured this out by reading the built-in Python documentation for the function (try help(json.load), or for the entire module (try help(json)), or by checking the documentation for those functions on http://docs.python.org .
Instead of json.load() use json.loads() and it would work:
ex:
import json
from json import dumps
strinjJson = '{"event_type": "affected_element_added"}'
data = json.loads(strinjJson)
print(data)
So, don't use json.load(data.read()) use json.loads(data.read()):
def findMailOfDev(fileName):
file=open(fileName,'r')
data=file.read();
data=json.loads(data)
return data['mail']
use json.loads() function , put the s after that ... just a mistake btw i just realized after i searched error
def getEntries (self, sub):
url = 'http://www.reddit.com/'
if (sub != ''):
url += 'r/' + sub
request = urllib2.Request (url +
'.json', None, {'User-Agent' : 'Reddit desktop client by /user/RobinJ1995/'})
response = urllib2.urlopen (request)
jsonStr = response.read()
return json.loads(jsonStr)['data']['children']
try this
Open the file as a text file first
json_data = open("data.json", "r")
Now load it to dict
dict_data = json.load(json_data)
If you need to convert string to json. Then use loads() method instead of load(). load() function uses to load data from a file so used loads() to convert string to json object.
j_obj = json.loads('["label" : "data"]')
I´m trying to save and load objects using pickle module.
First I declare my objects:
>>> class Fruits:pass
...
>>> banana = Fruits()
>>> banana.color = 'yellow'
>>> banana.value = 30
After that I open a file called 'Fruits.obj'(previously I created a new .txt file and I renamed 'Fruits.obj'):
>>> import pickle
>>> filehandler = open(b"Fruits.obj","wb")
>>> pickle.dump(banana,filehandler)
After do this I close my session and I began a new one and I put the next (trying to access to the object that it supposed to be saved):
file = open("Fruits.obj",'r')
object_file = pickle.load(file)
But I have this message:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python31\lib\pickle.py", line 1365, in load
encoding=encoding, errors=errors).load()
ValueError: read() from the underlying stream did notreturn bytes
I don´t know what to do because I don´t understand this message.
Does anyone know How I can load my object 'banana'?
Thank you!
EDIT:
As some of you have sugested I put:
>>> import pickle
>>> file = open("Fruits.obj",'rb')
There were no problem, but the next I put was:
>>> object_file = pickle.load(file)
And I have error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python31\lib\pickle.py", line 1365, in load
encoding=encoding, errors=errors).load()
EOFError
As for your second problem:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python31\lib\pickle.py", line
1365, in load encoding=encoding,
errors=errors).load() EOFError
After you have read the contents of the file, the file pointer will be at the end of the file - there will be no further data to read. You have to rewind the file so that it will be read from the beginning again:
file.seek(0)
What you usually want to do though, is to use a context manager to open the file and read data from it. This way, the file will be automatically closed after the block finishes executing, which will also help you organize your file operations into meaningful chunks.
Finally, cPickle is a faster implementation of the pickle module in C. So:
In [1]: import _pickle as cPickle
In [2]: d = {"a": 1, "b": 2}
In [4]: with open(r"someobject.pickle", "wb") as output_file:
...: cPickle.dump(d, output_file)
...:
# pickle_file will be closed at this point, preventing your from accessing it any further
In [5]: with open(r"someobject.pickle", "rb") as input_file:
...: e = cPickle.load(input_file)
...:
In [7]: print e
------> print(e)
{'a': 1, 'b': 2}
The following works for me:
class Fruits: pass
banana = Fruits()
banana.color = 'yellow'
banana.value = 30
import pickle
filehandler = open("Fruits.obj","wb")
pickle.dump(banana,filehandler)
filehandler.close()
file = open("Fruits.obj",'rb')
object_file = pickle.load(file)
file.close()
print(object_file.color, object_file.value, sep=', ')
# yellow, 30
You're forgetting to read it as binary too.
In your write part you have:
open(b"Fruits.obj","wb") # Note the wb part (Write Binary)
In the read part you have:
file = open("Fruits.obj",'r') # Note the r part, there should be a b too
So replace it with:
file = open("Fruits.obj",'rb')
And it will work :)
As for your second error, it is most likely cause by not closing/syncing the file properly.
Try this bit of code to write:
>>> import pickle
>>> filehandler = open(b"Fruits.obj","wb")
>>> pickle.dump(banana,filehandler)
>>> filehandler.close()
And this (unchanged) to read:
>>> import pickle
>>> file = open("Fruits.obj",'rb')
>>> object_file = pickle.load(file)
A neater version would be using the with statement.
For writing:
>>> import pickle
>>> with open('Fruits.obj', 'wb') as fp:
>>> pickle.dump(banana, fp)
For reading:
>>> import pickle
>>> with open('Fruits.obj', 'rb') as fp:
>>> banana = pickle.load(fp)
Always open in binary mode, in this case
file = open("Fruits.obj",'rb')
You can use anycache to do the job for you. Assuming you have a function myfunc which creates the instance:
from anycache import anycache
class Fruits:pass
#anycache(cachedir='/path/to/your/cache')
def myfunc()
banana = Fruits()
banana.color = 'yellow'
banana.value = 30
return banana
Anycache calls myfunc at the first time and pickles the result to a
file in cachedir using an unique identifier (depending on the the function name and the arguments) as filename.
On any consecutive run, the pickled object is loaded.
If the cachedir is preserved between python runs, the pickled object is taken from the previous python run.
The function arguments are also taken into account.
A refactored implementation works likewise:
from anycache import anycache
class Fruits:pass
#anycache(cachedir='/path/to/your/cache')
def myfunc(color, value)
fruit = Fruits()
fruit.color = color
fruit.value = value
return fruit
You didn't open the file in binary mode.
open("Fruits.obj",'rb')
Should work.
For your second error, the file is most likely empty, which mean you inadvertently emptied it or used the wrong filename or something.
(This is assuming you really did close your session. If not, then it's because you didn't close the file between the write and the read).
I tested your code, and it works.
It seems you want to save your class instances across sessions, and using pickle is a decent way to do this. However, there's a package called klepto that abstracts the saving of objects to a dictionary interface, so you can choose to pickle objects and save them to a file (as shown below), or pickle the objects and save them to a database, or instead of use pickle use json, or many other options. The nice thing about klepto is that by abstracting to a common interface, it makes it easy so you don't have to remember the low-level details of how to save via pickling to a file, or otherwise.
Note that It works for dynamically added class attributes, which pickle cannot do...
dude#hilbert>$ python
Python 2.7.6 (default, Nov 12 2013, 13:26:39)
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from klepto.archives import file_archive
>>> db = file_archive('fruits.txt')
>>> class Fruits: pass
...
>>> banana = Fruits()
>>> banana.color = 'yellow'
>>> banana.value = 30
>>>
>>> db['banana'] = banana
>>> db.dump()
>>>
Then we restart…
dude#hilbert>$ python
Python 2.7.6 (default, Nov 12 2013, 13:26:39)
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from klepto.archives import file_archive
>>> db = file_archive('fruits.txt')
>>> db.load()
>>>
>>> db['banana'].color
'yellow'
>>>
Klepto works on python2 and python3.
Get the code here:
https://github.com/uqfoundation
I'm not sure what's going on, as the script used to work (before I messed around with my python on my system...)
But when I try something along the lines of
import mechanize
browser = mechanize.Browser()
browser.open("http://google.com")
I get something like
<response_seek_wrapper at 0x10123fd88 whose wrapped object = <closeable_response at 0x101232170 whose fp = <socket._fileobject object at 0x1010bf5f0>>>
Does anyone know why this is and what the fix is?
thanks!
it's not an exception, is it?
nothing wrong is happening, you just got a return value, which is esentially a response object, equivalent to br.response().
see
>>> r = browser.open("http://google.com")
>>> r
<response_seek_wrapper at 0x9bb116c whose wrapped object = <closeable_response at 0x9bb426c whose fp = <socket._fileobject object at 0x9ba306c>>>
>>> r.info().headers
# see the response headers
vs
>>> browser.open("http://google.com")
>>> browser.response()
<response_seek_wrapper at 0x9c229cc whose wrapped object = <closeable_response at 0x9bb426c whose fp = <socket._fileobject object at 0x9ba306c>>>
>>> browser.response().info().headers
# see the response headers