I'm trying out Falcon for a small api project. Unfortunate i'm stuck on the json parsing stuff and code from the documentation examples does not work.
I have tried so many things i've found on Stack and Google but no changes.
I've tried the following codes that results in the errors below
import json
import falcon
class JSON_Middleware(object):
def process_request(self, req, resp):
raw_json = json.loads(req.stream.read().decode('UTF-8'))
"""Exception: AttributeError: 'str' object has no attribute 'read'"""
raw_json = json.loads(req.stream.read(), 'UTF-8')
"""Exception: TypeError: the JSON object must be str, not 'bytes'"""
raw_json = json.loads(req.stream, 'UTF-8')
"""TypeError: the JSON object must be str, not 'Body'"""
I'm on the way of giving up, but if somebody can tell me why this is happening and how to parse JSON in Falcon i would be extremely thankful.
Thanks
Environment:
OSX Sierra
Python 3.5.2
Falcon and other is the latest version from Pip
your code should work if other pieces of code are in place . a quick test(filename app.py):
import falcon
import json
class JSON_Middleware(object):
def process_request(self, req, resp):
raw_json = json.loads(req.stream.read())
print raw_json
class Test:
def on_post(self,req,resp):
pass
app = application = falcon.API(middleware=JSON_Middleware())
t = Test()
app.add_route('/test',t)
run with: gunicorn app
$ curl -XPOST 'localhost:8000' -d '{"Hello":"wold"}'
You have to invoke encode() on the bytes returned by read() with something like req.stream.read().encode('utf-8').
This way the bytes are converted to a str as expected by json.loads().
The other way not to bother with all this boring and error prone encode/decode and bytes/str stuff (which BTW differs in Py2 and Py3), is to use simplejson as a replacement for json. It is API compatible, so the only change is to replace import json with import simplejson as json in your code.
In addition, it simplifies the code since reading the body can be done with json.load(req.bounded_stream), which is much shorter and more readable than json.loads(req.bounded_stream.read().encode('utf-8')).
I now do it this way, and don't use the standard json module any more.
Related
I'm trying to change my python code from 2.7 to 3.6
So, I'm not familiar to python but I have error with urllib2
I have this error
Error Contents: name 'urllib2' is not defined
So I do this:
from urllib.request import urlopen
This is maybe ok, because urllib2 doesn't work on phyton 3?
But I have this:
class NoRedirection(urllib2.HTTPErrorProcessor):
def http_response(self, request, response):
return response
https_response = http_response
What I tried to change
class NoRedirection(urlopen.HTTPErrorProcessor):
But does't work. How to fix this?
**AttributeError: 'function' object has no attribute 'HTTPErrorProcessor'**
There is a separate module for errors found here. What you want to do is something along these lines
from urllib.error import HTTPError
class NoRedirection(HTTPError):
...
I try to make connection with the Google Adwords API using Python 3.6. I managed to install the libraries, got a developer token, client_customer_id, user_agent, client_id, client_secret and requested succesfully a refresh_token.
My googleads.yaml file looks like this:
adwords:
developer_token: hta...
client_customer_id: 235-...-....
user_agent: mycompany
client_id: 25785...apps.googleusercontent.com
client_secret: J9Da...
refresh_token: 1/ckhGH6...
When running the first python script get_campaigns.py, I get the very generic response TypeError: cannot use a string pattern on a bytes-like object in ...\Anaconda3\lib\site-packages\googleads-10.0.0-py3.6.egg\googleads\util.py", line 302, in filter
Other functions like traffic_estimator_service.get(selector) produce the same error. Furthermore, when starting the Python script get_campaigns.py, I get the following warning, which might explains something:
WARNING:googleads.common:Your default encoding, cp1252, is not UTF-8. Please run this script with UTF-8 encoding to avoid errors.
INFO:oauth2client.client:Refreshing access_token
INFO:googleads.common:Request summary - {'methodName': get, 'clientCustomerId': xxx-xxx-xxxx}
I tried many things, but still can't find what causes my error. My settings seem to be right, and I use the examples as provided here. Help is highly appreciated!
There are two solutions for now:
One:
Use Python2.7, solved this error for me.
Two:
For python 3
def method_waraper(self, record):
def filter(self, record):
if record.args:
arg = record.args[0]
if isinstance(arg, suds.transport.Request):
new_arg = suds.transport.Request(arg.url)
sanitized_headers = arg.headers.copy()
if self._AUTHORIZATION_HEADER in sanitized_headers:
sanitized_headers[self._AUTHORIZATION_HEADER] = self._REDACTED
new_arg.headers = sanitized_headers
msg = arg.message
if sys.version_info.major < 3:
msg = msg.decode('utf-8')
new_arg.message = self._DEVELOPER_TOKEN_SUB.sub(
self._REDACTED, str(msg, encoding='utf-8'))
record.args = (new_arg,)
return filter(self, record)
googleads.util._SudsTransportFilter.filter = method_waraper
This solution changes code provided by google and add utf encoding for the binary string, which solves our problem.
I'm porting ebay sdk to python3 and I've stumbled upon the following issue.
I'm using pycurl to send some HTTP requests.
Here is how I configure it:
self._curl = pycurl.Curl()
self._curl.setopt(pycurl.FOLLOWLOCATION, 1)
self._curl.setopt(pycurl.URL, str(request_url))
self._curl.setopt(pycurl.SSL_VERIFYPEER, 0)
self._response_header = io.StringIO()
self._response_body = io.StringIO()
self._curl.setopt(pycurl.CONNECTTIMEOUT, self.timeout)
self._curl.setopt(pycurl.TIMEOUT, self.timeout)
self._curl.setopt(pycurl.HEADERFUNCTION, self._response_header.write)
self._curl.setopt(pycurl.WRITEFUNCTION, self._response_body.write)
When I call self._curl.perform() I get the following error:
pycurl.error: (23, 'Failed writing body (1457 != 1460)')
As far as I know this means that there is an issue with the write function, but I can't figure out what it is exactly. Could be related to migration from StringIO module to io, but I'm not sure.
UPD:
I've tried the following:
def body(buf):
self._response_body.write(buf)
def header(buf):
self._response_header.write(buf)
self._curl.setopt(pycurl.HEADERFUNCTION, header)
self._curl.setopt(pycurl.WRITEFUNCTION, body)
and it works. I've tried to do the same trick with lambdas (instead of defining those awkward function, but it didn't work.
I believe the problem is that pycurl no longer functions with StringIO like desired. A solution is to use io.BytesIO instead. You can then get information written into the buffer and decode it into a string.
Using BytesIO with pycurl instead of StringIO:
e = io.BytesIO()
c.setopt(pycurl.WRITEFUNCTION, e.write)
Decoding byte information from the BytesIO object:
htmlString = e.getvalue().decode('UTF-8')
You can use any type of decoding you want, but this should give you a string object you can parse.
Hope this helps people using Python 3.
This question already has answers here:
How can I parse (read) and use JSON?
(5 answers)
Closed 29 days ago.
In Python I'm getting an error:
Exception: (<type 'exceptions.AttributeError'>,
AttributeError("'str' object has no attribute 'read'",), <traceback object at 0x1543ab8>)
Given python code:
def getEntries (self, sub):
url = 'http://www.reddit.com/'
if (sub != ''):
url += 'r/' + sub
request = urllib2.Request (url +
'.json', None, {'User-Agent' : 'Reddit desktop client by /user/RobinJ1995/'})
response = urllib2.urlopen (request)
jsonStr = response.read()
return json.load(jsonStr)['data']['children']
What does this error mean and what did I do to cause it?
The problem is that for json.load you should pass a file like object with a read function defined. So either you use json.load(response) or json.loads(response.read()).
Ok, this is an old thread but.
I had a same issue, my problem was I used json.load instead of json.loads
This way, json has no problem with loading any kind of dictionary.
Official documentation
json.load - Deserialize fp (a .read()-supporting text file or binary file containing a JSON document) to a Python object using this conversion table.
json.loads - Deserialize s (a str, bytes or bytearray instance containing a JSON document) to a Python object using this conversion table.
You need to open the file first. This doesn't work:
json_file = json.load('test.json')
But this works:
f = open('test.json')
json_file = json.load(f)
If you get a python error like this:
AttributeError: 'str' object has no attribute 'some_method'
You probably poisoned your object accidentally by overwriting your object with a string.
How to reproduce this error in python with a few lines of code:
#!/usr/bin/env python
import json
def foobar(json):
msg = json.loads(json)
foobar('{"batman": "yes"}')
Run it, which prints:
AttributeError: 'str' object has no attribute 'loads'
But change the name of the variablename, and it works fine:
#!/usr/bin/env python
import json
def foobar(jsonstring):
msg = json.loads(jsonstring)
foobar('{"batman": "yes"}')
This error is caused when you tried to run a method within a string. String has a few methods, but not the one you are invoking. So stop trying to invoke a method which String does not define and start looking for where you poisoned your object.
AttributeError("'str' object has no attribute 'read'",)
This means exactly what it says: something tried to find a .read attribute on the object that you gave it, and you gave it an object of type str (i.e., you gave it a string).
The error occurred here:
json.load(jsonStr)['data']['children']
Well, you aren't looking for read anywhere, so it must happen in the json.load function that you called (as indicated by the full traceback). That is because json.load is trying to .read the thing that you gave it, but you gave it jsonStr, which currently names a string (which you created by calling .read on the response).
Solution: don't call .read yourself; the function will do this, and is expecting you to give it the response directly so that it can do so.
You could also have figured this out by reading the built-in Python documentation for the function (try help(json.load), or for the entire module (try help(json)), or by checking the documentation for those functions on http://docs.python.org .
Instead of json.load() use json.loads() and it would work:
ex:
import json
from json import dumps
strinjJson = '{"event_type": "affected_element_added"}'
data = json.loads(strinjJson)
print(data)
So, don't use json.load(data.read()) use json.loads(data.read()):
def findMailOfDev(fileName):
file=open(fileName,'r')
data=file.read();
data=json.loads(data)
return data['mail']
use json.loads() function , put the s after that ... just a mistake btw i just realized after i searched error
def getEntries (self, sub):
url = 'http://www.reddit.com/'
if (sub != ''):
url += 'r/' + sub
request = urllib2.Request (url +
'.json', None, {'User-Agent' : 'Reddit desktop client by /user/RobinJ1995/'})
response = urllib2.urlopen (request)
jsonStr = response.read()
return json.loads(jsonStr)['data']['children']
try this
Open the file as a text file first
json_data = open("data.json", "r")
Now load it to dict
dict_data = json.load(json_data)
If you need to convert string to json. Then use loads() method instead of load(). load() function uses to load data from a file so used loads() to convert string to json object.
j_obj = json.loads('["label" : "data"]')
I need to post data to a Django server. I'd like to use pickle. (There're no security requirements -> small intranet app.)
First, I pickle the data on the client and sending it with urllib2
def dispatch(self, func_name, *args, **kwargs):
dispatch_url = urljoin(self.base_url, "api/dispatch")
pickled_args = cPickle.dumps(args, 2)
pickled_kwargs = cPickle.dumps(kwargs, 2)
data = urllib.urlencode({'func_name' : func_name,
'args' : pickled_args,
'kwargs': pickled_kwargs})
resp = self.opener.open(dispatch_url, data)
Recieving the data at the server works, too:
def dispatch(request):
func_name = request.POST["func_name"]
pickled_args = request.POST["args"]
pickled_kwargs = request.POST["kwargs"]
But unpickling raises an error:
cPickle.loads(pickled_args)
Traceback (most recent call last):
File "<string>", line 1, in <fragment>
TypeError: must be string, not unicode
Obviously the urllib.urlencode has created a unicode string. But how can I convert it back to be able to unpickling (laods) again?
By the way, using pickle format 0 (ascii) works. I can convert to string before unpickling, but I'd rather use format 2.
Also, recommendations about how to get binary data to a Django view are highly appreciated.
Obviously the urllib.urlencode has created a unicode string.
urllib.urlencode doesn't return Unicode string.
It might be a feature of the web framework you use that request.POST contains Unicode strings.
Don't use pickle to communicate between services it is not secure, brittle, not portable, hard to debug and it makes your components too coupled.