Why I can't use urlencode to encode json format data? - python

I have a problem about urlencode in python 2.7:
>>> import urllib
>>> import json
>>> urllib.urlencode(json.dumps({'title':"hello world!",'anonymous':False,'needautocategory':True}))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/urllib.py", line 1280, in urlencode
raise TypeError
TypeError: not a valid non-string sequence or mapping object

urlencode can encode a dict, but not a string. The output of json.dumps is a string.
Depending on what output you want, either don't encode the dict in JSON:
>>> urllib.urlencode({'title':"hello world!",'anonymous':False,'needautocategory':True})
'needautocategory=True&anonymous=False&title=hello+world%EF%BC%81'
or wrap the whole thing in a dict:
>>> urllib.urlencode({'data': json.dumps({'title':"hello world!",'anonymous':False,'needautocategory':True})})
'data=%7B%22needautocategory%22%3A+true%2C+%22anonymous%22%3A+false%2C+%22title%22%3A+%22hello+world%5Cuff01%22%7D'
or use quote_plus() instead (urlencode uses quote_plus for the keys and values):
>>> urllib.quote_plus(json.dumps({'title':"hello world!",'anonymous':False,'needautocategory':True}))
'%7B%22needautocategory%22%3A+true%2C+%22anonymous%22%3A+false%2C+%22title%22%3A+%22hello+world%5Cuff01%22%7D'

Because urllib.urlencode "converts a mapping object or a sequence of two-element tuples to a “percent-encoded” string...". Your string is neither of these.
I think you need urllib.quote or urllib.quote_plus.

For those of ya'll getting the error:
AttributeError: module 'urllib' has no attribute 'urlencode'
It's because urllib has been split up in Python 3
Here's the code for Python 3:
import urllib.parse
data = {
"title": "Hello world",
"anonymous": False,
"needautocategory": True
}
urllib.parse.urlencode(data)
# 'title=Hello+world&anonymous=False&needautocategory=True'

(1) Import libraries
import requests
import json
(2) Spec is a dictionary object
spec = {...}
(3) Convert dictionary object to json
data = json.dumps(spec, ensure_ascii=False)
(4) Finally, do request with parameter spec in json format
response = requests.get(
'http://localhost:8080/...',
params={'spec': data}
)
(5) Analyze response ...

json.dumps() returns a string.
urllib.urlencode() expects a query in the format of a mapping object or tuples. Note that it does not expect a string.
You're passing the first as the parameter for the second, resulting in the error.

Related

How to print from response object

I'm currently trying to get data with python from the Internet. Fortunately the Website I want to Approach Hands me the data in json Format.
Now this is my Code:
from GetWebJson import simple_get
import json
raw_data = simple_get('http://api.openweathermap.org/data/2.5/weather?q=Hamburg,de')
raw_data.encoding
data_json = raw_data.json
print(data_json["temp"])
The simple_get function executes "Get" from the requests library and Returns the Response object.
After formatting the raw data to json In want to print the value of the key "temp". But when i debug this Code I get the following error:
Traceback (most recent call last):
File "C:\Users\nikhi\source\repos\WebScraper\WebScraper\WebScraper.py", line 7, in
print(data_json["temp"])
TypeError: 'method' object is not subscriptable
Can't I use the variable data_json like a dict? If not, how do I convert the json, which contains lists and dicts to a printable Format?

Attribute error when encoding with base64

I have two keys(secret key and public key) that are generated using curve25519. I want to encode the two keys using base64.safe_b64encode but i keep getting an error. Is there any way I can encode using this?
This is my code:
import libnacl.public
import libnacl.secret
import libnacl.utils
from tinydb import TinyDB
from hashlib import sha256
import json
import base64
pikeys = libnacl.public.SecretKey()
piprivkey = pikeys.sk
pipubkey = pikeys.pk
piprivkey = base64.safe_b64encode(piprivkey)
pipubkey = base64.safe_b64encode(pipubkey)
print("encoded priv", piprivkey)
print("encoded pub", pipubkey)
This is the error I got:
Traceback (most recent call last):
File "/home/pi/Desktop/finalcode/pillar1.py", line 130, in <module>
File "/home/pi/Desktop/finalcode/pillar1.py", line 50, in generatepillar1key
piprivkey = base64.safe_b64encode(piprivkey)
AttributeError: 'module' object has no attribute 'safe_b64encode'
The reason you get this error is because the base64 library does not have a function named safe_base64encode. What do you even mean by safe_base64encode? Why do you want to encode both of your keys with base64? there is a urlsafe encoding function and there is the regular base64 encoding function.
encoded_data = base64.b64encode(data_to_encode)
or
encoded_data = base64.urlsafe_b64encode(data_to_encode)
The latter one will just have a different alphabet with - instead of + and _ instead of / so it's urlsafe. I'm not sure what you want to do but refer to the docs here
The error is telling you that the function safe_b64encode does not exist in the base64 module. Perhaps you meant to use base64.urlsafe_b64encode(s)?

Python3 and hmac . How to handle string not being binary

I had a script in Python2 that was working great.
def _generate_signature(data):
return hmac.new('key', data, hashlib.sha256).hexdigest()
Where data was the output of json.dumps.
Now, if I try to run the same kind of code in Python 3, I get the following:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.4/hmac.py", line 144, in new
return HMAC(key, msg, digestmod)
File "/usr/lib/python3.4/hmac.py", line 42, in __init__
raise TypeError("key: expected bytes or bytearray, but got %r" %type(key).__name__)
TypeError: key: expected bytes or bytearray, but got 'str'
If I try something like transforming the key to bytes like so:
bytes('key')
I get
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: string argument without an encoding
I'm still struggling to understand the encodings in Python 3.
You can use bytes literal: b'key'
def _generate_signature(data):
return hmac.new(b'key', data, hashlib.sha256).hexdigest()
In addition to that, make sure data is also bytes. For example, if it is read from file, you need to use binary mode (rb) when opening the file.
Not to resurrect an old question but I did want to add something I feel is missing from this answer, to which I had trouble finding an appropriate explanation/example of anywhere else:
Aquiles Carattino was pretty close with his attempt at converting the string to bytes, but was missing the second argument, the encoding of the string to be converted to bytes.
If someone would like to convert a string to bytes through some other means than static assignment (such as reading from a config file or a DB), the following should work:
(Python 3+ only, not compatible with Python 2)
import hmac, hashlib
def _generate_signature(data):
key = 'key' # Defined as a simple string.
key_bytes= bytes(key , 'latin-1') # Commonly 'latin-1' or 'ascii'
data_bytes = bytes(data, 'latin-1') # Assumes `data` is also an ascii string.
return hmac.new(key_bytes, data_bytes , hashlib.sha256).hexdigest()
print(
_generate_signature('this is my string of data')
)
try
codecs.encode()
which can be used both in python2.7.12 and 3.5.2
import hashlib
import codecs
import hmac
a = "aaaaaaa"
b = "bbbbbbb"
hmac.new(codecs.encode(a), msg=codecs.encode(b), digestmod=hashlib.sha256).hexdigest()
for python3 this is how i solved it.
import codecs
import hmac
def _generate_signature(data):
return hmac.new(codecs.encode(key), codecs.encode(data), codecs.encode(hashlib.sha256)).hexdigest()

Python - dump dict as a json string

What am I missing? I want to dump a dictionary as a json string.
I am using python 2.7
With this code:
import json
fu = {'a':'b'}
output = json.dump(fu)
I get the following error:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/gevent-1.0b2-py2.7-linux-x86_64.egg/gevent/greenlet.py", line 328, in run
result = self._run(*self.args, **self.kwargs)
File "/home/ubuntu/workspace/bitmagister-api/mab.py", line 117, in mabLoop
output = json.dump(fu)
TypeError: dump() takes at least 2 arguments (1 given)
<Greenlet at 0x7f4f3d6eec30: mabLoop> failed with TypeError
Use json.dumps to dump a str
>>> import json
>>> json.dumps({'a':'b'})
'{"a": "b"}'
json.dump dumps to a file
i think the problem is json.dump. try
json.dumps(fu)
You can use json.dumps.
Example:
import json
json.dumps({'zuckerberg':'tech','sachin':'cricket'})
This outputs:
'{"zuckerberg": "tech", "sachin": "cricket"}'
If you want to sort the keys, use sort_keys as the second argument to json.dumps:
json.dumps({'zuckerberg':'tech','sachin':'cricket'},sort_keys=True)
Outputs:
'{"sachin": "cricket", "zuckerberg": "tech"}'
message={"message":"Done", "result":"1"}
message_json = simplejson.dumps(message)
payload = message_json
##or
message={"message":"Done", "result":"1"}
message_json=jsonify(message)

Why do I get "'str' object has no attribute 'read'" when trying to use `json.load` on a string? [duplicate]

This question already has answers here:
How can I parse (read) and use JSON?
(5 answers)
Closed 29 days ago.
In Python I'm getting an error:
Exception: (<type 'exceptions.AttributeError'>,
AttributeError("'str' object has no attribute 'read'",), <traceback object at 0x1543ab8>)
Given python code:
def getEntries (self, sub):
url = 'http://www.reddit.com/'
if (sub != ''):
url += 'r/' + sub
request = urllib2.Request (url +
'.json', None, {'User-Agent' : 'Reddit desktop client by /user/RobinJ1995/'})
response = urllib2.urlopen (request)
jsonStr = response.read()
return json.load(jsonStr)['data']['children']
What does this error mean and what did I do to cause it?
The problem is that for json.load you should pass a file like object with a read function defined. So either you use json.load(response) or json.loads(response.read()).
Ok, this is an old thread but.
I had a same issue, my problem was I used json.load instead of json.loads
This way, json has no problem with loading any kind of dictionary.
Official documentation
json.load - Deserialize fp (a .read()-supporting text file or binary file containing a JSON document) to a Python object using this conversion table.
json.loads - Deserialize s (a str, bytes or bytearray instance containing a JSON document) to a Python object using this conversion table.
You need to open the file first. This doesn't work:
json_file = json.load('test.json')
But this works:
f = open('test.json')
json_file = json.load(f)
If you get a python error like this:
AttributeError: 'str' object has no attribute 'some_method'
You probably poisoned your object accidentally by overwriting your object with a string.
How to reproduce this error in python with a few lines of code:
#!/usr/bin/env python
import json
def foobar(json):
msg = json.loads(json)
foobar('{"batman": "yes"}')
Run it, which prints:
AttributeError: 'str' object has no attribute 'loads'
But change the name of the variablename, and it works fine:
#!/usr/bin/env python
import json
def foobar(jsonstring):
msg = json.loads(jsonstring)
foobar('{"batman": "yes"}')
This error is caused when you tried to run a method within a string. String has a few methods, but not the one you are invoking. So stop trying to invoke a method which String does not define and start looking for where you poisoned your object.
AttributeError("'str' object has no attribute 'read'",)
This means exactly what it says: something tried to find a .read attribute on the object that you gave it, and you gave it an object of type str (i.e., you gave it a string).
The error occurred here:
json.load(jsonStr)['data']['children']
Well, you aren't looking for read anywhere, so it must happen in the json.load function that you called (as indicated by the full traceback). That is because json.load is trying to .read the thing that you gave it, but you gave it jsonStr, which currently names a string (which you created by calling .read on the response).
Solution: don't call .read yourself; the function will do this, and is expecting you to give it the response directly so that it can do so.
You could also have figured this out by reading the built-in Python documentation for the function (try help(json.load), or for the entire module (try help(json)), or by checking the documentation for those functions on http://docs.python.org .
Instead of json.load() use json.loads() and it would work:
ex:
import json
from json import dumps
strinjJson = '{"event_type": "affected_element_added"}'
data = json.loads(strinjJson)
print(data)
So, don't use json.load(data.read()) use json.loads(data.read()):
def findMailOfDev(fileName):
file=open(fileName,'r')
data=file.read();
data=json.loads(data)
return data['mail']
use json.loads() function , put the s after that ... just a mistake btw i just realized after i searched error
def getEntries (self, sub):
url = 'http://www.reddit.com/'
if (sub != ''):
url += 'r/' + sub
request = urllib2.Request (url +
'.json', None, {'User-Agent' : 'Reddit desktop client by /user/RobinJ1995/'})
response = urllib2.urlopen (request)
jsonStr = response.read()
return json.loads(jsonStr)['data']['children']
try this
Open the file as a text file first
json_data = open("data.json", "r")
Now load it to dict
dict_data = json.load(json_data)
If you need to convert string to json. Then use loads() method instead of load(). load() function uses to load data from a file so used loads() to convert string to json object.
j_obj = json.loads('["label" : "data"]')

Categories