We Keep Coding

zabbix API json request with python urllib.request

I'm working on my python project and I migrated from python2.6 to python 3.6. So I had to replace urllib2 with urllib.request ( and .error and .parse ).
But I'm facing an issue I can't solve, here it is...
I want to send a request written in JSON like below :
import json
import urllib2
data= json.dumps({
"jsonrpc":"2.0",
"method":"user.login",
"params":{
"user":"guest",
"password":"password"
}
"id":1,
"auth":None
})
with urllib2 I faced no issue, I just had to create the request with :
req=urllib2.Request("http://myurl/zabbix/api_jsonrpc.php",data,{'Content-type':'application/json})
send it with:
response=urllib2.urlopen(req)
and it was good but now with urllib.request, I have met many error raised by the library. check what I did ( the request is the same within 'data') :
import json
import urllib.request
data= json.dumps({
"jsonrpc":"2.0",
"method":"user.login",
"params":{
"user":"guest",
"password":"password"
}
"id":1,
"auth":None
})
req = urllib.request.Request("http://myurl/zabbix/api_jsonrpc.php",data,{'Content-type':'application/json})
response = urllib.request.urlopen(req)
and I get this error :
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/tmp/Python-3.6.1/Lib/urllib/request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "/tmp/Python-3.6.1/Lib/urllib/request.py", line 524, in open
req = meth(req)
File "/tmp/Python-3.6.1/Lib/urllib/request.py", line 1248, in do_request_
raise TypeError(msg)
TypeError: POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.
So I inquired about this and learned that I must use the function urllib.parse.urlencode() to convert my request into bytes, so I tried to use it on my request :
import urllib.parse
dataEnc=urllib.parse.urlencode(data)
another error occured :
Traceback (most recent call last):
File "/tmp/Python-3.6.1/Lib/urllib/parse.py", line 842, in urlencode
raise TypeError
TypeError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/tmp/Python-3.6.1/Lib/urllib/parse.py", line 850, in urlencode
"or mapping object").with_traceback(tb)
File "/tmp/Python-3.6.1/Lib/urllib/parse.py", line 842, in urlencode
raise TypeError
TypeError: not a valid non-string sequence or mapping object
and I realized that json.dumps(data) just convert my array/dictionnary into a string, which is not valid for the urllib.parse.urlencode function, soooooo I retired the json.dumps from data and did this :
import json
import urllib.request
import urllib.parse
data= {
"jsonrpc":"2.0",
"method":"user.login",
"params":{
"user":"guest",
"password":"password"
}
"id":1,
"auth":None
}
dataEnc=urllib.parse.urlencode(data) #this one worked then
req=urllib.request.Request("http://myurl/zabbix/api_jsonrpc.php",data,{'Content-type':'application/json})
response = urllib.request.urlopen(req) #and this one too, but it was too beautiful
then I took a look in the response and got this :
b'{"jsonrpc":"2.0",
"error":{
"code":-32700,
"message":"Parse error",
"data":"Invalid JSON. An error occurred on the server while parsing the JSON text."}
,"id":1}
And I guess it's because the JSON message is not json.dumped !
There is always one element blocking me from doing the request correctly,
so I'm totally stuck with it, if any of you guys have an idea or an alternative I would be so happy.
best Regards
Gozu09

In fact you just need to pass your json data as a byte sequence like this:
data= {
"jsonrpc":"2.0",
"method":"user.login",
"params":{
"user":"guest",
"password":"password"
}
"id":1,
"auth":None
}
req = urllib.request.Request(
"http://myurl/zabbix/api_jsonrpc.php",
data=json.dumps(data).encode(), # Encode a string to a bytes sequence
headers={'Content-type':'application/json}
)
POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str
This error means that the data argument is expected to be an iterables of bytes.
st = "This is a string"
by = b"This is an iterable of bytes"
by2 = st.encode() # Convert my string to a bytes sequence
st2 = by.decode() # Convert my byte sequence into an UTF-8 string
json.dumps() returns a string, therefore you have to call json.dumps().encode() to convert it into a byte array.
By the way, urlencode is used when you want to convert a string that will be passed as an url argument (i.e: converting spaces characters to "%20"). The output of this method is a string, not a byte array

Not able to import json from commandline for Python

I am currently tring to work with import a json input that is accepted by Python through a commandline argument and I am trying to save the different values to JSON to a list. I am having issues with my code given below and have attached both the code and the error I get below. Any help much appreciated.
import sys
import json
def lookup1 ():
jsonData = json.loads(sys.argv[1])
print jsonData
jsonList = [jsonData['proxy'],jsonData['OS']]
print jsonList
lookup1()
The error is given below:
$ python dynamicMapper.py '{'proxy':1,'OS':2}'
Traceback (most recent call last):
File "dynamicMapper.py", line 9, in <module>
lookup1()
File "dynamicMapper.py", line 4, in lookup1
jsonData = json.loads(sys.argv[1])
File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 1 column 2 (char 1)
The commadline argunet that I give is python dynamicMapper.py '{'proxy':1,'OS':2}'
I am not able to find out what is causing this error and if my approach is right.

The script is working fine, you just need to call it the right way:
python dynamicMapper.py '{"proxy":1,"OS":2}'
{u'OS': 2, u'proxy': 1}
[1, 2]
In JSON the strings are quoted with double quotes instead of single quotes. You also need to quote the string passed to script so that shell understands it being a single argument.

convert json retrived from url

I am partially able to work with json saved as file:
#! /usr/bin/python3
import json
from pprint import pprint
json_file='a.json'
json_data=open(json_file)
data = json.load(json_data)
json_data.close()
print(data[10])
But I am trying to achieve the same from data directly from web. I am trying with the accepted answer here:
#! /usr/bin/python3
from urllib.request import urlopen
import json
from pprint import pprint
jsonget=urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee")
data = json.load(jsonget)
pprint(data)
which is giving me error:
Traceback (most recent call last):
File "i.py", line 10, in <module>
data = json.load(jsonget)
File "/usr/lib64/python3.5/json/__init__.py", line 268, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.5/json/__init__.py", line 312, in loads
s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
What is going wrong here?
Changing the code as par Charlie's reply to:
jsonget=str(urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee"))
data = json.load(jsonget)
pprint(jsonget)
breaks at json.load:
Traceback (most recent call last):
File "i.py", line 9, in <module>
data = json.load(jsonget)
File "/usr/lib64/python3.5/json/__init__.py", line 265, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'

It's actually telling you the answer: you're getting back a byte array, where in Python 3 a string is different because of dealing with unicode. In Python 2.7, it would work. You should be able to fix it by converting your bytes explicitly to a string with
jsonget=str(urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee")_

How to wrap single quote around python dictionary to make it JSON?

I am trying to parse JSON from Python. I am able to parse JSON properly if I am using single quote around json string but if I remove that single quote then it doesn't works for me -
#!/usr/bin/python
import json
# getting JSON string from a method which gives me like this
# so I need to wrap around this dict with single quote to deserialize the JSON
jsonStr = {"hello":"world"}
j = json.loads(`jsonStr`) #this doesnt work either?
shell_script = j['hello']
print shell_script
So my question is how to wrap that JSON string around single quote so that I am able to deserialize it properly?
The error that I get -
$ python jsontest.py
Traceback (most recent call last):
File "jsontest.py", line 7, in <module>
j = json.loads('jsonStr')
File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

I think you are confusing two different concepts here between dumps() and loads()
jsonStr = {"hello":"world"}
j = json.loads(json.dumps(jsonStr)) #this should work
shell_script = j['hello']
print shell_script
But yes, it's redundant since jsonStr is already an object. If you wanted to try loads() you need a valid json string as input, like such:
jsonStr = '{"hello":"world"}'
j = json.loads(jsonStr) #this should work
shell_script = j['hello']
print shell_script

jsonStr is a dictionary not a string. It's already "loaded".
You should be able to directly call
shell_script = jsonStr['hello']
print shell_script

Python 'str' object has no attribute 'read'

Python 3.3.2 import json & urllib.request
Json
[{"link":"www.google.com","orderid":"100000222"},
{"link":"www.google.com","orderid":"100000222"},
{"link":"www.google.com","orderid":"100000222"}]
print(response.info())
Date: Sun, 20 Oct 2013 07:06:51 GMT
Server: Apache
X-Powered-By: PHP/5.4.12
Content-Length: 145
Connection: close
Content-Type: application/json
Codes
url = "http://www.Link.com"
request = urllib.request.Request(url)
request.add_header('User-Agent','Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)')
request.add_header('Content-Type','application/json')
response = urllib.request.urlopen(request)
decodedRes = response.read().decode('utf-8')
json_object = json.load(decodedRes)
The following are my codes
Error
Traceback (most recent call last):
File "C:\Users\Jonathan\Desktop\python.py", line 57, in <module>
checkLink()
File "C:\Users\Jonathan\Desktop\python.py", line 50, in checkLink
json_object = json.load(decodedRes)
File "C:\Python33\lib\json\__init__.py", line 271, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
>>> .
Any idea how i can fix this issue?

Use json.loads instead of json.load.
json.loads(decodedRes)
json.load accept file-like object.
>>> import json
>>> json.load('{"a": 1}')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\json\__init__.py", line 286, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
>>> json.loads('{"a": 1}')
{u'a': 1}
Alternatively you can pass response object to json.load:
## decodedRes = response.read().decode('utf-8')
json_object = json.load(response)

Try replacing the json.load() with json.loads() . The former needs a file-stream thats the reason you are running into the attribute error.

Following code loads json file into document. Document will have values of dict type.
file_name = "my_file.json"
with open(file_name, 'r') as f:
document = json.loads(f.read())

It's very simple, You just need to import beautiful soup in another way.
Right now you are importing as
from beautifulSoup import BeautifulSoup
Change it to
from bs4 import BeautifulSoup