I am new to Python, How to convert this to a json string? I like to get the first line of the commitMessage in below output(op) ? thanks in advance
>>> op = subprocess.Popen('ssh -p 29999 server-name.com gerrit query --commit-message --format=JSON Ib3856dcf0826942787c3d5a076eb6888dae9k2be', shell=True, stdout=subprocess.PIPE, cwd='../').communicate()
>>>
>>> op
('{"project":"mtt/proprietary/fg","branch":"master","id":"Ib3856dcf0826942787c3d5a076eb6888dae9k2be","number":"1857599","subject":"store Gain","owner":{"name":"owner1","email":"one#mail.com","username":"user1"},"url":"https://server-name.com/1857599","commitMessage":"my commit message\\n\\nChange-Id: Ib949999d3f4d94299993d5a076eb681c4aaaa2be\\n","createdOn":1478281199,"lastUpdated":1478732989,"sortKey":"0041150d001ad179","open":false,"status":"MERGED"}\n{"type":"stats","rowCount":1,"runTimeMilliseconds":4}\n', None)
>>> js = json.loads(op)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer
Processes have two standard output streams: stdout and stderr. One for regular output and other for errors. communicate returns a tuple with two elements: first is stdout and second is stderr. You need the first:
json.loads(op[0])
Your op variable returned from subprocess is not a string, it is a tuple of 2 elements.
Just do: js = json.loads(op[0]) instead, to parse the first element, which is the json string you are attempting to parse
Related
I have a string that I need to load into a dictionary. I'm trying to use json.loads() to do this, but it is failing because I need to replace single quotes with double quotes because by default the strings use single quotes to wrap property names and values, although in some cases the value is wrapped in double quotes because it contains an single quote.
Here is a reproduceable example using Python3.8
>>> import json
>>> example = "{'msg': \"I'm a string\"}"
>>> json.loads(example.replace("'", '"'))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 12 (char 11)
Is there a better way to load a string into a dictionary? The json module seems to only work with double quotes and as that is not possible here (I have no control over the formatting of the strings), hence why I'm having to use the unreliable replace.
My desired result is to have a dictionary like this.
{'msg': "I'm a string"}
Use ast.literal_eval()Docs:
>>> import ast
>>> ast.literal_eval(example)
{'msg': "I'm a string"}
I am currently tring to work with import a json input that is accepted by Python through a commandline argument and I am trying to save the different values to JSON to a list. I am having issues with my code given below and have attached both the code and the error I get below. Any help much appreciated.
import sys
import json
def lookup1 ():
jsonData = json.loads(sys.argv[1])
print jsonData
jsonList = [jsonData['proxy'],jsonData['OS']]
print jsonList
lookup1()
The error is given below:
$ python dynamicMapper.py '{'proxy':1,'OS':2}'
Traceback (most recent call last):
File "dynamicMapper.py", line 9, in <module>
lookup1()
File "dynamicMapper.py", line 4, in lookup1
jsonData = json.loads(sys.argv[1])
File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 1 column 2 (char 1)
The commadline argunet that I give is python dynamicMapper.py '{'proxy':1,'OS':2}'
I am not able to find out what is causing this error and if my approach is right.
The script is working fine, you just need to call it the right way:
python dynamicMapper.py '{"proxy":1,"OS":2}'
{u'OS': 2, u'proxy': 1}
[1, 2]
In JSON the strings are quoted with double quotes instead of single quotes. You also need to quote the string passed to script so that shell understands it being a single argument.
I am just learning python and can not solve one issue.
The input json text like:
[1123771,10,7699,4357,'UMF Selfoss','Haukar Hafnarfjordur','2015,5,25,19,15,00','2015,5,25,20,16,37',-1,0,1,0,1,0,0,2,2,'8','7',,'True',0.25,'',25,'',2.75]
Then I trying to use python json module to parse it i get an error.
Here is the code:
js = json.loads("[1123771,10,7699,4357,'UMF Selfoss','Haukar Hafnarfjordur','2015,5,25,19,15,00','2015,5,25,20,16,37',-1,0,1,0,1,0,0,2,2,'8','7',,'True',0.25,'',25,'',2.75]")
The error is:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "C:\Python27\lib\json\__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\lib\json\decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
This json text successfuly parsed by another frameworks like json.net (C#).
So the question is what I doing wrong?
Your json needs to be valid in order to be able to parse it:
Use this tool :
http://jsonlint.com/
JSON only works with double quotes.
Also two consecutive commas would make your JSON invalid
It's wrong JSON format. Check it by using some online services.
I am trying to parse JSON from Python. I am able to parse JSON properly if I am using single quote around json string but if I remove that single quote then it doesn't works for me -
#!/usr/bin/python
import json
# getting JSON string from a method which gives me like this
# so I need to wrap around this dict with single quote to deserialize the JSON
jsonStr = {"hello":"world"}
j = json.loads(`jsonStr`) #this doesnt work either?
shell_script = j['hello']
print shell_script
So my question is how to wrap that JSON string around single quote so that I am able to deserialize it properly?
The error that I get -
$ python jsontest.py
Traceback (most recent call last):
File "jsontest.py", line 7, in <module>
j = json.loads('jsonStr')
File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
I think you are confusing two different concepts here between dumps() and loads()
jsonStr = {"hello":"world"}
j = json.loads(json.dumps(jsonStr)) #this should work
shell_script = j['hello']
print shell_script
But yes, it's redundant since jsonStr is already an object. If you wanted to try loads() you need a valid json string as input, like such:
jsonStr = '{"hello":"world"}'
j = json.loads(jsonStr) #this should work
shell_script = j['hello']
print shell_script
jsonStr is a dictionary not a string. It's already "loaded".
You should be able to directly call
shell_script = jsonStr['hello']
print shell_script
I'm currently trying to parse through a text file containing a number of Facebook chat fragments. The fragments are stored as below:-
{"t":"msg","c":"p_100002239013747","s":14,"ms":[{"msg":{"text":"2what is the best restauran
t in hong kong? ","time":1303115825598,"clientTime":1303115824391,"msgID":"1862585188"},"from":10000
2239013747,"to":635527479,"from_name":"David Robinson","from_first_name":"David","from_gender":1,"to_name":"Jason Yeung","to_first_name":"Jason","to_gender":2,"type":"msg"}]}
I've tried a number of ways to parse / open the JSON file but to no avail. Here is what I've tried thusfar:-
import json
data = []
with open("C:\\Users\\Me\\Desktop\\facebookchat.txt", 'r') as json_string:
for line in json_string:
data.append(json.loads(line))
error:
Traceback (most recent call last):
File "C:/Users/Amy/Desktop/facebookparser.py", line 6, in <module>
data.append(json.loads(line))
File "C:\Program Files\Python27\lib\json\__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "C:\Program Files\Python27\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Program Files\Python27\lib\json\decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Invalid control character at: line 1 column 91 (char 91)
and also:
import json
with open("C:\\Users\\Me\\Desktop\\facebookchat.txt", 'r') as json_file:
data = json.load(json_file)
... but I get exactly the same error as above.
Any suggestions? I've searched previous posts on here and tried the alternative solutions but to no avail. I'm aware I need to treat it as a dictionary file with for example, 'time' being a key and '1303115825598' being the respective time value but if I can't even process the json file into memory, there's no way I can parse it.
Where am I going wrong? Thanks
Your data contains newlines where JSON would not allow these. You'll have to stitch the lines back together again:
data = []
with open("C:\\Users\\Me\\Desktop\\facebookchat.txt", 'r') as json_string:
partial = ''
for line in json_string:
partial += line.rstrip('\n')
try:
data.append(json.loads(partial))
partial = ''
except ValueError:
continue # Not yet a complete JSON value
The code collects lines into partial, but minus the newline, and tries to decode the JSON. If that succeeds, partial is set to the empty string again to process the next entry. If it fails, we loop to the next line to append, until there is a complete JSON value to decode.