Decode double quotes in a json string to convert to json Python - python

Faced the following problem, I have a following string like this:
mystr = '[\\x22https://dosprn.co.il/\\x22, \\x22DOSPrn - הדפסה בעברית ב-DOS מדפסת Windows DOSPrn. \\\\\\x22Tools\\\\\\x22 Create Bootable USB Drive\\x22]'
I am trying to convert this string to JSON. However, this is not possible:
json.loads(mystr)
Traceback (most recent call last):
File "/Users/Tokyonight/PycharmProjects/WebCeoDev/venv/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-57-6f4efa0d20c6>", line 1, in <module>
json.loads(mystr)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)
import html, json
# Escape html objects. And unescape unicode.
mystr = '[\\x22https://dosprn.co.il/\\x22, \\x22DOSPrn - הדפסה בעברית ב-DOS מדפסת Windows DOSPrn. \\\\\\x22Tools\\\\\\x22 Create Bootable USB Drive\\x22]'
mystr = mystr.encode('ascii', 'xmlcharrefreplace').decode('unicode_escape')
mystr_to_json = json.loads(mystr)
print(mystr_to_json)
#['https://dosprn.co.il/', 'DOSPrn - הדפסה בעברית ב-DOS מדפסת Windows DOSPrn. "Tools" Create Bootable USB Drive']
mystr_to_json[1] = html.unescape(mystr_to_json[1])
print(mystr_to_json)
# ['https://dosprn.co.il/', 'DOSPrn - הדפסה בעברית ב-DOS מדפסת Windows DOSPrn. "Tools" Create Bootable USB Drive']
It works. But I think it's too cumbersome. I'm pretty sure there is an easier way to do this.
Anyone have any ideas how to do it better?
Thanks.

Related

Viusal studio python error when using requests package (error : json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0))

Im getting the error whenever i try to run this code in visual studio code.
Image of code
Error message:
Traceback (most recent call last):
File "c:\Users\climia1\Downloads\hangman.py", line 3, in <module>
r = requests.get('https://api.hypixel.net/skyblock/bazaar?key=29d9ecf6-71bb-49b1-b931-e0ed5f00f31d').json()
File "C:\Users\climia1\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\climia1\AppData\Local\Programs\Python\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\climia1\AppData\Local\Programs\Python\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\climia1\AppData\Local\Programs\Python\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
[Done] exited with code=1 in 18.946 seconds
it displays the same error on repl.it but it goes away after one run. with visual studio it stays. any help?
Your json file must be empty or isn't formatted properly
if you are reading or writing to a empty Json file make sure it contains {}
if you are writing or reading a json file with lot of data make sure all the tags are closed

I have a problem with json variable setting

i have this code here:
import json
with open("pass_file.txt", "r") as file:
password = json.loads(file.read())
it calls this error:
Traceback (most recent call last):
File "testdoc.py", line 9, in <module>
print(json.loads(file.read()))
File "C:\Program Files\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Program Files\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Program Files\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I dont know why this is happening because i have the same code on another file just with different variable name and file name and it works file. I did notice another similar question about a similar error but it didnt answer my question.
Thanks in advance :)
What is the content of you pass_file.txt ? The python code use json.loads so it expect JSON formated content in the pass_file.txt
For example for a string, the content of this file will be "hello world"
If you don't put quotes, the JSON parsing process will fail.

Not able to import json from commandline for Python

I am currently tring to work with import a json input that is accepted by Python through a commandline argument and I am trying to save the different values to JSON to a list. I am having issues with my code given below and have attached both the code and the error I get below. Any help much appreciated.
import sys
import json
def lookup1 ():
jsonData = json.loads(sys.argv[1])
print jsonData
jsonList = [jsonData['proxy'],jsonData['OS']]
print jsonList
lookup1()
The error is given below:
$ python dynamicMapper.py '{'proxy':1,'OS':2}'
Traceback (most recent call last):
File "dynamicMapper.py", line 9, in <module>
lookup1()
File "dynamicMapper.py", line 4, in lookup1
jsonData = json.loads(sys.argv[1])
File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 1 column 2 (char 1)
The commadline argunet that I give is python dynamicMapper.py '{'proxy':1,'OS':2}'
I am not able to find out what is causing this error and if my approach is right.
The script is working fine, you just need to call it the right way:
python dynamicMapper.py '{"proxy":1,"OS":2}'
{u'OS': 2, u'proxy': 1}
[1, 2]
In JSON the strings are quoted with double quotes instead of single quotes. You also need to quote the string passed to script so that shell understands it being a single argument.

Unable to parse json array

I am just learning python and can not solve one issue.
The input json text like:
[1123771,10,7699,4357,'UMF Selfoss','Haukar Hafnarfjordur','2015,5,25,19,15,00','2015,5,25,20,16,37',-1,0,1,0,1,0,0,2,2,'8','7',,'True',0.25,'',25,'',2.75]
Then I trying to use python json module to parse it i get an error.
Here is the code:
js = json.loads("[1123771,10,7699,4357,'UMF Selfoss','Haukar Hafnarfjordur','2015,5,25,19,15,00','2015,5,25,20,16,37',-1,0,1,0,1,0,0,2,2,'8','7',,'True',0.25,'',25,'',2.75]")
The error is:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "C:\Python27\lib\json\__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\lib\json\decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
This json text successfuly parsed by another frameworks like json.net (C#).
So the question is what I doing wrong?
Your json needs to be valid in order to be able to parse it:
Use this tool :
http://jsonlint.com/
JSON only works with double quotes.
Also two consecutive commas would make your JSON invalid
It's wrong JSON format. Check it by using some online services.

Reading scraped json file into python

I'm trying to read a JSON file I've scraped from a website into Python, with the eventual aim of turning it into a csv file for later statistical analysis.
An example of the file I want to read is:
globals.jsonpCallback('/feed/match/1-1-8vOt3JMq-1-2-yj0a9.dat', {"s":1,"d":{"bt":1,"sc":2,"st":{"notice":"finished","status-name":"Finished","result-name":"<p id=\"event-status\" class=\"result\"><span class=\"bold\" id=\"event-status-finished\">Final result 3:1<\/span><\/p>"},"oddsdata":{"back":{"E-1-2-0-0-0":{"handicapType":0,"handicapValue":"0.00","odds":{"49":{"0":1.32,"2":7,"1":4.8},"46":{"0":1.33,"2":7.3,"1":4.5},"9":{"0":1.25,"2":12,"1":5},"16":{"0":1.29,"2":11,"1":5},"164":{"0":1.31,"2":7.59,"1":4.5},"18":{"0":1.24,"2":11.6,"1":5.25},"23":{"0":1.27,"2":11.23,"1":4.75},"26":{"0":1.3,"2":10.5,"1":5},"5":{"0":1.23,"2":12,"1":5.75},"44":{"0":1.14,"2":4,"1":2.52},"1":{"0":1.3,"2":8.5,"1":4.4},"8":{"0":1.25,"2":9,"1":5},"3":{"0":1.25,"2":11,"1":4.5},"31":{"0":1.25,"2":9,"1":4.5},"60":{"0":1.29,"2":8.5,"1":4.5},"15":{"0":1.28,"2":10,"1":5.4},"34":{"0":1.25,"2":10,"1":5},"111":{"0":1.25,"2":10,"1":4.84},"68":{"0":1.26,"2":9.75,"1":4.84},"105":{"0":1.28,"2":8,"1":4.5},"71":{"0":1.3,"2":9,"1":5},"28":{"0":1.25,"2":12,"1":5.5},"33":{"0":1.3,"2":9.25,"1":4.84},"14":{"0":1.22,"2":10.93,"1":5.05},"21":{"0":1.33,"2":7.5,"1":4.33},"30":{"0":1.22,"2":8.5,"1":5.5},"24":{"0":1.3,"2":9,"1":4.95},"32":{"0":1.25,"2":9.5,"1":4.9},"2":{"0":1.25,"2":10,"1":5},"90":{"0":1.28,"2":10.25,"1":4.8},"56":{"0":1.3,"2":9.8,"1":4.8},"41":{"0":1.3,"2":8.9,"1":4.5},"128":{"0":1.27,"2":12,"1":5.5},"75":{"0":1.2,"2":10.5,"1":5.2}},"mixedParameterId":0,"OutcomeID":{"0":"s3nfxv464x0x1o7d5","2":"s3nfxv464x0x1o7d6","1":"s3nfxv498x0x0"},"opening_odds":{"49":{"0":null,"2":null,"1":null},"46":{"0":null,"2":null,"1":null},"9":{"0":null,"2":null,"1":null},"16":{"0":1.22,"2":null,"1":null},"164":{"0":null,"2":null,"1":null},"18":{"0":1.31,"2":9.8,"1":5.65},"23":{"0":null,"2":4.75,"1":11.23},"26":{"0":1.25,"2":10,"1":5.25},"5":{"0":1.22,"2":12.5,"1":5.5},"44":{"0":null,"2":null,"1":null},"1":{"0":null,"2":null,"1":null},"8":{"0":null,"2":null,"1":null},"3":{"0":null,"2":null,"1":null},"31":{"0":null,"2":null,"1":null},"60":{"0":null,"2":null,"1":null},"15":{"0":1.25,"2":null,"1":4.5},"34":{"0":null,"2":null,"1":null},"111":{"0":null,"2":null,"1":null},"68":{"0":1.27,"2":9.15,"1":4.65},"105":{"0":null,"2":null,"1":null},"71":{"0":1.29,"2":8,"1":4.5},"28":{"0":1.22,"2":10,"1":5},"33":{"0":1.4,"2":6.65,"1":4.25},"14":{"0":1.32,"2":7.8,"1":4.42},"21":{"0":null,"2":null,"1":null},"30":{"0":null,"2":null,"1":null},"24":{"0":null,"2":null,"1":null},"32":{"0":1.3,"2":8.5,"1":4.75},"2":{"0":1.35,"2":7.75,"1":4.3},"90":{"0":null,"2":null,"1":null},"56":{"0":null,"2":11,"1":4.59},"41":{"0":null,"2":null,"1":null},"128":{"0":1.25,"2":null,"1":5},"75":{"0":1.25,"2":8.9,"1":4.8}},"opening_change_time":{"49":{"0":false,"2":false,"1":false},"46":{"0":false,"2":false,"1":false},"9":{"0":false,"2":false,"1":false},"16":{"0":false,"2":false,"1":false},"164":{"0":false,"2":false,"1":false},"18":{"0":false,"2":false,"1":false},"23":{"0":false,"2":false,"1":false},"26":{"0":false,"2":false,"1":false},"5":{"0":false,"2":false,"1":false},"44":{"0":false,"2":false,"1":false},"1":{"0":false,"2":false,"1":false},"8":{"0":false,"2":false,"1":false},"3":{"0":false,"2":false,"1":false},"31":{"0":false,"2":false,"1":false},"60":{"0":false,"2":false,"1":false},"15":{"0":false,"2":false,"1":false},"34":{"0":false,"2":false,"1":false},"111":{"0":false,"2":false,"1":false},"68":{"0":false,"2":false,"1":false},"105":{"0":false,"2":false,"1":false},"71":{"0":false,"2":false,"1":false},"28":{"0":false,"2":false,"1":false},"33":{"0":false,"2":false,"1":false},"14":{"0":false,"2":false,"1":false},"21":{"0":false,"2":false,"1":false},"30":{"0":false,"2":false,"1":false},"24":{"0":false,"2":false,"1":false},"32":{"0":false,"2":false,"1":false},"2":{"0":false,"2":false,"1":false},"90":{"0":false,"2":false,"1":false},"56":{"0":false,"2":false,"1":false},"41":{"0":false,"2":false,"1":false},"128":{"0":false,"2":false,"1":false},"75":{"0":false,"2":false,"1":false}},"opening_volume":{"49":{"0":null,"2":null,"1":null},"46":{"0":null,"2":null,"1":null},"9":{"0":null,"2":null,"1":null},"16":{"0":null,"2":null,"1":null},"164":{"0":null,"2":null,"1":null},"18":{"0":null,"2":null,"1":null},"23":{"0":null,"2":null,"1":null},"26":{"0":null,"2":null,"1":null},"5":{"0":null,"2":null,"1":null},"44":{"0":null,"2":null,"1":null},"1":{"0":null,"2":null,"1":null},"8":{"0":null,"2":null,"1":null},"3":{"0":null,"2":null,"1":null},"31":{"0":null,"2":null,"1":null},"60":{"0":null,"2":null,"1":null},"15":{"0":null,"2":null,"1":null},"34":{"0":null,"2":null,"1":null},"111":{"0":null,"2":null,"1":null},"68":{"0":null,"2":null,"1":null},"105":{"0":null,"2":null,"1":null},"71":{"0":null,"2":null,"1":null},"28":{"0":null,"2":null,"1":null},"33":{"0":null,"2":null,"1":null},"14":{"0":null,"2":null,"1":null},"21":{"0":null,"2":null,"1":null},"30":{"0":null,"2":null,"1":null},"24":{"0":null,"2":null,"1":null},"32":{"0":null,"2":null,"1":null},"2":{"0":null,"2":null,"1":null},"90":{"0":null,"2":null,"1":null},"56":{"0":null,"2":null,"1":null},"41":{"0":null,"2":null,"1":null},"128":{"0":null,"2":null,"1":null},"75":{"0":null,"2":null,"1":null}},"volume":[],"change_time":{"49":{"0":1192344925,"2":1192344925,"1":1192344925},"46":{"0":1192356640,"2":1192356640,"1":1192356640},"9":{"0":1192374117,"2":1192374117,"1":1192374117},"16":{"0":1192643462,"2":1192643462,"1":1192643462},"164":{"0":1192356640,"2":1192356640,"1":1192356640},"18":{"0":1192643462,"2":1192642538,"1":1192643462},"23":{"0":1192528549,"2":1192533516,"1":1192533516},"26":{"0":1192643462,"2":1192527200,"1":1192643462},"5":{"0":1192617648,"2":1192617648,"1":1192617648},"44":{"0":1192054308,"2":1192054308,"1":1192054308},"1":{"0":1192383076,"2":1192383076,"1":1192383076},"8":{"0":1192364254,"2":1192364254,"1":1192364254},"3":{"0":1192430433,"2":1192430433,"1":1192430433},"31":{"0":1192365159,"2":1192365159,"1":1192365159},"60":{"0":1192389374,"2":1192389374,"1":1192389374},"15":{"0":1192610985,"2":1192310256,"1":1192610985},"34":{"0":1192366959,"2":1192366959,"1":1192366959},"111":{"0":1192487538,"2":1192487538,"1":1192487538},"68":{"0":1192473161,"2":1192473161,"1":1192473161},"105":{"0":1192354376,"2":1192354376,"1":1192354376},"71":{"0":1192610985,"2":1192610985,"1":1192610985},"28":{"0":1192570418,"2":1192570418,"1":1192570418},"33":{"0":1192397461,"2":1192397461,"1":1192397461},"14":{"0":1192641649,"2":1192641649,"1":1192641649},"21":{"0":1192362474,"2":1192362474,"1":1192362474},"30":{"0":1192356171,"2":1192356171,"1":1192356171},"24":{"0":1192419956,"2":1192419956,"1":1192419956},"32":{"0":1192635347,"2":1192635347,"1":1192635347},"2":{"0":1192449725,"2":1192449725,"1":1192449725},"90":{"0":1192426261,"2":1192426261,"1":1192426261},"56":{"0":1192366052,"2":1192621850,"1":1192621850},"41":{"0":1192436180,"2":1192436180,"1":1192436180},"128":{"0":1192624523,"2":1192441597,"1":1192624523},"75":{"0":1192437084,"2":1192437084,"1":1192437084}},"st":{"49":[3,60],"46":[3,60],"9":[3,60],"16":[3,60],"164":[3,60],"18":[3,60],"23":[3,60],"26":[3,60],"5":[3,60],"44":[3,60],"1":[3,60],"8":[3,60],"3":[3,60],"31":[3,60],"60":[3,60],"15":[3,60],"34":[3,60],"111":[3,60],"68":[3,60],"105":[3,60],"71":[3,60],"28":[3,60],"33":[3,60],"14":[3,60],"21":[3,60],"30":[3,60],"24":[3,60],"32":[3,60],"2":[3,60],"90":[3,60],"56":[3,60],"41":[3,60],"128":[3,60],"75":[3,60]},"bs":[],"act":{"49":true,"46":true,"9":true,"16":true,"164":true,"18":true,"23":true,"26":true,"5":true,"44":true,"1":true,"8":true,"3":true,"31":true,"60":true,"15":true,"34":false,"111":true,"68":true,"105":true,"71":true,"28":true,"33":true,"14":true,"21":true,"30":false,"24":true,"32":true,"2":true,"90":true,"56":true,"41":true,"128":true,"75":true},"actEx":{"44":{"0":true,"2":true,"1":true}}}},"lay":[]},"history":{"back":null,"lay":null},"hcl":true,"time-base":1192644000,"nav":{"1":{"2":["1","2","3","5","8","9","14","15","16","18","21","23","24","26","28","30","31","32","33","34","41","44","46","49","56","60","68","71","75","90","105","111","128","164"]},"4":{"2":["2","5","9","23","46","49","164"]}},"hasLiveOdds":false,"brokenParser":["30","34","103","154"],"hash":"c9fff233b9f89d1da10e81a9fdae7c14"},"refresh":16});
My current code looks like this (previous copy and paste job was inexplicably wrong):
import json
from pprint import pprint
json_data = open('/home/readejj/Documents/data/1-1-jFEwehyT-1-2-yjaf9.dat').read()
json_stuff = json_data[json_data.find('{'):json_data.rfind('}')+1]
json_stuff = json_stuff.replace("\"","'")
print "json_stuff",type(json_stuff), len(json_stuff), json_stuff[1:2000]
d = json.loads(json_stuff)
pprint(d)
This is adopted from: Reading JSON from a file?
The error I'm getting is:
Traceback (most recent call last):
File "read_json.py", line 16, in <module>
d = json.loads(json_stuff)
File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 1 column 1 (char 1)
If anyone can help that would be greatly appreciated - my apologies if it's a very basic error, I am not an experienced programmer.
UPDATE: It's been pointed out my replace line is bad - if removed, I get a different error:
Traceback (most recent call last):
File "read_oddsportal_json.py", line 19, in <module>
d = json.loads(json_stuff)
File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 1 column 99 (char 99)
This line:
json_stuff = json_stuff.replace("\"","'")
is bad. Delete it.
The JSON spec requires that strings are double-quote (") delimited, not single-quote.

Categories