I've got a file that contains a JSON object. It's been loaded the following way:
with open('data.json', 'r') as input_file:
input_data = input_file.read()
At this point input_data contains just a string, and now I proceed to parse it into JSON:
data_content = json.loads(input_data.decode('utf-8'))
data_content has the JSON representation of the string which is what I need, but for some reason not clear to me after json.loads it is altering the order original order of the keys, so for instance, if my file contained something like:
{ "z_id": 312312,
"fname": "test",
"program": "none",
"org": null
}
After json.loads the order is altered to let's say something like:
{ "fname": "test",
"program": None,
"z_id": 312312,
"org": "none"
}
Why is this happening? Is there a way to preserve the order? I'm using Python 2.7.
Dictionaries (objects) in python have no guaranteed order. So when parsed into a dict, the order is lost.
If the order is important for some reason, you can have json.loads use an OrderedDict instead, which is like a dict, but the order of keys is saved.
from collections import OrderedDict
data_content = json.loads(input_data.decode('utf-8'), object_pairs_hook=OrderedDict)
This is not an issue with json.load. Dictionaries in Python are not order enforced, so you will get it out of order; generally speaking, it doesn't matter, because you access elements based on strings, like "id".
Related
I am looking to parse some JSON into a dictionary but need to preserve order for one particular part of the dictionary.
I know that I can parse the entire JSON file into an ordered dictionary (ex. Can I get JSON to load into an OrderedDict?) but this is not quite what I'm looking for.
{
"foo": "bar",
"columns":
{
"col_1": [],
"col_2": []
}
}
In this example, I would want to parse the entire file in as a dictionary with the "columns" portion being an OrderedDict. Is it possible to get that granular with the JSON parsing tools while guaranteeing that order is preserved throughout? Thank you!
From the comments meanwhile, I gathered that a complete, nested OrderedDict is fine as well, but this could be a solution too, if you don't mind using some knowledge about the names of the columns:
import json
from collections import OrderedDict
def hook(partialjson):
if "col_1" in partialjson:
return OrderedDict(partialjson)
return dict(partialjson)
result = json.loads("YOUR JSON STRING", object_hook=hook)
Hope this helps!
I've noticed the order of elements in a JSON object not being the original order.
What about the elements of JSON lists? Is their order maintained?
Yes, the order of elements in JSON arrays is preserved. From RFC 7159 -The JavaScript Object Notation (JSON) Data Interchange Format
(emphasis mine):
An object is an unordered collection of zero or more name/value
pairs, where a name is a string and a value is a string, number,
boolean, null, object, or array.
An array is an ordered sequence of zero or more values.
The terms "object" and "array" come from the conventions of
JavaScript.
Some implementations do also preserve the order of JSON objects as well, but this is not guaranteed.
The order of elements in an array ([]) is maintained. The order of elements (name:value pairs) in an "object" ({}) is not, and it's usual for them to be "jumbled", if not by the JSON formatter/parser itself then by the language-specific objects (Dictionary, NSDictionary, Hashtable, etc) that are used as an internal representation.
Practically speaking, if the keys were of type NaN, the browser will not change the order.
The following script will output "One", "Two", "Three":
var foo={"3":"Three", "1":"One", "2":"Two"};
for(bar in foo) {
alert(foo[bar]);
}
Whereas the following script will output "Three", "One", "Two":
var foo={"#3":"Three", "#1":"One", "#2":"Two"};
for(bar in foo) {
alert(foo[bar]);
}
Some JavaScript engines keep keys in insertion order. V8, for instance, keeps all keys in insertion order except for keys that can be parsed as unsigned 32-bit integers.
This means that if you run either of the following:
var animals = {};
animals['dog'] = true;
animals['bear'] = true;
animals['monkey'] = true;
for (var animal in animals) {
if (animals.hasOwnProperty(animal)) {
$('<li>').text(animal).appendTo('#animals');
}
}
var animals = JSON.parse('{ "dog": true, "bear": true, "monkey": true }');
for (var animal in animals) {
$('<li>').text(animal).appendTo('#animals');
}
You'll consistently get dog, bear, and monkey in that order, on Chrome, which uses V8. Node.js also uses V8. This will hold true even if you have thousands of items. YMMV with other JavaScript engines.
Demo here and here.
"Is the order of elements in a JSON list maintained?" is not a good question. You need to ask "Is the order of elements in a JSON list maintained when doing [...] ?"
As Felix King pointed out, JSON is a textual data format. It doesn't mutate without a reason. Do not confuse a JSON string with a (JavaScript) object.
You're probably talking about operations like JSON.stringify(JSON.parse(...)). Now the answer is: It depends on the implementation. 99%* of JSON parsers do not maintain the order of objects, and do maintain the order of arrays, but you might as well use JSON to store something like
{
"son": "David",
"daughter": "Julia",
"son": "Tom",
"daughter": "Clara"
}
and use a parser that maintains order of objects.
*probably even more :)
I have a simple file (username.json) as shown below:
{"lastname": "doe", "firstname": "john"}
I use the following code to read the file:
with open(filename) as file_obj:
dictionary = json.load(file_obj)
print(dictionary['firstname'])
But when I print the dictionary value for the key "firstname" it prints nothing.
When I print the dictionary I get the following:
{u'lastname': u'doe', u'firstname': u'john'}
I know that "u" stands for unicode but for some reason I am not able to use the firstname and lastname keys.
UPDATE:
For some reason it works now!
json.loads converts a json object to the python equivalent.
This means it uses lists and dicts instead of arrays and objects. You are seeing the representation of the former.
doctionary["firstname"] will get you the value in first name (ie, "doe") while it's still a python object.
If you want to see json again, you'll need to pass it through json.dumps - but of course you won't be able to manipulate it as above when in that format.
I am trying to store a variable with Key Value data in a file using Python, but when I try printing it, it comes up in a different format.
I want the result to be printed like this-
data={"name":'name',"description": "This is my offering","icon":"/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library","version": "1.0.0",
"design": {
"#self": "#self"
}
}
This is the output I get while printing the data-
{'icon': '/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library', 'design': {'#self': '#self'}, 'name': 'name', 'version': '1.0.0', 'description': 'This is my offering'}
You haven't stated what is important to you when printing, nor how you are currently attempting to print.
There is no formatting within a dictionary. Any formatting in your code is merely to make the code look human readable and is not actually stored within your data dictionary (only formatting within each string element is retained, ie, between a pair of quotes).
If it is merely the format (multiple lines and indents) that you are concerned about, the easiest way to resolve that is to use either the Pretty Print module or the JSON module - either should do the job, depending on your preferences for how you want the data to look and how much control you want to have over the printed output format. In particular, the JSON output occupies more vertical screen space, but some people may think that it is marginally more human readable.
PrettyPrint pprint:
import pprint
data={"name":'name',"description": "This is my offering","icon":"/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library","version": "1.0.0","design": {"#self": "#self"}}
pp = pprint.PrettyPrinter(indent=4)
pp.pprint(data)
>>>
{ 'description': 'This is my offering',
'design': { '#self': '#self'},
'icon': '/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library',
'name': 'name',
'version': '1.0.0'}
>>>
JSON dumps:
import json
data={"name":'name',"description": "This is my offering","icon":"/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library","version": "1.0.0","design": {"#self": "#self"}}
print(json.dumps(data, indent=4))
>>>
{
"icon": "/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library",
"design": {
"#self": "#self"
},
"name": "name",
"version": "1.0.0",
"description": "This is my offering"
}
>>>
If you are concerned about the the order in which the items are printed, then you'll need to have an array that stores the keys in their preferred order (dictionaries don't have any inherant ordering), and then iterate through your keys and print the dictionary items out manually one by one (perhaps using a list comprehension on your keys array).
Python doesn't respect the indention or newlines you use to define your data structure, and ignores any format you had when print()ing it. You don't have very many options here, but perhaps you can use the json.dumps() function to format your code. The format does not match your expected output exactly, but it comes fairly close:
>>> data = {"name":'name',"description": "This is my offering","icon":"/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library","version": "1.0.0",
"design": {
"#self": "#self"
}
}
>>> import json
>>> print(json.dumps(data, indent=2))
{
"description": "This is my offering",
"version": "1.0.0",
"icon": "/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library",
"name": "name",
"design": {
"#self": "#self"
}
}
>>>
Note that a variable doesn't contain any formatting to be changed. The python interpreter reads text from the REPL or a file and turns them into instructions for the CPU of your machine. None of this involves formatting until you call print(). By default, print() calls str() which then provides some very basic formatting. For dictionaries, this includes the curly braces, colons, and commas. If you want anything more than this, you will need to do it yourself. Alternatively, you can find a Python module that helps reduce some of the tedium.
I want to dump this json to a file:
json.dumps(data)
This is the data:
{
"list":[
"one": { "id": "12","desc":"its 12","name":"pop"},
"two": {"id": "13","desc":"its 13","name":"kindle"}
]
}
I want id to be the first property after I dump it to file, but it is not. How can I fix this?
My guess is that it's because you're using a dictionary (hash-map). It's unsortable.
What you could do is:
from collections import OrderedDict
data = OrderedDict()
data['list'] = OrderedDict()
data['list']['one'] = OrderedDict()
data['list']['one']['id'] = '12'
data['list']['one']['idesc'] = ...
data['list']['two'] = ...
This makes it sorted by order of input.
It's "impossible" to know the output of a dict/hashmap because the nature (and speed) of a traditional dictionary makes the sort/access order vary depending on usage, items in the dictionary and a lot of other factors.
So you need to either pass your dictionary to a sort() function prior to sending it to json or use a slower version of the dictionary called OrderedDict (see above).
Many thanks goes out to #MarcoNawijn for checking the source of JSON that does not honor the sort structure of the dictionary, which means you'll have to build the JSON string yourself.
If the parser on the other end of your JSON string honors the order (which i doubt), you could pass this to a function that builds a regular text-string representation of your OrderedDict and formatting the string as per JSON standards. This will however take up more time than I have at this moment since i'm not 100% certain of the RFC for JSON strings.
You shouldnt worry about the order in which json is saved. The order will be changed when dumping. Better look at these too. JSON order mixed up
and
Is the order of elements in a JSON list maintained?