We Keep Coding

Python3 - Parse list of strings inside nested json

Python Noob here. I saw many similar questions but none of it my exact use case. I have a simple nested json, and I'm trying to access the element name present inside metadata. Below is my sample json.
{
"items": [{
"metadata": {
"name": "myname1"
}
},
{
"metadata": {
"name": "myname1"
}
}
]
}
Below is the code That I have tried so far, but not successfull.
import json
f = open('./myfile.json')
x = f.read()
data = json.loads(x)
for i in data['items']:
for j in i['metadata']:
print (j['name'])
It errors out stating below
File "pythonjson.py", line 8, in
print (j['name']) TypeError: string indices must be integers
When I printed print (type(j)) I received the following o/p <class 'str'>. So I can see that it is a list of strings and not an dictinoary. So now How can I parse through a list of strings? Any official documentation or guide would be much helpful to know the concept of this.

Your json is bad, and the python exception is clear and unambiguous. You have the basic string "name" and you are trying to ... do a lookup on that?
Let's cut out all the json and look at the real issue. You do not know how to iterate over a dict. You're actually iterating over the keys themselves. If you want to see their values too, you're going to need dict.items()
https://docs.python.org/3/tutorial/datastructures.html#looping-techniques
metadata = {"name": "myname1"}
for key, value in metadata.items():
if key == "name":
print ('the name is', value)
But why bother if you already know the key you want to look up?
This is literally why we have dict.
print ('the name is', metadata["name"])

You likely need:
import json
f = open('./myfile.json')
x = f.read()
data = json.loads(x)
for item in data['items']:
print(item["metadata"]["name"]
Your original JSON is not valid (colons missing).

to access contents of name use "i["metadata"].keys()" this will return all keys in "metadata".
Working code to access all values of the dictionary in "metadata".
for i in data['items']:
for j in i["metadata"].keys():
print (i["metadata"][j])
**update:**Working code to access contents of "name" only.
for i in data['items']:
print (i["metadata"]["name"])

Unicode characters printed as escape sequences inside python object

I am using python 2.7.13. My python script generates a dictionary, which contains Hebrew.
The code is as follows,
# -*- coding: utf-8 -*-
val = "אבג".decode('utf-8')
print val
dict = {
'attributes' : {
'OBJECTID_1' : 1,
'LOCID' : val
}
}
print dict
The results are as follows,
אבג
{'attributes': {'LOCID': u'\u05d0\u05d1\u05d2', 'OBJECTID_1': 1}}
The first result shows as expected, since we use 'print'. However, in the dictionary I created, the Hebrew shows as unicode.
Is there anyway to display actual Hebrew in my dictionary? Or is this expected?
Thanks

In python2, when you print out a list, you end up printing the repr of the contents of that list.
In python3, a string's repr is the same as its str return value. You can observe this below:
Python2
>>> val = "אבג".decode('utf-8')
>>> val # displays repr value
u'\u05d0\u05d1\u05d2'
>>> print val # displays str value
אבג
And, as mentioned,
>>> print [val]
[u'\u05d0\u05d1\u05d2']
Constrasting with python3, str objects do not have a decode function - they are already decoded.
>>> val = "אבג"
>>> val
'אבג'
>>> print(val)
אבג
>>> print([val])
['אבג']
You can see this is why it works now.
For your problem, if you want to view the character as it is when you print the dict, you can do this:
print dict['LOCID']
Side note, do not use dict to name variables since it shadows the very important builtin class you are using.

how to create a dictionary from a set of properly formatted tuples in python

Is there a simple way to create a dictionary from a list of formatted tuples. e.g. if I do something like:
d={"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}
This creates a dictionary called d. However, if I want to create a dictionary from a string which contains the same string, I can't do that
res=<some command that returns {"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}>
print res
# returns {"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}
d=dict(res)
This throws an error that says:
ValueError: dictionary update sequence element #0 has length 1; 2 is required

I strongly strongly suspect that you have json on your hands.
import json
d = json.loads('{"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}')
would give you what you want.

Use dict(zip(tuples))
>>> u = ("foo", "bar")
>>> v = ("blah", "zoop")
>>> d = dict(zip(u, v))
>>> d
{'foo': 'blah', 'bar': 'zoop'}
Note, if you have an odd number of tuples this will not work.

Based on what you gave is, res is
# returns {"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}
So the plan is to grab the string starting at the curly brace to the end and use json to decode it:
import json
# Discard the text before the curly brace
res = res[res.index('{'):]
# Turn that text into a dictionary
d = json.loads(res)

All you need to do in your particular case is
d = eval(res)
And please keep security in mind when using eval, especially if you're mixing it with ajax/json.
UPDATE
Since others pointed out you might be getting this data over the web and it isn't just a "how to make this work" question, use this:
import json
json.loads(res)

Navigating Multi-Dimensional JSON arrays in Python

I'm trying to figure out how to query a JSON array in Python. Could someone show me how to do a simple search and print through a fairly complex array please?
The example I'm using is here: http://eu.battle.net/api/wow/realm/status
I'd like to see, for example, how to find the 'Silvermoon' server, and print say its 'population', then the 'controlling-faction' within the 'Wintergrasp' array.
The array snippet currently looks like this:
{"type":"pve","population":"high","queue":false,"wintergrasp":{"area":1,"controlling-faction":0,"status":0,"next":1382350068792},"tol-barad":{"area":21,"controlling-faction":0,"status":0,"next":1382349141932},"status":true,"name":"Silvermoon","slug":"silvermoon","battlegroup":"Cyclone / Wirbelsturm","locale":"en_GB","timezone":"Europe/Paris"}
At the moment I can access the main array, but don't seem to be able to access sub-arrays without copying the whole thing to another new variable which seems wasteful. I'd like to be able to do something like
import urllib2
import json
req = urllib2.Request("http://eu.battle.net/api/wow/realm/status", None, {})
opener = urllib2.build_opener()
f = opener.open(req)
x = json.load(f) # open the file and read into a variable
# search and find the Silvermoon server
silvermoon_array = ????
# print the population
print silvermoon_array.????
# access the Wintergrasp sub-array
wintergrasp_sub = ????
print wintergrasp_sub.???? # print the controlling-faction variable
This would really help me get to grips with how to access other things.

Python's interactive mode is a great way to progressively explore structured data. It's easy to find how to access, say, the silvermoon server data:
>>> data=json.load(urllib2.urlopen("http://eu.battle.net/api/wow/realm/status"))
>>> type(data)
<type 'dict'>
>>> data.keys()
[u'realms']
>>> type(data['realms'])
<type 'list'>
>>> type(data['realms'][0])
<type 'dict'>
>>> data['realms'][0].keys()
[u'status', u'wintergrasp', u'battlegroup', u'name', u'tol-barad', u'locale', u'queue', u'timezone', u'type', u'slug', u'population']
>>> data['realms'][0]['name']
u'Aegwynn'
>>> [realm['name'] for realm in data['realms']].index('Silvermoon')
212
>>> silvermoon= data['realms'][212]
>>> silvermoon['population']
u'high'
>>> type(silvermoon['wintergrasp'])
<type 'dict'>
>>> silvermoon['wintergrasp'].keys()
[u'status', u'next', u'controlling-faction', u'area']
>>> silvermoon['wintergrasp']['controlling-faction']
>>> silvermoon['population']
u'high'
If you don't know about them yet, you should read up on dictionary.keys, list.index and list comprehensions to understand what's going on.
After figuring out the structure of the data, you can finally rewrite the data access to be a bit more readable and efficient:
realms= data['realms']
realm_from_name= dict( [(realm['name'], realm) for realm in realms])
print realm_from_name['Silvermoon']['population']
print realm_from_name['Silvermoon']['wintergrasp']['controlling-faction']
As to copying the array to another variable being wasteful, you should know that python passes value by reference. That means that there's no copying involved when you assign something to a new variable. Here's a simple explanation of passing by value vs passing by reference
Finally, you seem to be excessively worried with performance. Python's philosophy is get it right first, optimize later. When you have this working correctly and if you need better performance, optimize it, if it's worth the time.

This does what you want:
# -*- coding: utf-8 -*-
import json
import urllib2
def searchListOfDicts(listOfDicts, attr, value):
"""
Loops through a list of dictionaries and returns matching attribute value pair
You can also pass it slug, silvermoon or type, pve
returns a list containing all matching dictionaries
"""
matches = [record for record in listOfDicts if attr in record and record[attr] == value]
return matches
def myjsonDict():
"""
Opens url, grabs json and puts it inside a dictionary
"""
req = urllib2.Request("http://eu.battle.net/api/wow/realm/status", None, {})
opener = urllib2.build_opener()
f = opener.open(req)
json_dict = json.load(f)
return json_dict
jsonDict = myjsonDict()
#we want to search inside realms list
silverMoonServers = searchListOfDicts(jsonDict["realms"], "name", "Silvermoon")
#access first dictionary that matched "name, Silvermoon" query
print silverMoonServers[0]
print silverMoonServers[0]["wintergrasp"]
print silverMoonServers[0]["wintergrasp"]["controlling-faction"]

In Python, json.loads maps json objects to python dictionaries, and Arrays to list, so further manipulation is done just like with regular python dict and list structures.
Here is how you can do it with requests and lamdbas:
import json
import requests
response = requests.get("http://eu.battle.net/api/wow/realm/status")
json_data = json.loads(response.text)
# loop through the list of realms to find the one you need (realm_name)
get_realm = lambda realm_name, jd: [r for r in jd['realms']
if r['name'] == realm_name]
# extract data you need, if there is a match in the list of realms,
# return None otherwise
get_your_data = lambda realm: (
realm[0]['name'],
realm[0]['wintergrasp']['controlling-faction']
) if realm else None
print get_your_data(get_realm('Silvermoon', json_data))

Convert a python dict to a string and back

I am writing a program that stores data in a dictionary object, but this data needs to be saved at some point during the program execution and loaded back into the dictionary object when the program is run again.
How would I convert a dictionary object into a string that can be written to a file and loaded back into a dictionary object? This will hopefully support dictionaries containing dictionaries.

The json module is a good solution here. It has the advantages over pickle that it only produces plain text output, and is cross-platform and cross-version.
import json
json.dumps(dict)

If your dictionary isn't too big maybe str + eval can do the work:
dict1 = {'one':1, 'two':2, 'three': {'three.1': 3.1, 'three.2': 3.2 }}
str1 = str(dict1)
dict2 = eval(str1)
print(dict1 == dict2)
You can use ast.literal_eval instead of eval for additional security if the source is untrusted.

I use json:
import json
# convert to string
input_ = json.dumps({'id': id_ })
# load to dict
my_dict = json.loads(input_)

Why not to use Python 3's inbuilt ast library's function literal_eval. It is better to use literal_eval instead of eval
import ast
str_of_dict = "{'key1': 'key1value', 'key2': 'key2value'}"
ast.literal_eval(str_of_dict)
will give output as actual Dictionary
{'key1': 'key1value', 'key2': 'key2value'}
And If you are asking to convert a Dictionary to a String then, How about using str() method of Python.
Suppose the dictionary is :
my_dict = {'key1': 'key1value', 'key2': 'key2value'}
And this will be done like this :
str(my_dict)
Will Print :
"{'key1': 'key1value', 'key2': 'key2value'}"
This is the easy as you like.

Use the pickle module to save it to disk and load later on.

Convert dictionary into JSON (string)
import json
mydict = { "name" : "Don",
"surname" : "Mandol",
"age" : 43}
result = json.dumps(mydict)
print(result[0:20])
will get you:
{"name": "Don", "sur
Convert string into dictionary
back_to_mydict = json.loads(result)

In Chinese language you should do the following adjustments:
import codecs
fout = codecs.open("xxx.json", "w", "utf-8")
dict_to_json = json.dumps({'text':"中文"},ensure_ascii=False,indent=2)
fout.write(dict_to_json + '\n')

You may find the json.dumps() method needs help handling some object types.
Credit goes to the top answer of this post for the following:
import json
json.dumps(my_dictionary, indent=4, sort_keys=True, default=str)

I think you should consider using the shelve module which provides persistent file-backed dictionary-like objects. It's easy to use in place of a "real" dictionary because it almost transparently provides your program with something that can be used just like a dictionary, without the need to explicitly convert it to a string and then write to a file (or vice-versa).
The main difference is needing to initially open() it before first use and then close() it when you're done (and possibly sync()ing it, depending on the writeback option being used). Any "shelf" file objects create can contain regular dictionaries as values, allowing them to be logically nested.
Here's a trivial example:
import shelve
shelf = shelve.open('mydata') # open for reading and writing, creating if nec
shelf.update({'one':1, 'two':2, 'three': {'three.1': 3.1, 'three.2': 3.2 }})
shelf.close()
shelf = shelve.open('mydata')
print shelf
shelf.close()
Output:
{'three': {'three.1': 3.1, 'three.2': 3.2}, 'two': 2, 'one': 1}

If you care about the speed use ujson (UltraJSON), which has the same API as json:
import ujson
ujson.dumps([{"key": "value"}, 81, True])
# '[{"key":"value"},81,true]'
ujson.loads("""[{"key": "value"}, 81, true]""")
# [{u'key': u'value'}, 81, True]

I use yaml for that if needs to be readable (neither JSON nor XML are that IMHO), or if reading is not necessary I use pickle.
Write
from pickle import dumps, loads
x = dict(a=1, b=2)
y = dict(c = x, z=3)
res = dumps(y)
open('/var/tmp/dump.txt', 'w').write(res)
Read back
from pickle import dumps, loads
rev = loads(open('/var/tmp/dump.txt').read())
print rev

I figured out the problem was not with my dict object it was the keys and values that were of RubyString type after loading it with RubyMarshl 'loads' method
So i did this:
dic_items = dict.items()
new_dict = {str(key): str(value) for key, value in dic_items}