Is there an exception free way to access values from a dictionary containing lists. For example, if I have:
data = {
"object_1": {
"object_2": {
"list": [
{
"property": "hello"
}
]
}
}
}
How do I access the path data['object_1']['object_2']['list'][0]['property'] safely(i.e. return some default value if not possible to access without throwing error)? I am trying to avoid wrapping these in try-except's. I have seen the reduce based approach but it doesn't take into account having lists inside the dictionary.
In JS, I can write something like:
data.object_1?.object_2?.list[0]?.property ?? 'nothing_found'
Is there something similar in Python?
For dict you can use the get method. For lists you can just be careful with the index:
data.get('object_1', {}).get('object_2', {}).get('list', [{}])[0].get('property', default)
This is a bit awkward because it makes a new temporary dict or lost for each call get. It's also not super safe for lists, which don't have an equivalent method.
You can wrap the getter in a small routine to support lists too, but it's not really worth it. You're better off writing a one-off utility function that uses either exception handling or preliminary checking to handle the cases you want to react to:
def get(obj, *keys, default=None):
for key in keys:
try:
obj = obj[key]
except KeyError, IndexError:
return default
return obj
Exception handing has a couple of huge advantages over doing it the other way. For one thing, you don't have to do separate checks on the key depending on whether the object is a dict or list. For another, you can support almost any other reasonable type that supports __getitem__ indexing. To show what I mean, here is the asking for permission rather than forgiveness approach:
from collections.abc import Mapping, Sequence
from operator import index
def get(obj, *keys, default=None):
for key in keys:
if isinstance(obj, Mapping):
if key not in obj:
return default
elif isinstance(obj, Sequence):
try:
idx = index(key)
except TypeError:
return default
if len(obj) <= idx or len(obj) < -idx:
return default
obj = obj[key]
return obj
Observe how awkward and error-prone the checking is. Try passing in a custom object instead of a list, or a key that's not an integer. In Python, carefully used exceptions are your friend, and there's a reason it's pythonic to ask for forgiveness rather than for permission.
Uggh. Yeah, accessing such JSON data structures is just terrible,
it's a bit awkward.
Glom to the rescue!
There's two ways to win:
You can just specify ... , default=None) to avoid exceptions, ..or..
Use Coalesce.
print(glom(data, {'object_1.object_2.list': ['property']}, default=None))
In the below code, x will return None if 'object_1'/'object_2'/'list' key does not exist.
Also, if we are able to access 'list' key then we have x as Not None and we should ensure that the length of the list should be greater than zero and then we can search for 'property' key.
x = data.get('object_1', {}).get('object_2', {}).get('list')
if x is not None and len(x) > 0:
print(x[0].get('property'))
else:
print(None)
There is one way to do that, but it would involve the get method and would involve a lot of checking, or using temporary values.
One example lookup function would look like that:
def lookup(data):
object_1 = data.get("object_1")
if object_1 is None:
# return your default
object_2 = object_1.get('object_2')
# and so on...
In Python 3.10 and above, there is also structural pattern matching that can help, in which case you would do something like this:
match data:
case {'object_1': {'object_2': {'list': [{'property': x}]}}}:
print(x) # should print 'hello'
case _:
print(<your_default>)
Please remember that this only works with the latest versions of Python (the online Python console on Python.org is still only on Python3.9, and the code above would cause a syntax error).
Related
Some one know if its possible to get a dict inside a dict by using their keys as strings... Sorry for the bad explation i think the code would be more clear... don't know how to properly say this
s={"framework": {"challenger": {
"logo.qvg": "50c137c8199d0a807bc31b5c384e67d6c1c0837061bd7d39ed15943b"}}}
possible_folders=["tractor","svg","framework","common",'framework["challenger"]']
print(s['framework']['challenger']) # this line is working
print(s[possible_folders[-1]]) # here im getting keyerror-> due to the string...
I think what you want is a way to access nested dicts by a sequence of keys. There are many ways to do this. One I like is to define a simple function like:
def nested_get(d, *keys, *, default=None):
for k in keys:
try:
d = d[k]
except KeyError:
return default
return d
Then use a tuple like keys = ('framework', 'challenger'):
nested_get(d, *keys)
It's not possible. s is a dict with a single key, "framework". There is no way to index s to get something deeper inside one of s's values.
You can write a function that takes a list of keys and iteratively digs into the dict, however. (Iguananaut does this with better error handling.)
def chain_lookup(s, *keys):
d = s
for key in keys:
d = d[key]
return d
result = chain_lookup(s, "framework", "challenger")
# result == {"logo.qvg": "..."}
You can use the eval() function like:
s={"framework": {"challenger": {
"logo.qvg": "50c137c8199d0a807bc31b5c384e67d6c1c0837061bd7d39ed15943b"}}}
possible_folders=["tractor","svg","framework","common",'framework["challenger"]']
def get_entry(d, k):
eval_string = "d" +"".join([".get('" + i.replace('"',"").replace("[","").replace("]","") + "')" for i in k.split("[")])
return eval(eval_string)
get_entry(s,possible_folders[-1])
Out[32]: {'logo.qvg': '50c137c8199d0a807bc31b5c384e67d6c1c0837061bd7d39ed15943b'}
I am not saying I would recommend it, but it is the only solution that works with the String Syntax in your questions. The other answers are better, however require a change in your list
I agree with the above answers, because they're how you do it in a good programming way. Although I have a quick fix to make your exact command work, but it is NOT a good programming way, since it uses hardcoding and also eval function.
In the first line, you are accessing with print(s['framework']['challenger']), but in second line, you are accessing like s['framework["challenger"]' (see the brackets).
You can change the possible folders from
possible_folders=["tractor","svg","framework","common",'framework["challenger"]']
to
possible_folders=["tractor","svg","framework","common",'["framework"]["challenger"]']
eval("s"+possible_folders[-1])
First of all I strongly suggest reading this - I would go that way.
Having said that, below you will find the non-responsible way (using eval) of achieving this. Again, strongly suggest going that way.
possible_folders=["tractor","svg","framework","common","['framework']['challenger']"]
print(eval('s'+possible_folders[-1]))
I need a dictionary with a key of type string, but you don't need to enter the whole word to access the correspondent value, just some of the first characters, enough of them to don´t match any other key (similar to typing in a console, and press tab and the console guess the correct command because no other match it)
For example in the dictionary:
dicc = { 'suma':fun_suma, 'resta':fun_resta, 'producto':fun_producto }
I need that dicc['pr'], dicc['prod'], dicc['produc'], even just dicc['p'], return fun_producto (in this example, because the strings entered doesn't match any other key).
Is there any structure in python that makes that? If not, which algorithm do you suggest to build it? I know that behind any Python dictionary, there is an efficient tree of keys that lets found the asked key quickly. I would like to take advantage of that.
UPDATES:
Yes, I was wrong, Python dictionary uses a hash table, not a tree. I mixed my memories.
Yeah, dicc['p'] was a dramatic example, but I used it just to show that in some cases it could abbreviate so much the entry of characters. Of course it will be useful only if the keys are well-known and a finite set, like a parser.
You could make a simple dictionary subclass that overrides __getitem__() and returns values. You will probably want to prevent non-string keys, since dictionaries can have keys like numbers where prefixes don't make as much sense.
This might be a start, it will return the first key that matches the prefix:
from collections import UserDict
class PrefixDict(UserDict):
def __setitem__(self, name, value):
if not isinstance(name, str):
raise ValueError("Keys must be strings")
super().__setitem__(name, value)
def __getitem__(self, name):
try:
return next(self.data[k] for k in self.data.keys() if k.startswith(name))
except StopIteration as error:
raise KeyError(f"Prefix '{name}' not found") from None
dicc = PrefixDict({'suma':'fun_suma', 'resta':'fun_resta', 'producto':'fun_producto' })
dicc['pr']
#'fun_producto'
dicc["q"]
# raises KeyError
dicc[""]
# ??
Instead of forcing keys to be strings, you could instead test in __getitem__() and defer non-string keys to the parent.
There is a python dictionary:
a = {b:{c:{"x":1, "y":2, "z":3}}}
I want to know if a[b][c]["z"] exists,
but yet I don't know if a[b][c] or a[b] exist either.
So, if I do:
if "z" in a[b][c]:
I may get a "key c doesn't exist in a[b]" or "key b doesn't exist in a" error.
How to properly check if z exists in a[b][c] in this case?
The python way is not to bother with the check at all.
try:
value = a[b][c]["z"]
# do something with value
except KeyError:
print 'Sorry wrong key'
here any combination of a,b,"z" maybe missing from the dictionary but they will all be caught by the exception handler. OTH, the situation where the exact key exists will result in the rest of your code being executed.
You might also want to consider using defaultdict
The first argument provides the initial value for the default_factory
attribute; it defaults to None. All remaining arguments are treated
the same as if they were passed to the dict constructor, including
keyword arguments.
This gives you the assurance that notation like a[b][c]["z"] will always execute with out errors and maybe useful in some situations, particularly where you are trying to avoid boilerplatish code.
IF you can't use an exception for some reason (eg. lambda func, list comprehension, generator expression etc)
value = a.get(b, {}).get(c, {}).get("z", None)
But normally you should prefer to use the exception handler
Why don't you use your own function, it gives you flexibility, Recursion!
def _finditem(obj, key):
if key in obj: return obj[key]
for k, v in obj.items():
if isinstance(v,dict):
item = _finditem(v, key)
if item is not None:
return item
Of course, that will fail if you have None values in any of your dictionaries. In that case, you could set up a sentinel object() for this function and return that in the case that you don't find anything -- Then you can check against the sentinel to know if you found something or not.
I love python one liners:
u = payload.get("actor", {}).get("username", "")
Problem I face is, I have no control over what 'payload' contains, other than knowing it is a dictionary. So, if 'payload' does not have "actor", or it does and actor does or doesn't have "username", this one-liner is fine.
Problem of course arises when payload DOES have actor, but actor is not a dictionary.
Is there as pretty a way to do this comprehensively as a one liner, and consider the possibility that 'actor' may not be a dictionary?
Of course I can check the type using 'isinstance', but that's not as nice.
I'm not requiring a one liner per se, just asking for the most efficient way to ensure 'u' gets populated, without exception, and without prior knowledge of what exactly is in 'payload'.
Using EAFP
As xnx suggested, you can take advantage of the following python paradigm:
Easier to ask for forgiveness than permission
you can use it on KeyErrors as well:
try:
u = payload["actor"]["username"]
except (AttributeError, KeyError):
u = ""
Using a wrapper with forgiving indexing
Sometimes it would be nice to have something like null-conditional operators in Python. With some helper class this can be compressed into a one-liner expression:
class Forgive:
def __init__(self, value = None):
self.value = value
def __getitem__(self, name):
if self.value is None:
return Forgive()
try:
return Forgive(self.value.__getitem__(name))
except (KeyError, AttributeError):
return Forgive()
def get(self, default = None):
return default if self.value is None else self.value
data = {'actor':{'username': 'Joe'}}
print(Forgive(data)['actor']['username'].get('default1'))
print(Forgive(data)['actor']['address'].get('default2'))
ps: one could redefine __getattr__ as well besides __getitem__, so you could even write Forgive(data)['actor'].username.get('default1').
Why not use an Exception:
try:
u = payload.get("actor", {}).get("username", "")
except AttributeError:
u = ""
The answer hege_hegedus gave is correct, however there's one caveat in that exception handling is a lot slower than going through if..else statement.
For example if you're iterating over thousands of payload object and an actor entry is only occasionally not a dictionary, this code is perfectly valid.
However if you're iterating over thousands of payload objects and every other actor entry is not a dictionary then you'd be better off with this code.
u = ''
if 'actor' in payload and isinstance(payload['actor'], dict):
u = payload['actor'].get('username', '')
For more discussion go here -- https://mail.python.org/pipermail/tutor/2011-January/081143.html
UPDATE
Also the code statement can be re-written as a one-liner albeit not nearly as legible as two-line statement
u = payload['actor'].get('username', '') if 'actor' in payload and isinstance(payload['actor'], dict) else ''
If you really need to do it in 1 line, you'll have to implement the functionality yourself. Which is worth doing if you use this semantics many times in your program.
There are two ways to do it: function or custom dictionary-like object for payload.
1) Function handles the case of actor being not a dict. It can check for isinstance or do the try or whatever else -- it's not essential. The usage would look something like u = get("username", "", payload.get("actor", {})) or u = get("", payload, 'actor', 'username') (with arbitrary amount of nested calls for items in payload).
2) A class of custom objects is a powerful thing -- do it if you can and really need this abstraction in the program. A descendant of dict or UserDict (in Python3) can check for what it stores or outputs on __getitem__ calls.
I'm creating instances of a class Foo, and I'd like to be able to instantiate these in a general way from a variety of types. You can't pass Foo a dict or list. Note that Foo is from a 3rd party code base - I can't change Foo's code.
I know that type checking function arguments in Python is considered bad form. Is there a more Pythonic way to write the function below (i.e. without type checking)?
def to_foo(arg):
if isinstance(arg, dict):
return dict([(key,to_foo(val)) for key,val in arg.items()])
elif isinstance(arg, list):
return [to_foo(i) for i in arg]
else:
return Foo(arg)
Edit: Using try/except blocks is possible. For instance, you could do:
def to_foo(arg):
try:
return Foo(arg)
except ItWasADictError:
return dict([(key,to_foo(val)) for key,val in arg.items()])
except ItWasAListError:
return [to_foo(i) for i in arg]
I'm not totally satisfied by this for two reasons: first, type checking seems like it addresses more directly the desired functionality, whereas the try/except block here seems like it's getting to the same place but less directly. Second, what if the errors don't cleanly map like this? (e.g. if passing either a list or dict throws a TypeError)
Edit: a third reason I'm not a huge fan of the try/except method here is I need to go and find what exceptions Foo is going to throw in those cases, rather than being able to code it up front.
If you're using python 3.4 you can use functools.singledispatch, or a backport for a different python version
from functools import singledispatch
#singledispatch
def to_foo(arg):
return Foo(arg)
#to_foo.register(list)
def to_foo_list(arg):
return [Foo(i) for i in arg]
#to_foo.register(dict)
def to_foo_dict(arg):
return {key: Foo(val) for key, val in arg.items()}
This is a fairly new construct for python, but a common pattern in other languages. I'm not sure you'd call this pythonic or not, but it does feel better than writing isinstances everywhere. Though, in practise, the singledispatch is probably just doing the isinstance checks for you internally.
The pythonic way to deal with your issue is to go ahead and assume (first) that arg is Foo and except any error:
try:
x = Foo(arg)
except NameError:
#do other things
The phrase for this idea is "duck typing", and it's a popular pattern in python.