I am very new to Python and parsing data.
I can pull an external JSON feed into a Python dictionary and iterate over the dictionary.
for r in results:
print r['key_name']
As I walk through the results returned, I am getting an error when a key does not have a value (a value may not always exist for a record). If I print the results, it shows as
'key_name': None, 'next_key':.................
My code breaks on the error. How can I control for a key not having a value?
Any help will be greatly appreciated!
Brock
The preferred way, when applicable:
for r in results:
print r.get('key_name')
this will simply print None if key_name is not a key in the dictionary. You can also have a different default value, just pass it as the second argument:
for r in results:
print r.get('key_name', 'Missing: key_name')
If you want to do something different than using a default value (say, skip the printing completely when the key is absent), then you need a bit more structure, i.e., either:
for r in results:
if 'key_name' in r:
print r['key_name']
or
for r in results:
try: print r['key_name']
except KeyError: pass
the second one can be faster (if it's reasonably rare than a key is missing), but the first one appears to be more natural for many people.
There are two straightforward ways of reading from Python dict if key might not be present. for example:
dicty = {'A': 'hello', 'B': 'world'}
The pythonic way to access a key-value pair is:
value = dicty.get('C', 'default value')
The non-pythonic way:
value = dicty['C'] if dicty['C'] else 'default value'
even worse:
try:
value = dicty['C']
except KeyError as ke:
value = 'default value'
If possible, use the simplejson library for managing JSON data.
the initial question in this thread is why I wrote the Dictor library, it handles JSON fallback and None values gracefully without needing try/except or If blocks.
Also gives you additional options like ignore upper/lower case,
see,
https://github.com/perfecto25/dictor
use has_key() , and that will return true or false
[Updated to remove careless mistake]
You could also do something like this:
for r in (row for row in results if 'a' in row):
print r['a']
This uses a generator expression to pick "rows" out of "results" where "row" includes the key "a".
Here's a little test script:
results = [{'a':True}, {'b':True}, {'a':True}]
for r in (row for row in results if 'a' in row): print r['a']
You can use the built in function hasattr
key='key_name'
# or loop your keys
if hasattr(e, key):
print(e[key])
else:
print('No key for %s' % key)
Taken from https://stackoverflow.com/a/14923509/1265070
id = getattr(myobject, 'id', None)
Related
I am given an assignment when I am supposed to define a function that returns the second element of a tuple if the first element of a tuple matches with the argument of a function.
Specifically, let's say that I have a list of student registration numbers that goes by:
particulars = (("S12345", "John"), ("S23456", "Max"), ("S34567", "Mary"))
And I have defined a function that is supposed to take in the argument of reg_num, such as "S12345", and return the name of the student in this case, "John". If the number does not match at all, I need to print "Not found" as a message. In essence, I understand that I need to sort through the larger tuple, and compare the first element [0] of each smaller tuple, then return the [1] entry of each smaller tuple. Here's what I have in mind:
def get_student_name(reg_num, particulars):
for i in records:
if reg_num == particulars[::1][0]:
return particulars[i][1]
else:
print("Not found")
I know I'm wrong, but I can't tell why. I'm not well acquainted with how to sort through a tuple. Can anyone offer some advice, especially in syntax? Thank you very much!
When you write for i in particulars, in each iteration i is an item of the collection and not an index. As such you cannot do particulars[i] (and there is no need - as you already have the item). In addition, remove the else statement so to not print for every item that doesn't match condition:
def get_student_name(reg_num, particulars):
for i in particulars:
if reg_num == i[0]:
return i[1]
print("Not found")
If you would want to iterate using indices you could do (but less nice):
for i in range(len(particulars)):
if reg_num == particulars[i][0]:
return particulars[i][1]
Another approach, provided to help learn new tricks for manipulating python data structures:
You can turn you tuple of tuples:
particulars = (("S12345", "John"), ("S23456", "Max"), ("S34567", "Mary"))
into a dictionary:
>>> pdict = dict(particulars)
>>> pdict
{'S12345': 'John', 'S23456': 'Max', 'S34567': 'Mary'}
You can look up the value by supplying the key:
>>> r = 'S23456'
>>> dict(pdict)[r]
'Max'
The function:
def get_student_name(reg, s_data):
try:
return dict(s_data)[reg]
except:
return "Not Found"
The use of try ... except will catch errors and just return Not Found in the case where the reg is not in the tuple in the first place. It will also catch of the supplied tuple is not a series of PAIRS, and thus cannot be converted the way you expect.
You can read more about exceptions: the basics and the docs to learn how to respond differently to different types of error.
for loops in python
Gilad Green already answered your question with a way to fix your code and a quick explanation on for loops.
Here are five loops that do more or less the same thing; I invite you to try them out.
particulars = (("S12345", "John"), ("S23456", "Max"), ("S34567", "Mary"))
for t in particulars:
print("{} {}".format(t[0], t[1]))
for i in range(len(particulars)):
print("{}: {} {}".format(i, particulars[i][0], particulars[i][1]))
for i, t in enumerate(particulars):
print("{}: {} {}".format(i, t[0], t[1]))
for reg_value, student_name in particulars:
print("{} {}".format(reg_value, student_name))
for i, (reg_value, student_name) in enumerate(particulars):
print("{}: {} {}".format(i, reg_value, student_name))
Using dictionaries instead of lists
Most importantly, I would like to add that using an unsorted list to store your student records is not the most efficient way.
If you sort the list and maintain it in sorted order, then you can use binary search to search for reg_num much faster than browsing the list one item at a time. Think of this: when you need to look up a word in a dictionary, do you read all words one by one, starting by "aah", "aback", "abaft", "abandon", etc.? No; first, you open the dictionary somewhere in the middle; you compare the words on that page with your word; then you open it again to another page; compare again; every time you do that, the number of candidate pages diminishes greatly, and so you can find your word among 300,000 other words in a very small time.
Instead of using a sorted list with binary search, you could use another data structure, for instance a binary search tree or a hash table.
But, wait! Python already does that very easily!
There is a data structure in python called a dictionary. See the documentation on dictionaries. This structure is perfectly adapted to most situations where you have keys associated to values. Here the key is the reg_number, and the value is the student name.
You can define a dictionary directly:
particulars = {'S12345': 'John', 'S23456': 'Max', 'S34567': 'Mary'}
Or you can convert your list of tuples to a dictionary:
particulars = (("S12345", "John"), ("S23456", "Max"), ("S34567", "Mary"))
particulars_as_dict = dict(particulars)
Then you can check if an reg_number is in the dictionary, with they keyword in; you can return the student name using square brackets or with the method get:
>>> particulars = {'S12345': 'John', 'S23456': 'Max', 'S34567': 'Mary'}
>>> 'S23456' in particulars
True
>>> 'S98765' in particulars
False
>>>
>>> particulars['S23456']
'Max'
>>> particulars.get('S23456')
'Max'
>>> particulars.get('S23456', 'not found')
'Max'
>>>
>>> particulars['S98765']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'S98765'
>>> particulars.get('S98765')
None
>>> particulars.get('S98765', 'not found')
'not found'
What is the shortest way to check for the existence of a certain key/value pair in a dictionary if I don't know that the key exists?
So far, I've come up with:
if 'key' in my_dict and my_dict['key'] == 'value':
do_something()
This is really long with longer variable names or longer key/value names, such as:
if 'X-Powered-By' in self.request.headers and self.request.headers['X-Powered-By'] == 'NodeBB':
do_something()
What's a shorter way to check for the presence of a key and a corresponding value?
You can fetch the value, and compare it right away:
# default return value is None if key is not found
if mydict.get("key") == "somevalue"
or
# Or specify your own default value
if mydict.get("key", False) == "somevalue"
Check out Python dict.get docs.
Actually, none of the answers captures the full problem. If the value that is being queried for happens to be None or whatever default value one provides, the get()-based solutions fail. The following might be the most generally applicable solution, not relying on defaults, truly checking the existence of a key (unlike get()), and not over-'except'-ing KeyErrors (unlike the other try-except answer) while still using O(1) dict lookup (unlike items() approach):
try:
assert my_dict[key] == value:
except (KeyError, AssertionError):
do_sth_else() # or: pass
else:
do_something()
If you're just looking for existence of the key
if not 'key' in my_dict:
my_dict['key'] = 'some default value'
Ok, my suggestion, to make your code more readable:
headers = self.request.headers
if headers.get('X-Powered-By') == 'NodeBB':
do_something()
This could be another short code but definitely not efficient as a dict.get:
if ('X-Powered-By', 'NodeBB') in self.request.headers.items():
do_something()
I'm reading a file and putting contents into dictionary. I'm writing a method where I search for key and return its value. How do I throw exception if my key is not present in dictionary. For example below is the code I'm testing but I get output of re.search as None for non-match items.
Can I use has_key() method?
mylist = {'fruit':'apple','vegi':'carrot'}
for key,value in mylist.items():
found = re.search('vegi',key)
if found is None:
print("Not found")
else:
print("Found")
Found
Not found
Python trends towards the "Easier to Ask Forgiveness than Permission" model versus "Look Before You Leap". So in your code, don't search for the key before trying to pull it's value, just pull for it's value and handle the fallout as needed (and where needed).
*Assuming you're asking how to find one key, and return it's value.
EAFP approach:
def some_func(key)
my_dict = {'fruit':'apple', 'vegi':'carrot'}
return my_dict[key] # Raises KeyError if key is not in my_dict
If a LBYP is what you have to do, try this:
def some_func(key):
my_dict = {'fruit':'apple', 'vegi':'carrot'}
if not key in my_dict:
raise SomeException('my useful exceptions message')
else:
return my_dict[key]
The biggest problem with the LBYP approach is that it introduces a race condition; the 'key' may or may not exist between checking for it, then returning it's value (which is only possible when doing current work).
You can simply use 'in'.
mylist = {'fruit':'apple','vegi':'carrot'}
test = ['fruit', 'vegi', 'veg']
for value in test:
if value in mylist:
print(value + ' is in the dict, its value : ' + mylist[value])
else:
raise Exception(value + ' not in dict.')
# Console
# fruit is in the dict, its value: apple
# vegi is in the dict, its value: carrot
# Exception: veg is not in dict
#JRazor gave you several ways of using list comprehension, lambda and filter for doing what you call a "has_key() method" (I get SyntaxErrors when I copy/paste them to python 2.7 interpreter though?)
Here's the literal answer to your question: "How do I throw exception if my key is not present in dictionary?"
What many languages refer to as throw (an exception), python calls raise (an exception).
More info on that here.
In your case, you could add a custom exception like so:
mylist = {'fruit':'apple','vegi':'carrot'} # mylist is a dictionary. Just sayin'
if "key" not in mylist:
raise Exception("Key not found")
else:
print "Key found"
I'm using ConfigParser which returns a dictionary of configuration data as such:
{'general': {'UserKey': 'thisisatestkey'}}
If I want to simply print the value of the UserKey key (in this case thisisatestkey), then I generally just do a print "Your key is: {0}".format(mydictvar.get('UserKey')).
If I just print out the raw dict to a string I get the above. If I use the print statement above I get result of None since there is no key in the root of the dict called UserKey. If I .get('general') I just get: {'UserKey': 'thisisatestkey'}
Obviously I could do a fore loop like so:
keydic = cp.get_config_data()
for m, k in keydic.iteritems():
for s, v in k.iteritems():
userkey = v
and then print userkey which works fine. But I want to know how I can just avoid having to do the entire for loop first and just print the darned value right inline? Thanks!
You can use
mydictvar['general']['UserKey']
Or, if keys might be missing
mydictvar.get('general', {}).get('UserKey')
mydictvar['general'] returns a dictionary object; you can then just apply [...] to that value to retrieve the next key.
This works in string formatting too:
>>> mydictvar = {'general': {'UserKey': 'thisisatestkey'}}
>>> print "Your key is: {0[general][UserKey]}".format(mydictvar)
Your key is: thisisatestkey
simply without loop:
>>> my_dict = {'general': {'UserKey': 'thisisatestkey'}}
>>> my_dict['general']['UserKey']
'thisisatestkey'
If mysql has no output...
if record[0][0]:
will return an error
IndexError: tuple index out of range
the only solution i know to fix this issue is:
try:
if record[0][0]:
# Do stuff
except IndexError:
pass
but this looks like a very heavy wrapper script
only to find out if
record[0][0]
has no data. ( no value )
is there something lighter that can be done such as..
if record[0][0] = ""
?
UPDATE:
This is my MYSQL code:
a = _mysql.escape_string(a)
db=b()
db.query("select * from b where a='" + a + "' limit 1")
result = db.store_result()
record = result.fetch_row()
UPDATE:
turns out what worked is:
if record:
rather than
if record[0]:
or
if record[0][0]:
In the general case, if you want to check if an item exists in a list, just check that it exists. Exceptions are considered Pythonic code. Using another construct for access checking is likely to be less readable and suffer from performance problems.
However, if you're really interested in something else.. how about this?
>>> if record[0]:
... field = record[0][0]
This works because an empty list ([]) evaluates as False in an if statement.
>>> record = [[]]
>>> if record[0]: # returns an empty list, e.g. []
... field = record[0][0] # is not executed
A simpler alternative:
import MySQLdb
conn = MySQLdb.connect(passwd="sekret",db="foo")
cur = conn.cursor()
cur.execute("select * from b where a=%s limit 1", (a,))
for result in cur:
print(result)
Note the changes:
Use MySQLdb, not the underlying _mysql* API
Don't concatenate variables into SQL query strings, this will lead to SQL injection.
Iterate over the cursor to get the results
In Python, there is a way to get a default value from a dict but not from a list. E.g. in a dict:
x = mydict.get('key') # x will be None if there is no 'key'
(you can also provide a different default as a 2nd arg to get() method)
Now, it would be convenient to have something like that for lists. Getting an item from a list
is in some ways very similar to getting an item from a dict, but not exactly the same. Python
made a design decision to not have a similar method for lists.
Of course, you can make your own very easily. I sometimes use a function in my own library
called 'getitem', which returns a default arg; however it only looks up one level of a list,
because I feel multiple levels is too much of a corner case and it's probably worth using
an exception with multiple levels. But, for one level you can do:
def getitem(seq, index, default=None):
"""Get item from a `seq` at `index`, return default if index out of range."""
try : return seq[index]
except IndexError : return default
Note that there's a gotcha: you can't assume that getting None back means there is no
item, if your list may itself contain None values. Seems obvious but that's something
you have to remember.
You can easily extend this function to accept multiple indexes and handle multiple
levels of lists, but then you may ask: how do I know at which level there was an
IndexError?