We Keep Coding

Dynamically nesting a list, and related comprehension/mapping to find indices of string match

The context of what I'm doing: I'm translating if/then/else statements between 2 languages via a Python script (2x for now, but may eventually upgrade to 3x). I have a function that takes the if/then/else statement from the original language and breaks it into a list of [if_clause,then_clause,else_clause]. The thing is, there may be (and often are) nested if statements in the then and/or else clauses. For example, I would pass a string like...
if (sim_time<=1242) then (new_tmaxF0740) else if (sim_time<=2338) then (new_tmaxF4170) else (new_tmaxF7100)
...to my function, and it would return the list...
['(sim_time<=1242)','(new_tmaxF0740)','if (sim_time<=2338) then (new_tmaxF4170) else (new_tmaxF7100)']
So, as you can see, in this case the else clause needs to be further broken up by running it again through the same function I used to generate the list, this time only passing the last list element to that function. I am going about this by testing the original string to see if there are more than 1 if statements contained (I already have the regex for this) and my thought is to use a loop to create nested lists within the original list, that might then look like...
[if_clause,then_clause,[if_clause, then_clause, else_clause]]
These can be nested any number of times/to any dimension. My plan so far is to write a loop that looks for the next nested if statement (using a regex), and reassigns the list index where the if statement is found to the resultant list from applying my if_extract() function to break up the statement.
I feel like list comprehension may not do this, because to find the indices, it seems like the list comprehension statement might have to dynamically change. Maybe better suited for map, but I'm not sure how to apply? I ultimately want to iterate through the loop to return the index of the next (however deeply nested) if statement so I can continue breaking them apart with my function.

If I understand correctly, you could call your function recursively.
def split_if_then_else(str):
if check_if_if_in_string_function(str)
if_clause, then_clause, else_clause = split_str_core_function(str)
then_clause = split_if_then_else(str)
return [if_clause, then_clause, else_clause]
else:
return str
I didn't test it since I don't know what functions you are using exactly, but I think something like this should work

Am I using `all` correctly?

A user asked (Keyerror while using pandas in PYTHON 2.7) why he was having a KeyError while looking in a dictionary and how he could avoid this exception.
As an answer, I suggested him to check for the keys in the dictionary before. So, if he needed all the keys ['key_a', 'key_b', 'key_c'] in the dictionary, he could test it with:
if not all([x in dictionary for x in ['key_a', 'key_b', 'key_c']]):
continue
This way he could ignore dictionaries that didn't have the expected keys (the list of dictionaries is created out of JSON formatted lines loaded from a file). *Refer to the original question for more details, if relevant to this question.
A user more experienced in Python and SO, which I would consider an authority on the matter for its career and gold badges told me I was using all incorrectly. I was wondering if this is really the case (for what I can tell, that works as expected) and why, or if there is a better way to check if a couple of keys are all in a dictionary.

Yes that will work fine, but you don't even need the list comprehension
if not all(x in dictionary for x in ['key_a', 'key_b', 'key_c']):
continue
If you have the surrounding [], it will evaluate all the elements before calling all. If you remove them, the inner expression is a generator, and will short-circuit upon the first False.

A more idiomatic way to fill a dictionary in Python

Currently, I'm trying to fill a dictionary in Python but I think what I'm doing is a little bit redundant. Is there a more pythonic way to do the following:
if not pattern_file_map.get(which_match):
pattern_file_map[which_match] = [line]
else:
pattern_file_map[which_match].append(line)
where pattern_file_map is a dictionary.
I know that there is a certain idiom to use when checking if there is a key in a dictionary, like
this question, but I I just want to fill this dictionary with a lists.

You could use
pattern_file_map.setdefault(which_match, []).append(line)
instead.
Other people might suggest using a collections.defaultdict(list) instead, which is another option, but be warned that this might hide errors, since it will silently create all keys you access.

You could try using a collections.defaultdict.

Since you're adding to a maybe existing value:
pattern_file_map.setdefault(which_match, []).append(line)
If using dict.get():
li = pattern_file_map.get(which_match, [])
li.append(line)
pattern_file_map[which_match] = li
Of course, both cases are restricted to the case where your dict's values are lists.

Excel CSV into Nested Dictionary; List Comprehensions

I have a Excel CSV files with employee records in them. Something like this:
mail,first_name,surname,employee_id,manager_id,telephone_number
blah#blah.com,john,smith,503422,503423,+65(2)3423-2433
foo#blah.com,george,brown,503097,503098,+65(2)3423-9782
....
I'm using DictReader to put this into a nested dictionary:
import csv
gd_extract = csv.DictReader(open('filename 20100331 original.csv'), dialect='excel')
employees = dict([(row['employee_id'], row) for row in gp_extract])
Is the above the proper way to do it - it does work, but is it the Right Way? Something more efficient? Also, the funny thing is, in IDLE, if I try to print out "employees" at the shell, it seems to cause IDLE to crash (there's approximately 1051 rows).
2. Remove employee_id from inner dict
The second issue issue, I'm putting it into a dictionary indexed by employee_id, with the value as a nested dictionary of all the values - however, employee_id is also a key:value inside the nested dictionary, which is a bit redundant? Is there any way to exclude it from the inner dictionary?
3. Manipulate data in comprehension
Thirdly, we need do some manipulations to the imported data - for example, all the phone numbers are in the wrong format, so we need to do some regex there. Also, we need to convert manager_id to an actual manager's name, and their email address. Most managers are in the same file, while others are in an external_contractors CSV, which is similar but not quite the same format - I can import that to a separate dict though.
Are these two items things that can be done within the single list comprehension, or should I use a for loop? Or does multiple comprehensions work? (sample code would be really awesome here). Or is there a smarter way in Python do it?
Cheers,
Victor

Your first part has one simple issue (which might not even be an issue). You don't handle key collisions at all (unless you intend to simply overwrite).
>>> dict([('a', 'b'), ('a', 'c')])
{'a': 'c'}
If you're guaranteed that employee_id is unique, there isn't an issue though.
2) Sure you can exclude it, but no real harm done. Actually, especially in python, if employee_id is a string or int (or some other primitive), the inner dict's reference and the key actually reference the same thing. They both point to the same spot in memory. The only duplication is in the reference (which isn't that big). If you're worried about memory consumption, you probably don't have to.
3) Don't try to do too much in one list comprehension. Just use a for loop after the first list comprehension.
To sum it all up, it sounds like you're really worried about the performance of iterating over the loop twice. Don't worry about performance initially. Performance problems come from algorithm problems, not specific language constructs like for loops vs list comprehensions.
If you're familiar with Big O notation, the list comprehension and for loop after (if you decide to do that) both have a Big O of O(n). Add them together and you get O(2n), but as we know from Big O notation, we can simplify that to O(n). I've over simplified a lot here, but the point is, you really don't need to worry.
If there are performance concerns, raise them after you written the code and prove it to yourself with a code profiler.
response to comments
As for your #2 reply, python really doesn't have a lot of mechanisms for making one liners cute and extra snazzy. It's meant to force you into simply writing the code out vs sticking it all in one line. That being said, it's still possible to do quite a bit of work in one line. My suggestion is to not worry about how much code you can stick in one line. Python looks a lot more beautiful (IMO) when its written out, not jammed in one line.
As for your #1 reply, you could try something like this:
employees = {}
for row in gd_extract:
if row['employee_id'] in employees:
... handle duplicates in employees dictionary ...
else:
employees[row['employee_id']] = row
As for your #3 reply, not sure what you're looking for and what about the telephone numbers you'd like to fix, but... this may give you a start:
import re
retelephone = re.compile(r'[-\(\)\s]') # remove dashes, open/close parens, and spaces
for empid, row in employees.iteritems():
retelephone.sub('',row['telephone'])

Fastest way to uniqify a list in Python

Fastest way to uniqify a list in Python without preserving order? I saw many complicated solutions on the Internet - could they be faster than simply:
list(set([a,b,c,a]))

Going to a set only works for lists such that all their items are hashable -- so e.g. in your example if c = [], the code you give will raise an exception. For non-hashable, but comparable items, sorting the list, then using itertools.groupby to extract the unique items from it, is the best available solution (O(N log N)). If items are neither all hashable, nor all comparable, your only "last ditch" solution is O(N squared).
You can code a function to "uniquify" any list that uses the best available approach by trying each approach in order, with a try/except around the first and second (and a return of the result either at the end of the try clause, or, elegantly, in an else clause of the try statement;-).

set([a, b, c, a])
Leave it in that form if possible.

This updated post by Peter Bengtsson suggests two of the fastest ways to make a list of unique items in Python 3.6+ are:
# Unordered (hashable items)
list(set(seq))
# Order preserving
list(dict.fromkeys(seq))