I have just start with Python 2.7 for few months. I usually work with some 2D list in Python, simple task but I wonder that is there some more elegant way to do the same my job in Python 2.7?
Here is my task. I have a 2D list:
my_list = [["__cat_1", "__cat_2"],
["__cat_3", "__cat_4"]]
I want to convert the above 2D string list to 2D integer list.
expected_result = [[1, 2], [3, 4]]
As usual, I do as the following:
def convert_2D_string_list(my_list):
for a_group in my_list:
yield [int(k.replace("__cat_","")) for k in a_group]
But the above could not work when my input has 3 dimensions like:
my_second_list = [[["__cat_1", "__cat_2"], "__cat_12"],
[["__cat_3", "__cat_4"], "__cat_34"]]
If my input list is integer, I know the elegant way to convert for transform it. For example:
def convert_2D_int_list:
my_list = [[1, 2], [3, 4]]
import numpy as np
# Assume that, I want to add 1 for each of element
# Convert to numpy array
my_list = np.asarray(my_list)
my_list += 1
# my_list = [[2, 3], [4, 5]]
return my_list
What is the best practice for my convert_2D_string_list method?
So in case of it is not 2D list, but 3D list -> I will not afraid about the number of dimensions.
Any suggestion would be appreciated.
Thank you.
Numpy array works very well with NUMbers, it's a little more tricky with strings. np.vectorize is a work around for such situation, even no performance improvement is done. But it manages any number of dimensions, hope you will find it elegant.
a=np.array([[[['__cat_0', '__cat_1'],
['__cat_2', '__cat_3']],
[['__cat_4', '__cat_5'],
['__cat_6', '__cat_7']]],
[[['__cat_8', '__cat_9'],
['__cat_10', '__cat_11']],
[['__cat_12', '__cat_13'],
['__cat_14', '__cat_15']]]])
def f(str): return int(str[6:])
fv=np.vectorize(f)
print(fv(a))
gives :
[[[[ 0 1]
[ 2 3]]
[[ 4 5]
[ 6 7]]]
[[[ 8 9]
[10 11]]
[[12 13]
[14 15]]]]
Remark : a is np.vectorize(lambda n : '__cat_'+str(n))(np.arange(16).reshape((2,)*4)) ;)
Modify the values whilst recursively copying the list(s).
def visit(fn, xs):
return [visit(fn, x) if isinstance(x, list) else fn(x) for x in xs]
in this case the modification function fn is something like:
def categorize(s):
return int(re.match(r'__cat_(\d*)', s).group(1))
testing this:
my_second_list = [[["__cat_1", "__cat_2"], "__cat_12"],
[["__cat_3", "__cat_4"], "__cat_34"]]
print visit(categorize, my_second_list)
outputs:
> [[[1, 2], 12], [[3, 4], 34]]
I hope this may useful for you:
def con(x):
# you can do here whatever u want
return x.split('_')[-1]
y= [con(v) for v in my_list]
You can convert your string 2D list to int 2D list using map and list comprehensions
my_list = [["__cat_1", "__cat_2"],["__cat_3", "__cat_4"]]
my_list = [map(lambda z: int(z.split('_')[-1]), ls) for ls in my_list]
print my_list # [[1,2],[3,4]]
3d list:
my_list = [["__cat_1", "__cat_2", '__cat_3'],["__cat_1", "__cat_2", '__cat_3']]
my_list = [map(lambda z: int(z.split('_')[-1]), ls) for ls in my_list]
print my_list # [[1,2,3],[1,2,3],[1,2,3]]
Related
There are a few similar questions to this one but not exactly the same:
I want to dynamically decrease a given input array or list of lists. For example:
matrix = [[0,1,2], [3,4,5],[6,7,8]]
Starting at 0 I need to iterate through and remove the final index - the iterative. So the output I would like to store in a new list is:
#output
[0,1,2], ,[3,4], [6]]
[0,1,2], ,[3,4], [6]] ==> which then flattens to [0,1,2,3,4,6]
Here's what I'm currently going after:
def get_list(matrix, stop_index):
temp = []
for i in range(0, stop_index):
for m in matrix:
temp.append(matrix[0:stop_index])
outside_list.append(temp)
return outside_list
I believe I am seeing well my over reliance on packages and libraries, so I am really trying to do this without outside packages or imports
Thank you for any help! I don't forget to green check mark.
Using list comprehension
l = [[0,1,2], [3,4,5],[6,7,8]]
ll = [ x[:len(l)-l.index(x)] for x in l]
# [[0, 1, 2], [3, 4], [6]]
print([x for y in ll for x in y ])
# [0, 1, 2, 3, 4, 6]
Simpler syntax:
matrix = [[0,1,2], [3,4,5],[6,7,8]]
outside_list = list()
for i in range(len(matrix)):
# matrix[i] is used to access very sublist in the matrix,
#[:3-i] is to slice every sublist from the beginning to (3 - current position)
outside_list.append(matrix[i][:3-i])
print(outside_list)
Some useful refernces
List slicing https://stackoverflow.com/a/509295/8692977
List comprehension: https://stackoverflow.com/a/34835952/8692977
I am working through some code trying to understand some Python mechanics, which I just do not get. I guess it is pretty simple and I also now, what it does, but i do not know how it works. I understand the normal use of for-loops but this here... I do not know.
Remark: I know some Python, but I am not an expert.
np.array([[[S[i,j]] for i in range(order+1)] for j in range(order+1)])
The second piece of code, I have problems with is this one:
for i in range(len(u)):
for j in range(len(v)):
tmp+=[rm[i,j][k]*someFuction(name,u[i],v[j])[k] for k in range(len(rm[i,j])) if rm[i,j][k]]
How does the innermost for-loop work? And also what does the if do here?
Thank you for your help.
EDIT: Sorry that the code is so unreadable, I just try to understand it myself. S, rm are numpy matrices, someFunction returns an array with scalar entries, andtmp is just a help variable
There are quite a few different concepts inside your code. Let's start with the most basic ones. Python lists and numpy arrays have different methodologies for indexation. Also you can build a numpy array by providing it a list:
S_list = [[1,2,3], [4,5,6], [7,8,9]]
S_array = np.array(S_list)
print(S_list)
print(S_array)
print(S_list[0][2]) # indexing element 2 from list 0
print(S_array[0,2]) # indexing element at position 0,2 of 2-dimensional array
This results in:
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
[[1 2 3]
[4 5 6]
[7 8 9]]
3
3
So for your first line of code:
np.array([[[S[i,j]] for i in range(order+1)] for j in range(order+1)])
You are building a numpy array by providing it a list. This list is being built with the concept of list comprehension. So the code inside the np.array(...) method:
[[[S[i,j]] for i in range(order+1)] for j in range(order+1)]
... is equivalent to:
order = 2
full_list = []
for j in range(order+1):
local_list = []
for i in range(order+1):
local_list.append(S_array[i, j])
full_list.append(local_list)
print(full_list)
This results in:
[[1, 4, 7], [2, 5, 8], [3, 6, 9]]
As for your second snippet its important to notice that although typically numpy arrays have very specific and constant (for all the array) cell types you can actually give the data type object to a numpy array. So creating a 2-dimensional array of lists is possible. It is also possible to create a 3-dimensional array. Both are compatible with the indexation rm[i,j][k]. You can check this in the following example:
rm = np.array(["A", 3, [1,2,3]], dtype="object")
print(rm, rm[2][0]) # Acessing element 0 of list at position 2 of the array
rm2 = np.zeros((3, 3, 3))
print(rm2[0, 1][2]) # This is also valid
The following code:
[rm[i,j][k]*someFuction(name,u[i],v[j])[k] for k in range(len(rm[i,j])) if rm[i,j][k]]
... could be written as such:
some_list = []
for k in range(len(rm[i,j])):
if rm[i, j][k]: # Expecting a boolean value (or comparable)
a_list = rm[i,j][k]*someFuction(name,u[i],v[j])
some_list.append(a_list[k])
The final detail is the tmp+=some_list. When you sum two list they'll be concatenated as can been seen in this simple example:
tmp = []
tmp += [1, 2, 3]
print(tmp)
tmp += [4, 5, 6]
print(tmp)
Which results in this:
[1, 2, 3]
[1, 2, 3, 4, 5, 6]
Also notice that multiplying a list by a number will effectively be the same as summing the list several times. So 2*[1,2] will result in [1,2,1,2].
Its a list comprehension, albeit a pretty unreadable one. That was someome doing something very 'pythonic' in spite of readablity. Just look up list comprehensions and try to rewrite it yourself as a traditional for loop. list comprehensions are very useful, not sure I would have gone that route here.
The syntax for a list comprehension is
[var for var in iterable if optional condition]
So this bottom line can be rewritten like so:
for k in range(len(rm[i,j]):
if rm[i,j][k]:
tmp+= rm[i,j][k]*someFunction(name,u[i],v[j])[k]
I want to rearrange a list l into a list of n lists, where n is the number of columns.
e.g.,
l = [1,2,3,4,5,6,7,8,9,10]
n = 4
==> [[1,5,9],[2,6,10],[3,7][4,8]]
Can someone please help me out with an algorithm? Feel free to use any python awesomeness that's available; I'm sure theres some cool mechanism that's a good fit for this, i just can't think of it.
PS The example list just happened to be ordered numbers starting at 1. That's not my actual scenario.
There is indeed a cool mechanism for this in Python: the three-argument form of slicing, where the last argument is step size.
>>> l = [1,2,3,4,5,6,7,8,9,10]
>>> n = 4
>>> [l[i::n] for i in range(n)]
[[1, 5, 9], [2, 6, 10], [3, 7], [4, 8]]
l = [1,2,3,4,5,6,7,8,9,10]
n = 4
def f(l,n):
A = []
[A.append([]) for i in xrange(n)]
[ A [(i - 1) % n].append(i) for i in l]
return A
print f(l,n)
[[1, 5, 9], [2, 6, 10], [3, 7], [4, 8]]
The following function does what you want to achieve:
def rearrange(seq,n):
return [[v for i,v in enumerate(seq[x:]) if i%n==0] for x in xrange(len(seq))][:n]
Writing Python isn't a game of code golf, don't be afraid to use more than one line for the sake of readability.
l = [1,2,3,4,5,6,7,8]
def split_into_columns(input_list, num_of_cols=3):
retval = [ [] for _ in xrange(num_of_cols)] # build 3 columns
for i in xrange(len(input_list)): # iterate through original list
retval[i%num_of_cols].append(input_list[i]) # place in the "modulo 3" column
return retval
# here's a compressed, less readable version of that for-loop
#[retval[i%3].append(input_list[i]) for i in xrange(len(input_list))]
#return retval
print split_into_columns(l, 3)
Possible Duplicate:
How can I turn a list into an array in python?
How can I turn a list such as:
data_list = [0,1,2,3,4,5,6,7,8]
into a list of lists such as:
new_list = [ [0,1,2] , [3,4,5] , [6,7,8] ]
ie I want to group ordered elements in a list and keep them in an ordered list. How can I do this?
Thanks
This groups each 3 elements in the order they appear:
new_list = [data_list[i:i+3] for i in range(0, len(data_list), 3)]
Give us a better example if it is not what you want.
This assumes that data_list has a length that is a multiple of three
i=0
new_list=[]
while i<len(data_list):
new_list.append(data_list[i:i+3])
i+=3
Something like:
map (lambda x: data_list[3*x:(x+1)*3], range (3))
Based on the answer from Fred Foo, if you're already using numpy, you may use reshape to get a 2d array without copying the data:
import numpy
new_list = numpy.array(data_list).reshape(-1, 3)
new_list = [data_list[x:x+3] for x in range(0, len(data_list) - 2, 3)]
List comprehensions for the win :)
The following function expands the original context to include any desired list of lists structure:
def gen_list_of_lists(original_list, new_structure):
assert len(original_list) == sum(new_structure), \
"The number of elements in the original list and desired structure don't match"
list_of_lists = [[original_list[i + sum(new_structure[:j])] for i in range(new_structure[j])] \
for j in range(len(new_structure))]
return list_of_lists
Using the above:
data_list = [0,1,2,3,4,5,6,7,8]
new_list = gen_list_of_lists(original_list=data_list, new_structure=[3,3,3])
# The original desired outcome of [[0,1,2], [3,4,5], [6,7,8]]
new_list = gen_list_of_lists(original_list=data_list, new_structure=[2,3,3,1])
# [[0, 1], [2, 3, 4], [5, 6, 7], [8]]
The below one is more optimized and quite straightforward.
data_list = [0,1,2,3,4,5,6,7,8]
result =[]
i=0
while i <(len(data_list)-2):
result.append(data_list[i:i+3])
i+=3
print(result)
**output**
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]
Here is a generalized solution
import math
data_list = [0,1,2,3,4,5,6,7,8]
batch_size=3
n_batches=math.ceil(len(data_list)/batch_size)
[data_list[x*batch_size:min(x*batch_size+batch_size,len(data_list))]
for x in range(n_batches)]
It works even if the last sublist is not the same size as the rest (<batch_size)
Do you have any sort of selection criteria from your original list?
Python does allow you to do this:
new_list = []
new_list.append(data_list[:3])
new_list.append(data_list[3:6])
new_list.append(data_list[6:])
print new_list
# Output: [ [0,1,2] , [3,4,5] , [6,7,8] ]
How to handle nested lists in Python? I am having problem figuring out the syntax. Like example:
>>> l = [[1, 2, 3], [5, 6, 7]]
I want to square all the elements in this list. I tried:
[m*m for m in l]
But that doesn't work and throws up:
TypeError: can't multiply sequence by
non-int of type 'list'
because of the nested lists I guess?
How do I fix this?
>>> l = [[1, 2, 3], [5, 6, 7]]
>>> [[e*e for e in m] for m in l]
|-nested list-|
|---- complete list ---|
[[1, 4, 9], [25, 36, 49]]
Assuming you wanted the answer to look like this:
[[1, 4, 9], [25, 36, 49]]
You could do something like this:
l = [[1, 2, 3], [5, 6, 7]]
for x in range(len(l)):
for y in range(len(l[x])):
l[x][y] = l[x][y] * l[x][y]
print l
Obviously, the list comprehension answer is better.
[[1,2,3],[4,5,6]] != [1,2,3,4,5,6]
[map(lambda x: x *x,sl) for sl in l] #List comprhension
What you need is a recursive function, something like this:
def square(el):
if type(el) == list:
return [square(x) for x in el]
else:
return el**2;
I'd rather not get into the correctness of type(el) == list here, but you get the gist.
Of course, this is also doable with a list-comprehension, as many people have pointer out, provided that the structure is always the same. This recursive function can handle any level of recursion, and lists containing both lists and numbers.