Fast way to create two list of certain pattern - python

I'd like to create two list (appearantly it doesnt need to be a list, can be tuple, array, numpy.array, see edit below) of a certain partern, lets call them list_1 and list_2
list_1 has the patern that can be described as (i is an integer)
{i}, {i, i + 17}, {i, i+17 , i + 17 + 17}, {i, i + 17, i + 17 + 17, i + 17 + 17 + 17},
But the pattern occurs twice, which means that the resulting list will look like
list_1 = [i, i, i + 17, i, i + 17 , i + 34, i, i + 17, i + 34, i + 51,
i, i, i + 17, i, i + 17 , i + 34, i, i + 17, i + 34, i + 51]
Currently I'm doing it as (with i = 2 in this example)
some_limit = 5
list_1 = [17 * x + i for b in range(some_limit + 1) for x in range(b)]
list_1 += list_1
result
[2, 2, 19, 2, 19, 36, 2, 19, 36, 53, 2, 19, 36, 53, 70, 2, 2, 19, 2, 19, 36, 2, 19, 36, 53, 2, 19, 36, 53, 70]
When some_limit is a big number this takes time. Is there a faster way?
list_2 has a pattern that can be described as (j is an integer)
{j} {j+1, j+1}, {j+2, j+2, j+2}, {j+3, j+3, j+3, j+3}
This pattern also occours twice but with a shift which means that the resulting list will look like
list_2 = [j, j+1, j+1, j+2, j+2, j+2, j+3, j+3, j+3, j+3,
j+shift, j+1+shift, j+1+shift, j+2+shift, j+2+shift, j+2+shift, j+3+shift, j+3+shift, j+3+shift, j+3+shift,
Currently I'm doing it as (with j = 0 in this example)
some_limit = 5
arithemic_list = [k for k in range(some_limit + 1)]
rows_index_temp = [item -1 + some_limit * j * 2 for item, count in zip(arithemic_list, arithemic_list) for k in range(count)]
rows_index_temp += [some_limit + elem for elem in rows_index_temp]
result
[0, 1, 1, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 6, 6, 7, 7, 7, 8, 8, 8, 8, 9, 9, 9, 9, 9]
When some_limit is a big number this takes time. Is there a faster way?
Edit
This is used to create sparse matrices for the optimization package cvxopt. A matrix is built by providing elements with their row and column indicies. For example a 3x3 identity matrix is created by 3 lists, elements = [1,1,1], rows = [0, 1, 2] and columns = [0, 1, 2]. My matrix is very large which implies that the lists will be very large.
From the documentation
cvxopt.spmatrix(x, I, J[, size[, tc ] ])
I and J are sequences of integers (lists, tuples, array arrays, . . . ) or integer matrices (matrix objects with
typecode 'i'), containing the row and column indices of the nonzero entries. The lengths of I and J must
be equal. If they are matrices, they are treated as lists of indices stored in column-major order, i.e., as lists
list(I), respectively, list(J)
https://readthedocs.org/projects/cvxopt/downloads/pdf/1.2.0/
Example
Lets say we would like to create the following matrix:
[ 1.00e+00 0 0 0 0 ]
[ 2.00e+00 0 0 3.00e+00 0 ]
[ 0 0 0 0 4.00e+00]
We are only interested in the non-zero-elements and we can group them as
element row column
1 0 0
2 1 0
3 1 3
4 2 4
Thus, by having
elements = [1,2,3,4,0]
rows = [0,1,1,2, 3]
columns = [0,0, 3, 4, 4]
print(spmatrix(elements, rows, columns))
[ 1.00e+00 0 0 0 0 ]
[ 2.00e+00 0 0 3.00e+00 0 ]
[ 0 0 0 0 4.00e+00]
Notice that the order doesnt mather, as long as the element,row,column are grouped together. Thus this would be equivalent.
elements = [4,2,3,1]
rows = [2,1,1,0]
columns = [4,0, 3,0]
print(spmatrix(elements, rows, columns))
[ 1.00e+00 0 0 0 0 ]
[ 2.00e+00 0 0 3.00e+00 0 ]
[ 0 0 0 0 4.00e+00]
What am I trying to create? (some_limit = 3)
This is basically a concatination of two lower triangual matrix, with a constant shift between its elements. So how would we describe this matrix?
If we focus on the "upper" lower triangual part which contains -1:s.
We can group as
element row column
-1 0 2
-1 1 2
-1 1 19
-1 2 2
-1 2 19
-1 2 36
The "lower" triangular part with only 1:s can be grouped as
We can group as
element row column
1 3 2
1 4 2
1 4 19
1 5 2
1 5 19
1 5 36
Together this become
element row column
-1 0 2
-1 1 2
-1 1 19
-1 2 2
-1 2 19
-1 2 36
1 3 2
1 4 2
1 4 19
1 5 2
1 5 19
1 5 36
Or,
element row column
-1 0 2
-1 1 2
-1 1 2 + 17
-1 2 2
-1 2 2 + 17
-1 2 2 + 2*17
1 1 + 2 2
1 2 + 2 2
1 2 + 2 2 + 17
1 3 + 2 2
1 3 + 2 2 + 17
1 3 + 2 2 + 2*17
Since the order doesnt mather, as long as element, row, column is grouped togethere, there might be a "better" pattern which I havent consider.
The size of this matrix is directly correlated with the number of elements in the list describing the elements, rows, and columns. I wish to have some_limit >=4343 => the lenght of each of the list elements, rows, and columns will be
(4343*(4343 + 1)/2)*2 which explodes quite fast...
some_limit
4343
len(list_1)
18865992
#size_of_matrix
<34744x73831 sparse matrix, tc='d', nnz=75463969>

You can convert your list comprehensions to generator expressions, then you do not have to create the entire list at once but can generate values as needed. In particular note the additional loop I added so that you do not have to create the entire list, then add a copy to itself.
i, j, limit, shift = 2, 0, 5, 10
gen_1 = (17 * x + i for _ in range(2) # repeat twice
for b in range(limit) # len of subsegments
for x in range(b+1)) # multiplier
gen_2 = (j + b + s for s in (0, 10) # repeat with shift
for b in range(limit) # len of subsegments
for _ in range(b+1)) # repeat b times
You can then either iterate the values as needed (or still create a list from all the values):
for x in gen_1:
print(x)
print(list(gen_2))

Related

Function to use indexes in a matrix

I am trying to create a function which takes two inputs. One input is the matrix (n*m), and the second is K. K is a integer value. The distance between the cells A[3][2] and A[1][4] is |1-3| + |4-2| = 4. The expected output from the function is the count of cells with cell distance greater than K.
Cell here is each entry in the given matrix A. For example, A[0][0] is a cell and it has an entry value of 1 in the matrix.
I have created a function like this:
A = [[1, 0, 0],
[0, 0, 0],
[0, 0, 1],
[0, 1, 0]]
def findw(K, matrix):
m_c = matrix.copy()
result = 0
for i, j in zip(range(len(matrix)), range(len(m_c))):
for k, l in zip(range(len(matrix[i])), range(len(m_c[j]))):
D = abs(i - l) + abs(j - k)
print(i, k)
print(j, l)
print(D)
if D > K:
result += 1
return result
findw(1, A)
The output I got from the above function for the given matrix A with K = 1 is 9. But I am expecting 3. From the output I also realized that for both the matrices my function is always taking same value, for example (0,0) or (1,0), etc. See the print output below.
findw(1, A)
0 0
0 0
0
0 1
0 1
2
0 2
0 2
4
1 0
1 0
2
1 1
1 1
0
1 2
1 2
2
2 0
2 0
4
2 1
2 1
2
2 2
2 2
0
3 0
3 0
6
3 1
3 1
4
3 2
3 2
2
Out[120]: 9
It looks like my function is not iterating for cells where the indexes for both matrices are different. For example, matrix[0][0] and m_c[0][1].
How can I resolve this issue?
Working under the assumption that it is only the positions which have the value 1 that you care about, you could first enumerate those indices and then loop over the pairs of such things. itertools is a natural tool to use here:
from itertools import product, combinations
def D(p,q):
i,j = p
k,l = q
return abs(i-k) + abs(j-l)
def findw(k,matrix):
m = len(matrix)
n = len(matrix[0])
result = 0
indices = [(i,j) for i,j in product(range(m),range(n)) if matrix[i][j] == 1]
for p,q in combinations(indices,2):
d = D(p,q)
if d > k:
print(p,q,d)
result += 1
return result
#test:
A = [[1, 0, 0],
[0, 0, 0],
[0, 0, 1],
[0, 1, 0]]
print(findw(1, A))
Output:
(0, 0) (2, 2) 4
(0, 0) (3, 1) 4
(2, 2) (3, 1) 2
3

Numpy / Pandas slicing based on intervals

Trying to figure out a way to slice non-contiguous and non-equal length rows of a pandas / numpy matrix so I can set the values to a common value. Has anyone come across an elegant solution for this?
import numpy as np
import pandas as pd
x = pd.DataFrame(np.arange(12).reshape(3,4))
#x is the matrix we want to index into
"""
x before:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
"""
y = pd.DataFrame([[0,3],[2,2],[1,2],[0,0]])
#y is a matrix where each row contains a start idx and end idx per column of x
"""
0 1
0 0 3
1 2 3
2 1 3
3 0 1
"""
What I'm looking for is a way to effectively select different length slices of x based on the rows of y
x[y] = 0
"""
x afterwards:
array([[ 0, 1, 2, 0],
[ 0, 5, 0, 7],
[ 0, 0, 0, 11]])
Masking can still be useful, because even if a loop cannot be entirely avoided, the main dataframe x would not need to be involved in the loop, so this should speed things up:
mask = np.zeros_like(x, dtype=bool)
for i in range(len(y)):
mask[y.iloc[i, 0]:(y.iloc[i, 1] + 1), i] = True
x[mask] = 0
x
0 1 2 3
0 0 1 2 0
1 0 5 0 7
2 0 0 0 11
As a further improvement, consider defining y as a NumPy array if possible.
I customized this answer to your problem:
y_t = y.values.transpose()
y_t[1,:] = y_t[1,:] - 1 # or remove this line and change '>= r' below to '> r`
r = np.arange(x.shape[0])
mask = ((y_t[0,:,None] <= r) & (y_t[1,:,None] >= r)).transpose()
res = x.where(~mask, 0)
res
# 0 1 2 3
# 0 0 1 2 0
# 1 0 5 0 7
# 2 0 0 0 11

How to get the same output example in Python?

I'm a beginner trying to learn about lists and arrays, especially with dimensional lists. I'm really struggling on how to do the code for the list shown in the output example. I would appreciate some help putting me in the right direction.
Here's how I want the list to look like as a example
# If I put in 4 as a example, the result will be this:
[[0, 0, 0, 0, 0], [0, 1, 2, 3, 4], [0, 2, 4, 6, 8], [0, 3, 6, 9, 12], [0, 4, 8, 12, 16]]
0 0 0 0 0
0 1 2 3 4
0 2 4 6 8
0 3 6 9 12
0 4 8 12 16
my code down below:
def myNumber():
chooseNumber=int(input("\nPlease enter a number between [1, 12]: "))
while not (chooseNumber >= 1 and chooseNumber <= 12):
print("That number is not between [1, 12], try again.")
chooseNumber=int(input("Please enter a number between [1, 12]: "))
print("Your number is:", chooseNumber)
myNumber()
#---- the code below I am struggling with here ----#
n = 4
[[i*j for i in range(n+2)] for j in range (n+2)]
# x = [ [1,3],[2,4] ]
print([[i*j for i in range(n+2)] for j in range (n+2)])
here is how to produce your desired output if I understand correctly:
n = 4
lists = [[i*j for i in range(n+1)] for j in range(n+1)]
outstr = ''
for i in lists:
for n in i:
outstr += str(n) + " "
outstr += "\n"
print(outstr)
Output:
0 0 0 0 0
0 1 2 3 4
0 2 4 6 8
0 3 6 9 12
0 4 8 12 16
Let me know if this helps!

Multiplying Elements in a List

I want to multiply the first and second element of sum_row by 13 individually. And multiply the third and fourth by 11 individually and the last element by 9.
I guess my question really is how do I access the elements in lists, so I can use them for calculations later on?
matrix5x5 = [[1 for row in range (5)] for col in range (5)]
for row in matrix5x5:
for item in row:
print(item,end=" ")
print()
sum_row = [sum(i) for i in matrix5x5]
print(sum_row)
OUTPUT:
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
[5, 5, 5, 5, 5]
You can try this one:
sum_row = [1,1,1,1,1] # example
YourList = [13,13,11,11,9]
result = []
for i in range(0, len(sum_row)):
result.append(sum_row[i]*YourList[i])
print result
and the output going to be:
[13, 13, 11, 11, 9]
You can even try with [5,5,5,5,5] as the sum of each row.
You can use zip() function:
result = [a * b for a, b in zip(sum_row, [13,13,11,11,9])]
print(result)
# OUTPUT
# [65, 65, 55, 55, 45]
For vectorized calculations, use numpy:
import numpy as np
result = np.array(sum_row) * np.array([13,13,11,11,9])
result:
>>> result
array([65, 65, 55, 55, 45])
The simplest answer is:
l = [1,2,3,4,5]
a = l[0] * 13
b = l[1] * 13
c = l[2] * 11
d = l[3] * 11
e = l[4] * 9
print(a, b, c, d, e)
Your results will be 13 26 33 44 45.
Other users have provided much shorter and better ways of doing this, but you should try to understand what they did if you want to follow theirs.

Creating a python array where the outside elements increase in chronological order

I want to create a python array like this:
[[1, 10, 9, 8]
[2, 0, 0, 7]
[3, 4, 5, 6]]
However, I want to do it with a function so if the dimensions of the array change I still get the same output, where the outside elements increase numerically and the middle elements stay as zero.
this is an attempt with N and M as height and width:
N = 5
M = 7
m = []
# first row
m.append([1] + list(range(2*N + 2*M - 4, M + 2*N - 3, -1)))
# middle rows
for i in range(1, N-1):
row = M*[0]
row[0] = i+1
row[-1] = 2*N + M -2 - i
m.append(row)
# last row
m.append(list(range(N, N+M)))
for row in m:
strgs = ('{:2d}'.format(n) for n in row)
print(' '.join(strgs))
it prints:
1 20 19 18 17 16 15
2 0 0 0 0 0 14
3 0 0 0 0 0 13
4 0 0 0 0 0 12
5 6 7 8 9 10 11
as requested the very same in numpy
import numpy as np
m = np.zeros(shape=(N, M), dtype=int)
# first row
m[0] = [1] + list(range(2*N + 2*M - 4, M + 2*N - 3, -1))
# middle rows
for i, row in enumerate(m[1:-1], start=2):
row[0] = i
row[-1] = 2*N + M -1 - i
# last row
m[-1] = list(range(N, N + M))

Categories