We Keep Coding

Index pandas dataframe rows 4 at a time [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
I want to to be able to something along the lines of:
for i in range(0, len(df), 4):
curr = pd.DataFrame()
vcch = int(df.loc[i, 'IN_CUSTOM_SELECT'])
icch = int(df.loc[i+1, 'IN_CUSTOM_SELECT'])
vccl = int(df.loc[i+2, 'IN_CUSTOM_SELECT'])
iccl = int(df.loc[i+3, 'IN_CUSTOM_SELECT'])
idlpwr = (vcch * icch) + (vccl * iccl)
idlpwr = idlpwr / (10**6)
where I do some calculations based on the specific values of columns in combinations of rows of 4.

If you're just working with a regular autonumbered index, one easy option is to reshape your data and use pandas vectorized operations for the math:
In [196]: df = pd.DataFrame({'IN_CUSTOM_SELECT': np.random.random(24)})
In [197]: reshaped = df.set_index([df.index.map(lambda x: x // 4), df.index.map(lambda x: x % 4)]).unstack()['IN_CUSTOM_SELECT']
In [198]: reshaped['idlpwr'] = ((reshaped[0] * reshaped[1]) + (reshaped[2] * reshaped[3])) / 10**6
In [199]: reshaped
Out[199]:
0 1 2 3 idlpwr
0 0.788758 0.853356 0.627796 0.355143 8.960487e-07
1 0.312111 0.602934 0.908984 0.046183 2.301622e-07
2 0.842201 0.507629 0.541432 0.592680 7.484218e-07
3 0.506601 0.605108 0.497627 0.362006 4.866923e-07
4 0.308097 0.991945 0.822433 0.272082 5.293851e-07
5 0.573716 0.852356 0.009606 0.961437 4.982462e-07

Find row in array corresponding to combination of numbers [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I am working with q-tables and I got 30 * 30 * 30 * 30=810000 different states. I want a function that takes for example [10, 20, 16, 5] as input and this should correspond to a particular row representing this exact combination of state but I cannot figure out how to do that. One (bad) approach would be 10 * 20 * 15 * 5=15000 but then row number 15 000 would also be represented by 20 * 10 * 5 * 15 = 15000 and I do not want that. How can I make this work?

One way would be to represent your table as a 4-dimensional list:
self.q_table = [[[[0 for _ in range(30)] for _ in range(30)] for _ in range(30)] for _ in range(30)]
so you can do:
def func(self, coords):
[q, r, s, t] = coords
entry = self.q_table[q][r][s][t]
...
If you wanted to flatten it to a single-dimensional list, then you need to multiple each coordinate by an increasing power of 30 to make sure each combination of coordinates yields a unique result:
self.q_table = [0 for _ in range(30**4)]
def func(self, coords):
row_num = sum(i * 30**p for p, i in enumerate(coords))
entry = self.q_table[row_num]
...

How to combine numbers? [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 4 years ago.
Improve this question
x = 5
y = 8
I want to combine those variables and define an another variable, like:
z = 58

It depends on what you mean by "combine".
You can concatenate numbers as strings:
z = int(str(x) + str(y))
But you can also compute x * 10 + y:
z = x * 10 + y
This will give different results if y > 9, e.g. for x = 5 and y = 10, the first version will give 510, while the second version will give 60.

Convert them to strings and concatenate them, then convert them back to an integer:
z = int(str(x) + str(y))

If you're using Python 3.6 or later, this can be done quite concisely with format strings:
>>> x = 5
>>> y = 8
>>> z = int(f'{x}{y}')
>>> z
58
>>>
A more general solution would be something like:
>>> def join_ints(*args):
... return int(''.join(map(str, args)))
...
>>> join_ints(5, 6, 8, 3)
5683
>>>

How to split a 3D matrix/volume into a fixed size sub-volumes and then re-merge in python? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
How can I split a 3D numpy array into fixed size 3D sub-arrays, do some manipulation on the sub-arrays, and finally put them back in the same order to create the original big 3D volume?
e.g. big volume is nxnxm
so, I would like to split it to sub-vlumes of k x k x k, and do some manipulation on each sub volume and put them together again to create nxnxm

A simple solution would be to process your array with nested for-loops:
A = np.random.rand(5, 4)
print "A:", A
step = 2
newHeight = np.ceil(float(A.shape[0]) / step)
newWidth = np.ceil(float(A.shape[1]) / step)
B = np.zeros((newHeight, newWidth))
C = np.zeros(A.shape)
for i in range(B.shape[0]):
for j in range(B.shape[1]):
B[i, j] = np.mean(A[i*step:(i+1)*step, j*step:(j+1)*step])
C[i*step:(i+1)*step, j*step:(j+1)*step] = B[i, j]
print "B:", B
print "C:", C
Output:
A: [[ 0.86754517 0.65107995 0.01074822 0.18394825]
[ 0.03184878 0.07052286 0.44014168 0.84913463]
[ 0.2982024 0.94988568 0.33208104 0.28697172]
[ 0.36721371 0.9352932 0.22780242 0.13650031]
[ 0.84073176 0.33792535 0.53240018 0.54008341]]
B: [[ 0.40524919 0.37099319]
[ 0.63764875 0.24583887]
[ 0.58932856 0.53624179]]
C: [[ 0.40524919 0.40524919 0.37099319 0.37099319]
[ 0.40524919 0.40524919 0.37099319 0.37099319]
[ 0.63764875 0.63764875 0.24583887 0.24583887]
[ 0.63764875 0.63764875 0.24583887 0.24583887]
[ 0.58932856 0.58932856 0.53624179 0.53624179]]
A is the large input array
B is the small output array
C is the large output array
step is the size of each block, 20 in your case
newHeight and newWidth is the computed size of B: dividing the size of A by the window size step and rounding up
i*step:(i+1)*step and j*step:(j+1)*step are the vertical and horizontal ranges for each block in A and C, respectively.
I'm using a small array of 5x4 as well as two dimensions only for simplicity and readability of the example results. It should be not to hard to extend this approach to three dimensions.

Reading a CSV file to convert string to float [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 6 years ago.
Improve this question
I have a CSV file with data in the format:
Row 1:
[-0.74120803 0.6338942 ],[-1.01583889 0.20901699],[-1.02969154 0.14459244],[ 0.10362657 0.31347394],[ 1.69977092 -0.13384537],[ 1.39789431 -0.52155783],[ 0.02928792 0.24156825],[-1.03616494 0.33943 ],[ 0.84921822 0.47879992],[ 0.279905 0.96184517],[ 0.43602597 -0.27275052],[ 1.4766132 -0.48128695],[ 0.96219625 -0.44950686],[ 0.24356381 -0.0253022 ],[ 0.09346193 0.07808998],[ 0.26571546 -0.1678716 ],[ 0.03055046 1.05913456],[ 1.94137487e+00 -1.57339675e-03],[ 0.22311559 0.98762516],[ 2.00176133 0.13017485],......
Note that the data is of two rows: the first row contains both x and y coordinates and 2nd row contains their flag status.
Row 2
0,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1,0,1,0,1,1,0,0,1,0,0,0,1,0,1,0,0,1,1,1,0,0,1,0,1,1,0,0,0,1,1,1,1,0,1,0,0,0,0,0,0,0,0,1,0,0,0,1,1,1,0,0,1,1,.......
I want to store the data in 3 lists: x, y, and flag.
Thank you for helping with this part.

row1 = '[-0.74120803 0.6338942 ],[-1.01583889 0.20901699],[-1.02969154 0.14459244],[ 0.10362657 0.31347394],[ 1.69977092 -0.13384537],[ 1.39789431 -0.52155783],[ 0.02928792 0.24156825],[-1.03616494 0.33943 ],[ 0.84921822 0.47879992],[ 0.279905 0.96184517],[ 0.43602597 -0.27275052],[ 1.4766132 -0.48128695],[ 0.96219625 -0.44950686],[ 0.24356381 -0.0253022 ],[ 0.09346193 0.07808998],[ 0.26571546 -0.1678716 ],[ 0.03055046 1.05913456],[ 1.94137487e+00 -1.57339675e-03],[ 0.22311559 0.98762516],[ 2.00176133 0.13017485]'
row2 = '0,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1,0,1,0,1'
l = []
for xy, flag in zip(row1.split(','), row2.split(',')):
x, y = xy.strip('[] ').split(' ')
l.append((float(x), float(y), int(flag)))
print l
If you prefere 3 separate lists:
row1 = '[-0.74120803 0.6338942 ],[-1.01583889 0.20901699],[-1.02969154 0.14459244],[ 0.10362657 0.31347394],[ 1.69977092 -0.13384537],[ 1.39789431 -0.52155783],[ 0.02928792 0.24156825],[-1.03616494 0.33943 ],[ 0.84921822 0.47879992],[ 0.279905 0.96184517],[ 0.43602597 -0.27275052],[ 1.4766132 -0.48128695],[ 0.96219625 -0.44950686],[ 0.24356381 -0.0253022 ],[ 0.09346193 0.07808998],[ 0.26571546 -0.1678716 ],[ 0.03055046 1.05913456],[ 1.94137487e+00 -1.57339675e-03],[ 0.22311559 0.98762516],[ 2.00176133 0.13017485]'
row2 = '0,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1,0,1,0,1'
listX, listY = [], []
for xy in row1.split(','):
x, y = xy.strip('[] ').split(' ')
listX.append(float(x))
listY.append(float(y))
listFlag = [int(flag) for flag in row2.split(',')]
print listX, listY, listFlag

Two one-liners will do:
flags = [int(x) for x in row2.split(',')]
x, y = zip(*((float(value) for value in entry[1:-1].split()) for entry in row1.split(',')))
Now:
print(flags[:5])
print(list(x[:5]))
print(list(y[:5]))
Output:
[0, 0, 0, 1, 1]
[-0.74120803, -1.01583889, -1.02969154, 0.10362657, 1.69977092]
[0.6338942, 0.20901699, 0.14459244, 0.31347394, -0.13384537]