Getting column name where a condition matches in a row - python

I have a pandas dataframe which looks like this:
A B C D E F G H I
1 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
2 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
3 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Now, for each row, I have to check which column contains 1 and then record this column name in a new column. The final dataframe would look like this:
A B C D E F G H I IsTrue
1 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 B
2 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 A
3 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 B
Is there any faster and pythonic way to do it?


Here's one way using DataFrame.dot:
df['isTrue'] = df.astype(bool).dot(df.columns)
A B C D E F G H I isTrue
1 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 B
2 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 A
3 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 B
For an even better performance you can use:
df['isTrue'] = df.columns[df.to_numpy().argmax(1)]


What you described is the definition of idxmax
>>> df.idxmax(1)
1 B
2 A
3 B
dtype: object

Related

Python - Adding a column in a text file when certain criteria is met

I want to create a python script that will perform the following steps, however I'm not too sure on how to tackle it. I have a text file that looks like this (this was a .tsv file that was converted to a .txt):
trans_x trans_y trans_z rot_x rot_y rot_z non_steady_state_outlier00 non_steady_state_outlier01 motion_outlier00 motion_outlier01 motion_outlier02 motion_outlier03 motion_outlier04 motion_outlier05 motion_outlier06 motion_outlier07 motion_outlier08 motion_outlier09 motion_outlier10 motion_outlier11 motion_outlier12 motion_outlier13 motion_outlier14 motion_outlier15 motion_outlier16 motion_outlier17 motion_outlier18 motion_outlier19 motion_outlier20 motion_outlier21 motion_outlier22 motion_outlier23 motion_outlier24 motion_outlier25 motion_outlier26 motion_outlier27 motion_outlier28 motion_outlier29 motion_outlier30 motion_outlier31 motion_outlier32 motion_outlier33 motion_outlier34 motion_outlier35 motion_outlier36 motion_outlier37 motion_outlier38 motion_outlier39 motion_outlier40 motion_outlier41 motion_outlier42 motion_outlier43 motion_outlier44 motion_outlier45 motion_outlier46 motion_outlier47 motion_outlier48 motion_outlier49 motion_outlier50 motion_outlier51 motion_outlier52 motion_outlier53 motion_outlier54 motion_outlier55 motion_outlier56 motion_outlier57
-0.213045 0.188054 -0.0350898 -0.000904171 -0.001059 0.00130768 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.310081 -0.194011 0.00361878 0.000139216 0.000693861 -0.00162252 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.120743 -0.328317 0.00325342 -0.000296934 0.000299405 -0.000840907 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.17164 -0.36066 -0.00439546 -0.000190615 0.000769665 -0.000238167 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
What I need is a code that:
Goes through my .txt files and searches to see whether there is a column called non_steady_state_outlier02.
If non_steady_state_outlier02 doesn't exist, add a column with this name after non_steady_state_outlier01 and fill it out so that there is a 1 in the 3rd position and 0 everywhere else (e.g. 0 0 1 0 0 0...).
Many thanks in advance!

Assuming your table is like this:
etc.
Then looping over your directory of text files in python and using pandas for your condition will work nicely:
import pandas as pd
import os
for filename in os.listdir('.'):
if filename.endswith('.txt'):
df = pd.read_csv(filename)
if not 'non_steady_state_outlier02' in df.columns:
df['non_steady_state_outlier02'] = pd.Series([0, 0, 1, 0])
df.to_csv(filename.replace('.txt', '_new.txt'))
else:
pass
And if not .csv add your delimiter to pandas such as tabs:
df = pd.read_csv(filename, delimiter='\t)

Convert a list with multiple lines into data frame

I have list as given below
['7237 G 0.0 0.0 0.0 1.239999E-07 -3.733407E-07 2.212587E-07',
'7238 G 0.0 0.0 0.0 4.846863E-06 -9.922069E-07 8.458463E-06',
'7239 G 0.0 0.0 0.0 3.073667E-05 -3.717125E-06 2.890958E-05',
'7240 G 0.0 0.0 0.0 8.920388E-05 -7.419183E-06 6.471878E-05',
'7241 G 0.0 0.0 0.0 1.832756E-04 -1.062274E-05 1.152823E-04',
'7242 G 0.0 0.0 0.0 3.104847E-04 -1.263393E-05 1.780886E-04',
'7243 G 0.0 0.0 0.0 4.645259E-04 -1.285412E-05 2.493027E-04',
'7244 G 0.0 0.0 0.0 6.359626E-04 -1.083016E-05 3.242427E-04',
'7245 G 0.0 0.0 0.0 8.127339E-04 -6.187393E-06 3.976933E-04',
'7246 G 0.0 0.0 0.0 9.805798E-04 1.373597E-06 4.641845E-04',
'7247 G 0.0 0.0 0.0 1.123478E-03 1.204513E-05 5.182628E-04',
'7248 G 0.0 0.0 0.0 1.224244E-03 2.590691E-05 5.547877E-04',
'7249 G 0.0 0.0 0.0 1.265126E-03 4.292454E-05 5.691787E-04',
'7250 G 0.0 0.0 0.0 1.228418E-03 6.293558E-05 5.576411E-04',
'7251 G 0.0 0.0 0.0 1.097197E-03 8.563578E-05 5.173817E-04',
'7252 G 0.0 0.0 0.0 8.560375E-04 1.105952E-04 4.467830E-04',
'7253 G 0.0 0.0 0.0 4.916305E-04 1.372820E-04 3.454892E-04',
'7254 G 0.0 0.0 0.0 -6.634202E-06 1.650501E-04 2.144534E-04',
'7255 G 0.0 0.0 0.0 -6.459849E-04 1.931538E-04 5.597509E-05',
'7256 G 0.0 0.0 0.0 -1.429897E-03 2.208132E-04 -1.264037E-04',
'7257 G 0.0 0.0 0.0 -2.357971E-03 2.472607E-04 -3.280706E-04',
'7258 G 0.0 0.0 0.0 -3.425660E-03 2.716504E-04 -5.434573E-04',
'7259 G 0.0 0.0 0.0 -4.623938E-03 2.931546E-04 -7.662221E-04',
'7260 G 0.0 0.0 0.0 -5.939634E-03 3.110625E-04 -9.895572E-04',
'7261 G 0.0 0.0 0.0 -7.355830E-03 3.247523E-04 -1.206506E-03',
'7262 G 0.0 0.0 0.0 -8.851902E-03 3.335908E-04 -1.410110E-03',
'7263 G 0.0 0.0 0.0 -1.040373E-02 3.370576E-04 -1.593686E-03',
'7264 G 0.0 0.0 0.0 -1.198441E-02 3.348549E-04 -1.751147E-03',
'7265 G 0.0 0.0 0.0 -1.356535E-02 3.268293E-04 -1.877329E-03',
'7266 G 0.0 0.0 0.0 -1.511621E-02 3.128250E-04 -1.967968E-03',
'7267 G 0.0 0.0 0.0 -1.660536E-02 2.928187E-04 -2.019983E-03',
'7268 G 0.0 0.0 0.0 -1.800119E-02 2.671044E-04 -2.031644E-03',
'7269 G 0.0 0.0 0.0 -1.927284E-02 2.360256E-04 -2.002611E-03',
'7270 G 0.0 0.0 0.0 -2.039046E-02 1.999231E-04 -1.934050E-03',
'7271 G 0.0 0.0 0.0 -2.132561E-02 1.593439E-04 -1.828405E-03',
'7272 G 0.0 0.0 0.0 -2.205253E-02 1.149674E-04 -1.689621E-03',
'7273 G 0.0 0.0 0.0 -2.254844E-02 6.753532E-05 -1.522828E-03',
'7274 G 0.0 0.0 0.0 -2.279390E-02 1.774249E-05 -1.334391E-03',
'7275 G 0.0 0.0 0.0 -2.277315E-02 -3.355248E-05 -1.131589E-03',
'7276 G 0.0 0.0 0.0 -2.247470E-02 -8.549847E-05 -9.224852E-04',
'7277 G 0.0 0.0 0.0 -2.189133E-02 -1.372712E-04 -7.157133E-04',
'7278 G 0.0 0.0 0.0 -2.102014E-02 -1.880403E-04 -5.202099E-04',
'7279 G 0.0 0.0 0.0 -1.986319E-02 -2.369646E-04 -3.449890E-04',
'7280 G 0.0 0.0 0.0 -1.842674E-02 -2.833290E-04 -1.988935E-04',
'7281 G 0.0 0.0 0.0 -1.672131E-02 -3.264680E-04 -9.035443E-05',
'7282 G 0.0 0.0 0.0 -1.476152E-02 -3.656876E-04 -2.717614E-05',
'7283 G 0.0 0.0 0.0 -1.256647E-02 -4.004281E-04 -1.629253E-05',
'7284 G 0.0 0.0 0.0 -1.015823E-02 -4.303372E-04 -6.362354E-05',
'7285 G 0.0 0.0 0.0 -7.561759E-03 -4.550700E-04 -1.739045E-04',
'7286 G 0.0 0.0 0.0 -4.804914E-03 -4.742260E-04 -3.505420E-04',
'7287 G 0.0 0.0 0.0 -1.918424E-03 -4.876927E-04 -5.954905E-04',
'7288 G 0.0 0.0 0.0 1.066243E-03 -4.954925E-04 -9.092903E-04',
'7289 G 0.0 0.0 0.0 4.116441E-03 -4.976777E-04 -1.290999E-03',
'7290 G 0.0 0.0 0.0 7.198711E-03 -4.941604E-04 -1.738162E-03',
'7291 G 0.0 0.0 0.0 1.027845E-02 -4.851927E-04 -2.246741E-03',
'7292 G 0.0 0.0 0.0 1.332275E-02 -4.711772E-04 -2.811515E-03',
'7293 G 0.0 0.0 0.0 1.629910E-02 -4.523546E-04 -3.425953E-03',
'7294 G 0.0 0.0 0.0 1.917596E-02 -4.289898E-04 -4.082270E-03',
'7295 G 0.0 0.0 0.0 2.192276E-02 -4.014440E-04 -4.771517E-03',
'7296 G 0.0 0.0 0.0 2.451170E-02 -3.703599E-04 -5.484174E-03',
'7297 G 0.0 0.0 0.0 2.691683E-02 -3.361543E-04 -6.209807E-03',
'7298 G 0.0 0.0 0.0 2.911462E-02 -2.990009E-04 -6.937434E-03',
'7299 G 0.0 0.0 0.0 3.108352E-02 -2.596472E-04 -7.655686E-03',
'7300 G 0.0 0.0 0.0 3.280532E-02 -2.184632E-04 -8.353029E-03',
'7301 G 0.0 0.0 0.0 3.426476E-02 -1.759197E-04 -9.017997E-03',
'7302 G 0.0 0.0 0.0 3.544938E-02 -1.324717E-04 -9.639216E-03',
'7303 G 0.0 0.0 0.0 3.634929E-02 -8.843491E-05 -1.020561E-02',
'7304 G 0.0 0.0 0.0 3.695816E-02 -4.435903E-05 -1.070667E-02',
'7305 G 0.0 0.0 0.0 3.727290E-02 -6.557301E-07 -1.113260E-02',
'7306 G 0.0 0.0 0.0 3.729299E-02 4.243319E-05 -1.147457E-02',
'7307 G 0.0 0.0 0.0 3.702102E-02 8.445788E-05 -1.172442E-02',
'7308 G 0.0 0.0 0.0 3.646299E-02 1.251660E-04 -1.187534E-02',
'7309 G 0.0 0.0 0.0 3.562659E-02 1.642386E-04 -1.192169E-02',
'7310 G 0.0 0.0 0.0 3.452363E-02 2.017011E-04 -1.185930E-02',
'7311 G 0.0 0.0 0.0 3.316473E-02 2.369071E-04 -1.168458E-02',
'7312 G 0.0 0.0 0.0 3.156928E-02 2.713725E-04 -1.139660E-02',
'7313 G 0.0 0.0 0.0 2.973885E-02 3.023380E-04 -1.099048E-02',
'7314 G 0.0 0.0 0.0 2.771325E-02 3.401653E-04 -1.047144E-02',
'7315 G 0.0 0.0 0.0 2.543261E-02 3.599349E-04 -9.820674E-03',
'7316 G 0.0 0.0 0.0 2.303620E-02 4.483988E-04 -9.053966E-03',
'7317 G 0.0 0.0 0.0 2.015015E-02 3.680175E-04 -8.111417E-03',
'7318 G 0.0 0.0 0.0 1.734203E-02 8.962593E-04 -6.898442E-03',
'7319 G 0.0 0.0 0.0 1.293533E-02 -1.037660E-04 -5.614001E-03',
'7320 G 0.0 0.0 0.0 6.715167E-03 2.935881E-03 -1.681344E-03',
'7321 G 0.0 0.0 0.0 -1.925452E-06 -9.375238E-06 -2.608838E-06',
'7322 G -6.823360E-05 1.435983E-05 6.082949E-05 7.756484E-06 -2.577170E-05 1.456521E-05',
'7323 G -1.968978E-04 6.570999E-05 2.270545E-04 4.888926E-05 -5.037818E-05 4.911626E-05',
'7324 G -3.873697E-04 1.604050E-04 5.172742E-04 1.267640E-04 -7.556767E-05 1.001224E-04',
'7325 G -6.331790E-04 2.944199E-04 9.284513E-04 2.409916E-04 -9.749331E-05 1.644443E-04',
'7326 G -9.221990E-04 4.581800E-04 1.442364E-03 3.869134E-04 -1.133447E-04 2.379998E-04',
'7327 G -1.238365E-03 6.386000E-04 2.028668E-03 5.564194E-04 -1.209288E-04 3.159248E-04',
'7328 G -1.563143E-03 8.205113E-04 2.647201E-03 7.385519E-04 -1.184654E-04 3.928505E-04',
'7329 G -1.876570E-03 9.876314E-04 3.249530E-03 9.199430E-04 -1.045441E-04 4.631632E-04',
'7330 G -2.158222E-03 1.123439E-03 3.780576E-03 1.085245E-03 -7.814648E-05 5.212718E-04',
'7331 G -2.388134E-03 1.212101E-03 4.180369E-03 1.217595E-03 -3.870377E-05 5.618648E-04',
'7332 G -2.547863E-03 1.239300E-03 4.386411E-03 1.299286E-03 1.387789E-05 5.802092E-04',
'7333 G -2.621319E-03 1.193072E-03 4.335866E-03 1.312376E-03 7.919144E-05 5.723748E-04',
'7334 G -2.595507E-03 1.064444E-03 3.967895E-03 1.239348E-03 1.563191E-04 5.354343E-04',
'7335 G -2.461227E-03 8.481031E-04 3.226334E-03 1.063860E-03 2.438154E-04 4.676474E-04',
'7336 G -2.213612E-03 5.428195E-04 2.062120E-03 7.714584E-04 3.397792E-04 3.685738E-04']
I want to convert this list into a Data frame.

The below works:
import pandas as pd
lst = ['7237 G 0.0 0.0 0.0 1.239999E-07 -3.733407E-07 2.212587E-07',
'7238 G 0.0 0.0 0.0 4.846863E-06 -9.922069E-07 8.458463E-06',
'7239 G 0.0 0.0 0.0 3.073667E-05 -3.717125E-06 2.890958E-05',
'7240 G 0.0 0.0 0.0 8.920388E-05 -7.419183E-06 6.471878E-05',
'7241 G 0.0 0.0 0.0 1.832756E-04 -1.062274E-05 1.152823E-04',
'7242 G 0.0 0.0 0.0 3.104847E-04 -1.263393E-05 1.780886E-04',
'7243 G 0.0 0.0 0.0 4.645259E-04 -1.285412E-05 2.493027E-04',
'7244 G 0.0 0.0 0.0 6.359626E-04 -1.083016E-05 3.242427E-04',
'7245 G 0.0 0.0 0.0 8.127339E-04 -6.187393E-06 3.976933E-04',
'7246 G 0.0 0.0 0.0 9.805798E-04 1.373597E-06 4.641845E-04',
'7247 G 0.0 0.0 0.0 1.123478E-03 1.204513E-05 5.182628E-04',
'7248 G 0.0 0.0 0.0 1.224244E-03 2.590691E-05 5.547877E-04',
'7249 G 0.0 0.0 0.0 1.265126E-03 4.292454E-05 5.691787E-04',
'7250 G 0.0 0.0 0.0 1.228418E-03 6.293558E-05 5.576411E-04',
'7251 G 0.0 0.0 0.0 1.097197E-03 8.563578E-05 5.173817E-04',
'7252 G 0.0 0.0 0.0 8.560375E-04 1.105952E-04 4.467830E-04',
'7253 G 0.0 0.0 0.0 4.916305E-04 1.372820E-04 3.454892E-04',
'7254 G 0.0 0.0 0.0 -6.634202E-06 1.650501E-04 2.144534E-04',
'7255 G 0.0 0.0 0.0 -6.459849E-04 1.931538E-04 5.597509E-05',
'7256 G 0.0 0.0 0.0 -1.429897E-03 2.208132E-04 -1.264037E-04',
'7257 G 0.0 0.0 0.0 -2.357971E-03 2.472607E-04 -3.280706E-04',
'7258 G 0.0 0.0 0.0 -3.425660E-03 2.716504E-04 -5.434573E-04',
'7259 G 0.0 0.0 0.0 -4.623938E-03 2.931546E-04 -7.662221E-04',
'7260 G 0.0 0.0 0.0 -5.939634E-03 3.110625E-04 -9.895572E-04',
'7261 G 0.0 0.0 0.0 -7.355830E-03 3.247523E-04 -1.206506E-03',
'7262 G 0.0 0.0 0.0 -8.851902E-03 3.335908E-04 -1.410110E-03',
'7263 G 0.0 0.0 0.0 -1.040373E-02 3.370576E-04 -1.593686E-03',
'7264 G 0.0 0.0 0.0 -1.198441E-02 3.348549E-04 -1.751147E-03',
'7265 G 0.0 0.0 0.0 -1.356535E-02 3.268293E-04 -1.877329E-03',
'7266 G 0.0 0.0 0.0 -1.511621E-02 3.128250E-04 -1.967968E-03',
'7267 G 0.0 0.0 0.0 -1.660536E-02 2.928187E-04 -2.019983E-03',
'7268 G 0.0 0.0 0.0 -1.800119E-02 2.671044E-04 -2.031644E-03',
'7269 G 0.0 0.0 0.0 -1.927284E-02 2.360256E-04 -2.002611E-03',
'7270 G 0.0 0.0 0.0 -2.039046E-02 1.999231E-04 -1.934050E-03',
'7271 G 0.0 0.0 0.0 -2.132561E-02 1.593439E-04 -1.828405E-03',
'7272 G 0.0 0.0 0.0 -2.205253E-02 1.149674E-04 -1.689621E-03',
'7273 G 0.0 0.0 0.0 -2.254844E-02 6.753532E-05 -1.522828E-03',
'7274 G 0.0 0.0 0.0 -2.279390E-02 1.774249E-05 -1.334391E-03',
'7275 G 0.0 0.0 0.0 -2.277315E-02 -3.355248E-05 -1.131589E-03',
'7276 G 0.0 0.0 0.0 -2.247470E-02 -8.549847E-05 -9.224852E-04',
'7277 G 0.0 0.0 0.0 -2.189133E-02 -1.372712E-04 -7.157133E-04',
'7278 G 0.0 0.0 0.0 -2.102014E-02 -1.880403E-04 -5.202099E-04',
'7279 G 0.0 0.0 0.0 -1.986319E-02 -2.369646E-04 -3.449890E-04',
'7280 G 0.0 0.0 0.0 -1.842674E-02 -2.833290E-04 -1.988935E-04',
'7281 G 0.0 0.0 0.0 -1.672131E-02 -3.264680E-04 -9.035443E-05',
'7282 G 0.0 0.0 0.0 -1.476152E-02 -3.656876E-04 -2.717614E-05',
'7283 G 0.0 0.0 0.0 -1.256647E-02 -4.004281E-04 -1.629253E-05',
'7284 G 0.0 0.0 0.0 -1.015823E-02 -4.303372E-04 -6.362354E-05',
'7285 G 0.0 0.0 0.0 -7.561759E-03 -4.550700E-04 -1.739045E-04',
'7286 G 0.0 0.0 0.0 -4.804914E-03 -4.742260E-04 -3.505420E-04',
'7287 G 0.0 0.0 0.0 -1.918424E-03 -4.876927E-04 -5.954905E-04',
'7288 G 0.0 0.0 0.0 1.066243E-03 -4.954925E-04 -9.092903E-04',
'7289 G 0.0 0.0 0.0 4.116441E-03 -4.976777E-04 -1.290999E-03',
'7290 G 0.0 0.0 0.0 7.198711E-03 -4.941604E-04 -1.738162E-03',
'7291 G 0.0 0.0 0.0 1.027845E-02 -4.851927E-04 -2.246741E-03',
'7292 G 0.0 0.0 0.0 1.332275E-02 -4.711772E-04 -2.811515E-03',
'7293 G 0.0 0.0 0.0 1.629910E-02 -4.523546E-04 -3.425953E-03',
'7294 G 0.0 0.0 0.0 1.917596E-02 -4.289898E-04 -4.082270E-03',
'7295 G 0.0 0.0 0.0 2.192276E-02 -4.014440E-04 -4.771517E-03',
'7296 G 0.0 0.0 0.0 2.451170E-02 -3.703599E-04 -5.484174E-03',
'7297 G 0.0 0.0 0.0 2.691683E-02 -3.361543E-04 -6.209807E-03',
'7298 G 0.0 0.0 0.0 2.911462E-02 -2.990009E-04 -6.937434E-03',
'7299 G 0.0 0.0 0.0 3.108352E-02 -2.596472E-04 -7.655686E-03',
'7300 G 0.0 0.0 0.0 3.280532E-02 -2.184632E-04 -8.353029E-03',
'7301 G 0.0 0.0 0.0 3.426476E-02 -1.759197E-04 -9.017997E-03',
'7302 G 0.0 0.0 0.0 3.544938E-02 -1.324717E-04 -9.639216E-03',
'7303 G 0.0 0.0 0.0 3.634929E-02 -8.843491E-05 -1.020561E-02',
'7304 G 0.0 0.0 0.0 3.695816E-02 -4.435903E-05 -1.070667E-02',
'7305 G 0.0 0.0 0.0 3.727290E-02 -6.557301E-07 -1.113260E-02',
'7306 G 0.0 0.0 0.0 3.729299E-02 4.243319E-05 -1.147457E-02',
'7307 G 0.0 0.0 0.0 3.702102E-02 8.445788E-05 -1.172442E-02',
'7308 G 0.0 0.0 0.0 3.646299E-02 1.251660E-04 -1.187534E-02',
'7309 G 0.0 0.0 0.0 3.562659E-02 1.642386E-04 -1.192169E-02',
'7310 G 0.0 0.0 0.0 3.452363E-02 2.017011E-04 -1.185930E-02',
'7311 G 0.0 0.0 0.0 3.316473E-02 2.369071E-04 -1.168458E-02',
'7312 G 0.0 0.0 0.0 3.156928E-02 2.713725E-04 -1.139660E-02',
'7313 G 0.0 0.0 0.0 2.973885E-02 3.023380E-04 -1.099048E-02',
'7314 G 0.0 0.0 0.0 2.771325E-02 3.401653E-04 -1.047144E-02',
'7315 G 0.0 0.0 0.0 2.543261E-02 3.599349E-04 -9.820674E-03',
'7316 G 0.0 0.0 0.0 2.303620E-02 4.483988E-04 -9.053966E-03',
'7317 G 0.0 0.0 0.0 2.015015E-02 3.680175E-04 -8.111417E-03',
'7318 G 0.0 0.0 0.0 1.734203E-02 8.962593E-04 -6.898442E-03',
'7319 G 0.0 0.0 0.0 1.293533E-02 -1.037660E-04 -5.614001E-03',
'7320 G 0.0 0.0 0.0 6.715167E-03 2.935881E-03 -1.681344E-03',
'7321 G 0.0 0.0 0.0 -1.925452E-06 -9.375238E-06 -2.608838E-06',
'7322 G -6.823360E-05 1.435983E-05 6.082949E-05 7.756484E-06 -2.577170E-05 1.456521E-05',
'7323 G -1.968978E-04 6.570999E-05 2.270545E-04 4.888926E-05 -5.037818E-05 4.911626E-05',
'7324 G -3.873697E-04 1.604050E-04 5.172742E-04 1.267640E-04 -7.556767E-05 1.001224E-04',
'7325 G -6.331790E-04 2.944199E-04 9.284513E-04 2.409916E-04 -9.749331E-05 1.644443E-04',
'7326 G -9.221990E-04 4.581800E-04 1.442364E-03 3.869134E-04 -1.133447E-04 2.379998E-04',
'7327 G -1.238365E-03 6.386000E-04 2.028668E-03 5.564194E-04 -1.209288E-04 3.159248E-04',
'7328 G -1.563143E-03 8.205113E-04 2.647201E-03 7.385519E-04 -1.184654E-04 3.928505E-04',
'7329 G -1.876570E-03 9.876314E-04 3.249530E-03 9.199430E-04 -1.045441E-04 4.631632E-04',
'7330 G -2.158222E-03 1.123439E-03 3.780576E-03 1.085245E-03 -7.814648E-05 5.212718E-04',
'7331 G -2.388134E-03 1.212101E-03 4.180369E-03 1.217595E-03 -3.870377E-05 5.618648E-04',
'7332 G -2.547863E-03 1.239300E-03 4.386411E-03 1.299286E-03 1.387789E-05 5.802092E-04',
'7333 G -2.621319E-03 1.193072E-03 4.335866E-03 1.312376E-03 7.919144E-05 5.723748E-04',
'7334 G -2.595507E-03 1.064444E-03 3.967895E-03 1.239348E-03 1.563191E-04 5.354343E-04',
'7335 G -2.461227E-03 8.481031E-04 3.226334E-03 1.063860E-03 2.438154E-04 4.676474E-04',
'7336 G -2.213612E-03 5.428195E-04 2.062120E-03 7.714584E-04 3.397792E-04 3.685738E-04']
lst = [x.split() for x in lst]
df = pd.DataFrame(lst)
print(df)
output
0 1 2 ... 5 6 7
0 7237 G 0.0 ... 1.239999E-07 -3.733407E-07 2.212587E-07
1 7238 G 0.0 ... 4.846863E-06 -9.922069E-07 8.458463E-06
2 7239 G 0.0 ... 3.073667E-05 -3.717125E-06 2.890958E-05
3 7240 G 0.0 ... 8.920388E-05 -7.419183E-06 6.471878E-05
4 7241 G 0.0 ... 1.832756E-04 -1.062274E-05 1.152823E-04
5 7242 G 0.0 ... 3.104847E-04 -1.263393E-05 1.780886E-04
6 7243 G 0.0 ... 4.645259E-04 -1.285412E-05 2.493027E-04
7 7244 G 0.0 ... 6.359626E-04 -1.083016E-05 3.242427E-04
8 7245 G 0.0 ... 8.127339E-04 -6.187393E-06 3.976933E-04
9 7246 G 0.0 ... 9.805798E-04 1.373597E-06 4.641845E-04
10 7247 G 0.0 ... 1.123478E-03 1.204513E-05 5.182628E-04
...

Iterating over pandas DataFrame with identical columns header

I am trying to iterate through rows and columns of the Pandas DataFrame and write that result in a new DataFrame if some condition is met. I am able to iterate on the following DataFrame which has different names for row and column.
W0O5 W1O5 W2O5 W3O5
W0O5 0.0 0.0 0.0 0.0
W1O5 0.0 0.0 1.0 0.0
W2O5 0.0 1.0 0.0 0.0
W3O5 0.0 0.0 0.0 0.0
I used the following approach
for i in pandas_df.index:
for j in pandas_df.columns:
print(i, j)
print(pandas_df.at[i, j])
if pandas_df.at[i, j] ==1:
single_pandas_df.at['WO5', 'WO5_corner'] =1
where single_pandas_df  is the new DataFrame I created, on which I want to add the value at corresponding row and column.
However, when I try to iterate through  DataFrame containing identical header for row and columns as below:
WO5 WO5 WO5 WO5
WO5 0.0 0.0 0.0 0.0
WO5 0.0 0.0 1.0 0.0
WO5 0.0 1.0 0.0 0.0
WO5 0.0 0.0 0.0 0.0
I get the AttributeError saying
AttributeError: 'BlockManager' object has no attribute 'T'
I know the error is due to duplicate column names. I was curious is there any way to handle such case in pandas. I have all of my DataFrames as in the second case and I need to get the values of each index from row and column.
Thanks in advance.
Update after Yolos comment:
Actually I have many such DataFrames as below
DyO7 DyO7 DyO6 DyO7 DyO7 DyO6
DyO7 0.0 3.0 1.0 2.0 1.0 0.0
DyO7 3.0 0.0 0.0 1.0 0.0 1.0
DyO6 1.0 0.0 0.0 0.0 1.0 0.0
DyO7 2.0 1.0 0.0 0.0 3.0 1.0
DyO7 1.0 0.0 1.0 3.0 0.0 0.0
DyO6 0.0 1.0 0.0 1.0 0.0 0.0
and next one as
TaO6 TaO6
TaO6 0.0 1.0
TaO6 1.0 0.0
In these DataFrames 1 ,2 and 3 represents the corner, edge and face sharing. So if (i,j) item in the DataFrame is 1, it goes to "..."_corner, if 2 it goes to edge and 3 goes to face.
my initial single_pandas DataFrame looks like following
DyO6_corner DyO6_edge DyO6_face DyO7_corner DyO7_edge DyO7_face TaO6_corner TaO6_edge TaO6_face WO5_corner WO5_edge WO5_face
DyO6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
DyO7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
TaO6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
WO5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
As from my above script after updating this single_pandas DataFrame, there will be 1 at ('WO5', 'WO5_corner') and it becomes:
DyO6_corner DyO6_edge DyO6_face DyO7_corner DyO7_edge DyO7_face TaO6_corner TaO6_edge TaO6_face WO5_corner WO5_edge WO5_face
DyO6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
DyO7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
TaO6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
WO5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1 0.0 0.0

Need to add column names to numpy array

I am trying to create connect 4 game with 6/7 arrray in python, and i need column headers so that column 0 is named a, column 2 is named b, and so on. The purpose of this is for the moves to be initiated by typing 'a' (drops token in first column) 'b' (drops token in second) etc.... This is my code to create the array
def clear_board():
board = np.zeros((6,7))
return board

If you need column names, the easiest way is to use a pandas Dataframe instead of a numpy array:
import pandas as pd
def clear_board():
board = pd.DataFrame(np.zeros((6,7)),columns=list('ABCDEFG'))
return board
>>> clear_board()
A B C D E F G
0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
1 0.0 0.0 0.0 0.0 0.0 0.0 0.0
2 0.0 0.0 0.0 0.0 0.0 0.0 0.0
3 0.0 0.0 0.0 0.0 0.0 0.0 0.0
4 0.0 0.0 0.0 0.0 0.0 0.0 0.0
5 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Beyond that, take a look at the options provided in this answer

"Cannot reindex from a duplicate axis" when groupby.apply() on MultiIndex columns

I'm playing around with computing subtotals within a DataFrame that looks like this (note the MultiIndex):
0 1 2 3 4 5
A 1 0.0 0.0 0.0 0.0 0.0 0.0
2 0.0 0.0 0.0 0.0 0.0 0.0
B 1 0.0 0.0 0.0 0.0 0.0 0.0
2 0.0 0.0 0.0 0.0 0.0 0.0
I can successfully add the subtotals with the following code:
(
df
.groupby(level=0)
.apply(
lambda df: pd.concat(
[df.xs(df.name), df.sum().to_frame('Total').T]
)
)
)
And it looks like this:
0 1 2 3 4 5
A 1 0.0 0.0 0.0 0.0 0.0 0.0
2 0.0 0.0 0.0 0.0 0.0 0.0
Total 0.0 0.0 0.0 0.0 0.0 0.0
B 1 0.0 0.0 0.0 0.0 0.0 0.0
2 0.0 0.0 0.0 0.0 0.0 0.0
Total 0.0 0.0 0.0 0.0 0.0 0.0
However, when I work with the transposed DataFrame, it does not work. The DataFrame looks like:
A B
1 2 1 2
0 0.0 0.0 0.0 0.0
1 0.0 0.0 0.0 0.0
2 0.0 0.0 0.0 0.0
3 0.0 0.0 0.0 0.0
4 0.0 0.0 0.0 0.0
5 0.0 0.0 0.0 0.0
And I use the following code:
(
df2
.groupby(level=0, axis=1)
.apply(
lambda df: pd.concat(
[df.xs(df.name, axis=1), df.sum(axis=1).to_frame('Total')],
axis=1
)
)
)
I have specified axis=1 everywhere I can think of, but I get an error:
ValueError: cannot reindex from a duplicate axis
I would expect the output to be:
A B
1 2 Total 1 2 Total
0 0.0 0.0 0.0 0.0 0.0 0.0
1 0.0 0.0 0.0 0.0 0.0 0.0
2 0.0 0.0 0.0 0.0 0.0 0.0
3 0.0 0.0 0.0 0.0 0.0 0.0
4 0.0 0.0 0.0 0.0 0.0 0.0
5 0.0 0.0 0.0 0.0 0.0 0.0
Is this a bug? Or have I not specified the axis correctly everywhere? As a workaround, I can obviously transpose the DataFrame, produce the totals, and transpose back, but I'd like to know why it's not working here, and submit a bug report if necessary.
The problem DataFrame can be generated with:
df2 = pd.DataFrame(
np.zeros([6, 4]),
columns=pd.MultiIndex.from_product([['A', 'B'], [1, 2]])
)

Categories