We Keep Coding

What is the most efficient way to swap the values of two columns of a 2D list in python when the number of rows is in the tens of thousands?

for example if I have an original list:
A B
1 3
2 4
to be turned into
A B
3 1
4 2

two cents worth:
3 ways to do it
you could add a 3rd column C, copy A to C, then delete A. This would take more memory.
you could create a swap function for the values in a row, then wrap it into a loop.
you could just swap the labels of the columns. This is probably the most efficient way.

You could use rename:
df2 = df.rename(columns={'A': 'B', 'B': 'A'})
output:
B A
0 1 3
1 2 4
If order matters:
df2 = df.rename(columns={'A': 'B', 'B': 'A'})[df.columns]
output:
A B
0 3 1
1 4 2

Use DataFrame.rename with dictionary for swapping columnsnames, last check orcer by selecting columns:
df = df.rename(columns=dict(zip(df.columns, df.columns[::-1])))[df.columns]
print (df)
A B
0 3 1
1 4 2

You can also just simple use masking to change the values.
import pandas as pd
df = pd.DataFrame({"A":[1,2],"B":[3,4]})
df[["A","B"]] = df[["B","A"]].values
df
A B
0 3 1
1 4 2

for more than 2 columns:
df = pd.DataFrame({'A':[1,2,3],'B':[4,5,6],'C':[7,8,9], 'D':[10,11,12]})
print(df)
'''
A B C D
0 1 4 7 10
1 2 5 8 11
2 3 6 9 12
'''
df = df.set_axis(df.columns[::-1],axis=1)[df.columns]
print(df)
'''
A B C D
0 10 7 4 1
1 11 8 5 2
2 12 9 6 3

I assume that your list is like this:
my_list = [[1, 3], [2, 4]]
So you can use this code:
print([[each_element[1], each_element[0]] for each_element in my_list])
The output is:
[[3, 1], [4, 2]]

Getting the total for some columns (independently) in a data frame with python [duplicate]

I have the following DataFrame:
In [1]:
df = pd.DataFrame({'a': [1, 2, 3],
'b': [2, 3, 4],
'c': ['dd', 'ee', 'ff'],
'd': [5, 9, 1]})
df
Out [1]:
a b c d
0 1 2 dd 5
1 2 3 ee 9
2 3 4 ff 1
I would like to add a column 'e' which is the sum of columns 'a', 'b' and 'd'.
Going across forums, I thought something like this would work:
df['e'] = df[['a', 'b', 'd']].map(sum)
But it didn't.
I would like to know the appropriate operation with the list of columns ['a', 'b', 'd'] and df as inputs.

You can just sum and set param axis=1 to sum the rows, this will ignore none numeric columns:
In [91]:
df = pd.DataFrame({'a': [1,2,3], 'b': [2,3,4], 'c':['dd','ee','ff'], 'd':[5,9,1]})
df['e'] = df.sum(axis=1)
df
Out[91]:
a b c d e
0 1 2 dd 5 8
1 2 3 ee 9 14
2 3 4 ff 1 8
If you want to just sum specific columns then you can create a list of the columns and remove the ones you are not interested in:
In [98]:
col_list= list(df)
col_list.remove('d')
col_list
Out[98]:
['a', 'b', 'c']
In [99]:
df['e'] = df[col_list].sum(axis=1)
df
Out[99]:
a b c d e
0 1 2 dd 5 3
1 2 3 ee 9 5
2 3 4 ff 1 7

If you have just a few columns to sum, you can write:
df['e'] = df['a'] + df['b'] + df['d']
This creates new column e with the values:
a b c d e
0 1 2 dd 5 8
1 2 3 ee 9 14
2 3 4 ff 1 8
For longer lists of columns, EdChum's answer is preferred.

Create a list of column names you want to add up.
df['total']=df.loc[:,list_name].sum(axis=1)
If you want the sum for certain rows, specify the rows using ':'

This is a simpler way using iloc to select which columns to sum:
df['f']=df.iloc[:,0:2].sum(axis=1)
df['g']=df.iloc[:,[0,1]].sum(axis=1)
df['h']=df.iloc[:,[0,3]].sum(axis=1)
Produces:
a b c d e f g h
0 1 2 dd 5 8 3 3 6
1 2 3 ee 9 14 5 5 11
2 3 4 ff 1 8 7 7 4
I can't find a way to combine a range and specific columns that works e.g. something like:
df['i']=df.iloc[:,[[0:2],3]].sum(axis=1)
df['i']=df.iloc[:,[0:2,3]].sum(axis=1)

You can simply pass your dataframe into the following function:
def sum_frame_by_column(frame, new_col_name, list_of_cols_to_sum):
frame[new_col_name] = frame[list_of_cols_to_sum].astype(float).sum(axis=1)
return(frame)
Example:
I have a dataframe (awards_frame) as follows:
...and I want to create a new column that shows the sum of awards for each row:
Usage:
I simply pass my awards_frame into the function, also specifying the name of the new column, and a list of column names that are to be summed:
sum_frame_by_column(awards_frame, 'award_sum', ['award_1','award_2','award_3'])
Result:

Following syntax helped me when I have columns in sequence
awards_frame.values[:,1:4].sum(axis =1)

You can use the function aggragate or agg:
df[['a','b','d']].agg('sum', axis=1)
The advantage of agg is that you can use multiple aggregation functions:
df[['a','b','d']].agg(['sum', 'prod', 'min', 'max'], axis=1)
Output:
sum prod min max
0 8 10 1 5
1 14 54 2 9
2 8 12 1 4

The shortest and simplest way here is to use
df.eval('e = a + b + d')

Merging a dataframe with another dataframe with constant values from the first dataframe

I would like to merge two data frames, df1 and df2:
import pandas as pd
df1 = pd.DataFrame({
'A': ['a'],
'B': ['b'],
'C': ['c']
})
df2 = pd.DataFrame({
'W': [1, 2, 3],
'X': [4, 5, 6],
'Y': [7, 8, 9],
'Z': [10, 11, 12]
})
df1: (will always have only one row)
df2: (can have any number of rows)
In a way that all the columns of df1 are added to the df2 dataframe with all the rows having the same values present in the df1 dataframe.
I have tried:
df3 = pd.concat([df1,df2], sort=False, axis=1)
But this is giving me NaN's:
But i want all the rows to have the same constant value that is present in df1 like:
I would also like to maintain having the new columns from df1 be before the columns of df2 as above.
What might be the most efficient way to achieve this.

We can do an outer merge on an artificially created key:
df1.assign(key=1).merge(df2.assign(key=1), on='key').drop('key', axis=1)
A B C W X Y Z
0 a b c 1 4 7 10
1 a b c 2 5 8 11
2 a b c 3 6 9 12

Use DataFrame.assign with selecting first row and then change order of columns by DataFrame.reindex:
df3 = df2.assign(**df1.iloc[0]).reindex(df1.columns.union(df2.columns, sort=False),axis=1)
print (df3)
A B C W X Y Z
0 a b c 1 4 7 10
1 a b c 2 5 8 11
2 a b c 3 6 9 12
Or add rows to df1 by df2.index with method='ffill':
df3 = pd.concat([df1.reindex(df2.index, method='ffill'),df2], sort=False, axis=1)
print (df3)
A B C W X Y Z
0 a b c 1 4 7 10
1 a b c 2 5 8 11
2 a b c 3 6 9 12

How to delete the row in a dataframe panda based on the row names of another dataframe?

I want to short my data, the whole data shape is 30000x480. And I want to drop some rows based on the row names of another data frame.
Help me to solve it and get the solution for:
df1
Row a b
A 1 2
B 3 4
C 5 6
D 7 8
E 9 10
F 11 12
G 13 14
df2
Row a b
C 5 6
D 7 8
F 11 12
G 13 14
So, I want to delete the rows in df1 that doesn't exist on the df2, it's hard to delete as manually because the data is very big

For better understanding, taking the same data given. Let me put the same question in a different context for a better understanding as below:
Question : Want to delete the rows in df1 that doesn't exist on the df2
New way : you need the rows of df1 that are present in df2 (or) in a way you need the common rows of both df1 & df2, try this
>>> import pandas as pd
>>> df2 = pd.DataFrame({'Row': ['C', 'D', 'F','G'], 'a': [5, 7, 11, 13], 'b' : [6, 8, 12, 14]})
>>> df1 = pd.DataFrame({'Row' : ['A', 'B', 'C', 'D'], 'a': [1,3,5,7], 'b': [2,4,6, 8]})
>>> df1
Row a b
0 A 1 2
1 B 3 4
2 C 5 6
3 D 7 8
>>> df2
Row a b
0 C 5 6
1 D 7 8
2 F 11 12
3 G 13 14
>>> pd.merge(df1, df2, 'inner')
Row a b
0 C 5 6
1 D 7 8
>>>

pandas reorder only a specific row

I have a Dataframe file in which I want to switch the order of columns in only the third row while keeping other rows the same.
Under some condition, I have to switch orders for my project, but here is an example that probably has no real meaning.
Suppose the dataset is
df = pd.DataFrame({'A': [0, 1, 2, 3, 4],
'B': [5, 6, 7, 8, 9],
'C': ['a', 'b', 'c', 'd', 'e']})
df
out[1]:
A B C
0 0 5 a
1 1 6 b
2 2 7 c
3 3 8 d
4 4 9 e
I want to have the output:
A B C
0 0 5 a
1 1 6 b
2 **7 2** c
3 3 8 d
4 4 9 e
How do I do it?
I have tried:
new_order = [1, 0, 2] # specify new order of the third row
i = 2 # specify row number
df.iloc[i] = df[df.columns[new_order]].loc[i] # reorder the third row only and assign new values to df
I observed from the output of the right-hand side that the columns are reordering as I wanted:
df[df.columns[new_order]].loc[i]
Out[2]:
B 7
A 2
C c
Name: 2, dtype: object
But when assigned to df again, it did nothing. I guess it's because of the name matching.
Can someone help me? Thanks in advance!