Drop rows from pandas DataFrame based on alternating columns - python

I'm attempting to remove all rows in a dataframe between an entry and exit point in a timeseries of price data based on a bool entry and exit columns.
data = {'Entry': [True,True,True,False,False,False,False,True, False, False, False],
'Exit': [False,False,True,False,False,True,True,False, False, False, True]}
df = pd.DataFrame(data)
Entry Exit
0 True False
1 True False
2 True True
3 False False
4 False False
5 False True
6 False True
7 True False
8 False False
9 False False
10 False True
So given the above I want to be left with
Entry Exit
0 True False
2 True True
7 True False
10 False True
I need to get the first True from the Entry column, then the following True in the Exit column, followed by the next True in the Entry column and so on.

You can do it the old fashion way using zip:
df = pd.DataFrame(data)
group = None
idx = []
for num, (a, b) in enumerate(zip(df["Entry"], df["Exit"])):
if a is True and not group:
idx.append(num)
group = True
if b is True and group:
if idx[-1] != num:
idx.append(num)
group = False
print (idx) # [0, 2, 7, 10]
print (df.loc[idx])
Entry Exit
0 True False
2 True True
7 True False
10 False True

Try this:
entry = df[df['Entry']]
exit = df[df['Exit']]
idx = []
pos = 0
for i in range(entry.shape[0]):
if i % 2 == 0:
print("bad")
idx.append([entry.iloc[pos][0],entry.iloc[pos][1]])
else:
idx.append([exit.iloc[pos][0],exit.iloc[pos][1]])
pos += 1
,Hope this help

Related

How to make a matrix using index and the value in python?

I have a dataset, which has two columns:
index Value
0 True
1 True
2 False
3 True
Is it possible to obtain a matrix that looks like
index 0 1 2 3
0 True True False True
1 True True False True
2 False False False False
3 True True False True
I tried pd.crosstab, still not able to get the matrix, can anyone please help?
A possible way:
m = np.tile(df['Value'], len(df)).reshape(-1, len(df)) * df[['Value']].values
out = pd.DataFrame(m)
print(out)
# Output
0 1 2 3
0 True True False True
1 True True False True
2 False False False False
3 True True False True
First, convert the values of Value columns to a numpy array using to_numpy. Then take advantage of numpy broadcasting by creating an extra axis with [:,None] and computing the bitwise and operation:
vals = df['Value'].to_numpy()
res = pd.DataFrame(vals[:,None] & vals, index=df.index)
Output:
>>> res
0 1 2 3
index
0 True True False True
1 True True False True
2 False False False False
3 True True False True

I want to change a boolean variable based on other boolean variables for every observation in a Python data frame

A data summary looks like this from the data frame. This just shows a subset where multiple A, B and C variables are True. I only want one of them to be true. I created the Multiples variable to select the ones to change and a summary looks like this table.
A B C Multiples
ID
197 True True False True
215 True True False True
225 True False True True
234 True True False True
265 True True False True
321 False True True True
Here is an example of what the improved data would look like. In every row only one of A, B or C is True and it's the rightmost one. I need to know how make that change in Python. Don't worry about the Multiples variable. When I re-compute that, with A, B and C as they are below, all the Multiples will be False. I already have the code to do that.
A B C Multiples
ID
197 False True False False
215 False True False False
225 False False True False
234 False True False False
265 False True False False
321 False False True False
I've searched the web and this site and can't find anything that works, at least that I understand.
Here is my current Python code:
for index, item in enumerate(df['A']):
if ((df['Multiples'] == True) & (df['C'] == True | df['B'] == True)):
df['Multiples'] = False
If you want to have Multiples as True if only one column is True:
import pandas as pd
df = pd.DataFrame({
"A": [True, True, True, False],
"B": [True, True, False, False],
"C": [True, False, False, False]
})
df['Multiples'] = 0+df['A']+df['B']+df['C'] == 1
print(df)
That will output:
A B C Multiples
0 True True True False
1 True True False False
2 True False False True
3 False False False False
To check for more than one True column use > 1 instead of == 1.
I managed to solve the problem myself with the following code:
def clear_redundant(row):
# no need to change rows with only one True value
if not row['Multiples']:
return
# if the last column is true, both earlier should be false
if row['C']:
row['A'] = False
row['B'] = False
# if the middle column is true, the first should be false
elif row['B']:
row['A'] = False
df.apply(clear_redundant, axis="columns")

Merge the True False Value based On Condition

i have df like this
a b c
0 True False True
1 False False False
2 True True True
i want this
a b c Result
0 True False True True
1 False False False False
2 True True True True
if any one Value True then Result True ele false
You can use any():
df['result'] = df.any(1)
# or with pd.assign
df = df.assign(result = df.any(1))
both will print:
a b c result
0 True False True True
1 False False False False
2 True True True True
Note that 1 is short for axis=1, i.e. perform operation row-wise
It's quite easy...
if a or b or c:
#do stuff
or you could also use
if a | b | c:
#do stuff
Use any with (axis=1) to check the existance of any True in each row.
df['result'] = df.any(axis=1)
If values are string rather than boolean then:
df['result'] = df.eq('True').any(axis=1)

How to check if column A row 1 and column B in row 1 has value true in pandas Datarame?

how can i imitate following excel formula in python DataFrame?
=IF(AND(A1=TRUE,B1=TRUE),TRUE,FALSE)
A B C
TRUE FALSE FALSE
TRUE TRUE TRUE
FALSE FALSE FALSE
FALSE TRUE FALSE
i tried this,
def check(sig1,sig2):
if sig1 == True and sig2 == True:
return True
else:
return False
df['chk'] = df.apply(check,df['up_signal1',df['up_signal2']],axis=1)
You can do this:
# DataFrame that checks all possible combinations
df = pd.DataFrame({
'up_signal1': [False, False, True, True],
'up_signal2': [False, True, False, True]
})
df['chk'] = df.up_signal1 & df.up_signal2
df
up_signal1 up_signal2 chk
0 False False False
1 False True False
2 True False False
3 True True True

How to change the first occurrence of 'True' in a row to false in pandas

I'm trying to change the first instance of True to False in my DataFrame dependent on row:
A B C
Number
1 True True True
2 False True True
3 False False True
A B C
Number
1 False True True
2 False False True
3 False False False
Every time I try using the for index, row in target_df.iterrows(): line it ends up never finding any 'True' when I look through the row.
Thanks in advance!
You can use the cumulative sum of the Boolean values (False corresponds to 0; True to 1) for each row, along with DataFrame.mask():
>>> condition = df.cumsum(axis=1) == 1
>>> df.mask(condition, False)
a b c
0 False True True
1 False False True
2 False False False
df.mask(self, cond, other=nan)
Return an object of same shape as self and whose corresponding entries
are from self where cond is False and otherwise are from other.
In this case, condition is False everywhere except the points at which you want to switch True -> False:
>>> condition
a b c
0 True False False
1 False True False
2 False False True
One other option would be to use NumPy:
>>> row, col = np.where(df.cumsum(axis=1) == 1)
>>> df.values[row, col] = False
>>> df
a b c
0 False True True
1 False False True
2 False False False

Categories