I have code as below.
import pandas as pd
import numpy as np
data = [['Alex',10,5,0],['Bob',12,4,1],['Clarke',13,6,0],['brke',15,1,0]]
df = pd.DataFrame(data,columns=['Name','Age','weight','class'],dtype=float)
df_numeric=df.select_dtypes(include='number')#, exclude=None)[source]
df_non_numeric=df.select_dtypes(exclude='number')
df_non_numeric['class']=df_numeric['class'].copy()
it gives me below message
__main__:1: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
i want to have df_non_numeric independent from df_numeric
i used df_numeric['class'].copy() based upon suggestions given in other posts.
How could i avoid the message?
I think you need copy because DataFrame.select_dtypes is slicing operation, filtering by types of column, check Question 3:
df_numeric=df.select_dtypes(include='number').copy()
df_non_numeric=df.select_dtypes(exclude='number').copy()
If you modify values in df_non_numeric later you will find that the modifications do not propagate back to the original data (df), and that Pandas does warning.
Related
I am trying to convert a column in a pandas dataframe to datetime format as follows:
df["date"] = pd.to_datetime(df["date"])
Although this works as expected, pandas gives the following warning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
if sys.path[0] == '':
Is there a better way to to a datetime conversion of a pandas column that does not produce this warning?
This should get rid of the warning:
df.loc["date"] = pd.to_datetime(df["date"])
Pandas discourages it if you set a slice of a dataset. Generally, using .loc is the best way to go when accessing your data.
I am using pandas 1.0.1 and I am creating a new column that converts the date column to a datetime column and I am getting the warning below. I tried using data.loc[:, "Datetime"] as well and I still got the same warning. Please how could this be avoided?
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
data["Datetime"] = pd.to_datetime(data["Date"], infer_datetime_format=True)
Most likely you created your source DataFrame as a view of another
DataFrame (only some columns and / or only some rows).
Find in your code the place where your DataFrame is created and append .copy() there.
Then your DataFrame will be created as a fully independent DataFrame (with its
own data buffer) and this warning should not appear any more.
I have a column in my data frame where I have emails and not emails.
with this slice I can only get the fields that are without email:
df[~df['email'].str.contains('#', case=False)]['email']
But when I try to replace it with a value of my preference:
df[~df['email'].str.contains('#', case=False)]['email'] = 'No'
The column does not receive the change.
I don't get any error, just the following warning:
/home/rockstar/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:1: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
"""Entry point for launching an IPython kernel.
follows an image of my complete dataframe:
Also, df[~df['email'].str.contains('#',case=False)] = 'No' works perfectly but I end up losing data from the rest of the line
Refer the following example code.
import pandas as pd
df = pd.DataFrame({"E-mail":["abc#de", "abcde"]})
df['E-mail'].loc[~df['E-mail'].str.contains('#', case = False)] = 'No'
I have a dataframe as below from which I take a slice called NDCSPart_df using NDCSPart_df = Register_df.iloc[:, :17]
This NDCSPart_df needs to be updated by the latest dataframe NOTES_df of same column length but some with different values, and the same or larger number of rows.
I compare a row of NDCSPart_df and NOTES_df using the "MainDocID" to identify any changes and if there are any changes, the row in NDCSPart_df will be assigned the value of the row with the same "MainDocID" in NOTES_df.
for i in ChangedDocumentIDDict.keys():
NDCSPart_df.loc[NDCSPart_df["MainDocID"]==i,:].update(NOTES_df.loc[NOTES_df["MainDocID"]==i,:])
which gives me the following warning,
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py:5516:
SettingWithCopyWarning: A value is trying to be set on a copy of a
slice from a DataFrame. Try using .loc[row_indexer,col_indexer] =
value instead
See the caveats in the documentation:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
self[col] = expressions.where(mask, this, that)
Likewise I tried the following code:
for i in ChangedDocumentIDDict.keys():
NDCSPart_df.loc[NDCSPart_df["MainDocID"]==i,:]= NOTES_df.loc[NOTES_df["MainDocID"]==i,:]
with similar warning:
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexing.py:190:
SettingWithCopyWarning: A value is trying to be set on a copy of a
slice from a DataFrame
See the caveats in the documentation:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
self._setitem_with_indexer(indexer, value)
C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:3:
SettingWithCopyWarning: A value is trying to be set on a copy of a
slice from a DataFrame
See the caveats in the documentation:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
This is separate from the ipykernel package so we can avoid doing
imports until
But my concern is with the fact that assignment fails with NaN values populated which should have values of the row against index 78 ofNOTES_df as indicated in the second snapshot.
I am using Python 3.7.3, pandas 0.24.2 and I have tried Python 3.6.6, pandas 0.23.4 with the same results.
My question is:
How am I using .loc incorrectly?
How can I assign the rows of NOTES_df to NDCSPart_df?
This is more like the index of two df after filter with condition is different, so we need adding the .values more info
for i in ChangedDocumentIDDict.keys():
NDCSPart_df.loc[NDCSPart_df["MainDocID"]==i,:]= NOTES_df.loc[NOTES_df["MainDocID"]==i,:].values
I have the following code:
block_table[[compared_attribute]] = block_table[[compared_attribute]].astype(int)
I want to change the datatype of a column. The code is working, but I get a warning from Python: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
self[k1] = value[k2]
I looked into this warning and I was reading it may be creating a copy of the dataframe, instead of just overwriting it, so I tried the following solutions with no luck...
block_table.loc[[compared_attribute]] = block_table[[compared_attribute]].astype(int)
block_table.loc[:,compared_attribute] = block_table[[compared_attribute]].astype(int)
It should be as simple as:
block_table.loc[:,compared_attribute] = block_table[compared_attribute].astype(int)
This is assuming compared attributes is by columns otherwise, switch the colon and compared_attribute in the loc part.
Also quite hard to answer without an example of what the data looks like and what compared_attribute looks like.