Having issues with building a find and replace tool in python. Goal is to search a column in an excel file for a string and swap out every letter of the string based on the key value pair of the dictionary, then write the entire new string back to the same cell. So "ABC" should convert to "BCD". I have to find and replace any occurrence of individual characters.
The below code runs without debugging, but newvalue never creates and I don't know why. No issues writing data to the cell if newvalue gets created.
input: df = pd.DataFrame({'Code1': ['ABC1', 'B5CD', 'C3DE']})
expected output: df = pd.DataFrame({'Code1': ['BCD1', 'C5DE', 'D3EF']})
mycolumns = ["Col1", "Col2"]
mydictionary = {'A': 'B', 'B': 'C', 'C': 'D'}
for x in mycolumns:
# 1. If the mycolumn value exists in the headerlist of the file
if x in headerlist:
# 2. Get column coordinate
col = df.columns.get_loc(x) + 1
# 3. iterate through the rows underneath that header
for ind in df.index:
# 4. log the row coordinate
rangerow = ind + 2
# 5. get the original value of that coordinate
oldval = df[x][ind]
for count, y in enumerate(oldval):
# 6. generate replacement value
newval = df.replace({y: mydictionary}, inplace=True, regex=True, value=None)
print("old: " + str(oldval) + " new: " + str(newval))
# 7. update the cell
ws.cell(row=rangerow, column=col).value = newval
else:
print("not in the string")
else:
# print(df)
print("column doesn't exist in workbook, moving on")
else:
print("done")
wb.save(filepath)
wb.close()
I know there's something going on with enumerate and I'm probably not stitching the string back together after I do replacements? Or maybe a dictionary is the wrong solution to what I am trying to do, the key:value pair is what led me to use it. I have a little programming background but ery little with python. Appreciate any help.
newvalue never creates and I don't know why.
DataFrame.replace with inplace=True will return None.
>>> df = pd.DataFrame({'Code1': ['ABC1', 'B5CD', 'C3DE']})
>>> df = df.replace('ABC1','999')
>>> df
Code1
0 999
1 B5CD
2 C3DE
>>> q = df.replace('999','zzz', inplace=True)
>>> print(q)
None
>>> df
Code1
0 zzz
1 B5CD
2 C3DE
>>>
An alternative could b to use str.translate on the column (using its str attribute) to encode the entire Series
>>> df = pd.DataFrame({'Code1': ['ABC1', 'B5CD', 'C3DE']})
>>> mydictionary = {'A': 'B', 'B': 'C', 'C': 'D'}
>>> table = str.maketrans('ABC','BCD')
>>> df
Code1
0 ABC1
1 B5CD
2 C3DE
>>> df.Code1.str.translate(table)
0 BCD1
1 C5DD
2 D3DE
Name: Code1, dtype: object
>>>
Related
I have a dataframe generated from unformatted csv. So I need format some datas (e.g there is some strings as 12.323,03 for float format and i'm trying to convert them 12323.03 for convert string to float in python)
I'm trying to do it as:
for column in data:
if(data[column].name != 'blabla' and data[column].name != 'otherblabla'):
for row_value in data[column]:
if type(row_value) == str:
float_format = row_value.replace('.','').replace(',','.')
row_value = row_value.replace(row_value, float_format)
float format: converts string "12.323,03" to "12323.03".
But row values are not affected. What am i missing?
To affect the new value, you must locate it with
df.loc[row_index,column_name] = row_value
To do so, try an enumerate.
for row_index, row_value in enumerate(data_column):
Here an example to understand it:
df = pd.DataFrame({'A':[1,2,3,4],'B':[5,6,7,8]})
print('Before change')
print(df)
for i,j in enumerate(df['B']):
if j == 6:
df.loc[i,'B'] = 4
print('Afetr Change')
print(df)
The variable row_value is a single value of a row/column pair in the original df which does not reference back that df position. As the other answer has pointed out with your approach you need to locate the value in order to change the df.
Additionally, I want to mention that the second replace of row_value could be substituted with simply row_value = float_format. Also, I share with you an approach using apply which I consider cleaner and you might find useful:
df = pd.DataFrame(
{
'c1': ['100,12', 1.230, '30.000,4'],
'c2': ['5.367,46', '10', 7.3],
'c3': ['a', 'b', 'c']
}
)
cols = ['c1', 'c2']
for col in cols:
df[col] = df[col].apply(
lambda x: float(x.replace('.','').replace(',','.')) if type(x) == str else x
)
This results in:
c1 c2 c3
0 100.12 5367.46 a
1 1.23 10.00 b
2 30000.40 7.30 c
c1 float64
c2 float64
c3 object
dtype: object
How to filter out a row value in a column B, if another column C has a specific text say "ABC" ? in this case "google.com" would be filtered out.
A B C D
0 True facebook.com kxy 19999
1 True google.com ABC 21212
2 False yahoo.com PoP 3213231
Everytime there is "ABC" in Col C, row value from col B should be appended in a list.
pseudocode:
dataset = pd.read_csv('xyz.csv')
path = []
for value in dataset.C:
if dataset['C'] == 'abc':
#append path with row value of Col B
else:
#not append path
path = dataset.loc[dataset.C == 'ABC', 'B'].tolist()
will give you the desired list in one go.
as an alternative you can use where and list:
path = list(data.B.where(data.C == 'ABC').dropna())
print(path)
# ['google.com']
inds will be a pandas series with boolean values, indicating whether a row's value in column 'C' is equal to 'ABC'.
Once we know that, we can subset dataset and take the values of column 'B':
inds = dataset['C'] == 'ABC'
list(dataset.loc[inds, 'B'])
Direct asnwer:
filtered_values = dataset.loc[dataset["C"]=='ABC']['B'].tolist()
For understanding purposes:
First get the rows where C="ABC"
filtered_rows = dataset.loc[dataset["C"]=='ABC']
filtered_rows
Output:
A B C D
1 True google.com ABC 21212
From these rows get the values of only column B and convert this Series into a list with .tolist() function
filtered_values = filtered_rows["B"].tolist()
filtered_values
Output:
['google.com']
I have created a dataframe which contains the columns Name and Mains.
data = [['Anshu', '8321-1328-11'], ['Hero', '83211-1128-11'], ['Naman', '65432-8765-4']]
df = pd.DataFrame(data, columns = ['Name', 'Mains'])
I want to update the Mains column into a new column with an updated value i.e. df['new_mains'] with the following condition: if the number is seprated by 4-4-2, then should be added with 0 and updated separated number will be 5-4-2? is it possible to do so in pandas?
Pretty sure it could be done. For example,
def my_func(strn):
a,b,c = strn.split('-')
new_a = '0'+a if len(a)==4 else a
new_b = '0'+b if len(b)==3 else b
new_c = '9'+c if len(c)==1 else c
return '-'.join([new_a,new_b,new_c])
And then,
df['New_Mains'] = df['Mains'].apply(my_func)
Note: Going off the assumption 'a' is either of length 4 or 5. If 'a', 'b', 'c' are of any other length then, you can also do something like (works for current scenario as well)
new_a = '0'*5-len(a) + a
new_b = '0'*4-len(b) + b
new_c = '9'*2-len(c) + c
More on str.split here. Basically, in your case the string reads something like "99999-9999-99" with "-" as a separator. So,
"99999-9999-99".split('-') #would return
['99999', '9999', '99']
where a = '99999', b = '9999', c = '99'.
new_a, new_b, new_c are variables to hold new values of a, b and c after checking for the conditional statements. Finally join the strings new_a, new_b, new_c to look like original strings from 'Mains' column. More on str.join
I am querying a single value from my data frame which seems to be 'dtype: object'. I simply want to print the value as it is with out printing the index or other information as well. How do I do this?
col_names = ['Host', 'Port']
df = pd.DataFrame(columns=col_names)
df.loc[len(df)] = ['a', 'b']
t = df[df['Host'] == 'a']['Port']
print(t)
OUTPUT:
EXPECTED OUTPUT:
b
If you can guarantee only one result is returned, use loc and call item:
>>> df.loc[df['Host'] == 'a', 'Port'].item()
'b'
Or, similarly,
>>> df.loc[df['Host'] == 'a', 'Port'].values[0]
'b'
...to get the first value (similarly, .values[1] for the second). Which is better than df.loc[df['Host'] == 'a', 'Port'][0] because, if your DataFrame looks like this,
Host Port
1 a b
Then "KeyError: 0" will be thrown—
df.loc[df['Host'] == 'a', 'Port'][0]
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
Alternatively, use at:
>>> df.at[df['Host'].eq('a').idxmax(), 'Port']
'b'
The drawback is that if 'a' doesn't exist, idxmax will return the first index (and return an incorrect result).
t = df['Host'].values[0]
will give you the first value. If you need a string, just do:
t = str(df['Host'].values[0])
As mentioned in my comment, using [1] should work afterwards, to pull the variable you're looking for.
t = df[df['Host'] == 'a']['Port'][1]
it should work simply..
>>> df
Host Port
0 a b
>>> df[df['Host'] == 'a']['Port'][0] # will choose the first index simply which is 'b'
'b'
OR, use with print which will strip off the surrounded single ticks.
>>> print(df[df['Host'] == 'a']['Port'][0])
b
This will easier because you have just choose the desired Index even if you have Multiple values across Port columns
Example:
>>> df
Host Port
0 a b
1 c c
Looking for distinct a & c based on Index:
>>> df[df['Host'] == 'a']['Port'][0]
'b'
>>> df[df['Host'] == 'c']['Port'][1]
'c'
In my pandas DataFrame I want to add a new column (NewCol), based on some conditions that follow from data of another column (OldCol).
To be more specific, my column OldCol contains three types of strings:
BB_sometext
sometext1
sometext 1
I want to differentiate between these three types of strings. Right now, I did this using the following code:
df['NewCol'] = pd.Series()
for i in range(0, len(df)):
if str(df.loc[i, 'OldCol']).split('_')[0] == "BB":
df.loc[i, 'NewCol'] = "A"
elif len(str(df.loc[i, 'OldCol']).split(' ')) == 1:
df.loc[i, 'NewCol'] = "B"
else:
df.loc[i, 'NewCol'] = "C"
Even though this code seems to work, I'm sure there is a better way to do something like this, as this seems very inefficient. Does anyone know a better way to do this? Thanks in advance.
In general, you need something like the following formulation:
>>> df.loc[boolean_test, 'NewCol'] = desired_result
Or, for multiple conditions (Note the parentheses around each condition, and the rather unpythonic & instead of and):
>>> df.loc[(boolean_test1) & (boolean_test2), 'NewCol'] = desired_result
Example
Let's start with an example Data.Frame:
>>> df = pd.DataFrame(dict(OldCol=['sometext1', 'sometext 1', 'BB_ccc', 'sometext1']))
Then you'd do:
>>> df.loc[df['OldCol'].str.split('_').str[0] == 'BB', 'NewCol'] = "A"
To set all BB_ columns to A. You could even (optionally, for readability) separate out the boolean condition onto its own line:
>>> oldcol_starts_BB = df['OldCol'].str.split('_').str[0] == 'BB'
>>> df.loc[oldcol_starts_BB, 'NewCol'] = "A"
I like this method become it means the reader doesn't have to work out the logic hidden within the split('_').str[0] part.
Then, to set all columns with no space, which are still not set (i.e. where isnull is true):
>>> oldcol_has_no_space = df['OldCol'].str.find(' ') < 0
>>> newcol_is_null = df['NewCol'].isnull()
>>> df.loc[(oldcol_has_no_space) & (newcol_is_null), 'NewCol'] = 'C'
Then finally, set all remaining values of NewCol to B:
>>> df.loc[df['NewCol'].isnull(), 'NewCol'] = 'B'
>>> df
OldCol NewCol
0 sometext1 C
1 sometext 1 B
2 BB_ccc A
3 sometext1 C