This code works for single string (inputx) but I can't get it to work when I replace it with the name of the column in my dataframe. What I want to do is split the string in column DESC where the capitalized words (at beginning of string) is place into column break2 and the remainder of the description is placed in column break3. Any assistance is appreciated. Thanks.
Example:
What I want output to look like (but with the different DESC from each row
Code that works for hardcoded string:
inputx= "STOCK RECORD INQUIRY This is a system that keeps track of the positions, location and ownership of the securities that the broker holds"
pos = re.search("[a-z]", inputx[::1]).start()
Before_df['break1'] = pos
Before_df['break2'] = inputx[:(pos-1)]
Before_df['break3'] = inputx[(pos-1):]
But if I replace with dataframe column, I get error message: TypeError: expected string or bytes-like object
inputx = Before_df['DESC']
pos = re.search("[a-z]", inputx[::1]).start()
Before_df['break1'] = pos
Before_df['break2'] = inputx[:(pos-1)]
Before_df['break3'] = inputx[(pos-1):]
You can use regex in the df.str.split method
df[['result','result2','result3']] = df['yourcol'].str.split("([a-z])", expand= True)
If you absolutely must use re.search (which sounds a little like homework...)
for i in df.index:
df.at[i, 'columnName'] = re.search("[a-z]", df.at[i, 'inputColumn'][::1]).start()
The reason for looping instead of using df.apply() is because dataframes do not like to be changed during an apply
Related
I have the following line
open.loc[(open['Genesis'] == 'Incomplete Name') & (open['Date'] >= last_date),'Result'] = 'Missing information'
'last_date' is a variable equal to a specific date week (2022-25)
'open' is my dataframe
'Result' is a column of my dataframe, as well as 'Date' and 'Genesis'
What I'm trying to do is to transform that line of code into something where instead of specifying the value of 'Incomplete' in the ['Genesis'], I could have something like the 'LIKE' operator in SQL, to get more information, since the ['Genesis'] column has values like:
'Incomplete Name'
'Incomplete color'
'Incomplete material'
And to replace the ['Result'] value with the ['Genesis'] value itself, since I do not want to manually specify every possible outcome of that column.
I've tried something like the following but I'm struggling to make it work:
`for word in open['Genesis']:
if word.startswith('Incomplete') and open['Date'] >= last_date:
open['Result'] = open['Genesis']`
Thanks in advance!
You can use the in operator in python. Basically the expression returns true if the incomplete value is "subset" of the other complete value.
So you can do:
open.loc[('Incomplete Name' in open['Genesis']) & (open['Date'] >= last_date),'Result'] = 'Missing information'
** EDIT **
try this one:
for word in words:
if word in open['Genesis'] and open['Date'] >= last_date:
open['Result'] = open['Genesis']`
So I have a dataframe called reactions_drugs
and I want to create a table called new_r_d where I keep track of how often a see a symptom for a given medication like
Here is the code I have but I am running into errors such as "Unable to coerce to Series, length must be 3 given 0"
new_r_d = pd.DataFrame(columns = ['drugname', 'reaction', 'count']
for i in range(len(reactions_drugs)):
name = reactions_drugs.drugname[i]
drug_rec_act = reactions_drugs.drug_rec_act[i]
for rec in drug_rec_act:
row = new_r_d.loc[(new_r_d['drugname'] == name) & (new_r_d['reaction'] == rec)]
if row == []:
# create new row
new_r_d.append({'drugname': name, 'reaction': rec, 'count': 1})
else:
new_r_d.at[row,'count'] += 1
Assuming the rows in your current reactions (drug_rec_act) column contain one string enclosed in a list, you can convert the values in that column to lists of strings (by splitting each string on the comma delimiter) and then utilize the explode() function and value_counts() to get your desired result:
df['drug_rec_act'] = df['drug_rec_act'].apply(lambda x: x[0].split(','))
df_long = df.explode('drug_rec_act')
result = df_long.groupby('drugname')['drug_rec_act'].value_counts().reset_index(name='count')
I am doing some data mining. I have a database that looks like this (pulling out three lines):
100324822$10032482$1$PS$BENICAR$OLMESARTAN MEDOXOMIL$1$Oral$UNK$$$Y$$$$021286$$$TABLET$
1014687010$10146870$2$SS$BENICAR HCT$HYDROCHLOROTHIAZIDE\OLMESARTAN MEDOXOMIL$1$Oral$1/2 OF 40/25MG TABLET$$$Y$$$$$.5$DF$FILM-COATED TABLET$QD
115700162$11570016$5$C$Olmesartan$OLMESARTAN$1$Unknown$UNK$$$U$U$$$$$$$
My Code looks like this :
with open('DRUG20Q4.txt') as fileDrug20Q4:
drugTupleList20Q4 = [tuple(map(str, i.split('$'))) for i in fileDrug20Q4]
drug20Q4 = []
for entryDrugPrimaryID20Q4 in drugTupleList20Q4:
drug20Q4.append((entryDrugPrimaryID20Q4[0], entryDrugPrimaryID20Q4[3], entryDrugPrimaryID20Q4[5]))
fileDrug20Q4.close()
drugNameDataFrame20Q4 = pd.DataFrame(drug20Q4, columns = ['PrimaryID', 'Role', 'Drug Name']) drugNameDataFrame20Q4 = pd.DataFrame(drugNameDataFrame20Q4.loc[drugNameDataFrame20Q4['Drug Name'] == 'OLMESARTAN'])
Currently the code will pull only entries with the exact name "OLMESARTAN" out, how do I capture all the variations, for instance "OLMESARTAN MEDOXOMIL" etc? I can't simply list all the varieties as there's an infinite amount of variations, so I would need something that captures anything with the term "OLMESARTAN" within it.
Thanks!
You can use str.contains to get what you are looking for.
Here's an example (using some string I found in the documentation):
import pandas as pd
df = pd.DataFrame()
item = 'Return boolean Series or Index based on whether a given pattern or regex is contained within a string of a Series or Index.'
df['test'] = item.split(' ')
df[df['test'].str.contains('de')]
This outputs:
test
4 Index
22 Index.
I'm trying to loop through a list of dataframes and perform operations on them. In the final command I want to rename the dataframe as the original key plus '_rand_test'. I'm getting the error:
SyntaxError: cannot assign to operator
Is there a way to do this?
segments = [main_h, main_m, main_l]
seg_name = ['main_h', 'main_m', 'main_l']
for i in segments:
control = pd.DataFrame(i.groupby('State', group_keys=False).apply(lambda x : x.sample(frac = .1)))
control['segment'] = 'control'
test= i[~i.index.isin(control.index)]
test['segment'] = 'test'
seg_name[i]+'_rand_test' = pd.concat([control,test])
The error is because you are trying to perform addition on the left side of an = sign, which you can never do. If you want to rename the dataframe you could just do it on the next line. I'm unsure of what exactly you're trying to rename based off of the code, but if it's just the corresponding string in the seg_name list then the next line would look like this:
seg_name[segments.index(i)] += 'rand_test'
The reason for the segments.index(i) is because you're looping over the elements in segments, not their indexes, so you need to get the index of the element.
Maybe this will work for you?
Create an empty list befor you run the loop and fill that list with append function. And then you rename all the elements of the new list.
segments = [main_h, main_m, main_l]
seg_name = ['main_h', 'main_m', 'main_l']
new_list= []
for i in segments:
control = pd.DataFrame(i.groupby('State', group_keys=False).apply(lambda x : x.sample(frac = .1)))
control['segment'] = 'control'
test= i[~i.index.isin(control.index)]
test['segment'] = 'test'
new_list.append(df)
new_names_list=[item +'_rand_test' for item in new_list]
I am trying to replace a value inside a string column which is between two specific wording
For example, from this dataframe I want to change
df
seller_name url
Lucas http://sanyo.mapi/s3/e42390aac371?item_title=Branded%20boys%20Clothing&seller_name=102392852&buyer_item=106822419_1056424990
To this
url
http://sanyo.mapi/s3/e42390aac371?item_title=Branded%20boys%20Clothing&seller_name=Lucas&buyer_item=106822419_1056424990
Look in the URL in the seller_name= part I replaced by the real name, I changed the numbers for the real name.
I imagine something like changing from seller_name= to the first and that it see from seller_name.
this is just an example of what i want to do but really i have many of rows in my dataframe and length of the numbers inside the seller name is not always the same
Use apply and replace the string with seller name
Sample df
import pandas as pd
df=pd.DataFrame({'seller_name':['Lucas'],'url':['http://sanyo.mapi/s3/e42390aac371?item_title=Branded%20boys%20Clothing&seller_name=102392852&buyer_item=106822419_1056424990']})
import re
def myfunc(row):
return(re.sub('(seller_name=\d{1,})','seller_name='+row.seller_name,row.url))
df['url']=df.apply(lambda x: myfunc(x),axis=1)
seller_name = 'Lucas'
url = 'http://sanyo.mapi/s3/e42390aac371?item_title=Branded%20boys%20Clothing&seller_name=102392852&buyer_item=106822419_1056424990'
a = url.index('seller_name=')
b = url.index('&', a)
out = url.replace(url[a+12:b],seller_name)
print(out)
Try This one:
This solution doesn't assume the order of your query parameters, or the length of the ID you're replacing. All it assumes is that your query is &-delimited, and that you have the seller_name parameter, present.
split_by_amps = url.split('&')
for i in range(len(split_by_amps)):
if (split_by_amps[i].startswith('seller_name')):
split_by_amps[i] += 'seller_name=' + 'Lucas'
break
result = '&'.join(split_by_amps)
You can use regular expressions to substitute the code for the name:
import pandas as pd
import re
#For example use a dictionary to map codes to names
seller_dic = {102392852:'Lucas'}
for i in range(len(df['url'])):
#very careful with this, if a url doesn't have this structure it will throw
#an error, you may want to handle exceptions
code = re.search(r'seller_name=\d+&',df['url'][i]).group(0)
code = code.replace("seller_name=","")
code = code.replace("&","")
name = 'seller_name=' + seller_dic[code] + '&'
url = re.sub(r'seller_name=\d+&', name, df['url'][i])
df['url'][i] = url