What is the best way to search for matching words inside a string?
Right now I do something like the following:
if re.search('([h][e][l][l][o])',file_name_tmp, re.IGNORECASE):
Which works but its slow as I have probably around 100 different regex statements searching for full words so I'd like to combine several using a | separator or something.
>>> words = ('hello', 'good\-bye', 'red', 'blue')
>>> pattern = re.compile('(' + '|'.join(words) + ')', re.IGNORECASE)
>>> sentence = 'SAY HeLLo TO reD, good-bye to Blue.'
>>> print pattern.findall(sentence)
['HeLLo', 'reD', 'good-bye', 'Blue']
Can you try:
if 'hello' in longtext:
or
if 'HELLO' in longtext.upper():
to match hello/Hello/HELLO.
If you are trying to check 'hello' or a complete word in a string, you could also do
if 'hello' in stringToMatch:
... # Match found , do something
To find various strings, you could also use find all
>>>toMatch = 'e3e3e3eeehellloqweweemeeeeefe'
>>>regex = re.compile("hello|me",re.IGNORECASE)
>>>print regex.findall(toMatch)
>>>[u'me']
>>>toMatch = 'e3e3e3eeehelloqweweemeeeeefe'
>>>print regex.findall(toMatch)
>>>[u'hello', u'me']
>>>toMtach = 'e3e3e3eeeHelLoqweweemeeeeefe'
>>>print regex.findall(toMatch)
>>>[u'HelLo', u'me']
You say you want to search for WORDS. What is your definition of a "word"? If you are looking for "meet", do you really want to match the "meet" in "meeting"? If not, you might like to try something like this:
>>> import re
>>> query = ("meet", "lot")
>>> text = "I'll meet a lot of friends including Charlotte at the town meeting"
>>> regex = r"\b(" + "|".join(query) + r")\b"
>>> re.findall(regex, text, re.IGNORECASE)
['meet', 'lot']
>>>
The \b at each end forces it to match only at word boundaries, using re's definition of "word" -- "isn't" isn't a word, it's two words separated by an apostrophe. If you don't like that, look at the nltk package.
Related
I'm learning about regular expression. I don't know how to combine different regular expression to make a single generic regular expression.
I want to write a single regular expression which works for multiple cases. I know this is can be done with naive approach by using or " | " operator.
I don't like this approach. Can anybody tell me better approach?
You need to compile all your regex functions. Check this example:
import re
re1 = r'\d+\.\d*[L][-]\d*\s[A-Z]*[/]\d*'
re2 = '\d*[/]\d*[A-Z]*\d*\s[A-Z]*\d*[A-Z]*'
re3 = '[A-Z]*\d+[/]\d+[A-Z]\d+'
re4 = '\d+[/]\d+[A-Z]*\d+\s\d+[A-Z]\s[A-Z]*'
sentences = [string1, string2, string3, string4]
for sentence in sentences:
generic_re = re.compile("(%s|%s|%s|%s)" % (re1, re2, re3, re4)).findall(sentence)
To findall with an arbitrary series of REs all you have to do is concatenate the list of matches which each returns:
re_list = [
'\d+\.\d*[L][-]\d*\s[A-Z]*[/]\d*', # re1 in question,
...
'\d+[/]\d+[A-Z]*\d+\s\d+[A-z]\s[A-Z]*', # re4 in question
]
matches = []
for r in re_list:
matches += re.findall( r, string)
For efficiency it would be better to use a list of compiled REs.
Alternatively you could join the element RE strings using
generic_re = re.compile( '|'.join( re_list) )
I see lots of people are using pipes, but that seems to only match the first instance. If you want to match all, then try using lookaheads.
Example:
>>> fruit_string = "10a11p"
>>> fruit_regex = r'(?=.*?(?P<pears>\d+)p)(?=.*?(?P<apples>\d+)a)'
>>> re.match(fruit_regex, fruit_string).groupdict()
{'apples': '10', 'pears': '11'}
>>> re.match(fruit_regex, fruit_string).group(0)
'10a,11p'
>>> re.match(fruit_regex, fruit_string).group(1)
'11'
(?= ...) is a look ahead:
Matches if ... matches next, but doesn’t consume any of the string. This is called a lookahead assertion. For example, Isaac (?=Asimov) will match 'Isaac ' only if it’s followed by 'Asimov'.
.*?(?P<pears>\d+)p
find a number followed a p anywhere in the string and name the number "pears"
You might not need to compile both regex patterns. Here is a way, let's see if it works for you.
>>> import re
>>> text = 'aaabaaaabbb'
>>> A = 'aaa'
>>> B = 'bbb'
>>> re.findall(A+B, text)
['aaabbb']
>>>
further read read_doc
If you need to squash multiple regex patterns together the result can be annoying to parse--unless you use P<?> and .groupdict() but doing that can be pretty verbose and hacky. If you only need a couple matches then doing something like the following could be mostly safe:
bucket_name, blob_path = tuple(item for item in matches.groups() if item is not None)
I am writing a script that introduces misspellings into sentence. I am using python re module to replace the original word with the misspelling. The script looks like this:
# replacing original word by error
pattern = re.compile(r'%s' % original_word)
replace_by = r'\1' + err
modified_sentence = re.sub(pattern, replace_by, sentence, count=1)
But the problem is this will replace even if original_word was part of another word for example:
If i had
original_word = 'in'
err = 'il'
sentence = 'eating food in'
it would replace the occurrence of 'in' in eating like:
> 'eatilg food in'
I was checking in the re documentation but it doesn't give any example on how to include regex options, for example:
If my pattern is:
regex_pattern = '\b%s\b' % original_word
this would solve the problem as \b represents 'word boundary'. But it doesn't seem to work.
I tried to find to find a work around it by doing:
pattern = re.compile(r'([^\w])%s' % original_word)
but that does not work. For example :
original_word = 'to'
err = 'vo'
sentence = 'I will go tomorrow to the'
it replaces it to:
> I will go vomorrow to the
Thank you, any help appreciated
See here for an example of word boundaries in python re module. It looks like you were close just need to put it all together. The following script gives you the output you want...
import re
original_word = 'to'
err = 'vo'
sentence = 'I will go tomorrow to the'
pattern = re.compile(r'\b%s\b' % re.escape(original_word))
modified_sentence = re.sub(pattern, err, sentence, count=1)
print modified_sentence
Output --> I will go tomorrow vo the
I have many fill-in-the-blank sentences in strings,
e.g. "6d) We took no [pains] to hide it ."
How can I efficiently parse this string (in Python) to be
"We took no to hide it"?
I also would like to be able to store the word in brackets (e.g. "pains") in a list for use later. I think the regex module could be better than Python string operations like split().
This will give you all the words inside the brackets.
import re
s="6d) We took no [pains] to hide it ."
matches = re.findall('\[(.*?)\]', s)
Then you can run this to remove all bracketed words.
re.sub('\[(.*?)\]', '', s)
just for fun (to do the gather and substitution in one iteration)
matches = []
def subber(m):
matches.append(m.groups()[0])
return ""
new_text = re.sub("\[(.*?)\]",subber,s)
print new_text
print matches
import re
s = 'this is [test] string'
m = re.search(r"\[([A-Za-z0-9_]+)\]", s)
print m.group(1)
Output
'test'
For your example you could use this regex:
(.*\))(.+)\[(.+)\](.+)
You will get four groups that you can use to create your resulting string and save the 3. group for later use:
6d)
We took no
pains
to hide it .
I used .+ here because I don't know if your strings always look like your example. You can change the .+ to alphanumeric or sth. more special to your case.
import re
s = '6d) We took no [pains] to hide it .'
m = re.search(r"(.*\))(.+)\[(.+)\](.+)", s)
print(m.group(2) + m.group(4)) # "We took no to hide it ."
print(m.group(3)) # pains
import re
m = re.search(".*\) (.*)\[.*\] (.*)","6d) We took no [pains] to hide it .")
if m:
g = m.groups()
print g[0] + g[1]
Output :
We took no to hide it .
I am trying to write code that will take a string and remove specific data from it. I know that the data will look like the line below, and I only need the data within the " " marks, not the marks themselves.
inputString = 'type="NN" span="123..145" confidence="1.0" '
Is there a way to take a Substring of a string within two characters to know the start and stop points?
You can extract all the text between pairs of " characters using regular expressions:
import re
inputString='type="NN" span="123..145" confidence="1.0" '
pat=re.compile('"([^"]*)"')
while True:
mat=pat.search(inputString)
if mat is None:
break
strings.append(mat.group(1))
inputString=inputString[mat.end():]
print strings
or, easier:
import re
inputString='type="NN" span="123..145" confidence="1.0" '
strings=re.findall('"([^"]*)"', inputString)
print strings
Output for both versions:
['NN', '123..145', '1.0']
fields = inputString.split('"')
print fields[1], fields[3], fields[5]
You could split the string at each space to get a list of 'key="value"' substrings and then use regular expressions to parse the substrings.
Using your input string:
>>> input_string = 'type="NN" span="123..145" confidence="1.0" '
>>> input_string_split = input_string.split()
>>> print input_string_split
[ 'type="NN"', 'span="123..145"', 'confidence="1.0"' ]
Then use regular expressions:
>>> import re
>>> pattern = r'"([^"]+)"'
>>> for substring in input_string_split:
match_obj = search(pattern, substring)
print match_obj.group(1)
NN
123..145
1.0
The regular expression '"([^"]+)"' matches anything within quotation marks (provided there is at least one character). The round brackets indicate the bit of the regular expression that you are interested in.
This question already has answers here:
How to get a string after a specific substring?
(9 answers)
Closed 2 months ago.
I want to find words that appear after a keyword (specified and searched by me) and print out the result. I know that i am suppose to use regex to do it, and i tried it out too, like this:
import re
s = "hi my name is ryan, and i am new to python and would like to learn more"
m = re.search("^name: (\w+)", s)
print m.groups()
The output is just:
"is"
But I want to get all the words and punctuations that comes after the word "name".
Instead of using regexes you could just (for example) separate your string with str.partition(separator) like this:
mystring = "hi my name is ryan, and i am new to python and would like to learn more"
keyword = 'name'
before_keyword, keyword, after_keyword = mystring.partition(keyword)
>>> before_keyword
'hi my '
>>> keyword
'name'
>>> after_keyword
' is ryan, and i am new to python and would like to learn more'
You have to deal with the needless whitespaces separately, though.
Your example will not work, but as I understand the idea:
regexp = re.compile("name(.*)$")
print regexp.search(s).group(1)
# prints " is ryan, and i am new to python and would like to learn more"
This will print all after "name" and till end of the line.
An other alternative...
import re
m = re.search('(?<=name)(.*)', s)
print m.groups()
import re
s = "hi my name is ryan, and i am new to python and would like to learn more"
m = re.search("^name: (\w+)", s)
print m.group(1)
Instead of "^name: (\w+)" use:
"^name:(.*)"
What you have used regarding your output:
re.search("name (\w+)", s)
What you have to use (match all):
re.search("name (.*)", s)
You could simply do
s = "hi my name is ryan, and i am new to python and would like to learn more"
s.split('name')
This will split your string and return a list like this ['hi my', 'is ryan, and i am new to python and would like to learn more']
depending on what you want to do this may help or not.
This will work out for u : work name\s\w+\s(\w+)
>>> s = 'hi my name is ryan, and i am new to python and would like to learn more'
>>> m = re.search('name\s\w+\s(\w+)',s)
>>> m.group(0)
'name is ryan'
>>>> m.group(1)
'ryan'
Without using regex, you can
strip punctuation (consider making everything single case, including search term)
split your text into individual words
find index of searched word
get word from array (index + 1 for word after, index - 1 for word before )
Code snippet:
import string
s = 'hi my name is ryan, and i am new to python and would like to learn more'
t = 'name'
i = s.translate(string.maketrans("",""), string.punctuation).split().index(t)
print s.split()[i+1]
>> is
For multiple occurences, you need to save multiple indices:
import string
s = 'hi my NAME is ryan, and i am new to NAME python and would like to learn more'
t = 'NAME'
il = [i for i, x in enumerate(s.translate(string.maketrans("",""), string.punctuation).split()) if x == t]
print [s.split()[x+1] for x in il]
>> ['is', 'python']