Related
For an exercise, I have to create a simple profanity filter in order to learn about classes.
The filter gets initialized with a list of offensive keywords and a replacement template. Every occurrence of any of these words should be replaced with a string that is generated from the template. If the word size is shorter than the template, a substring should be used that starts from the beginning, for longer sizes, the template should be repeated as often as necessary.
The following are my results so far, with an example.
class ProfanityFilter:
def __init__(self, keywords, template):
self.__keywords = sorted(keywords, key=len, reverse=True)
self.__template = template
def filter(self, msg):
def __replace_letters__(old_word, replace_str):
replaced_word = ""
old_index = 0
replace_index = 0
while old_index <= len(old_word):
if replace_index == len(replace_str):
replace_index = 0
else:
replaced_word += replace_str[replace_index]
replace_index += 1
old_index += 1
return replaced_word
for keyword in self.__keywords:
idx = 0
while idx < len(msg):
index_l = msg.lower().find(keyword.lower(), idx)
if index_l == -1:
break
msg = msg[:index_l] + __replace_letters__(keyword, self.__template) + msg[index_l + len(keyword):]
idx = index_l + len(keyword)
return msg
f = ProfanityFilter(["duck", "shot", "batch", "mastard"], "?#$")
offensive_msg = "this mastard shot my duck"
clean_msg = f.filter(offensive_msg)
print(clean_msg) # should be: "this ?#$?#$? ?#$? my ?#$?"
The example should print:
this ?#$?#$? ?#$? my ?#$?
But it prints:
this ?#$?#$ ?#$? my ?#$?
For some reason it replaces the word "mastard" with 6 symbols instead of 7 (one for each letter). It works for the other keywords, why not for this one?
Also if you see anything else that seems off, feel free to tell me. Do keep in mind tho that I am a beginner and my "toolbox" is quite small atm.
Your problem is in the index logic. You have two errors
Each time you reach the end of the replacement string, you skip a letter in the profanity:
while old_index <= len(old_word):
if replace_index == len(replace_str):
replace_index = 0
# You don't replace a letter; you just reset the new index, but ...
else:
replaced_word += replace_str[replace_index]
replace_index += 1
old_index += 1 # ... but you still advance the old index.
The reason you didn't notice this is that you have a second bug: you run your old_index from 0 through len(old_word), which is one more character than you started with. For the canonical four-letter word (or words of 5 or 6 characters), the two errors cancel each other. You didn't see this because you didn't test enough. For instance, using:
f = ProfanityFilter(["StackOverflow", "PC"], "?#$")
offensive_msg = "StackOverflow on PC rulez!"
clean_msg = f.filter(offensive_msg)
Output:
?#$?#$?#$?# on ?#$ rulez!
The input words are 13 and 2 letters; the replacements are 11 and 3.
Fix those two errors: make old_index stay in bounds, and increment it only when you make a replacement.
while old_index < len(old_word):
if replace_index == len(replace_str):
replace_index = 0
else:
replaced_word += replace_str[replace_index]
replace_index += 1
old_index += 1
Future improvements:
Refactor this into a for loop.
Don't reset your replace_index; in fact, get rid of it. Simply use old_index % len(replace_str).
I'd do this with a regular expression instead, since re.sub() has a handy API for dynamic replacements:
import re
class ProfanityFilter:
def __init__(self, keywords, template):
# Build a regular expression that will match all of the profane words
self.keyword_re = re.compile("|".join(re.escape(keyword) for keyword in keywords), re.I)
self.template = template
def _generate_replacement(self, word):
l = len(word)
# Figure out how many times to repeat the template
repeats = (l // len(self.template)) + 1
# Since we may end up with a string longer than the original,
# slice to the correct length.
return (self.template * repeats)[:l]
def filter(self, msg):
# Replace all occurrences of the regular expression with
# a dynamically computed replacement value.
return self.keyword_re.sub(
lambda m: self._generate_replacement(m.group(0)),
msg,
)
f = ProfanityFilter(["duck", "shot", "batch", "mastard"], "?#$")
offensive_msg = "this mastard shot my duck"
print(f.filter(offensive_msg))
Couldn't make a one-liner, but here's a terrible implementation anway. Don't do what VoNWooDSoN does:
def replace(msg, keywords=["duck", "shot", "batch", "mastard"], template="?#$"):
for keyword in keywords * len(msg)):
msg = (template*len(keyword))[:len(keyword)].join([msg[:msg.find(keyword)], msg[msg.find(keyword)+len(keyword):]]) if msg.find(keyword) > 0 else msg
return msg
offensive_msg = "this mastard shot my duck"
clean_msg = replace(offensive_msg)
print(clean_msg) # should be: "this ?#$?#$? ?#$? my ?#$?"
print(clean_msg=="this ?#$?#$? ?#$? my ?#$?")
edit
So, I guess that 3.8 has assignment expressions... So, but this'd be the one liner then (probably).
print ((lambda msg: [msg := (("?#$"*len(keyword))[:len(keyword)].join([msg[:msg.find(keyword)], msg[msg.find(keyword)+len(keyword):]]) if msg.find(keyword) > 0 else msg) for keyword in ["duck", "shot", "batch", "mastard"]])("this mastard shot my duck")[-1])
def interleave(s1,s2): #This function interleaves s1,s2 together
guess = 0
total = 0
while (guess < len(s1)) and (guess < len(s2)):
x = s1[guess]
y = s2[guess]
m = x + y
print ((m),end ="")
guess += 1
if (len(s1) == len(s2)):
return ("")
elif(len(s1) > len(s2)):
return (s1[guess:])
elif(len(s2) > len(s1)):
return (s2[guess:])
print (interleave("Smlksgeneg n a!", "a ie re gsadhm"))
For some reason, my test function gives an assertion error eventhough the output is the same as the code below.
Eg - "Smlksgeneg n a!", "a ie re gsadhm" returns "Sam likes green eggs and ham!"
but an assertion error still comes out
def testInterleave():
print("Testing interleave()...", end="")
assert(interleave("abcdefg", "abcdefg")) == ("aabbccddeeffgg")
assert(interleave("abcde", "abcdefgh") == "aabbccddeefgh")
assert(interleave("abcdefgh","abcde") == "aabbccddeefgh")
assert(interleave("Smlksgeneg n a!", "a ie re gsadhm") ==
"Sam likes green eggs and ham!")
assert(interleave("","") == "")
print("Passed!")
testInterleave()
You are confusing what is printed by interleave() from what is returned by it. The assert is testing the returned value. For example, when s1 and s2 are the same length, your code prints the interleave (on the print((m),end="") line) but returns an empty string (in the line return ("")
If you want interleave to return the interleaved string, you need to collect the x and y variables (not very well named if they are always holding characters) into a single string and return that.
The problem is that your function just prints the interleaved portion of the resulting string, it doesn't return it, it only returns the tail of the longer string.
Here's a repaired and simplified version of your code. You don't need to do those if... elif tests. Also, your code has a lot of superfluous parentheses (and one misplaced parenthesis), which I've removed.
def interleave(s1, s2):
''' Interleave strings s1 and s2 '''
guess = 0
result = ""
while (guess < len(s1)) and (guess < len(s2)):
x = s1[guess]
y = s2[guess]
result += x + y
guess += 1
return result + s1[guess:] + s2[guess:]
def testInterleave():
print("Testing interleave()...", end="")
assert interleave("abcdefg", "abcdefg") == "aabbccddeeffgg"
assert interleave("abcde", "abcdefgh") == "aabbccddeefgh"
assert interleave("abcdefgh","abcde") == "aabbccddeefgh"
assert (interleave("Smlksgeneg n a!", "a ie re gsadhm")
== "Sam likes green eggs and ham!")
assert interleave("", "") == ""
print("Passed!")
print(interleave("Smlksgeneg n a!", "a ie re gsadhm"))
testInterleave()
output
Sam likes green eggs and ham!
Testing interleave()...Passed!
Here's a slightly improved version of interleave. It uses a list to store the result, rather than using repeated string concatenation. Using lists to build string like this is a common Python practice because it's more efficient than repeated string concatenation using + or +=; OTOH,+ and += on strings have been optimised so that they're fairly efficient for short strings (up to 1000 chars or so).
def interleave(s1, s2):
result = []
i = 0
for i, t in enumerate(zip(s1, s2)):
result.extend(t)
i += 1
result.extend(s1[i:] + s2[i:])
return ''.join(result)
That i = 0 is necessary in case either s1 or s2 are empty strings. When that happens the for loop isn't entered and so i won't get assigned a value.
Finally, here's a compact version using a list comprehension and the standard itertools.zip_longest function.
def interleave(s1, s2):
return ''.join([u+v for u,v in zip_longest(s1, s2, fillvalue='')])
t = 8
string = "1 2 3 4 3 3 2 1"
string.replace(" ","")
string2 = [x for x in string]
print string2
for n in range(t-1):
string2.remove(' ')
print string2
def remover(ca):
newca = []
print len(ca)
if len(ca) == 1:
return ca
else:
for i in ca:
newca.append(int(i) - int(min(ca)))
for x in newca:
if x == 0:
newca.remove(0)
print newca
return remover(newca)
print (remover(string2))
It's supposed to be a program that takes in a list of numbers, and for every number in the list it subtracts from it, the min(list). It works fine for the first few iterations but not towards the end. I've added print statements here and there to help out.
EDIT:
t = 8
string = "1 2 3 4 3 3 2 1"
string = string.replace(" ","")
string2 = [x for x in string]
print len(string2)
def remover(ca):
newca = []
if len(ca) == 1: return()
else:
for i in ca:
newca.append(int(i) - int(min(ca)))
while 0 in newca:
newca.remove(0)
print len(newca)
return remover(newca)
print (remover(string2))
for x in newca:
if x == 0:
newca.remove(0)
Iterating over a list and removing things from it at the same time can lead to strange and unexpected behvaior. Try using a while loop instead.
while 0 in newca:
newca.remove(0)
Or a list comprehension:
newca = [item for item in newca if item != 0]
Or create yet another temporary list:
newnewca = []
for x in newca:
if x != 0:
newnewca.append(x)
print newnewca
return remover(newnewca)
(Not a real answer, JFYI:)
Your program can be waaay shorter if you decompose it into proper parts.
def aboveMin(items):
min_value = min(items) # only calculate it once
return differenceWith(min_value, items)
def differenceWith(min_value, items):
result = []
for value in items:
result.append(value - min_value)
return result
The above pattern can, as usual, be replaced with a comprehension:
def differenceWith(min_value, items):
return [value - min_value for value in items]
Try it:
>>> print aboveMin([1, 2, 3, 4, 5])
[0, 1, 2, 3, 4]
Note how no item is ever removed, and that data are generally not mutated at all. This approach helps reason about programs a lot; try it.
So IF I've understood the description of what you expect,
I believe the script below would result in something closer to your goal.
Logic:
split will return an array composed of each "number" provided to raw_input, while even if you used the output of replace, you'd end up with a very long number (you took out the spaces that separated each number from one another), and your actual split of string splits it in single digits number, which does not match your described intent
you should test that each input provided is an integer
as you already do a print in your function, no need for it to return anything
avoid adding zeros to your new array, just test first
string = raw_input()
array = string.split()
intarray = []
for x in array:
try:
intarray.append(int(x))
except:
pass
def remover(arrayofint):
newarray = []
minimum = min(arrayofint)
for i in array:
if i > minimum:
newarray.append(i - minimum)
if len(newarray) > 0:
print newarray
remover(newarray)
remover(intarray)
I'd like to compare 2 strings and keep the matched, splitting off where the comparison fails.
So if I have 2 strings:
string1 = "apples"
string2 = "appleses"
answer = "apples"
Another example, as the string could have more than one word:
string1 = "apple pie available"
string2 = "apple pies"
answer = "apple pie"
I'm sure there is a simple Python way of doing this but I can't work it out, any help and explanation appreciated.
For completeness, difflib in the standard-library provides loads of sequence-comparison utilities. For instance find_longest_match which finds the longest common substring when used on strings. Example use:
from difflib import SequenceMatcher
string1 = "apple pie available"
string2 = "come have some apple pies"
match = SequenceMatcher(None, string1, string2).find_longest_match()
print(match) # -> Match(a=0, b=15, size=9)
print(string1[match.a:match.a + match.size]) # -> apple pie
print(string2[match.b:match.b + match.size]) # -> apple pie
If you're using a version older than 3.9, you'need to call find_longest_match() with the following arguments:
SequenceMatcher(None, string1, string2).find_longest_match(0, len(string1), 0, len(string2))
One might also consider os.path.commonprefix that works on characters and thus can be used for any strings.
import os
common = os.path.commonprefix(['apple pie available', 'apple pies'])
assert common == 'apple pie'
As the function name indicates, this only considers the common prefix of two strings.
def common_start(sa, sb):
""" returns the longest common substring from the beginning of sa and sb """
def _iter():
for a, b in zip(sa, sb):
if a == b:
yield a
else:
return
return ''.join(_iter())
>>> common_start("apple pie available", "apple pies")
'apple pie'
Or a slightly stranger way:
def stop_iter():
"""An easy way to break out of a generator"""
raise StopIteration
def common_start(sa, sb):
return ''.join(a if a == b else stop_iter() for a, b in zip(sa, sb))
Which might be more readable as
def terminating(cond):
"""An easy way to break out of a generator"""
if cond:
return True
raise StopIteration
def common_start(sa, sb):
return ''.join(a for a, b in zip(sa, sb) if terminating(a == b))
Its called Longest Common Substring problem. Here I present a simple, easy to understand but inefficient solution. It will take a long time to produce correct output for large strings, as the complexity of this algorithm is O(N^2).
def longestSubstringFinder(string1, string2):
answer = ""
len1, len2 = len(string1), len(string2)
for i in range(len1):
match = ""
for j in range(len2):
if (i + j < len1 and string1[i + j] == string2[j]):
match += string2[j]
else:
if (len(match) > len(answer)): answer = match
match = ""
return answer
print(longestSubstringFinder("apple pie available", "apple pies"))
print(longestSubstringFinder("apples", "appleses"))
print(longestSubstringFinder("bapples", "cappleses"))
Output
apple pie
apples
apples
Fix bugs with the first's answer:
def longestSubstringFinder(string1, string2):
answer = ""
len1, len2 = len(string1), len(string2)
for i in range(len1):
for j in range(len2):
lcs_temp = 0
match = ''
while ((i+lcs_temp < len1) and (j+lcs_temp<len2) and string1[i+lcs_temp] == string2[j+lcs_temp]):
match += string2[j+lcs_temp]
lcs_temp += 1
if len(match) > len(answer):
answer = match
return answer
print(longestSubstringFinder("dd apple pie available", "apple pies"))
print(longestSubstringFinder("cov_basic_as_cov_x_gt_y_rna_genes_w1000000", "cov_rna15pcs_as_cov_x_gt_y_rna_genes_w1000000")
print(longestSubstringFinder("bapples", "cappleses"))
print(longestSubstringFinder("apples", "apples"))
The same as Evo's, but with arbitrary number of strings to compare:
def common_start(*strings):
""" Returns the longest common substring
from the beginning of the `strings`
"""
def _iter():
for z in zip(*strings):
if z.count(z[0]) == len(z): # check all elements in `z` are the same
yield z[0]
else:
return
return ''.join(_iter())
The fastest way I've found is to use suffix_trees package:
from suffix_trees import STree
a = ["xxxabcxxx", "adsaabc"]
st = STree.STree(a)
print(st.lcs()) # "abc"
This script requests you the minimum common substring length and gives all common substrings in two strings. Also, it eliminates shorter substrings that longer substrings include already.
def common_substrings(str1,str2):
len1,len2=len(str1),len(str2)
if len1 > len2:
str1,str2=str2,str1
len1,len2=len2,len1
#short string=str1 and long string=str2
min_com = int(input('Please enter the minumum common substring length:'))
cs_array=[]
for i in range(len1,min_com-1,-1):
for k in range(len1-i+1):
if (str1[k:i+k] in str2):
flag=1
for m in range(len(cs_array)):
if str1[k:i+k] in cs_array[m]:
#print(str1[k:i+k])
flag=0
break
if flag==1:
cs_array.append(str1[k:i+k])
if len(cs_array):
print(cs_array)
else:
print('There is no any common substring according to the parametres given')
common_substrings('ciguliuana','ciguana')
common_substrings('apples','appleses')
common_substrings('apple pie available','apple pies')
Try:
import itertools as it
''.join(el[0] for el in it.takewhile(lambda t: t[0] == t[1], zip(string1, string2)))
It does the comparison from the beginning of both strings.
def matchingString(x,y):
match=''
for i in range(0,len(x)):
for j in range(0,len(y)):
k=1
# now applying while condition untill we find a substring match and length of substring is less than length of x and y
while (i+k <= len(x) and j+k <= len(y) and x[i:i+k]==y[j:j+k]):
if len(match) <= len(x[i:i+k]):
match = x[i:i+k]
k=k+1
return match
print matchingString('apple','ale') #le
print matchingString('apple pie available','apple pies') #apple pie
A Trie data structure would work the best, better than DP.
Here is the code.
class TrieNode:
def __init__(self):
self.child = [None]*26
self.endWord = False
class Trie:
def __init__(self):
self.root = self.getNewNode()
def getNewNode(self):
return TrieNode()
def insert(self,value):
root = self.root
for i,character in enumerate(value):
index = ord(character) - ord('a')
if not root.child[index]:
root.child[index] = self.getNewNode()
root = root.child[index]
root.endWord = True
def search(self,value):
root = self.root
for i,character in enumerate(value):
index = ord(character) - ord('a')
if not root.child[index]:
return False
root = root.child[index]
return root.endWord
def main():
# Input keys (use only 'a' through 'z' and lower case)
keys = ["the","anaswe"]
output = ["Not present in trie",
"Present in trie"]
# Trie object
t = Trie()
# Construct trie
for key in keys:
t.insert(key)
# Search for different keys
print("{} ---- {}".format("the",output[t.search("the")]))
print("{} ---- {}".format("these",output[t.search("these")]))
print("{} ---- {}".format("their",output[t.search("their")]))
print("{} ---- {}".format("thaw",output[t.search("thaw")]))
if __name__ == '__main__':
main()
Let me know in case of doubts.
In case we have a list of words that we need to find all common substrings I check some of the codes above and the best was https://stackoverflow.com/a/42882629/8520109 but it has some bugs for example 'histhome' and 'homehist'. In this case, we should have 'hist' and 'home' as a result. Furthermore, it differs if the order of arguments is changed. So I change the code to find every block of substring and it results a set of common substrings:
main = input().split(" ") #a string of words separated by space
def longestSubstringFinder(string1, string2):
'''Find the longest matching word'''
answer = ""
len1, len2 = len(string1), len(string2)
for i in range(len1):
for j in range(len2):
lcs_temp=0
match=''
while ((i+lcs_temp < len1) and (j+lcs_temp<len2) and string1[i+lcs_temp] == string2[j+lcs_temp]):
match += string2[j+lcs_temp]
lcs_temp+=1
if (len(match) > len(answer)):
answer = match
return answer
def listCheck(main):
'''control the input for finding substring in a list of words'''
string1 = main[0]
result = []
for i in range(1, len(main)):
string2 = main[i]
res1 = longestSubstringFinder(string1, string2)
res2 = longestSubstringFinder(string2, string1)
result.append(res1)
result.append(res2)
result.sort()
return result
first_answer = listCheck(main)
final_answer = []
for item1 in first_answer: #to remove some incorrect match
string1 = item1
double_check = True
for item2 in main:
string2 = item2
if longestSubstringFinder(string1, string2) != string1:
double_check = False
if double_check:
final_answer.append(string1)
print(set(final_answer))
main = 'ABACDAQ BACDAQA ACDAQAW XYZCDAQ' #>>> {'CDAQ'}
main = 'homehist histhome' #>>> {'hist', 'home'}
def LongestSubString(s1,s2):
if len(s1)<len(s2) :
s1,s2 = s2,s1
maxsub =''
for i in range(len(s2)):
for j in range(len(s2),i,-1):
if s2[i:j] in s1 and j-i>len(maxsub):
return s2[i:j]
Returns the first longest common substring:
def compareTwoStrings(string1, string2):
list1 = list(string1)
list2 = list(string2)
match = []
output = ""
length = 0
for i in range(0, len(list1)):
if list1[i] in list2:
match.append(list1[i])
for j in range(i + 1, len(list1)):
if ''.join(list1[i:j]) in string2:
match.append(''.join(list1[i:j]))
else:
continue
else:
continue
for string in match:
if length < len(list(string)):
length = len(list(string))
output = string
else:
continue
return output
**Return the comman longest substring**
def longestSubString(str1, str2):
longestString = ""
maxLength = 0
for i in range(0, len(str1)):
if str1[i] in str2:
for j in range(i + 1, len(str1)):
if str1[i:j] in str2:
if(len(str1[i:j]) > maxLength):
maxLength = len(str1[i:j])
longestString = str1[i:j]
return longestString
This is the classroom problem called 'Longest sequence finder'. I have given some simple code that worked for me, also my inputs are lists of a sequence which can also be a string:
def longest_substring(list1,list2):
both=[]
if len(list1)>len(list2):
small=list2
big=list1
else:
small=list1
big=list2
removes=0
stop=0
for i in small:
for j in big:
if i!=j:
removes+=1
if stop==1:
break
elif i==j:
both.append(i)
for q in range(removes+1):
big.pop(0)
stop=1
break
removes=0
return both
As if this question doesn't have enough answers, here's another option:
from collections import defaultdict
def LongestCommonSubstring(string1, string2):
match = ""
matches = defaultdict(list)
str1, str2 = sorted([string1, string2], key=lambda x: len(x))
for i in range(len(str1)):
for k in range(i, len(str1)):
cur = match + str1[k]
if cur in str2:
match = cur
else:
match = ""
if match:
matches[len(match)].append(match)
if not matches:
return ""
longest_match = max(matches.keys())
return matches[longest_match][0]
Some example cases:
LongestCommonSubstring("whose car?", "this is my car")
> ' car'
LongestCommonSubstring("apple pies", "apple? forget apple pie!")
> 'apple pie'
This isn't the most efficient way to do it but it's what I could come up with and it works. If anyone can improve it, please do. What it does is it makes a matrix and puts 1 where the characters match. Then it scans the matrix to find the longest diagonal of 1s, keeping track of where it starts and ends. Then it returns the substring of the input string with the start and end positions as arguments.
Note: This only finds one longest common substring. If there's more than one, you could make an array to store the results in and return that Also, it's case sensitive so (Apple pie, apple pie) will return pple pie.
def longestSubstringFinder(str1, str2):
answer = ""
if len(str1) == len(str2):
if str1==str2:
return str1
else:
longer=str1
shorter=str2
elif (len(str1) == 0 or len(str2) == 0):
return ""
elif len(str1)>len(str2):
longer=str1
shorter=str2
else:
longer=str2
shorter=str1
matrix = numpy.zeros((len(shorter), len(longer)))
for i in range(len(shorter)):
for j in range(len(longer)):
if shorter[i]== longer[j]:
matrix[i][j]=1
longest=0
start=[-1,-1]
end=[-1,-1]
for i in range(len(shorter)-1, -1, -1):
for j in range(len(longer)):
count=0
begin = [i,j]
while matrix[i][j]==1:
finish=[i,j]
count=count+1
if j==len(longer)-1 or i==len(shorter)-1:
break
else:
j=j+1
i=i+1
i = i-count
if count>longest:
longest=count
start=begin
end=finish
break
answer=shorter[int(start[0]): int(end[0])+1]
return answer
First a helper function adapted from the itertools pairwise recipe to produce substrings.
import itertools
def n_wise(iterable, n = 2):
'''n = 2 -> (s0,s1), (s1,s2), (s2, s3), ...
n = 3 -> (s0,s1, s2), (s1,s2, s3), (s2, s3, s4), ...'''
a = itertools.tee(iterable, n)
for x, thing in enumerate(a[1:]):
for _ in range(x+1):
next(thing, None)
return zip(*a)
Then a function the iterates over substrings, longest first, and tests for membership. (efficiency not considered)
def foo(s1, s2):
'''Finds the longest matching substring
'''
# the longest matching substring can only be as long as the shortest string
#which string is shortest?
shortest, longest = sorted([s1, s2], key = len)
#iterate over substrings, longest substrings first
for n in range(len(shortest)+1, 2, -1):
for sub in n_wise(shortest, n):
sub = ''.join(sub)
if sub in longest:
#return the first one found, it should be the longest
return sub
s = "fdomainster"
t = "exdomainid"
print(foo(s,t))
>>>
domain
>>>
def LongestSubString(s1,s2):
left = 0
right =len(s2)
while(left<right):
if(s2[left] not in s1):
left = left+1
else:
if(s2[left:right] not in s1):
right = right - 1
else:
return(s2[left:right])
s1 = "pineapple"
s2 = "applc"
print(LongestSubString(s1,s2))
Python has string.find() and string.rfind() to get the index of a substring in a string.
I'm wondering whether there is something like string.find_all() which can return all found indexes (not only the first from the beginning or the first from the end).
For example:
string = "test test test test"
print string.find('test') # 0
print string.rfind('test') # 15
#this is the goal
print string.find_all('test') # [0,5,10,15]
For counting the occurrences, see Count number of occurrences of a substring in a string.
There is no simple built-in string function that does what you're looking for, but you could use the more powerful regular expressions:
import re
[m.start() for m in re.finditer('test', 'test test test test')]
#[0, 5, 10, 15]
If you want to find overlapping matches, lookahead will do that:
[m.start() for m in re.finditer('(?=tt)', 'ttt')]
#[0, 1]
If you want a reverse find-all without overlaps, you can combine positive and negative lookahead into an expression like this:
search = 'tt'
[m.start() for m in re.finditer('(?=%s)(?!.{1,%d}%s)' % (search, len(search)-1, search), 'ttt')]
#[1]
re.finditer returns a generator, so you could change the [] in the above to () to get a generator instead of a list which will be more efficient if you're only iterating through the results once.
>>> help(str.find)
Help on method_descriptor:
find(...)
S.find(sub [,start [,end]]) -> int
Thus, we can build it ourselves:
def find_all(a_str, sub):
start = 0
while True:
start = a_str.find(sub, start)
if start == -1: return
yield start
start += len(sub) # use start += 1 to find overlapping matches
list(find_all('spam spam spam spam', 'spam')) # [0, 5, 10, 15]
No temporary strings or regexes required.
Here's a (very inefficient) way to get all (i.e. even overlapping) matches:
>>> string = "test test test test"
>>> [i for i in range(len(string)) if string.startswith('test', i)]
[0, 5, 10, 15]
Use re.finditer:
import re
sentence = input("Give me a sentence ")
word = input("What word would you like to find ")
for match in re.finditer(word, sentence):
print (match.start(), match.end())
For word = "this" and sentence = "this is a sentence this this" this will yield the output:
(0, 4)
(19, 23)
(24, 28)
Again, old thread, but here's my solution using a generator and plain str.find.
def findall(p, s):
'''Yields all the positions of
the pattern p in the string s.'''
i = s.find(p)
while i != -1:
yield i
i = s.find(p, i+1)
Example
x = 'banananassantana'
[(i, x[i:i+2]) for i in findall('na', x)]
returns
[(2, 'na'), (4, 'na'), (6, 'na'), (14, 'na')]
You can use re.finditer() for non-overlapping matches.
>>> import re
>>> aString = 'this is a string where the substring "is" is repeated several times'
>>> print [(a.start(), a.end()) for a in list(re.finditer('is', aString))]
[(2, 4), (5, 7), (38, 40), (42, 44)]
but won't work for:
In [1]: aString="ababa"
In [2]: print [(a.start(), a.end()) for a in list(re.finditer('aba', aString))]
Output: [(0, 3)]
Come, let us recurse together.
def locations_of_substring(string, substring):
"""Return a list of locations of a substring."""
substring_length = len(substring)
def recurse(locations_found, start):
location = string.find(substring, start)
if location != -1:
return recurse(locations_found + [location], location+substring_length)
else:
return locations_found
return recurse([], 0)
print(locations_of_substring('this is a test for finding this and this', 'this'))
# prints [0, 27, 36]
No need for regular expressions this way.
If you're just looking for a single character, this would work:
string = "dooobiedoobiedoobie"
match = 'o'
reduce(lambda count, char: count + 1 if char == match else count, string, 0)
# produces 7
Also,
string = "test test test test"
match = "test"
len(string.split(match)) - 1
# produces 4
My hunch is that neither of these (especially #2) is terribly performant.
this is an old thread but i got interested and wanted to share my solution.
def find_all(a_string, sub):
result = []
k = 0
while k < len(a_string):
k = a_string.find(sub, k)
if k == -1:
return result
else:
result.append(k)
k += 1 #change to k += len(sub) to not search overlapping results
return result
It should return a list of positions where the substring was found.
Please comment if you see an error or room for improvment.
This does the trick for me using re.finditer
import re
text = 'This is sample text to test if this pythonic '\
'program can serve as an indexing platform for '\
'finding words in a paragraph. It can give '\
'values as to where the word is located with the '\
'different examples as stated'
# find all occurances of the word 'as' in the above text
find_the_word = re.finditer('as', text)
for match in find_the_word:
print('start {}, end {}, search string \'{}\''.
format(match.start(), match.end(), match.group()))
This thread is a little old but this worked for me:
numberString = "onetwothreefourfivesixseveneightninefiveten"
testString = "five"
marker = 0
while marker < len(numberString):
try:
print(numberString.index("five",marker))
marker = numberString.index("five", marker) + 1
except ValueError:
print("String not found")
marker = len(numberString)
You can try :
>>> string = "test test test test"
>>> for index,value in enumerate(string):
if string[index:index+(len("test"))] == "test":
print index
0
5
10
15
You can try :
import re
str1 = "This dress looks good; you have good taste in clothes."
substr = "good"
result = [_.start() for _ in re.finditer(substr, str1)]
# result = [17, 32]
When looking for a large amount of key words in a document, use flashtext
from flashtext import KeywordProcessor
words = ['test', 'exam', 'quiz']
txt = 'this is a test'
kwp = KeywordProcessor()
kwp.add_keywords_from_list(words)
result = kwp.extract_keywords(txt, span_info=True)
Flashtext runs faster than regex on large list of search words.
This function does not look at all positions inside the string, it does not waste compute resources. My try:
def findAll(string,word):
all_positions=[]
next_pos=-1
while True:
next_pos=string.find(word,next_pos+1)
if(next_pos<0):
break
all_positions.append(next_pos)
return all_positions
to use it call it like this:
result=findAll('this word is a big word man how many words are there?','word')
src = input() # we will find substring in this string
sub = input() # substring
res = []
pos = src.find(sub)
while pos != -1:
res.append(pos)
pos = src.find(sub, pos + 1)
Whatever the solutions provided by others are completely based on the available method find() or any available methods.
What is the core basic algorithm to find all the occurrences of a
substring in a string?
def find_all(string,substring):
"""
Function: Returning all the index of substring in a string
Arguments: String and the search string
Return:Returning a list
"""
length = len(substring)
c=0
indexes = []
while c < len(string):
if string[c:c+length] == substring:
indexes.append(c)
c=c+1
return indexes
You can also inherit str class to new class and can use this function
below.
class newstr(str):
def find_all(string,substring):
"""
Function: Returning all the index of substring in a string
Arguments: String and the search string
Return:Returning a list
"""
length = len(substring)
c=0
indexes = []
while c < len(string):
if string[c:c+length] == substring:
indexes.append(c)
c=c+1
return indexes
Calling the method
newstr.find_all('Do you find this answer helpful? then upvote
this!','this')
This is solution of a similar question from hackerrank. I hope this could help you.
import re
a = input()
b = input()
if b not in a:
print((-1,-1))
else:
#create two list as
start_indc = [m.start() for m in re.finditer('(?=' + b + ')', a)]
for i in range(len(start_indc)):
print((start_indc[i], start_indc[i]+len(b)-1))
Output:
aaadaa
aa
(0, 1)
(1, 2)
(4, 5)
Here's a solution that I came up with, using assignment expression (new feature since Python 3.8):
string = "test test test test"
phrase = "test"
start = -1
result = [(start := string.find(phrase, start + 1)) for _ in range(string.count(phrase))]
Output:
[0, 5, 10, 15]
I think the most clean way of solution is without libraries and yields:
def find_all_occurrences(string, sub):
index_of_occurrences = []
current_index = 0
while True:
current_index = string.find(sub, current_index)
if current_index == -1:
return index_of_occurrences
else:
index_of_occurrences.append(current_index)
current_index += len(sub)
find_all_occurrences(string, substr)
Note: find() method returns -1 when it can't find anything
The pythonic way would be:
mystring = 'Hello World, this should work!'
find_all = lambda c,s: [x for x in range(c.find(s), len(c)) if c[x] == s]
# s represents the search string
# c represents the character string
find_all(mystring,'o') # will return all positions of 'o'
[4, 7, 20, 26]
>>>
if you only want to use numpy here is a solution
import numpy as np
S= "test test test test"
S2 = 'test'
inds = np.cumsum([len(k)+len(S2) for k in S.split(S2)[:-1]])- len(S2)
print(inds)
if you want to use without re(regex) then:
find_all = lambda _str,_w : [ i for i in range(len(_str)) if _str.startswith(_w,i) ]
string = "test test test test"
print( find_all(string, 'test') ) # >>> [0, 5, 10, 15]
please look at below code
#!/usr/bin/env python
# coding:utf-8
'''黄哥Python'''
def get_substring_indices(text, s):
result = [i for i in range(len(text)) if text.startswith(s, i)]
return result
if __name__ == '__main__':
text = "How much wood would a wood chuck chuck if a wood chuck could chuck wood?"
s = 'wood'
print get_substring_indices(text, s)
def find_index(string, let):
enumerated = [place for place, letter in enumerate(string) if letter == let]
return enumerated
for example :
find_index("hey doode find d", "d")
returns:
[4, 7, 13, 15]
Not exactly what OP asked but you could also use the split function to get a list of where all the substrings don't occur. OP didn't specify the end goal of the code but if your goal is to remove the substrings anyways then this could be a simple one-liner. There are probably more efficient ways to do this with larger strings; regular expressions would be preferable in that case
# Extract all non-substrings
s = "an-example-string"
s_no_dash = s.split('-')
# >>> s_no_dash
# ['an', 'example', 'string']
# Or extract and join them into a sentence
s_no_dash2 = ' '.join(s.split('-'))
# >>> s_no_dash2
# 'an example string'
Did a brief skim of other answers so apologies if this is already up there.
def count_substring(string, sub_string):
c=0
for i in range(0,len(string)-2):
if string[i:i+len(sub_string)] == sub_string:
c+=1
return c
if __name__ == '__main__':
string = input().strip()
sub_string = input().strip()
count = count_substring(string, sub_string)
print(count)
I runned in the same problem and did this:
hw = 'Hello oh World!'
list_hw = list(hw)
o_in_hw = []
while True:
o = hw.find('o')
if o != -1:
o_in_hw.append(o)
list_hw[o] = ' '
hw = ''.join(list_hw)
else:
print(o_in_hw)
break
Im pretty new at coding so you can probably simplify it (and if planned to used continuously of course make it a function).
All and all it works as intended for what i was doing.
Edit: Please consider this is for single characters only, and it will change your variable, so you have to create a copy of the string in a new variable to save it, i didnt put it in the code cause its easy and its only to show how i made it work.
By slicing we find all the combinations possible and append them in a list and find the number of times it occurs using count function
s=input()
n=len(s)
l=[]
f=input()
print(s[0])
for i in range(0,n):
for j in range(1,n+1):
l.append(s[i:j])
if f in l:
print(l.count(f))
To find all the occurence of a character in a give string and return as a dictionary
eg: hello
result :
{'h':1, 'e':1, 'l':2, 'o':1}
def count(string):
result = {}
if(string):
for i in string:
result[i] = string.count(i)
return result
return {}
or else you do like this
from collections import Counter
def count(string):
return Counter(string)