extract substring pattern - python

I have long file like 1200 sequences
>3fm8|A|A0JLQ2
CFLVNLNADPALNELLVYYLKEHTLIGSANSQDIQLCGMGILPEHCIIDITSEGQVMLTP
QKNTRTFVNGSSVSSPIQLHHGDRILWGNNHFFRLNLP
>2ht9|A|A0JLT0
LATAPVNQIQETISDNCVVIFSKTSCSYCTMAKKLFHDMNVNYKVVELDLLEYGNQFQDA
LYKMTGERTVPRIFVNGTFIGGATDTHRLHKEGKLLPLVHQCYL
I want to read each possible pattern has cysteine in middle and has in the beginning five string and follow by other five string such as xxxxxCxxxxx
the output should be like this:
QDIQLCGMGIL
ILPEHCIIDIT
TISDNCVVIFS
FSKTSCSYCTM
this is the pogram only give position of C . it is not work like what I want
pos=[]
def find(ch,string1):
for i in range(len(string1)):
if ch == string1[i]:
pos.append(i)
return pos
z=find('C','AWERQRTCWERTYCTAAAACTTCTTT')
print z

You need to return outside the loop, you are returning on the first match so you only ever get a single character in your list:
def find(ch,string1):
pos = []
for i in range(len(string1)):
if ch == string1[i]:
pos.append(i)
return pos # outside
You can also use enumerate with a list comp in place of your range logic:
def indexes(ch, s1):
return [index for index, char in enumerate(s1)if char == ch and 5 >= index <= len(s1) - 6]
Each index in the list comp is the character index and each char is the actual character so we keep each index where char is equal to ch.
If you want the five chars that are both sides:
In [24]: s="CFLVNLNADPALNELLVYYLKEHTLIGSANSQDIQLCGMGILPEHCIIDITSEGQVMLTP QKNTRTFVNGSSVSSPIQLHHGDRILWGNNHFFRLNLP"
In [25]: inds = indexes("C",s)
In [26]: [s[i-5:i+6] for i in inds]
Out[26]: ['QDIQLCGMGIL', 'ILPEHCIIDIT']
I added checking the index as we obviously cannot get five chars before C if the index is < 5 and the same from the end.
You can do it all in a single function, yielding a slice when you find a match:
def find(ch, s):
ln = len(s)
for i, char in enumerate(s):
if ch == char and 5 <= i <= ln - 6:
yield s[i- 5:i + 6]
Where presuming the data in your question is actually two lines from yoru file like:
s="""">3fm8|A|A0JLQ2CFLVNLNADPALNELLVYYLKEHTLIGSANSQDIQLCGMGILPEHCIIDITSEGQVMLTPQKNTRTFVNGSSVSSPIQLHHGDRILWGNNHFFRLNLP
>2ht9|A|A0JLT0LATAPVNQIQETISDNCVVIFSKTSCSYCTMAKKLFHDMNVNYKVVELDLLEYGNQFQDALYKMTGERTVPRIFVNGTFIGGATDTHRLHKEGKLLPLVHQCY"""
Running:
for line in s.splitlines():
print(list(find("C" ,line)))
would output:
['0JLQ2CFLVNL', 'QDIQLCGMGIL', 'ILPEHCIIDIT']
['TISDNCVVIFS', 'FSKTSCSYCTM', 'TSCSYCTMAKK']
Which gives six matches not four as your expected output suggest so I presume you did not include all possible matches.
You can also speed up the code using str.find, starting at the last match index + 1 for each subsequent match
def find(ch, s):
ln, i = len(s) - 6, s.find(ch)
while 5 <= i <= ln:
yield s[i - 5:i + 6]
i = s.find(ch, i + 1)
Which will give the same output. Of course if the strings cannot overlap you can start looking for the next match much further in the string each time.

My solution is based on regex, and shows all possible solutions using regex and while loop. Thanks to #Smac89 for improving it by transforming it into a generator:
import re
string = """CFLVNLNADPALNELLVYYLKEHTLIGSANSQDIQLCGMGILPEHCIIDITSEGQVMLTPQKNTRTFVNGSSVSSPIQLHHGDRILWGNNHFFRLNLP
LATAPVNQIQETISDNCVVIFSKTSCSYCTMAKKLFHDMNVNYKVVELDLLEYGNQFQDA LYKMTGERTVPRIFVNGTFIGGATDTHRLHKEGKLLPLVHQCYL"""
# Generator
def find_cysteine2(string):
# Create a loop that will utilize regex multiple times
# in order to capture matches within groups
while True:
# Find a match
data = re.search(r'(\w{5}C\w{5})',string)
# If match exists, let's collect the data
if data:
# Collect the string
yield data.group(1)
# Shrink the string to not include
# the previous result
location = data.start() + 1
string = string[location:]
# If there are no matches, stop the loop
else:
break
print [x for x in find_cysteine2(string)]
# ['QDIQLCGMGIL', 'ILPEHCIIDIT', 'TISDNCVVIFS', 'FSKTSCSYCTM', 'TSCSYCTMAKK']

Related

How to find the most amount of shared characters in two strings? (Python)

yamxxopd
yndfyamxx
Output: 5
I am not quite sure how to find the number of the most amount of shared characters between two strings. For example (the strings above) the most amount of characters shared together is "yamxx" which is 5 characters long.
xx would not be a solution because that is not the most amount of shared characters. In this case the most is yamxx which is 5 characters long so the output would be 5.
I am quite new to python and stack overflow so any help would be much appreciated!
Note: They should be the same order in both strings
Here is simple, efficient solution using dynamic programming.
def longest_subtring(X, Y):
m,n = len(X), len(Y)
LCSuff = [[0 for k in range(n+1)] for l in range(m+1)]
result = 0
for i in range(m + 1):
for j in range(n + 1):
if (i == 0 or j == 0):
LCSuff[i][j] = 0
elif (X[i-1] == Y[j-1]):
LCSuff[i][j] = LCSuff[i-1][j-1] + 1
result = max(result, LCSuff[i][j])
else:
LCSuff[i][j] = 0
print (result )
longest_subtring("abcd", "arcd") # prints 2
longest_subtring("yammxdj", "nhjdyammx") # prints 5
This solution starts with sub-strings of longest possible lengths. If, for a certain length, there are no matching sub-strings of that length, it moves on to the next lower length. This way, it can stop at the first successful match.
s_1 = "yamxxopd"
s_2 = "yndfyamxx"
l_1, l_2 = len(s_1), len(s_2)
found = False
sub_length = l_1 # Let's start with the longest possible sub-string
while (not found) and sub_length: # Loop, over decreasing lengths of sub-string
for start in range(l_1 - sub_length + 1): # Loop, over all start-positions of sub-string
sub_str = s_1[start:(start+sub_length)] # Get the sub-string at that start-position
if sub_str in s_2: # If found a match for the sub-string, in s_2
found = True # Stop trying with smaller lengths of sub-string
break # Stop trying with this length of sub-string
else: # If no matches found for this length of sub-string
sub_length -= 1 # Let's try a smaller length for the sub-strings
print (f"Answer is {sub_length}" if found else "No common sub-string")
Output:
Answer is 5
s1 = "yamxxopd"
s2 = "yndfyamxx"
# initializing counter
counter = 0
# creating and initializing a string without repetition
s = ""
for x in s1:
if x not in s:
s = s + x
for x in s:
if x in s2:
counter = counter + 1
# display the number of the most amount of shared characters in two strings s1 and s2
print(counter) # display 5

Finding repeated character combinations in string

I have a string that holds a very long sentence without whitespaces/spaces.
mystring = "abcdthisisatextwithsampletextforasampleabcd"
I would like to find all of the repeated substrings that contains minimum 4 chars.
So I would like to achieve something like this:
'text' 2 times
'sample' 2 times
'abcd' 2 times
As both abcd,text and sample can be found two times in the mystring they were recognized as properly matched substrings with more than 4 char length. It's important that I am seeking repeated substrings, finding only existing English words is not a requirement.
The answers I found are helpful for finding duplicates in texts with whitespaces, but I couldn't find a proper resource that covers the situation when there are no spaces and whitespaces in the string. How can this be done in the most efficient way?
Let's go through this step by step. There are several sub-tasks you should take care of:
Identify all substrings of length 4 or more.
Count the occurrence of these substrings.
Filter all substrings with 2 occurrences or more.
You can actually put all of them into a few statements. For understanding, it is easier to go through them one at a time.
The following examples all use
mystring = "abcdthisisatextwithsampletextforasampleabcd"
min_length = 4
1. Substrings of a given length
You can easily get substrings by slicing - for example, mystring[4:4+6] gives you the substring from position 4 of length 6: 'thisis'. More generically, you want substrings of the form mystring[start:start+length].
So what values do you need for start and length?
start must...
cover all substrings, so it must include the first character: start in range(0, ...).
not map to short substrings, so it can stop min_length characters before the end: start in range(..., len(mystring) - min_length + 1).
length must...
cover the shortest substring of length 4: length in range(min_length, ...).
not exceed the remaining string after i: length in range(..., len(mystring) - i + 1))
The +1 terms come from converting lengths (>=1) to indices (>=0).
You can put this all together into a single comprehension:
substrings = [
mystring[i:i+j]
for i in range(0, len(mystring) - min_length + 1)
for j in range(min_length, len(mystring) - i + 1)
]
2. Count substrings
Trivially, you want to keep a count for each substring. Keeping anything for each specific object is what dicts are made for. So you should use substrings as keys and counts as values in a dict. In essence, this corresponds to this:
counts = {}
for substring in substrings:
try: # increase count for existing keys, set for new keys
counts[substring] += 1
except KeyError:
counts[substring] = 1
You can simply feed your substrings to collections.Counter, and it produces something like the above.
>>> counts = collections.Counter(substrings)
>>> print(counts)
Counter({'abcd': 2, 'abcdt': 1, 'abcdth': 1, 'abcdthi': 1, 'abcdthis': 1, ...})
Notice how the duplicate 'abcd' maps to the count of 2.
3. Filtering duplicate substrings
So now you have your substrings and the count for each. You need to remove the non-duplicate substrings - those with a count of 1.
Python offers several constructs for filtering, depending on the output you want. These work also if counts is a regular dict:
>>> list(filter(lambda key: counts[key] > 1, counts))
['abcd', 'text', 'samp', 'sampl', 'sample', 'ampl', 'ample', 'mple']
>>> {key: value for key, value in counts.items() if value > 1}
{'abcd': 2, 'ampl': 2, 'ample': 2, 'mple': 2, 'samp': 2, 'sampl': 2, 'sample': 2, 'text': 2}
Using Python primitives
Python ships with primitives that allow you to do this more efficiently.
Use a generator to build substrings. A generator builds its member on the fly, so you never actually have them all in-memory. For your use case, you can use a generator expression:
substrings = (
mystring[i:i+j]
for i in range(0, len(mystring) - min_length + 1)
for j in range(min_length, len(mystring) - i + 1)
)
Use a pre-existing Counter implementation. Python comes with a dict-like container that counts its members: collections.Counter can directly digest your substring generator. Especially in newer version, this is much more efficient.
counts = collections.Counter(substrings)
You can exploit Python's lazy filters to only ever inspect one substring. The filter builtin or another generator generator expression can produce one result at a time without storing them all in memory.
for substring in filter(lambda key: counts[key] > 1, counts):
print(substring, 'occurs', counts[substring], 'times')
Nobody is using re! Time for an answer [ab]using the regular expression built-in module ;)
import re
Finding all the maximal substrings that are repeated
repeated_ones = set(re.findall(r"(.{4,})(?=.*\1)", mystring))
This matches the longest substrings which have at least a single repetition after (without consuming). So it finds all disjointed substrings that are repeated while only yielding the longest strings.
Finding all substrings that are repeated, including overlaps
mystring_overlap = "abcdeabcdzzzzbcde"
# In case we want to match both abcd and bcde
repeated_ones = set()
pos = 0
while True:
match = re.search(r"(.{4,}).*(\1)+", mystring_overlap[pos:])
if match:
repeated_ones.add(match.group(1))
pos += match.pos + 1
else:
break
This ensures that all --not only disjoint-- substrings which have repetition are returned. It should be much slower, but gets the work done.
If you want in addition to the longest strings that are repeated, all the substrings, then:
base_repetitions = list(repeated_ones)
for s in base_repetitions:
for i in range(4, len(s)):
repeated_ones.add(s[:i])
That will ensure that for long substrings that have repetition, you have also the smaller substring --e.g. "sample" and "ample" found by the re.search code; but also "samp", "sampl", "ampl" added by the above snippet.
Counting matches
Because (by design) the substrings that we count are non-overlapping, the count method is the way to go:
from __future__ import print_function
for substr in repeated_ones:
print("'%s': %d times" % (substr, mystring.count(substr)))
Results
Finding maximal substrings:
With the question's original mystring:
{'abcd', 'text', 'sample'}
with the mystring_overlap sample:
{'abcd'}
Finding all substrings:
With the question's original mystring:
{'abcd', 'ample', 'mple', 'sample', 'text'}
... and if we add the code to get all substrings then, of course, we get absolutely all the substrings:
{'abcd', 'ampl', 'ample', 'mple', 'samp', 'sampl', 'sample', 'text'}
with the mystring_overlap sample:
{'abcd', 'bcde'}
Future work
It's possible to filter the results of the finding all substrings with the following steps:
take a match "A"
check if this match is a substring of another match, call it "B"
if there is a "B" match, check the counter on that match "B_n"
if "A_n = B_n", then remove A
go to first step
It cannot happen that "A_n < B_n" because A is smaller than B (is a substring) so there must be at least the same number of repetitions.
If "A_n > B_n" it means that there is some extra match of the smaller substring, so it is a distinct substring because it is repeated in a place where B is not repeated.
Script (explanation where needed, in comments):
from collections import Counter
mystring = "abcdthisisatextwithsampletextforasampleabcd"
mystring_len = len(mystring)
possible_matches = []
matches = []
# Range `start_index` from 0 to 3 from the left, due to minimum char count of 4
for start_index in range(0, mystring_len-3):
# Start `end_index` at `start_index+1` and range it throughout the rest of
# the string
for end_index in range(start_index+1, mystring_len+1):
current_string = mystring[start_index:end_index]
if len(current_string) < 4: continue # Skip this interation, if len < 4
possible_matches.append(mystring[start_index:end_index])
for possible_match, count in Counter(possible_matches).most_common():
# Iterate until count is less than or equal to 1 because `Counter`'s
# `most_common` method lists them in order. Once 1 (or less) is hit, all
# others are the same or lower.
if count <= 1: break
matches.append((possible_match, count))
for match, count in matches:
print(f'\'{match}\' {count} times')
Output:
'abcd' 2 times
'text' 2 times
'samp' 2 times
'sampl' 2 times
'sample' 2 times
'ampl' 2 times
'ample' 2 times
'mple' 2 times
Here's a Python3 friendly solution:
from collections import Counter
min_str_length = 4
mystring = "abcdthisisatextwithsampletextforasampleabcd"
all_substrings =[mystring[start_index:][:end_index + 1] for start_index in range(len(mystring)) for end_index in range(len(mystring[start_index:]))]
counted_substrings = Counter(all_substrings)
not_counted_final_candidates = [item[0] for item in counted_substrings.most_common() if item[1] > 1 and len(item[0]) >= min_str_length]
counted_final_candidates = {item: counted_substrings[item] for item in not_counted_final_candidates}
print(counted_final_candidates)
Bonus: largest string
sub_sub_strings = [substring1 for substring1 in not_counted_final_candidates for substring2 in not_counted_final_candidates if substring1!=substring2 and substring1 in substring2 ]
largest_common_string = list(set(not_counted_final_candidates) - set(sub_sub_strings))
Everything as a function:
from collections import Counter
def get_repeated_strings(input_string, min_str_length = 2, calculate_largest_repeated_string = True ):
all_substrings = [input_string[start_index:][:end_index + 1]
for start_index in range(len(input_string))
for end_index in range(len(input_string[start_index:]))]
counted_substrings = Counter(all_substrings)
not_counted_final_candidates = [item[0]
for item in counted_substrings.most_common()
if item[1] > 1 and len(item[0]) >= min_str_length]
counted_final_candidates = {item: counted_substrings[item] for item in not_counted_final_candidates}
### This is just a bit of bonus code for calculating the largest repeating sting
if calculate_largest_repeated_string == True:
sub_sub_strings = [substring1 for substring1 in not_counted_final_candidates for substring2 in
not_counted_final_candidates if substring1 != substring2 and substring1 in substring2]
largest_common_strings = list(set(not_counted_final_candidates) - set(sub_sub_strings))
return counted_final_candidates, largest_common_strings
else:
return counted_final_candidates
Example:
mystring = "abcdthisisatextwithsampletextforasampleabcd"
print(get_repeated_strings(mystring, min_str_length= 4))
Output:
({'abcd': 2, 'text': 2, 'samp': 2, 'sampl': 2, 'sample': 2, 'ampl': 2, 'ample': 2, 'mple': 2}, ['abcd', 'text', 'sample'])
CODE:
pattern = "abcdthisisatextwithsampletextforasampleabcd"
string_more_4 = []
k = 4
while(k <= len(pattern)):
for i in range(len(pattern)):
if pattern[i:k+i] not in string_more_4 and len(pattern[i:k+i]) >= 4:
string_more_4.append( pattern[i:k+i])
k+=1
for i in string_more_4:
if pattern.count(i) >= 2:
print(i + " -> " + str(pattern.count(i)) + " times")
OUTPUT:
abcd -> 2 times
text -> 2 times
samp -> 2 times
ampl -> 2 times
mple -> 2 times
sampl -> 2 times
ample -> 2 times
sample -> 2 times
Hope this helps as my code length was short and it is easy to understand. Cheers!
This is in Python 2 because I'm not doing Python 3 at this time. So you'll have to adapt it to Python 3 yourself.
#!python2
# import module
from collections import Counter
# get the indices
def getIndices(length):
# holds the indices
specific_range = []; all_sets = []
# start building the indices
for i in range(0, length - 2):
# build a set of indices of a specific range
for j in range(1, length + 2):
specific_range.append([j - 1, j + i + 3])
# append 'specific_range' to 'all_sets', reset 'specific_range'
if specific_range[j - 1][1] == length:
all_sets.append(specific_range)
specific_range = []
break
# return all of the calculated indices ranges
return all_sets
# store search strings
tmplst = []; combos = []; found = []
# string to be searched
mystring = "abcdthisisatextwithsampletextforasampleabcd"
# mystring = "abcdthisisatextwithtextsampletextforasampleabcdtext"
# get length of string
length = len(mystring)
# get all of the indices ranges, 4 and greater
all_sets = getIndices(length)
# get the search string combinations
for sublst in all_sets:
for subsublst in sublst:
tmplst.append(mystring[subsublst[0]: subsublst[1]])
combos.append(tmplst)
tmplst = []
# search for matching string patterns
for sublst in all_sets:
for subsublst in sublst:
for sublstitems in combos:
if mystring[subsublst[0]: subsublst[1]] in sublstitems:
found.append(mystring[subsublst[0]: subsublst[1]])
# make a dictionary containing the strings and their counts
d1 = Counter(found)
# filter out counts of 2 or more and print them
for k, v in d1.items():
if v > 1:
print k, v
$ cat test.py
import collections
import sys
S = "abcdthisisatextwithsampletextforasampleabcd"
def find(s, min_length=4):
"""
Find repeated character sequences in a provided string.
Arguments:
s -- the string to be searched
min_length -- the minimum length of the sequences to be found
"""
counter = collections.defaultdict(int)
# A repeated sequence can't be longer than half the length of s
sequence_length = len(s) // 2
# populate counter with all possible sequences
while sequence_length >= min_length:
# Iterate over the string until the number of remaining characters is
# fewer than the length of the current sequence.
for i, x in enumerate(s[:-(sequence_length - 1)]):
# Window across the string, getting slices
# of length == sequence_length.
candidate = s[i:i + sequence_length]
counter[candidate] += 1
sequence_length -= 1
# Report.
for k, v in counter.items():
if v > 1:
print('{} {} times'.format(k, v))
return
if __name__ == '__main__':
try:
s = sys.argv[1]
except IndexError:
s = S
find(s)
$ python test.py
sample 2 times
sampl 2 times
ample 2 times
abcd 2 times
text 2 times
samp 2 times
ampl 2 times
mple 2 times
This is my approach to this problem:
def get_repeated_words(string, minimum_len):
# Storing count of repeated words in this dictionary
repeated_words = {}
# Traversing till last but 4th element
# Actually leaving `minimum_len` elements at end (in this case its 4)
for i in range(len(string)-minimum_len):
# Starting with a length of 4(`minimum_len`) and going till end of string
for j in range(i+minimum_len, len(string)):
# getting the current word
word = string[i:j]
# counting the occurrences of the word
word_count = string.count(word)
if word_count > 1:
# storing in dictionary along with its count if found more than once
repeated_words[word] = word_count
return repeated_words
if __name__ == '__main__':
mystring = "abcdthisisatextwithsampletextforasampleabcd"
result = get_repeated_words(mystring, 4)
This is how I would do it, but I don't know any other way:
string = "abcdthisisatextwithsampletextforasampleabcd"
l = len(string)
occurences = {}
for i in range(4, l):
for start in range(l - i):
substring = string[start:start + i]
occurences[substring] = occurences.get(substring, 0) + 1
for key in occurences.keys():
if occurences[key] > 1:
print("'" + key + "'", str(occurences[key]), "times")
Output:
'sample' 2 times
'ampl' 2 times
'sampl' 2 times
'ample' 2 times
'samp' 2 times
'mple' 2 times
'text' 2 times
Efficient, no, but easy to understand, yes.
Here is simple solution using the more_itertools library.
Given
import collections as ct
import more_itertools as mit
s = "abcdthisisatextwithsampletextforasampleabcd"
lbound, ubound = len("abcd"), len(s)
Code
windows = mit.flatten(mit.windowed(s, n=i) for i in range(lbound, ubound))
filtered = {"".join(k): v for k, v in ct.Counter(windows).items() if v > 1}
filtered
Output
{'abcd': 2,
'text': 2,
'samp': 2,
'ampl': 2,
'mple': 2,
'sampl': 2,
'ample': 2,
'sample': 2}
Details
The procedures are:
build sliding windows of varying sizes lbound <= n < ubound
count all occurrences and filter replicates
more_itertools is a third-party package installed by > pip install more_itertools.
s = 'abcabcabcdabcd'
d = {}
def get_repeats(s, l):
for i in range(len(s)-l):
ss = s[i: i+l]
if ss not in d:
d[ss] = 1
else:
d[ss] = d[ss]+1
return d
get_repeats(s, 3)

Removing duplicates if paired adjacently

Deleting any pair of adjacent letters with same value. For example, string "aabcc" would become either "aab" or "bcc" after operation.
Sample input = aaabccddd
Sample output = abd
Confused how to iterate the list or the string in a way to match the duplicates and removing them, here is the way I am trying and I know it is wrong.
S = input()
removals = []
for i in range(0, len(S)):
if i + 1 >= len(S):
break
elif S[i] == S[i + 1]:
removals.append(i)
# removals is to store all the indexes that are to be deleted.
removals.append(i + 1)
i += 1
print(i)
Array = list(S)
set(removals) #removes duplicates from removals
for j in range(0, len(removals)):
Array.pop(removals[j]) # Creates IndexOutOfRange error
This is a problem from Hackerrank: Super Reduced String
Removing paired letters can be reduced to reducing runs of letters to an empty sequence if there are an even number of them, 1 if there are an odd number. aaaaaa becomes empty, aaaaa is reduced to a.
To do this on any sequence, use itertools.groupby() and count the group size:
# only include a value if their consecutive count is odd
[v for v, group in groupby(sequence) if sum(1 for _ in group) % 2]
then repeat until the size of the sequence no longer changes:
prev = len(sequence) + 1
while len(sequence) < prev:
prev = len(sequence)
sequence = [v for v, group in groupby(sequence) if sum(1 for _ in group) % 2]
However, since Hackerrank gives you text it'd be faster if you did this with a regular expression:
import re
even = re.compile(r'(?:([a-z])\1)+')
prev = len(text) + 1
while len(text) < prev:
prev = len(text)
text = even.sub(r'', text)
[a-z] in a regex matches a lower-case letter, (..)groups that match, and\1references the first match and will only match if that letter was repeated.(?:...)+asks for repeats of the same two characters.re.sub()` replaces all those patterns with empty text.
The regex approach is good enough to pass that Hackerrank challenge.
You can use stack in order to achieve O(n) time complexity. Iterate over the characters in a string and for each character check if the top of stack contains the same character. In case it does pop the character from stack and move to next item. Otherwise push the character to the stack. Whatever remains in the stack is the result:
s = 'aaabccddd'
stack = []
for c in s:
if stack and stack[-1] == c:
stack.pop()
else:
stack.append(c)
print ''.join(stack) if stack else 'Empty String' # abd
Update Based on the discussion I ran couple of tests to measure the speed of regex and stack based solutions with input length of 100. Tests were run on Python 2.7 on Windows 8:
All same
Regex: 0.0563033799756
Stack: 0.267807865445
Nothing to remove
Regex: 0.075074750044
Stack: 0.183467329017
Worst case
Regex: 1.9983200193
Stack: 0.196362265609
Alphabet
Regex: 0.0759905517997
Stack: 0.182778728207
Code used for benchmarking:
import re
import timeit
def reduce_regexp(text):
even = re.compile(r'(?:([a-z])\1)+')
prev = len(text) + 1
while len(text) < prev:
prev = len(text)
text = even.sub(r'', text)
return text
def reduce_stack(s):
stack = []
for c in s:
if stack and stack[-1] == c:
stack.pop()
else:
stack.append(c)
return ''.join(stack)
CASES = [
['All same', 'a' * 100],
['Nothing to remove', 'ab' * 50],
['Worst case', 'ab' * 25 + 'ba' * 25],
['Alphabet', ''.join([chr(ord('a') + i) for i in range(25)] * 4)]
]
for name, case in CASES:
print(name)
res = timeit.timeit('reduce_regexp(case)',
setup='from __main__ import reduce_regexp, case; import re',
number=10000)
print('Regex: {}'.format(res))
res = timeit.timeit('reduce_stack(case)',
setup='from __main__ import reduce_stack, case',
number=10000)
print('Stack: {}'.format(res))

Why won't my for loop work? (Python)

Yes, this is homework. I'm just trying to understand why this doesn't seem to work.
I'm trying to find the longest substring in a string that's in alphabetical order. I make a list of random letters, and say the length is 19. When I run my code, it prints out indices 0 through 17. (I know this happens because I subtract 1 from the range) However, when I leave off that -1, it tells me the "string index is out of range." Why does that happen?
s = 'cntniymrmbhfinjttbiuqhib'
sub = ''
longest = []
for i in range(len(s) - 1):
if s[i] <= s[i+1]:
sub += s[i]
longest.append(sub)
elif s[i-1] <= s[i]:
sub += s[i]
longest.append(sub)
sub = ' '
else:
sub = ' '
print(longest)
print ('Longest substring in alphabetical order is: ' + max(longest, key=len))
I've also tried a few other methods
If I just say:
for i in s:
it throws an error, saying "string indices must be integers, not str." This seems like a much simpler way to iterate through the string, but how would I compare individual letters this way?
This is Python 2.7 by the way.
Edit: I'm sure my if/elif statements could be improved but that's the first thing I could think of. I can come back to that later if need be.
The issue is the line if s[i] <= s[i+1]:. If i=18 (the final iteration of your loop without the -1 in it). Then i+1=19 is out of bounds.
Note that the line elif s[i-1] <= s[i]: is also probably not doing what you want it to. When i=0 we have i-1 = -1. Python allows negative indices to mean counting from the back of the indexed object so s[-1] is the last character in the list (s[-2] would be the second last etc.).
A simpler way to get the previous and next character is to use zip whilst slicing the string to count from the first and second characters respectively.
zip works like this if you haven't seen it before:
>>> for char, x in zip(['a','b','c'], [1,2,3,4]):
>>> print char, x
'a' 1
'b' 2
'c' 3
So you can just do:
for previous_char, char, next_char in zip(string, string[1:], string[2:]):
To iterate over all the triples of characters without messing up at the ends.
However there is a much simpler way to do this. Instead of comparing the current character in the string to other characters in the string you should compare it with the last character in the current string of alphabetised characters for example:
s = "abcdabcdefa"
longest = [s[0]]
current = [s[0]]
for char in s[1:]:
if char >= current[-1]: # current[-1] == current[len(current)-1]
current.append(char)
else:
current=[char]
if len(longest) < len(current):
longest = current
print longest
This avoids having to do any fancy indexing.
I'm sure my if/elif statements could be improved but that's the first
thing I could think of. I can come back to that later if need be.
#or1426's solution creates a list of the currently longest sorted sequence and copies it over to longest whenever a longer sequence is found. This creates a new list every time a longer sequence is found, and appends to a list for every character. This is actually very fast in Python, but see below.
#Deej's solution keeps the currently longest sorted sequence in a string variable, and every time a longer substring is found (even if it's a continuation of the current sequence) the substring is saved to a list. The list ends up having all sorted substrings of the original string, and the longest is found by using a call to max.
Here is a faster solution that only keeps track of the indices of the currently largest sequence, and only makes changes to longest when it finds a character that is not in sorted order:
def bjorn4(s):
# we start out with s[0] being the longest sorted substring (LSS)
longest = (0, 1) # the slice-indices of the longest sorted substring
longlen = 1 # the length of longest
cur_start = 0 # the slice-indices of the *current* LSS
cur_stop = 1
for ch in s[1:]: # skip the first ch since we handled it above
end = cur_stop-1 # cur_stop is a slice index, subtract one to get the last ch in the LSS
if ch >= s[end]: # if ch >= then we're still in sorted order..
cur_stop += 1 # just extend the current LSS by one
else:
# we found a ch that is not in sorted order
if longlen < (cur_stop-cur_start):
# if the current LSS is longer than longest, then..
longest = (cur_start, cur_stop) # store current in longest
longlen = longest[1] - longest[0] # precompute longlen
# since we can't add ch to the current LSS we must create a new current around ch
cur_start, cur_stop = cur_stop, cur_stop+1
# if the LSS is at the end, then we'll not enter the else part above, so
# check for it after the for loop
if longlen < (cur_stop - cur_start):
longest = (cur_start, cur_stop)
return s[longest[0]:longest[1]]
How much faster? It's almost twice as fast as orl1426 and three times faster than deej. As always that depends on your input. The more chunks of sorted substrings that exist, the faster the above algorithm will be compared to the others. E.g. on an input string of length 100000 containing alternating 100 random chars and 100 in-order chars, I get:
bjorn4: 2.4350001812
or1426: 3.84699988365
deej : 7.13800001144
if I change it to alternating 1000 random chars and 1000 sorted chars, then I get:
bjorn4: 23.129999876
or1426: 38.8380000591
deej : MemoryError
Update:
Here is a further optimized version of my algorithm, with the comparison code:
import random, string
from itertools import izip_longest
import timeit
def _randstr(n):
ls = []
for i in range(n):
ls.append(random.choice(string.lowercase))
return ''.join(ls)
def _sortstr(n):
return ''.join(sorted(_randstr(n)))
def badstr(nish):
res = ""
for i in range(nish):
res += _sortstr(i)
if len(res) >= nish:
break
return res
def achampion(s):
start = end = longest = 0
best = ""
for c1, c2 in izip_longest(s, s[1:]):
end += 1
if c2 and c1 <= c2:
continue
if (end-start) > longest:
longest = end - start
best = s[start:end]
start = end
return best
def bjorn(s):
cur_start = 0
cur_stop = 1
long_start = cur_start
long_end = cur_stop
for ch in s[1:]:
if ch < s[cur_stop-1]:
if (long_end-long_start) < (cur_stop-cur_start):
long_start = cur_start
long_end = cur_stop
cur_start = cur_stop
cur_stop += 1
if (long_end-long_start) < (cur_stop-cur_start):
return s[cur_start:cur_stop]
return s[long_start:long_end]
def or1426(s):
longest = [s[0]]
current = [s[0]]
for char in s[1:]:
if char >= current[-1]: # current[-1] == current[len(current)-1]
current.append(char)
else:
current=[char]
if len(longest) < len(current):
longest = current
return ''.join(longest)
if __name__ == "__main__":
print 'achampion:', round(min(timeit.Timer(
"achampion(rstr)",
setup="gc.enable();from __main__ import achampion, badstr; rstr=badstr(30000)"
).repeat(15, 50)), 3)
print 'bjorn:', round(min(timeit.Timer(
"bjorn(rstr)",
setup="gc.enable();from __main__ import bjorn, badstr; rstr=badstr(30000)"
).repeat(15, 50)), 3)
print 'or1426:', round(min(timeit.Timer(
"or1426(rstr)",
setup="gc.enable();from __main__ import or1426, badstr; rstr=badstr(30000)"
).repeat(15, 50)), 3)
With output:
achampion: 0.274
bjorn: 0.253
or1426: 0.486
changing the data to be random:
achampion: 0.350
bjorn: 0.337
or1426: 0.565
and sorted:
achampion: 0.262
bjorn: 0.245
or1426: 0.503
"no, no, it's not dead, it's resting"
Now Deej has an answer I feel more comfortable posting answers to homework.
Just reordering #Deej's logic a little you can simplify to:
sub = ''
longest = []
for i in range(len(s)-1): # -1 simplifies the if condition
sub += s[i]
if s[i] <= s[i+1]:
continue # Keep adding to sub until condition fails
longest.append(sub) # Only add to longest when condition fails
sub = ''
max(longest, key=len)
But as mentioned by #thebjorn this has the issue of keeping every ascending partition in a list (in memory). You could fix this by using a generator, and I only put the rest here for instructional purposes:
def alpha_partition(s):
sub = ''
for i in range(len(s)-1):
sub += s[i]
if s[i] <= s[i+1]:
continue
yield sub
sub = ''
max(alpha_partition(s), key=len)
This certainly wont be the fastest solution (string construction and indexing) but it's quite simple to change, use zip to avoid the indexing into the string and indexes to avoid string construction and addition:
from itertools import izip_longest # For py3.X use zip_longest
def alpha_partition(s):
start = end = 0
for c1, c2 in izip_longest(s, s[1:]):
end += 1
if c2 and c1 <= c2:
continue
yield s[start:end]
start = end
max(alpha_partition(s), key=len)
Which should operate pretty efficiently and be only slightly slower than the iterative indexing approach from #thebjorn due to the generator overhead.
Using s*100
alpha_partition(): 1000 loops, best of 3: 448 µs per loop
#thebjorn: 1000 loops, best of 3: 389 µs per loop
For reference turning the generator into an iterative function:
from itertools import izip_longest # For py3.X use zip_longest
def best_alpha_partition(s):
start = end = longest = 0
best = ""
for c1, c2 in izip_longest(s, s[1:]):
end += 1
if c2 and c1 <= c2:
continue
if (end-start) > longest:
longest = end - start
best = s[start:end]
start = end
return best
best_alpha_partition(s)
best_alpha_partition(): 1000 loops, best of 3: 306 µs per loop
I personally prefer the generator form because you would use exactly the same generator for finding the minimum, the top 5, etc. very reusable vs. the iterative function which only does one thing.
ok, so after reading your responses and trying all kinds of different things, I finally came up with a solution that gets exactly what I need. It's not the prettiest code, but it works. I'm sure the solutions mentioned would work as well, however I couldn't figure them out. Here's what I did:
s = 'inaciaebganawfiaefc'
sub = ''
longest = []
for i in range(len(s)):
if (i+1) < len(s) and s[i] <= s[i+1]:
sub += s[i]
longest.append(sub)
elif i >= 0 and s[i-1] <= s[i]:
sub += s[i]
longest.append(sub)
sub = ''
else:
sub = ''
print ('Longest substring in alphabetical order is: ' + max(longest, key=len))

How do you reverse the words in a string using python (manually)? [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Reverse the ordering of words in a string
I know there are methods that python already provides for this, but I'm trying to understand the basics of how those methods work when you only have the list data structure to work with. If I have a string hello world and I want to make a new string world hello, how would I think about this?
And then, if I can do it with a new list, how would I avoid making a new list and do it in place?
Split the string, make a reverse iterator then join the parts back.
' '.join(reversed(my_string.split()))
If you are concerned with multiple spaces, change split() to split(' ')
As requested, I'm posting an implementation of split (by GvR himself from the oldest downloadable version of CPython's source code: Link)
def split(s,whitespace=' \n\t'):
res = []
i, n = 0, len(s)
while i < n:
while i < n and s[i] in whitespace:
i = i+1
if i == n:
break
j = i
while j < n and s[j] not in whitespace:
j = j+1
res.append(s[i:j])
i = j
return res
I think now there are more pythonic ways of doing that (maybe groupby) and the original source had a bug (if i = n:, corrrected to ==)
Original Answer
from array import array
def reverse_array(letters, first=0, last=None):
"reverses the letters in an array in-place"
if last is None:
last = len(letters)
last -= 1
while first < last:
letters[first], letters[last] = letters[last], letters[first]
first += 1
last -= 1
def reverse_words(string):
"reverses the words in a string using an array"
words = array('c', string)
reverse_array(words, first=0, last=len(words))
first = last = 0
while first < len(words) and last < len(words):
if words[last] != ' ':
last += 1
continue
reverse_array(words, first, last)
last += 1
first = last
if first < last:
reverse_array(words, first, last=len(words))
return words.tostring()
Answer using list to match updated question
def reverse_list(letters, first=0, last=None):
"reverses the elements of a list in-place"
if last is None:
last = len(letters)
last -= 1
while first < last:
letters[first], letters[last] = letters[last], letters[first]
first += 1
last -= 1
def reverse_words(string):
"""reverses the words in a string using a list, with each character
as a list element"""
characters = list(string)
reverse_list(characters)
first = last = 0
while first < len(characters) and last < len(characters):
if characters[last] != ' ':
last += 1
continue
reverse_list(characters, first, last)
last += 1
first = last
if first < last:
reverse_list(characters, first, last=len(characters))
return ''.join(characters)
Besides renaming, the only change of interest is the last line.
You have a string:
str = "A long string to test this algorithm"
Split the string (at word boundary -- no arguments to split):
splitted = str.split()
Reverse the array obtained -- either using ranges or a function
reversed = splitted[::-1]
Concatenate all words with spaces in between -- also known as joining.
result = " ".join(reversed)
Now, you don't need so many temps, combining them into one line gives:
result = " ".join(str.split()[::-1])
str = "hello world"
" ".join(str.split()[::-1])

Categories