I have a very long string of text with () and [] in it. I'm trying to remove the characters between the parentheses and brackets but I cannot figure out how.
The list is similar to this:
x = "This is a sentence. (once a day) [twice a day]"
This list isn't what I'm working with but is very similar and a lot shorter.
You can use re.sub function.
>>> import re
>>> x = "This is a sentence. (once a day) [twice a day]"
>>> re.sub("([\(\[]).*?([\)\]])", "\g<1>\g<2>", x)
'This is a sentence. () []'
If you want to remove the [] and the () you can use this code:
>>> import re
>>> x = "This is a sentence. (once a day) [twice a day]"
>>> re.sub("[\(\[].*?[\)\]]", "", x)
'This is a sentence. '
Important: This code will not work with nested symbols
Explanation
The first regex groups ( or [ into group 1 (by surrounding it with parentheses) and ) or ] into group 2, matching these groups and all characters that come in between them. After matching, the matched portion is substituted with groups 1 and 2, leaving the final string with nothing inside the brackets. The second regex is self explanatory from this -> match everything and substitute with the empty string.
-- modified from comment by Ajay Thomas
Run this script, it works even with nested brackets.
Uses basic logical tests.
def a(test_str):
ret = ''
skip1c = 0
skip2c = 0
for i in test_str:
if i == '[':
skip1c += 1
elif i == '(':
skip2c += 1
elif i == ']' and skip1c > 0:
skip1c -= 1
elif i == ')'and skip2c > 0:
skip2c -= 1
elif skip1c == 0 and skip2c == 0:
ret += i
return ret
x = "ewq[a [(b] ([c))]] This is a sentence. (once a day) [twice a day]"
x = a(x)
print x
print repr(x)
Just incase you don't run it,
Here's the output:
>>>
ewq This is a sentence.
'ewq This is a sentence. '
Here's a solution similar to #pradyunsg's answer (it works with arbitrary nested brackets):
def remove_text_inside_brackets(text, brackets="()[]"):
count = [0] * (len(brackets) // 2) # count open/close brackets
saved_chars = []
for character in text:
for i, b in enumerate(brackets):
if character == b: # found bracket
kind, is_close = divmod(i, 2)
count[kind] += (-1)**is_close # `+1`: open, `-1`: close
if count[kind] < 0: # unbalanced bracket
count[kind] = 0 # keep it
else: # found bracket to remove
break
else: # character is not a [balanced] bracket
if not any(count): # outside brackets
saved_chars.append(character)
return ''.join(saved_chars)
print(repr(remove_text_inside_brackets(
"This is a sentence. (once a day) [twice a day]")))
# -> 'This is a sentence. '
This should work for parentheses. Regular expressions will "consume" the text it has matched so it won't work for nested parentheses.
import re
regex = re.compile(".*?\((.*?)\)")
result = re.findall(regex, mystring)
or this would find one set of parentheses, simply loop to find more:
start = mystring.find("(")
end = mystring.find(")")
if start != -1 and end != -1:
result = mystring[start+1:end]
You can split, filter, and join the string again. If your brackets are well defined the following code should do.
import re
x = "".join(re.split("\(|\)|\[|\]", x)[::2])
You can try this. Can remove the bracket and the content exist inside it.
import re
x = "This is a sentence. (once a day) [twice a day]"
x = re.sub("\(.*?\)|\[.*?\]","",x)
print(x)
Expected ouput :
This is a sentence.
For anyone who appreciates the simplicity of the accepted answer by jvallver, and is looking for more readability from their code:
>>> import re
>>> x = 'This is a sentence. (once a day) [twice a day]'
>>> opening_braces = '\(\['
>>> closing_braces = '\)\]'
>>> non_greedy_wildcard = '.*?'
>>> re.sub(f'[{opening_braces}]{non_greedy_wildcard}[{closing_braces}]', '', x)
'This is a sentence. '
Most of the explanation for why this regex works is included in the code. Your future self will thank you for the 3 additional lines.
(Replace the f-string with the equivalent string concatenation for Python2 compatibility)
The RegEx \(.*?\)|\[.*?\] removes bracket content by finding pairs, first it remove paranthesis and then square brackets. I also works fine for the nested brackets as it acts in sequence. Ofcourse, it would break in case of bad brackets scenario.
_brackets = re.compile("\(.*?\)|\[.*?\]")
_spaces = re.compile("\s+")
_b = _brackets.sub(" ", "microRNAs (miR) play a role in cancer ([1], [2])")
_s = _spaces.sub(" ", _b.strip())
print(_s)
# OUTPUT: microRNAs play a role in cancer
This question's answers are a community effort. Edit existing answers to improve this post. It is not currently accepting new answers or interactions.
Example:
>>> convert('CamelCase')
'camel_case'
Camel case to snake case
import re
name = 'CamelCaseName'
name = re.sub(r'(?<!^)(?=[A-Z])', '_', name).lower()
print(name) # camel_case_name
If you do this many times and the above is slow, compile the regex beforehand:
pattern = re.compile(r'(?<!^)(?=[A-Z])')
name = pattern.sub('_', name).lower()
To handle more advanced cases specially (this is not reversible anymore):
def camel_to_snake(name):
name = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', name)
return re.sub('([a-z0-9])([A-Z])', r'\1_\2', name).lower()
print(camel_to_snake('camel2_camel2_case')) # camel2_camel2_case
print(camel_to_snake('getHTTPResponseCode')) # get_http_response_code
print(camel_to_snake('HTTPResponseCodeXYZ')) # http_response_code_xyz
To add also cases with two underscores or more:
def to_snake_case(name):
name = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', name)
name = re.sub('__([A-Z])', r'_\1', name)
name = re.sub('([a-z0-9])([A-Z])', r'\1_\2', name)
return name.lower()
Snake case to pascal case
name = 'snake_case_name'
name = ''.join(word.title() for word in name.split('_'))
print(name) # SnakeCaseName
There's an inflection library in the package index that can handle these things for you. In this case, you'd be looking for inflection.underscore():
>>> inflection.underscore('CamelCase')
'camel_case'
I don't know why these are all so complicating.
for most cases, the simple expression ([A-Z]+) will do the trick
>>> re.sub('([A-Z]+)', r'_\1','CamelCase').lower()
'_camel_case'
>>> re.sub('([A-Z]+)', r'_\1','camelCase').lower()
'camel_case'
>>> re.sub('([A-Z]+)', r'_\1','camel2Case2').lower()
'camel2_case2'
>>> re.sub('([A-Z]+)', r'_\1','camelCamelCase').lower()
'camel_camel_case'
>>> re.sub('([A-Z]+)', r'_\1','getHTTPResponseCode').lower()
'get_httpresponse_code'
To ignore the first character simply add look behind (?!^)
>>> re.sub('(?!^)([A-Z]+)', r'_\1','CamelCase').lower()
'camel_case'
>>> re.sub('(?!^)([A-Z]+)', r'_\1','CamelCamelCase').lower()
'camel_camel_case'
>>> re.sub('(?!^)([A-Z]+)', r'_\1','Camel2Camel2Case').lower()
'camel2_camel2_case'
>>> re.sub('(?!^)([A-Z]+)', r'_\1','getHTTPResponseCode').lower()
'get_httpresponse_code'
If you want to separate ALLCaps to all_caps and expect numbers in your string you still don't need to do two separate runs just use | This expression ((?<=[a-z0-9])[A-Z]|(?!^)[A-Z](?=[a-z])) can handle just about every scenario in the book
>>> a = re.compile('((?<=[a-z0-9])[A-Z]|(?!^)[A-Z](?=[a-z]))')
>>> a.sub(r'_\1', 'getHTTPResponseCode').lower()
'get_http_response_code'
>>> a.sub(r'_\1', 'get2HTTPResponseCode').lower()
'get2_http_response_code'
>>> a.sub(r'_\1', 'get2HTTPResponse123Code').lower()
'get2_http_response123_code'
>>> a.sub(r'_\1', 'HTTPResponseCode').lower()
'http_response_code'
>>> a.sub(r'_\1', 'HTTPResponseCodeXYZ').lower()
'http_response_code_xyz'
It all depends on what you want so use the solution that best suits your needs as it should not be overly complicated.
nJoy!
Avoiding libraries and regular expressions:
def camel_to_snake(s):
return ''.join(['_'+c.lower() if c.isupper() else c for c in s]).lstrip('_')
>>> camel_to_snake('ThisIsMyString')
'this_is_my_string'
stringcase is my go-to library for this; e.g.:
>>> from stringcase import pascalcase, snakecase
>>> snakecase('FooBarBaz')
'foo_bar_baz'
>>> pascalcase('foo_bar_baz')
'FooBarBaz'
I think this solution is more straightforward than previous answers:
import re
def convert (camel_input):
words = re.findall(r'[A-Z]?[a-z]+|[A-Z]{2,}(?=[A-Z][a-z]|\d|\W|$)|\d+', camel_input)
return '_'.join(map(str.lower, words))
# Let's test it
test_strings = [
'CamelCase',
'camelCamelCase',
'Camel2Camel2Case',
'getHTTPResponseCode',
'get200HTTPResponseCode',
'getHTTP200ResponseCode',
'HTTPResponseCode',
'ResponseHTTP',
'ResponseHTTP2',
'Fun?!awesome',
'Fun?!Awesome',
'10CoolDudes',
'20coolDudes'
]
for test_string in test_strings:
print(convert(test_string))
Which outputs:
camel_case
camel_camel_case
camel_2_camel_2_case
get_http_response_code
get_200_http_response_code
get_http_200_response_code
http_response_code
response_http
response_http_2
fun_awesome
fun_awesome
10_cool_dudes
20_cool_dudes
The regular expression matches three patterns:
[A-Z]?[a-z]+: Consecutive lower-case letters that optionally start with an upper-case letter.
[A-Z]{2,}(?=[A-Z][a-z]|\d|\W|$): Two or more consecutive upper-case letters. It uses a lookahead to exclude the last upper-case letter if it is followed by a lower-case letter.
\d+: Consecutive numbers.
By using re.findall we get a list of individual "words" that can be converted to lower-case and joined with underscores.
Personally I am not sure how anything using regular expressions in python can be described as elegant. Most answers here are just doing "code golf" type RE tricks. Elegant coding is supposed to be easily understood.
def to_snake_case(not_snake_case):
final = ''
for i in xrange(len(not_snake_case)):
item = not_snake_case[i]
if i < len(not_snake_case) - 1:
next_char_will_be_underscored = (
not_snake_case[i+1] == "_" or
not_snake_case[i+1] == " " or
not_snake_case[i+1].isupper()
)
if (item == " " or item == "_") and next_char_will_be_underscored:
continue
elif (item == " " or item == "_"):
final += "_"
elif item.isupper():
final += "_"+item.lower()
else:
final += item
if final[0] == "_":
final = final[1:]
return final
>>> to_snake_case("RegularExpressionsAreFunky")
'regular_expressions_are_funky'
>>> to_snake_case("RegularExpressionsAre Funky")
'regular_expressions_are_funky'
>>> to_snake_case("RegularExpressionsAre_Funky")
'regular_expressions_are_funky'
''.join('_'+c.lower() if c.isupper() else c for c in "DeathToCamelCase").strip('_')
re.sub("(.)([A-Z])", r'\1_\2', 'DeathToCamelCase').lower()
Here's my solution:
def un_camel(text):
""" Converts a CamelCase name into an under_score name.
>>> un_camel('CamelCase')
'camel_case'
>>> un_camel('getHTTPResponseCode')
'get_http_response_code'
"""
result = []
pos = 0
while pos < len(text):
if text[pos].isupper():
if pos-1 > 0 and text[pos-1].islower() or pos-1 > 0 and \
pos+1 < len(text) and text[pos+1].islower():
result.append("_%s" % text[pos].lower())
else:
result.append(text[pos].lower())
else:
result.append(text[pos])
pos += 1
return "".join(result)
It supports those corner cases discussed in the comments. For instance, it'll convert getHTTPResponseCode to get_http_response_code like it should.
I don't get idea why using both .sub() calls? :) I'm not regex guru, but I simplified function to this one, which is suitable for my certain needs, I just needed a solution to convert camelCasedVars from POST request to vars_with_underscore:
def myFunc(...):
return re.sub('(.)([A-Z]{1})', r'\1_\2', "iTriedToWriteNicely").lower()
It does not work with such names like getHTTPResponse, cause I heard it is bad naming convention (should be like getHttpResponse, it's obviously, that it's much easier memorize this form).
For the fun of it:
>>> def un_camel(input):
... output = [input[0].lower()]
... for c in input[1:]:
... if c in ('ABCDEFGHIJKLMNOPQRSTUVWXYZ'):
... output.append('_')
... output.append(c.lower())
... else:
... output.append(c)
... return str.join('', output)
...
>>> un_camel("camel_case")
'camel_case'
>>> un_camel("CamelCase")
'camel_case'
Or, more for the fun of it:
>>> un_camel = lambda i: i[0].lower() + str.join('', ("_" + c.lower() if c in "ABCDEFGHIJKLMNOPQRSTUVWXYZ" else c for c in i[1:]))
>>> un_camel("camel_case")
'camel_case'
>>> un_camel("CamelCase")
'camel_case'
Using regexes may be the shortest, but this solution is way more readable:
def to_snake_case(s):
snake = "".join(["_"+c.lower() if c.isupper() else c for c in s])
return snake[1:] if snake.startswith("_") else snake
This is not a elegant method, is a very 'low level' implementation of a simple state machine (bitfield state machine), possibly the most anti pythonic mode to resolve this, however re module also implements a too complex state machine to resolve this simple task, so i think this is a good solution.
def splitSymbol(s):
si, ci, state = 0, 0, 0 # start_index, current_index
'''
state bits:
0: no yields
1: lower yields
2: lower yields - 1
4: upper yields
8: digit yields
16: other yields
32 : upper sequence mark
'''
for c in s:
if c.islower():
if state & 1:
yield s[si:ci]
si = ci
elif state & 2:
yield s[si:ci - 1]
si = ci - 1
state = 4 | 8 | 16
ci += 1
elif c.isupper():
if state & 4:
yield s[si:ci]
si = ci
if state & 32:
state = 2 | 8 | 16 | 32
else:
state = 8 | 16 | 32
ci += 1
elif c.isdigit():
if state & 8:
yield s[si:ci]
si = ci
state = 1 | 4 | 16
ci += 1
else:
if state & 16:
yield s[si:ci]
state = 0
ci += 1 # eat ci
si = ci
print(' : ', c, bin(state))
if state:
yield s[si:ci]
def camelcaseToUnderscore(s):
return '_'.join(splitSymbol(s))
splitsymbol can parses all case types: UpperSEQUENCEInterleaved, under_score, BIG_SYMBOLS and cammelCasedMethods
I hope it is useful
Take a look at the excellent Schematics lib
https://github.com/schematics/schematics
It allows you to created typed data structures that can serialize/deserialize from python to Javascript flavour, eg:
class MapPrice(Model):
price_before_vat = DecimalType(serialized_name='priceBeforeVat')
vat_rate = DecimalType(serialized_name='vatRate')
vat = DecimalType()
total_price = DecimalType(serialized_name='totalPrice')
So many complicated methods...
Just find all "Titled" group and join its lower cased variant with underscore.
>>> import re
>>> def camel_to_snake(string):
... groups = re.findall('([A-z0-9][a-z]*)', string)
... return '_'.join([i.lower() for i in groups])
...
>>> camel_to_snake('ABCPingPongByTheWay2KWhereIsOurBorderlands3???')
'a_b_c_ping_pong_by_the_way_2_k_where_is_our_borderlands_3'
If you don't want make numbers like first character of group or separate group - you can use ([A-z][a-z0-9]*) mask.
A horrendous example using regular expressions (you could easily clean this up :) ):
def f(s):
return s.group(1).lower() + "_" + s.group(2).lower()
p = re.compile("([A-Z]+[a-z]+)([A-Z]?)")
print p.sub(f, "CamelCase")
print p.sub(f, "getHTTPResponseCode")
Works for getHTTPResponseCode though!
Alternatively, using lambda:
p = re.compile("([A-Z]+[a-z]+)([A-Z]?)")
print p.sub(lambda x: x.group(1).lower() + "_" + x.group(2).lower(), "CamelCase")
print p.sub(lambda x: x.group(1).lower() + "_" + x.group(2).lower(), "getHTTPResponseCode")
EDIT: It should also be pretty easy to see that there's room for improvement for cases like "Test", because the underscore is unconditionally inserted.
Lightely adapted from https://stackoverflow.com/users/267781/matth
who use generators.
def uncamelize(s):
buff, l = '', []
for ltr in s:
if ltr.isupper():
if buff:
l.append(buff)
buff = ''
buff += ltr
l.append(buff)
return '_'.join(l).lower()
This simple method should do the job:
import re
def convert(name):
return re.sub(r'([A-Z]*)([A-Z][a-z]+)', lambda x: (x.group(1) + '_' if x.group(1) else '') + x.group(2) + '_', name).rstrip('_').lower()
We look for capital letters that are precedeed by any number of (or zero) capital letters, and followed by any number of lowercase characters.
An underscore is placed just before the occurence of the last capital letter found in the group, and one can be placed before that capital letter in case it is preceded by other capital letters.
If there are trailing underscores, remove those.
Finally, the whole result string is changed to lower case.
(taken from here, see working example online)
Here's something I did to change the headers on a tab-delimited file. I'm omitting the part where I only edited the first line of the file. You could adapt it to Python pretty easily with the re library. This also includes separating out numbers (but keeps the digits together). I did it in two steps because that was easier than telling it not to put an underscore at the start of a line or tab.
Step One...find uppercase letters or integers preceded by lowercase letters, and precede them with an underscore:
Search:
([a-z]+)([A-Z]|[0-9]+)
Replacement:
\1_\l\2/
Step Two...take the above and run it again to convert all caps to lowercase:
Search:
([A-Z])
Replacement (that's backslash, lowercase L, backslash, one):
\l\1
I was looking for a solution to the same problem, except that I needed a chain; e.g.
"CamelCamelCamelCase" -> "Camel-camel-camel-case"
Starting from the nice two-word solutions here, I came up with the following:
"-".join(x.group(1).lower() if x.group(2) is None else x.group(1) \
for x in re.finditer("((^.[^A-Z]+)|([A-Z][^A-Z]+))", "stringToSplit"))
Most of the complicated logic is to avoid lowercasing the first word. Here's a simpler version if you don't mind altering the first word:
"-".join(x.group(1).lower() for x in re.finditer("(^[^A-Z]+|[A-Z][^A-Z]+)", "stringToSplit"))
Of course, you can pre-compile the regular expressions or join with underscore instead of hyphen, as discussed in the other solutions.
Concise without regular expressions, but HTTPResponseCode=> httpresponse_code:
def from_camel(name):
"""
ThisIsCamelCase ==> this_is_camel_case
"""
name = name.replace("_", "")
_cas = lambda _x : [_i.isupper() for _i in _x]
seq = zip(_cas(name[1:-1]), _cas(name[2:]))
ss = [_x + 1 for _x, (_i, _j) in enumerate(seq) if (_i, _j) == (False, True)]
return "".join([ch + "_" if _x in ss else ch for _x, ch in numerate(name.lower())])
Without any library :
def camelify(out):
return (''.join(["_"+x.lower() if i<len(out)-1 and x.isupper() and out[i+1].islower()
else x.lower()+"_" if i<len(out)-1 and x.islower() and out[i+1].isupper()
else x.lower() for i,x in enumerate(list(out))])).lstrip('_').replace('__','_')
A bit heavy, but
CamelCamelCamelCase -> camel_camel_camel_case
HTTPRequest -> http_request
GetHTTPRequest -> get_http_request
getHTTPRequest -> get_http_request
Very nice RegEx proposed on this site:
(?<!^)(?=[A-Z])
If python have a String Split method, it should work...
In Java:
String s = "loremIpsum";
words = s.split("(?<!^)(?=[A-Z])");
Just in case someone needs to transform a complete source file, here is a script that will do it.
# Copy and paste your camel case code in the string below
camelCaseCode ="""
cv2.Matx33d ComputeZoomMatrix(const cv2.Point2d & zoomCenter, double zoomRatio)
{
auto mat = cv2.Matx33d::eye();
mat(0, 0) = zoomRatio;
mat(1, 1) = zoomRatio;
mat(0, 2) = zoomCenter.x * (1. - zoomRatio);
mat(1, 2) = zoomCenter.y * (1. - zoomRatio);
return mat;
}
"""
import re
def snake_case(name):
s1 = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', name)
return re.sub('([a-z0-9])([A-Z])', r'\1_\2', s1).lower()
def lines(str):
return str.split("\n")
def unlines(lst):
return "\n".join(lst)
def words(str):
return str.split(" ")
def unwords(lst):
return " ".join(lst)
def map_partial(function):
return lambda values : [ function(v) for v in values]
import functools
def compose(*functions):
return functools.reduce(lambda f, g: lambda x: f(g(x)), functions, lambda x: x)
snake_case_code = compose(
unlines ,
map_partial(unwords),
map_partial(map_partial(snake_case)),
map_partial(words),
lines
)
print(snake_case_code(camelCaseCode))
Wow I just stole this from django snippets. ref http://djangosnippets.org/snippets/585/
Pretty elegant
camelcase_to_underscore = lambda str: re.sub(r'(?<=[a-z])[A-Z]|[A-Z](?=[^A-Z])', r'_\g<0>', str).lower().strip('_')
Example:
camelcase_to_underscore('ThisUser')
Returns:
'this_user'
REGEX DEMO
def convert(name):
return reduce(
lambda x, y: x + ('_' if y.isupper() else '') + y,
name
).lower()
And if we need to cover a case with already-un-cameled input:
def convert(name):
return reduce(
lambda x, y: x + ('_' if y.isupper() and not x.endswith('_') else '') + y,
name
).lower()
Not in the standard library, but I found this module that appears to contain the functionality you need.
If you use Google's (nearly) deterministic Camel case algorithm, then one does not need to handle things like HTMLDocument since it should be HtmlDocument, then this regex based approach is simple. It replace all capitals or numbers with an underscore. Note does not handle multi digit numbers.
import re
def to_snake_case(camel_str):
return re.sub('([A-Z0-9])', r'_\1', camel_str).lower().lstrip('_')
def convert(camel_str):
temp_list = []
for letter in camel_str:
if letter.islower():
temp_list.append(letter)
else:
temp_list.append('_')
temp_list.append(letter)
result = "".join(temp_list)
return result.lower()
Use: str.capitalize() to convert first letter of the string (contained in variable str) to a capital letter and returns the entire string.
Example:
Command: "hello".capitalize()
Output: Hello
In a python application, I need to replace adjacent identical occurrences of whitespace separated tokens that match a regex, e.g. for a pattern such as "a\w\w"
"xyz abc abc zzq ak9 ak9 ak9 foo abc" --> "xyz abc*2 zzq ak9*3 foo bar abc"
EDIT
My example above didn't make it clear that tokens which don't match the regex should not be aggregated. A better example is
"xyz xyz abc abc zzq ak9 ak9 ak9 foo foo abc"
--> "xyz xyz abc*2 zzq ak9*3 foo foo bar abc"
END EDIT
I have working code posted below, but it seems more complicated than it should be.
I'm not looking for a round of code golf, but I would be interested in a solution that's more readable using standard Python libraries with similar performance.
In my application, it's safe to assume that the input strings will be less than 10000 chars long and that any given string will contain only a handful, say < 10, of the possible strings that match the pattern.
import re
def fm_pattern_factory(ptnstring):
"""
Return a regex that matches two or more occurrences
of ptnstring separated by whitespace.
>>> fm_pattern_factory('abc').match(' abc abc ') is None
False
>>> fm_pattern_factory('abc').match('abc') is None
True
"""
ptn = r"\s*({}(?:\s+{})+)\s*".format(ptnstring, ptnstring)
return re.compile(ptn)
def fm_gather(target, ptnstring):
"""
Replace adjacent occurences of ptnstring in target with
ptnstring*N where n is the number occurrences.
>>> fm_gather('xyz abc abc def abc', 'abc')
'xyz abc*2 def abc'
>>> fm_gather('xyz abc abc def abc abc abc qrs', 'abc')
'xyz abc*2 def abc*3 qrs'
"""
ptn = fm_pattern_factory(ptnstring)
result = []
index = 0
for match in ptn.finditer(target):
result.append(target[index:match.start()+1])
repl = "{}*{}".format(ptnstring, match.group(1).count(ptnstring))
result.append(repl)
index = match.end() - 1
result.append(target[index:])
return "".join(result)
def fm_gather_all(target, ptn):
"""
Apply fm_gather() to all distinct matches for ptn.
>>> s = "x abc abc y abx abx z acq"
>>> ptn = re.compile(r"a..")
>>> fm_gather_all(s, ptn)
'x abc*2 y abx*2 z acq'
"""
ptns = set(ptn.findall(target))
for p in ptns:
target = fm_gather(target, p)
return "".join(target)
Sorry, I was working on the answer before seeing you first comment. If this doesn't answer your question, let me know, and I'll remove it or will try to modify it accordingly.
For the simple input provided in the question (what in the code below is stored in the my_string variable), you could maybe try a different approach: Walk your input list and keep a "bucket" of <matching_word, num_of_occurrences>:
my_string="xyz abc abc zzq ak9 ak9 ak9 foo abc"
my_splitted_string=my_string.split(' ')
occurrences = []
print ("my_splitted_string is a %s now containing: %s"
% (type(my_splitted_string), my_splitted_string))
current_bucket = [my_splitted_string[0], 1]
occurrences.append(current_bucket)
for i in range(1, len(my_splitted_string)):
current_word = my_splitted_string[i]
print "Does %s match %s?" % (current_word, current_bucket[0])
if current_word == current_bucket[0]:
current_bucket[1] += 1
print "It does. Aggregating"
else:
current_bucket = [current_word, 1]
occurrences.append(current_bucket)
print "It doesn't. Creating a new 'bucket'"
print "Collected occurrences: %s" % occurrences
# Now re-collect:
re_collected_str=""
for occurrence in occurrences:
if occurrence[1] > 1:
re_collected_str += "%s*%d " % (occurrence[0], occurrence[1])
else:
re_collected_str += "%s " % (occurrence[0])
print "Compressed string: '%s'"
This outputs:
my_splitted_string is a <type 'list'> now containing: ['xyz', 'abc', 'abc', 'zzq', 'ak9', 'ak9', 'ak9', 'foo', 'abc']
Does abc match xyz?
It doesn't. Creating a new 'bucket'
Does abc match abc?
It does. Aggregating
Does zzq match abc?
It doesn't. Creating a new 'bucket'
Does ak9 match zzq?
It doesn't. Creating a new 'bucket'
Does ak9 match ak9?
It does. Aggregating
Does ak9 match ak9?
It does. Aggregating
Does foo match ak9?
It doesn't. Creating a new 'bucket'
Does abc match foo?
It doesn't. Creating a new 'bucket'
Collected occurrences: [['xyz', 1], ['abc', 2], ['zzq', 1], ['ak9', 3], ['foo', 1], ['abc', 1]]
Compressed string: 'xyz abc*2 zzq ak9*3 foo abc '
(beware of the final blank space)
The following seems to be solid and has good performance in my application. Thanks to BorrajaX for an answer that pointed out benefits of not scanning the input string more often than absolutely necessary.
The function below also preserves newlines and whitespace in the output. I forgot to state that in my question, but it turns out to be desirable in my app which needs to produce some human-readable intermediate output.
def gather_token_sequences(masterptn, target):
"""
Find all sequences in 'target' of two or more identical adjacent tokens
that match 'masterptn'. Count the number of tokens in each sequence.
Return a new version of 'target' with each sequence replaced by one token
suffixed with '*N' where N is the count of tokens in the sequence.
Whitespace in the input is preserved (except where consumed within replaced
sequences).
>>> mptn = r'ab\w'
>>> tgt = 'foo abc abc'
>>> gather_token_sequences(mptn, tgt)
'foo abc*2'
>>> tgt = 'abc abc '
>>> gather_token_sequences(mptn, tgt)
'abc*2 '
>>> tgt = '\\nabc\\nabc abc\\ndef\\nxyz abx\\nabx\\nxxx abc'
>>> gather_token_sequences(mptn, tgt)
'\\nabc*3\\ndef\\nxyz abx*2\\nxxx abc'
"""
# Emulate python's strip() function except that the leading and trailing
# whitespace are captured for final output. This guarantees that the
# body of the remaining string will start and end with a token, which
# slightly simplifies the subsequent matching loops.
stripped = re.match(r'^(\s*)(\S.*\S)(\s*)$', target, flags=re.DOTALL)
head, body, tail = stripped.groups()
# Init the result list and loop variables.
result = [head]
i = 0
token = None
while i < len(body):
## try to match master pattern
match = re.match(masterptn, body[i:])
if match is None:
## Append char and advance.
result += body[i]
i += 1
else:
## Start new token sequence
token = match.group(0)
esc = re.escape(token) # might have special chars in token
ptn = r"((?:{}\s+)+{})".format(esc, esc)
seq = re.match(ptn, body[i:])
if seq is None: # token is not repeated.
result.append(token)
i += len(token)
else:
seqstring = seq.group(0)
replacement = "{}*{}".format(token, seqstring.count(token))
result.append(replacement)
i += len(seq.group(0))
result.append(tail)
return ''.join(result)
I have a name as a string, in this example "markus johansson".
I'm trying to code a program that makes 'm' and 'j' uppercase:
name = "markus johansson"
for i in range(1, len(name)):
if name[0] == 'm':
name[0] = "M"
if name[i] == " ":
count = name[i] + 1
if count == 'j':
name[count] = 'J'
I'm pretty sure this should work, but it gives me this error:
File "main.py", line 5 in <module>
name[0] = "M"
TypeError: 'str' object does support item assignment
I know there is a library function called .title(), but I want to do "real programming".
How do I fix this?
I guess that what you're trying to achieve is:
from string import capwords
capwords(name)
Which yields:
'Markus Johansson'
EDIT: OK, I see you want to tear down a open door.
Here's low level implementation.
''.join([char.upper() if prev==' ' else char for char,prev in zip(name,' '+name)])
>>> "markus johansson".title()
'Markus Johansson'
Built in string methods are the way to go.
EDIT:
I see you want to re-invent the wheel. Any particular reason ?
You can choose from any number of convoluted methods like:
' '.join(j[0].upper()+j[1:] for j in "markus johansson".split())
Standard Libraries are still the way to go.
string.capwords() (defined in string.py)
# Capitalize the words in a string, e.g. " aBc dEf " -> "Abc Def".
def capwords(s, sep=None):
"""capwords(s, [sep]) -> string
Split the argument into words using split, capitalize each
word using capitalize, and join the capitalized words using
join. Note that this replaces runs of whitespace characters by
a single space.
"""
return (sep or ' ').join(x.capitalize() for x in s.split(sep))
str.title() (defined in stringobject.c)
PyDoc_STRVAR(title__doc__,
"S.title() -> string\n\
\n\
Return a titlecased version of S, i.e. words start with uppercase\n\
characters, all remaining cased characters have lowercase.");
static PyObject*
string_title(PyStringObject *self)
{
char *s = PyString_AS_STRING(self), *s_new;
Py_ssize_t i, n = PyString_GET_SIZE(self);
int previous_is_cased = 0;
PyObject *newobj = PyString_FromStringAndSize(NULL, n);
if (newobj == NULL)
return NULL;
s_new = PyString_AsString(newobj);
for (i = 0; i < n; i++) {
int c = Py_CHARMASK(*s++);
if (islower(c)) {
if (!previous_is_cased)
c = toupper(c);
previous_is_cased = 1;
} else if (isupper(c)) {
if (previous_is_cased)
c = tolower(c);
previous_is_cased = 1;
} else
previous_is_cased = 0;
*s_new++ = c;
}
return newobj;
}
str.title() in pure Python
class String(str):
def title(self):
s = []
previous_is_cased = False
for c in self:
if c.islower():
if not previous_is_cased:
c = c.upper()
previous_is_cased = True
elif c.isupper():
if previous_is_cased:
c = c.lower()
previous_is_cased = True
else:
previous_is_cased = False
s.append(c)
return ''.join(s)
Example:
>>> s = ' aBc dEf '
>>> import string
>>> string.capwords(s)
'Abc Def'
>>> s.title()
' Abc Def '
>>> s
' aBc dEf '
>>> String(s).title()
' Abc Def '
>>> String(s).title() == s.title()
True
Strings are immutable. They can't be changed. You must create a new string with the changed content.
If you want to make every 'j' uppercase:
def make_uppercase_j(char):
if char == 'j':
return 'J'
else:
return char
name = "markus johansson"
''.join(make_uppercase_j(c) for c in name)
If you're looking into more generic solution for names, you should also look at following examples:
John Adams-Smith
Joanne d'Arc
Jean-Luc de'Breu
Donatien Alphonse François de Sade
Also some parts of the names shouldn't start with capital letters, like:
Herbert von Locke
Sander van Dorn
Edwin van der Sad
so, if you're looking into creating a more generic solution, keep all those little things in mind.
(This would be a perfect place to run a test-driven development, with all those conditions your method/function must follow).
If I understand your original algorithm correctly, this is what you want to do:
namn = list("markus johansson")
if namn[0] == 'm':
namn[0] = "M"
count = 0
for i in range(1, len(namn)):
if namn[i] == " ":
count = i + 1
if count and namn[count] == 'j':
namn[count] = 'J'
print ''.join(namn)
Of course, there's a million better ways ("wannabe" ways) to do what you're trying to do, like as shown in vartec's answer. :)
As it stands, your code only works for names that start with a J and an M for the first and last names, respectively.
Plenty of good suggestions, so I'll be in good company adding my own 2 cents :-)
I'm assuming you want something a little more generic that can handle more than just names starting with 'm' and 'j'. You'll probably also want to consider hyphenated names (like Markus Johnson-Smith) which have caps after the hyphen too.
from string import lowercase, uppercase
name = 'markus johnson-smith'
state = 0
title_name = []
for c in name:
if c in lowercase and not state:
c = uppercase[lowercase.index(c)]
state = 1
elif c in [' ', '-']:
state = 0
else:
state = 1 # might already be uppercase
title_name.append(c)
print ''.join(title_name)
Last caveat is the potential for non-ascii characters. Using the uppercase and lowercase properties of the string module is good in this case becase their contents change depending on the user's locale (ie: system-dependent, or when locale.setlocale() is called). I know you want to avoid using upper() for this exercise, and that's quite neat... as an FYI, upper() uses the locale controlled by setlocale() too, so the practice of use uppercase and lowercase is a good use of the API without getting too high-level. That said, if you need to handle, say, French names on a system running an English locale, you'll need a more robust implementation.
"real programming"?
I would use .title(), and I'm a real programmer.
Or I would use regular expressions
re.sub(r"(^|\s)[a-z]", lambda m: m.group(0).upper(), "this is a set of words")
This says "If the start of the text or a whitespace character is followed by a lower-case letter" (in English - other languages are likely not supported), then for each match convert the match text to upper-case. Since the match text is the space and the lower-case letter, this works just fine.
If you want it as low-level code then the following works. Here I only allow space as the separator (but you may want to support newline and other characters). On the other hand, "string.lowercase" is internationalized, so if you're in another locale then it will, for the most part, still work. If you don't want that then use string.ascii_lowercase.
import string
def title(s):
# Capitalize the first character
if s[:1] in string.lowercase:
s = s[0].upper() + s[1:]
# Find spaces
offset = 0
while 1:
offset = s.find(" ", offset)
# Reached the end of the string or the
# last character is a space
if offset == -1 or offset == len(s)-1:
break
if s[offset+1:offset+2] in string.lowercase:
# Is it followed by a lower-case letter?
s = s[:offset+1] + s[offset+1].upper() + s[offset+2:]
# Skip the space and the letter
offset += 2
else:
# Nope, so start searching for the next space
offset += 1
return s
To elaborate on my comment to this answer, this question can only be an exercise for curiosity's sake. Real names have special capitalization rules: the "van der" in "Johannes Diderik van der Waals" is never capitalized, "Farrah Fawcett-Majors" has the "M", and "Cathal Ó hEochaidh" uses the non-ASCII Ó and h, which modify "Eochaidh" to mean "grandson of Eochaidh".
string = 'markus johansson'
string = ' '.join(substring[0].upper() + substring[1:] for substring in string.split(' '))
# string == 'Markus Johansson'