I'm wondering if there's any way in python or perl to build a regex where you can define a set of options can appear at most once in any order. So for example I would like a derivative of foo(?: [abc])*, where a, b, c could only appear once. So:
foo a b c
foo b c a
foo a b
foo b
would all be valid, but
foo b b
would not be
You may use this regex with a capture group and a negative lookahead:
For Perl, you can use this variant with forward referencing:
^foo((?!.*\1) [abc])+$
RegEx Demo
RegEx Details:
^: Start
foo: Match foo
(: Start a capture group #1
(?!.*\1): Negative lookahead to assert that we don't match what we have in capture group #1 anywhere in input
[abc]: Match a space followed by a or b or c
)+: End capture group #1. Repeat this group 1+ times
$: End
As mentioned earlier, this regex is using a feature called Forward Referencing which is a back-reference to a group that appears later in the regex pattern. JGsoft, .NET, Java, Perl, PCRE, PHP, Delphi, and Ruby allow forward references but Python doesn't.
Here is a work-around of same regex for Python that doesn't use forward referencing:
^foo(?!.* ([abc]).*\1)(?: [abc])+$
Here we use a negative lookahead before repeated group to check and fail the match if there is any repeat of allowed substrings i.e. [abc].
RegEx Demo 2
You can assert that there is no match for a second match for a space and a letter at the right:
foo(?!(?: [abc])*( [abc])(?: [abc])*\1)(?: [abc])*
foo Match literally
(?! Negative lookahead
(?: [abc])* Match optional repetitions of a space and a b or c
( [abc]) Capture group, use to compare with a backreference for the same
(?: [abc])* Match again a space and either a b or c
\1 Backreference to group 1
) Close lookahead
(?: [abc])* Match optional repetitions or a space and either a b or c
Regex demo
If you don't want to match only foo, you can change the quantifier to 1 or more (?: [abc])+
A variant in perl reusing the first subpattern using (?1) which refers to the capture group ([abc])
^foo ([abc])(?: (?!\1)((?1))(?: (?!\1|\2)(?1))?)?$
Regex demo
If it doesn't have to be a regex:
import collections
# python >=3.10
def is_a_match(sentence):
words = sentence.split()
return (
(len(words) > 0)
and (words[0] == 'foo')
and (collections.Counter(words) <= collections.Counter(['foo', 'a', 'b', 'c']))
)
# python <3.10
def is_a_match(sentence):
words = sentence.split()
return (
(len(words) > 0)
and (words[0] == 'foo')
and not (collections.Counter(words) - collections.Counter(['foo', 'a', 'b', 'c']))
)
# TESTING
#foo a b c True
#foo b c a True
#foo a b True
#foo b True
#foo b b False
Or with a set and the walrus operator:
def is_a_match(sentence):
words = sentence.split()
return (
(len(words) > 0)
and (words[0] == 'foo')
and (
(s := set(words[1:])) <= set(['a', 'b', 'c'])
and len(s) == len(words) - 1
)
)
You can do it using references to previously captured groups.
foo(?: ([abc]))?(?: (?!\1)([abc]))?(?: (?!\1|\2)([abc]))?$
This gets quite long with many options. Such a regex can be generated dynamically, if necessary.
def match_sequence_without_repeats(options, seperator):
def prevent_previous(n):
if n == 0:
return ""
groups = "".join(rf"\{i}" for i in range(1, n + 1))
return f"(?!{groups})"
return "".join(
f"(?:{seperator}{prevent_previous(i)}([{options}]))?"
for i in range(len(options))
)
print(f"foo{match_sequence_without_repeats('abc', ' ')}$")
Here is a modified version of anubhava's answer, using a backreference (which works in Python, and is easier to understand at least for me) instead of a forward reference.
Match using [abc] inside a capturing group, then check that the text matched by the capturing group does not appear again anywhere after it:
^foo(?:( [abc])(?!.*\1))+$
regex demo
^: Start
foo: Match foo
(?:: Start non-capturing group (?:( [abc])(?!.*\1))
( [abc]): Capturing Group 1, matching a space followed by either a, b, or c
(?!.*\1): Negative lookahead, failing to match if the text matched by the first capturing group occurs after zero or more characters matched by .
)+: End non-capturing group and match it 1 or more times
$: End
I have assumed that the elements of the string can be in any order and appear any number of times. For example, 'a foo' should match and 'a foo b foo' should not.
You can do that with a series of alternations employing lookaheads, one for each substring of interest, but it becomes a bit of a dog's breakfast when there are many strings to consider. Let's suppose you wanted to match zero or one "foo"'s and/or zero or one "a"'s. You could use the following regular expression:
^(?:(?!.*\bfoo\b)|(?=(?:(?!\bfoo\b).)*\bfoo\b(?!(.*\bfoo\b))))(?:(?!.*\ba\b)|(?=(?:(?!\ba\b).)*\ba\b(?!(.*\ba\b))))
Start your engine!
This matches, for example, 'foofoo', 'aa' and afooa. If they are not to be matched remove the word breaks (\b).
Notice that this expression begins by asserting the start of the string (^) followed by two positive lookaheads, one for 'foo' and one for 'a'. To also check for, say, 'c' one would tack on
(?:(?!.*\bc\b)|(?=(?:(?!\bc\b).)*\bc\b(?!(.*\bc\b))))
which is the same as
(?:(?!.*\ba\b)|(?=(?:(?!\ba\b).)*\ba\b(?!(.*\ba\b))))
with \ba\b changed to \bc\b.
It would be nice to be able to use back-references but I don't see how that could be done.
By hovering over the regular expression in the link an explanation is provided for each element of the expression. (If this is not clear I am referring to the cursor.)
Note that
(?!\bfoo\b).
matches a character provided it does not begin the word 'foo'. Therefore
(?:(?!\bfoo\b).)*
matches a substring that does not contain 'foo' and does not end with 'f' followed by 'oo'.
Would I advocate this approach in practice, as opposed to using simple string methods? Let me ponder that.
If the order of the strings doesn't matter, and you want to make sure every string occurs only once, you can turn the list into a set in Python:
my_lst = ['a', 'a', 'b', 'c']
my_set = set(lst)
print(my_set)
# {'a', 'c', 'b'}
There is not much to add to the above answers except here is a regex that does not use back or forward references. Instead it uses 3 separate negative lookahead assertions to ensure that the input does not contain 2 occurrences of either a or b or c. The regex also allows for liberal uses of spaces.
^foo(?![^a]*a[^a]*a)(?![^b]*b[^b]*b)(?![^c]*c[^c]*c)( +[abc])* *$
See Regex Demo
^ - Matches start of string
(?![^a]*a[^a]*a) - Negative lookahead assertion that what follows does not contain two occurrences of a
(?![^b]*b[^b]*b) - Negative lookahead assertion that what follows does not contain two occurrences of b
(?![^c]*c[^c]*c) - Negative lookahead assertion that what follows does not contain two occurrences of c
( +[abc])* - Matches 0 or more occurrences of: 1 or more spaces followed by an a or b or c
* - Matches 0 or more occurrences of space
7 $ - Matches the end of the string
The regex looks "clunky" but is very straightforward. With input foo a b c the successful match is done in 35 steps and with input foo b b the unsuccessful match is done in 13 steps. Thich compares favorably with the other answers.
This is two texts:
1) 'provider:sipoutilp1.ym.ms'
2) 'provider:sipoutqtm.ym.ms'
I would like to print ilp when reaches to the fist line and qtm when reaches to the second line.
This is my solution but it is not working.
RE_PROVIDER = re.compile(r'(?P<provider>\((ilp+|qtm+)')
or in the line below,
182938,DOMINICAN REPUBLIC-MOBILE
to DOMINICAN REPUBLIC , can I use the same approach re.compile?
Thank you for any help.
Your regex is not correct because you have a open parenthesis before your keywords, since there is no such character in your lines.
As a more general way you can get capture the alphabetical character after sipout or provider:sipout.
>>> s1 = 'provider:sipoutilp1.ym.ms'
>>> s2 = 'provider:sipoutqtm.ym.ms'
>>> RE_PROVIDER = re.compile(r'(?P<provider>(?<=sipout)(ilp|qtm))')
>>> RE_PROVIDER.search(s1).groupdict()
{'provider': 'ilp'}
>>> RE_PROVIDER.search(s2).groupdict()
{'provider': 'qtm'}
(?<=sipout) is a positive look-behind which will makes the regex engine match the patter which is precede with sipout.
After edit:
If you want to match multiple strings with different structure, you have to use a optional preceding patterns for matching your keywords, and due to this point that you cannot use unfixed length patterns within look-behind you cannot use it for this aim. So instead you can use a capture group trick.
You can define the optional preceding patterns within a none capture group and your keyword within a capture group then after match get the second matched gorup (group(1), group(0) is the whole of your match).
>>> RE_PROVIDER = re.compile(r'(?:sipout|\d+,)(?P<provider>(ilp|qtm|[A-Z\s]+))')
>>> RE_PROVIDER.search(s1).groupdict()
{'provider': 'ilp'}
>>> RE_PROVIDER.search(s2).groupdict()
{'provider': 'qtm'}
>>> s3 = "182938,DOMINICAN REPUBLIC-MOBILE"
>>> RE_PROVIDER.search(s3).groupdict()
{'provider': 'DOMINICAN REPUBLIC'}
Note that gorupdict doesn't works in this case because it will returns
There are two different codes which produce two different result but I don't know how those differences arise.
>>>re.findall('[a-z]+','abc')
['abc']
and this one with group:
>>> re.findall('([a-z])+','abc')
['c']
why the second code yield character c ?
In your last regex pattern (([a-z])+), you are repeating a capturing group (()). And doing this will return only last iteration. So you get the last letter, which is c
But in your first pattern ([a-z]+), you are repeating a character class ([]), and this doesn't behave the same as a capturing group. It returns all the iterations.
When the parenthesis were used in the below program output is
['www.google.com'].
import re
teststring = "href=\"www.google.com\""
m=re.findall('href="(.*?)"',teststring)
print m;
If parenthesis is removed in findall function output is ['href="www.google.com"'].
import re
teststring = "href=\"www.google.com\""
m=re.findall('href=".*?"',teststring)
print m;
Would be helpful if someone explained how it works.
The re.findall() documentation is quite clear on the difference:
Return all non-overlapping matches of pattern in string, as a list of strings. […] If one or more groups are present in the pattern, return a list of groups; this will be a list of tuples if the pattern has more than one group.
So .findall() returns a list containing one of three types of values, depending on the number of groups in the pattern:
0 capturing groups in the pattern (no (...) parenthesis): the whole matched string ('href="www.google.com"' in your second example).
1 capturing group in the pattern: return the captured group ('www.google.com' in your first example).
more than 1 capturing group in the pattern: return a tuple of all matched groups.
Use non-capturing groups ((?:...)) if you don't want that behaviour, or add groups if you want more information. For example, adding a group around the href= part would result in a list of tuples with two elements each:
>>> re.findall('(href=)"(.*?)"', teststring)
[('href=', 'www.google.com')]
Referring to http://docs.python.org/howto/regex.html section Non-capturing and named groups, I saw an example whose output is not obvious to me.
>>> import re
>>> m = re.match(r"([abc])+", "abc")
>>> m.groups()
('c',)
Here I am not able to understand why the group(1) is 'c' and also why I see I dangling comma at the end. Can somebody help?
I don't know about the dangling comma, but your first group is c, because you let the group repeat itself by putting the + after the group, and not after the character group.
This way, the regex first matches a, which is assigned to group 1. Then, it matches b, which is also assigned to group 1. And at last, it matches c, assigned to group 1 and finishes; thus leaving your group 1 to be c.
If you write ([abc]+), your group 1 will be abc.