Searching multiple repeating patterns of text using regular exressions - python

I am trying to search for texts from a document, which have repeating portions and occur multiple times in the document. However, using the regex.match, it shows only the first match from the document and not others.
The patterns which I want to search looks like:
clauses 5.3, 12 & 15
clause 10 C, 10 CA & 10 CC
The following line shows the regular expression which I am using.
regex_crossref_multiple_1=r'(clause|Clause|clauses|Clauses)\s*\d+[.]?\d*\s*[a-zA-Z]*((,|&|and)\s*\d+[.]?\d*\s*[A-Z]*)+'
The code used for matching and the results are shown below:
cross=regex.search(regex_crossref_multiple_1,des)
(des is string containing text)
For printing the results, I am using print(cross.group()).
Result:
clauses 5.3, 12 & 15
However, there are other patterns as well in des which I am not getting in the result.
Please let me know what can be the problem.
The input string(des) is can be found from following link.
https://docs.google.com/document/d/1LPmYaD6VE724OYoXDGPfInvx8WTu5JfrTqTOIv8zAlg/edit?usp=sharing
In case, the contractor completes the work ahead of stipulated date of
completion or justified extended date of completion as determined
under clauses 5.3, 12 & 15, a bonus # 0.5 % (zero point five per cent) of
the tendered value per month computed on per day basis, shall be
payable to the contractor, subject to a maximum limit of 2 % (two
percent) of the tendered value. Provided that justified time for extra
work shall be calculated on pro-rata basis as cost of extra work excluding
amount payable/ paid under clause 10 C, 10 CA & 10 CC X stipulated
period /tendered value. The amount of bonus, if payable, shall be paid
along with final bill after completion of work. Provided always that
provision of the Clause 2A shall be applicable only when so provided in
‘Schedule F’

You could match clauses followed by an optional digits part and optional chars A-Z and then use a repeating pattern to match the optional following comma and the digits.
For the last part of the pattern you can optionally match either a ,, & or and followed by a digit and optional chars A-Z.
\b[Cc]lauses?\s+\d+(?:\.\d+)?(?:\s*[A-Z]+)?(?:,\s+\d+(?:\.\d+)?(?:\s*[A-Z]+)?)*(?:\s+(?:[,&]|and)\s+\d+(?:\.\d+)?(?:\s*[A-Z]+)?)?\b
Explanation
\b Word boundary
[Cc]lauses?\s+\d+(?:\.\d+)? Match clauses followed by digits and optional decimal part
(?:\s*[A-Z]+)? Optionally match whitespace chars and 1+ chars A-Z
(?: Non capture group
,\s+\d+(?:\.\d+)? Match a comma, digits and optional decimal part
(?:\s*[A-Z]+)? Optionally match whitespace chars and 1+ chars A-Z
)* Close group and repeat 0+ times
(?: Non capture group
\s+(?:[,&]|and) Match 1+ whitespace char and either ,, & or and
\s+\d+(?:\.\d+)? Match 1+ whitespace chars, 1+ digits with an optional decimal part
(?:\s*[A-Z]+)? Match optional whitespace chars and 1+ chars A-Z
)? Close group and make optional
\b Word boundary
Regex demo

Related

Regex to match phone number 5ABXXYYZZ

I am using regex to match 9 digits phone numbers.
I have this pattern 5ABXXYYZZ that I want to match.
What I tried
I have this regex that matches two repetitions only 5ABCDYYZZ
S_P_2 = 541982277
S_P_2_pattern = re.sub(r"(?!.*(\d)\1(\d)\2(\d)\3).{4}\d(\d)\4(\d)\5", "Special", str(S_P_2))
print(S_P_2_pattern)
What I want to achieve
I would like to update it to match three repetitions 5ABXXYYZZ sample 541882277
Try:
^5\d\d(?:(\d)\1(?!.*\1)){3}$
See an online demo
^5\d\d - Start-line anchor and literal 5 before two random digits;
(?:(\d)\1(?!.*\1)){3} - Non-capture group matched three times with nested capture group followed by itself directly but (due to negative lookahead) again after 0+ chars;
$ - End-line anchor.

Using regex to get multiple number in string in python

I have 3 strings
"1:1 (1 stock – 1 voting right)"
"1 stock – 1 voting right"
"01 stock – 01 voting right"
How can i get all the numbers in string and return it to format "num:num" when I use regex in Python. Can you give me a general regex to process all case?
Example:
If case is first string, it will return 1:1 because it already have 1:1
If case is in other, it will also return 1:1
Thank you
My two cents assuming:
At least two integers seperated by at least a single non-number;
Order of appearance of these integers is kept as input for 1st:2nd integer in final format;
Remove possible leading zeros from integers.
Try:
^\D*0*(\d+)\D+0*(\d+).*$
Replace with \1:\2. See an online demo
The above would match:
^ - Start-line anchor;
\D*0* - 0+ (Greedy) non-digits followed by 0+ zeros to prevent the following capture group from holding leading zeros;
(\d+) - A 1st capture group to catch the 1st integer of 1+ digits;
\D+0* - 1+ (Greedy) non-digits followed by 0+ zeros to prevent the following capture group from holding leading zeros;
(\d+) - A 2nd capture group to catch the 1st integer of 1+ digits;
.*$ - 0+ (Greedy) characters upto end-line anchor.

Making Regex combination of multiple chars and numbers combined

How to write a regex that will combine numbers and chars in a string in any order?
For example, If I want to read some kind of invoice number, I have example like this:
example 1: 32ah3af
example 2: 32ahPP-A2ah3af
example 3: 3A63af-3HHx3B-APe5y5-9OPiis
example 4: 3A63af 3HHx3B APe5y5 9OPiis
So each 'block' have length between 3 and 7 chars (letters or numbers) that can be in any order (letters can be lowercase or uppercase). Each. 'block' can start with letter or with number.
It can have one "block" or max 4 blocks that are separated with ' ' or -.
I know that I can make separators like: \s or \-, but I have no idea how to make these kind of blocks that have (or do not have) separator.
I tried with something like this:
([0-9]?[A-z]?){3,7}
But it does not work
You could use
^[A-Za-z0-9]{3,7}(?:[ -][A-Za-z0-9]{3,7}){0,3}\b
The pattern matches:
^ Start of string
[A-Za-z0-9]{3,7} Match 3-7 times either a lower or uppercase char a-z or number 0-9
(?: Non capture group
[ -][A-Za-z0-9]{3,7} Match either a space or - and 3-7 times either a lower or uppercase char a-z or number 0-9
){0,3} Close the non capture group and repeat 0-3 times to have a maximum or 4 occurrences
\b A word boundary to prevent a partial match
Regex demo
Note that [A-z] matches more than [A-Za-z0-9]
As long as you want to only capture / search for the invoice ids, the suggestion from Hao Wu is valid:
r'\w{3,7}'
for regex (check here).
If you can drop the remaining part, then this should be enough.
You can more precisely capture the whole string with example 1:
r'example (\d+): ((\w{3,7}[\- ]?)+)'
See here how it works. Please note how capturing groups are represented.

regex to find a pair of adjacent digits with different digits around them

I'm a beginner to regex and I am trying to make an expression to find if there are two of the same digits next to each other, and the digit behind and in front of the pair is different.
For example,
123456678 should match as there is a double 6,
1234566678 should not match as there is no double with different surrounding numbers.
12334566 should match because there are two 3s.
So far i have this which works only with 1, and as long as the double is not at the start or end of the string, however I can deal with that by adding a letter at the start and end.
^.*([^1]11[^1]).*$
I know i can use [0-9] instead of the 1s but the problem is having them all be the same digit.
Thank you!
I have divided my answer into four sections.
The first section contains my solution to the problem. Readers interested in nothing else may skip the other sections.
The remaining three sections are concerned with identifying the pairs of equal digits that are preceded by a different digit and are followed by a different digit. The first of the three sections matches them; the other two capture them in a group.
I've included the last section because I wanted to share The Greatest Regex Trick Ever with those unfamiliar with it, because I find it so very cool and clever, yet simple. It is documented here. Be forewarned that, to build suspense, the author at that link has included a lengthy preamble before the drum-roll reveal.
Determine if a string contains two consecutive equal digits that are preceded by a different digit and are followed by a different digit
You can test the string as follows:
import re
r = r'(\d)(?!\1)(\d)\2(?!\2)\d'
arr = ["123456678", "1123455a666788"]
for s in arr:
print(s, bool(re.search(r, s)) )
displays
123456678 True
1123455a666788 False
Run Python code | Start your engine!1
The regex engine performs the following operations.
(\d) : match a digit and save to capture group 1 (preceding digit)
(?!\1) : next character cannot equal content of capture group 1
(\d) : match a digit in capture group 2 (first digit of pair)
\2 : match content of capture group 2 (second digit of pair)
(?!\2) : next character cannot equal content of capture group 2
\d : match a digit
(?!\1) and (?!\2) are negative lookaheads.
Use Python's regex module to match pairs of consecutive digits that have the desired property
You can use the following regular expression with Python’s regex module to obtain the matching pairs of digits.
r'(\d)(?!\1)\K(\d)\2(?=\d)(?!\2)'
Regex Engine
The regex engine performs the following operations.
(\d) : match a digit and save to capture group 1 (preceding digit)
(?!\1) : next character cannot equal content of capture group 1
\K : forget everything matched so far and reset start of match
(\d) : match a digit in capture group 2 (first digit of pair)
\2 : match content of capture group 2 (second digit of pair)
(?=\d) : next character must be a digit
(?!\2) : next character cannot equal content of capture group 2
(?=\d) is a positive lookahead. (?=\d)(?!\2) could be replaced with (?!\2|$|\D).
Save pairs of consecutive digits that have the desired property to a capture group
Another way to obtain the matching pairs of digits, which does not require the regex module, is to extract the contents of capture group 2 from matches of the following regular expression.
r'(\d)(?!\1)((\d)\3)(?!\3)(?=\d)'
Re engine
The following operations are performed.
(\d) : match a digit in capture group 1
(?!\1) : next character does not equal last character
( : begin capture group 2
(\d) : match a digit in capture group 3
\3 : match the content of capture group 3
) : end capture group 2
(?!\3) : next character does not equal last character
(?=\d) : next character is a digit
Use The Greatest Regex Trick Ever to identify pairs of consecutive digits that have the desired property
We use the following regular expression to match the string.
r'(\d)(?=\1)|\d(?=(\d)(?!\2))|\d(?=\d(\d)\3)|\d(?=(\d{2})\d)'
When there is a match, we pay no attention to which character was matched, but examine the content of capture group 4 ((\d{2})), as I will explain below.
The Trick in action
The first three components of the alternation correspond to the ways that a string of four digits can fail to have the property that the second and third digits are equal, the first and second are unequal and the third and fourth are equal. They are:
(\d)(?=\1) : assert first and second digits are equal
\d(?=(\d)(?!\2)) : assert second and third digits are not equal
\d(?=\d(\d)\3) : assert third and fourth digits are equal
It follows that if there is a match of a digit and the first three parts of the alternation fail the last part (\d(?=(\d{2})\d)) must succeed, and the capture group it contains (#4) must contain the two equal digits that have the required properties. (The final \d is needed to assert that the pair of digits of interest is followed by a digit.)
If there is a match how do we determine if the last part of the alternation is the one that is matched?
When this regex matches a digit we have no interest in what digit that was. Instead, we look to capture group 4 ((\d{2})). If that group is empty we conclude that one of the first three components of the alternation matched the digit, meaning that the two digits following the matched digit do not have the properties that they are equal and are unequal to the digits that precede and follow them.
If, however, capture group 4 is not empty, it means that none of the first three parts of the alternation matched the digit, so the last part of the alternation must have matched and the two digits following the matched digit, which are held in capture group 4, have the desired properties.
1. Move the cursor around for detailed explanations.
With regex, it is much more convenient to use a PyPi regex module with the (*SKIP)(*FAIL) based pattern:
import regex
rx = r'(\d)\1{2,}(*SKIP)(*F)|(\d)\2'
l = ["123456678", "1234566678"]
for s in l:
print(s, bool(regex.search(rx, s)) )
See the Python demo. Output:
123456678 True
1234566678 False
Regex details
(\d)\1{2,}(*SKIP)(*F) - a digit and then two or more occurrences of the same digit
| - or
(\d)\2 - a digit and then the same digit.
The point is to match all chunks of identical 3 or more digits and skip them, and then match a chunk of two identical digits.
See the regex demo.
Inspired by the answer or Wiktor Stribiżew, another variation of using an alternation with re is to check for the existence of the capturing group which contains a positive match for 2 of the same digits not surrounded by the same digit.
In this case, check for group 3.
((\d)\2{2,})|\d(\d)\3(?!\3)\d
Regex demo | Python demo
( Capture group 1
(\d)\2{2,} Capture group 2, match 1 digit and repeat that same digit 2+ times
) Close group
| Or
\d(\d) Match a digit, capture a digit in group 3
\3(?!\3)\d Match the same digit as in group 3. Match the 4th digit, but is should not be the same as the group 3 digit
For example
import re
pattern = r"((\d)\2{2,})|\d(\d)\3(?!\3)\d"
strings = ["123456678", "12334566", "12345654554888", "1221", "1234566678", "1222", "2221", "66", "122", "221", "111"]
for s in strings:
match = re.search(pattern, s)
if match and match.group(3):
print ("Match: " + match.string)
else:
print ("No match: " + s)
Output
Match: 123456678
Match: 12334566
Match: 12345654554888
Match: 1221
No match: 1234566678
No match: 1222
No match: 2221
No match: 66
No match: 122
No match: 221
No match: 111
If for example 2 or 3 digits only is also ok to match, you could check for group 2
(\d)\1{2,}|(\d)\2
Python demo
You can also use a simple way .
import re
l=["123456678",
"1234566678",
"12334566 "]
for i in l:
matches = re.findall(r"((.)\2+)", i)
if any(len(x[0])!=2 for x in matches):
print "{}-->{}".format(i, False)
else:
print "{}-->{}".format(i, True)
You can customize this based on you rules.
Output:
123456678-->True
1234566678-->False
12334566 -->True

Capturing repeated pattern in Python

I'm trying to implement some kind of markdown like behavior for a Python log formatter.
Let's take this string as example:
**This is a warning**: Virus manager __failed__
A few regexes later the string has lost the markdown like syntax and been turned into bash code:
\033[33m\033[1mThis is a warning\033[0m: Virus manager \033[4mfailed\033[0m\033[0m
But that should be compressed to
\033[33;1mThis is a warning\033[0m: Virus manager \033[4mfailed\033[0m
I tried these, beside many other non working solutions:
(\\033\[([\d]+)m){2,} => Capture: \033[33m\033[1m with g1 '\033[1m' and g2 '1' and \033[0m\033[0mwith g1 '\033[0m' and g2 '0'
(\\033\[([\d]+)m)+ many results, not ok
(?:(\\033\[([\d]+)m)+) many results, although this is the recommended way for repeated patterns if I understood correctly, not ok
and others..
My goal is to have as results:
Input
\033[33m\033[1mThis is a warning\033[0m: Virus manager \033[4mfailed\033[0m\033[0m
Output
Match 1
033[33m\033[1m
Group1: 33
Group2: 1
Match 2
033[0m\033[0m
Group1: 0
Group2: 0
In other words, capture the ones that are "duplicated" and not the ones alone, so I can fuse them with a regex sub.
You want to match consectuively repeating \033[\d+m chunks of text and join the numbers after [ with a semi-colon.
You may use
re.sub(r'(?:\\033\[\d+m){2,}', lambda m: r'\033['+";".join(set(re.findall(r"\[(\d+)", m.group())))+'m', text)
See the Python demo online
The (?:\\033\[\d+m){2,} pattern will match two or more sequences of \033[ + one or more digits + m chunks of texts and then, the match will be passed to the lambda expression, where the output will be: 1) \033[, 2) all the numbers after [ extracted with re.findall(r"\[(\d+)", m.group()) and deduplicated with the set, and then 3) m.
The patterns in the string to be modified have not been made clear from the question. For example, is 033 fixed or might it be 025 or even 25? I've made certain assumptions in using the regex
r" ^(\\0(\d+)\[\2)[a-z]\\0\2\[(\d[a-z].+)
to obtain two capture groups that are to be combined, separated by a semi-colon. I've attempted to make clear my assumptions below, in part to help the OP modify this regex to satisfy alternative requirements.
Demo
The regex performs the following operations:
^ # match beginning of line
( # begin cap grp 1
\\0 # match '\0'
(\d+) # match 1+ digits in cap grp 2
\[ # match '['
\2 # match contents of cap grp 2
) # end cap grp 1
[a-z] # match a lc letter
\\0 # match '\0'
\2 # match contents of cap grp 2
\[ # match '['
(\d[a-z].+) # match a digit, then lc letter then 1+ chars to the
# end of the line in cap grp 3
As you see, the portion of the string captured in group 1 is
\033[33
I've assumed that the part of this string that is now 033 must be two or more digits beginning with a zero, and the second appearance of a string of digits consists of the same digits after the zero. This is done by capturing the digits following '0' (33) in capture group 2 and then using a back-reference \2.
The next part of the string is to be replaced and therefore is not captured:
m\\033[
I've assumed that m must be one lower case letter (or should it be a literal m?), the backslash and zero and required and the following digits must again match the content of capture group 2.
The remainder of the string,
1mThis is a warning\033[0m: Virus manager \033[4mfailed\033[0m\033[0m
is captured in capture group 3. Here I've assumed it begins with one digit (perhaps it should be \d+) followed by one lower case letter that needn't be the same as the lower case letter matched earlier (though that could be enforced with another capture group). At that point I match the remainder of the line with .+, having given up matching patterns in that part of the string.
One may alternatively have just two capture groups, the capture group that is now #2, becoming #1, and #2 being the part of the string that is to be replaced with a semicolon.
This is pretty straightforward for the cases you desribe here; simply write out from left to right what you want to match and capture. Repeating capturing blocks won't help you here, because only the most recently captured values would be returned as a result.
\\033\[(\d+)m\\033\[(\d+)m

Categories