This question already has answers here:
re.findall behaves weird
(3 answers)
Closed 3 years ago.
I m generally curious why re.findall makes sutch weid stuff as finding empty strings, tuples (what that suppose to mean). It seems it does not take clausures () normally, als o interpretes | wrong like ab | cd is (ab)| (cd) , not a (b|c)d like you would think normally. Because of that i cant define regex what i need.
But in this example ie see clear wrong behaviour on the simple pattern:
([a-zA-Z0-9]+\.+)+[a-zA-Z0-9]{1,3}
what describes simple urls like gskinner.com, www.capitolconnection.org what you can see on regex help in https://regexr.com/ , i recognize with re.findall :
hotmail.
living.
item.
2.
4S.
means letters then just. How can that be?
Full code, where i try to filter out jonk from the text is :
import re
singles = r'[()\.\/$%=0-9,?!=; \t\n\r\f\v\":\[\]><]'
digits_str = singles + r'[()\-\.\/$%=0-9 \t\n\r\f\v\'\":\[\]]*'
#small_word = '[a-zA-Z0-9]{1,3}'
#junk_then_small_word = singles + small_word + '(' + singles + small_word + ')*'
email = singles + '\S+#\S*'
http_str = r'[^\.]+\.+[^\.]+\.+([^\.]+\.+)+?'
http = '(http|https|www)' + http_str
web_address = '([a-zA-Z0-9]+\.+)+[a-zA-Z0-9]{1,3}'
pat = email + '|' + digits_str
d_pat = re.compile(web_address)
text = '''"Lucy Gonzalez" test-defis-wtf <stagecoachmama#hotmail.com> on 11/28/2000 01:02:22 PM
http://www.living.com/shopping/item/item.jhtml?.productId=LC-JJHY-2.00-10.4S.I will send checks
directly to the vendor for any bills pre 4/20. I will fax you copies. I will also try and get the payphone transferred.
www.capitolconnection.org <http://www.capitolconnection.org>.
and/or =3D=3D=3D=3D=3D=3D=3D= O\'rourke'''
print('findall:')
for x in re.findall(d_pat,text):
print(x)
print('split:')
for x in re.split(d_pat,text):
print(x)
From the documentation of re.findall:
If one or more groups are present in the pattern, return a list of groups; this will be a list of tuples if the pattern has more than one group.
Your regex has groups, namely the part in parenthesis. If you want to display the entire match, put your regex in one big group (put parenthesis around the whole thing) and then do print(x[0]) instead of print(x).
I'm guessing that our expression has to be modified here, and that might be the problem, for instance, if we wish to match the desired patterns we would start with an expression similar to:
([a-zA-Z0-9]+)\.
if we wish to have 1 to 3 chars after the ., we would expand it to:
([a-zA-Z0-9]+)\.([a-zA-Z0-9]{1,3})?
Demo 1
Demo 2
Test
# coding=utf8
# the above tag defines encoding for this document and is for Python 2.x compatibility
import re
regex = r"([a-zA-Z0-9]+)\.([a-zA-Z0-9]{1,3})?"
test_str = ("hotmail.\n"
"living.\n"
"item.\n"
"2.\n"
"4S.\n"
"hotmail.com\n"
"living.org\n"
"item.co\n"
"2.321\n"
"4S.123")
matches = re.finditer(regex, test_str, re.MULTILINE)
for matchNum, match in enumerate(matches, start=1):
print ("Match {matchNum} was found at {start}-{end}: {match}".format(matchNum = matchNum, start = match.start(), end = match.end(), match = match.group()))
for groupNum in range(0, len(match.groups())):
groupNum = groupNum + 1
print ("Group {groupNum} found at {start}-{end}: {group}".format(groupNum = groupNum, start = match.start(groupNum), end = match.end(groupNum), group = match.group(groupNum)))
# Note: for Python 2.7 compatibility, use ur"" to prefix the regex and u"" to prefix the test string and substitution.
Related
I am trying to find a amino acid pattern (B-C or M-D, where '-' could be any alphabet other than 'P') in a protein sequence let say 'VATLDSCBACSKVNDNVKNKVKVKNVKMLDHHHV'. Protein sequence in in a fasta file.
I have tried a lot but couldn't find any solution.
I tried a lot. the following code is one of them
import Bio
from Bio import SeqIO
seqs= SeqIO.parse(X, 'fasta') ### to read the sequences from fasta file
for aa in seqs:
x=aa.seq ## gives the sequences as a string (.seq is a build in function of Biopython)
for val, i in enumerate(x):
if i=='B':
if (x[val+2])=='C':
if x[val+1]!='P':
pattern=((x[val]:x[val+2])) ## trying to print full sequence B-C
But unfortunately none of them work.
It would be great if someone can help me out with this problem.
>>> x = 'VATLDSCBACSKVNDNVKNKVKVKNVKMLDHHHV'
>>> import re
>>> m = re.search('B(.+?)C', x)
>>> m
<_sre.SRE_Match object at 0x10262aeb0>
>>> m = re.search('B(.+?)C', x).group(0)
>>> m
'BAC'
>>> m = re.search('M(.+?)D', x).group(0)
>>> m
'MLD'
>>> re.search(r"(?<=M).*?(?=D)", x).group(0)
'L'
>>> re.search(r"(?<=B).*?(?=C)", x).group(0)
'A'
A common solution for pattern matching is the usage of regex.
A possible regex for your problem is B[^P]C|M[^P]D.
The following code has been generated by regex101 with the regex I propose and the test string you gave us. It find all matching pattern with their positions in the original string.
# coding=utf8
# the above tag defines encoding for this document and is for Python 2.x compatibility
import re
regex = r"B[^P]C|M[^P]D"
test_str = "VATLDSCBACSKVNDNVKNKVKVKNVKMLDHHHV"
matches = re.finditer(regex, test_str, re.MULTILINE)
for matchNum, match in enumerate(matches, start=1):
print ("Match {matchNum} was found at {start}-{end}: {match}".format(matchNum = matchNum, start = match.start(), end = match.end(), match = match.group()))
for groupNum in range(0, len(match.groups())):
groupNum = groupNum + 1
print ("Group {groupNum} found at {start}-{end}: {group}".format(groupNum = groupNum, start = match.start(groupNum), end = match.end(groupNum), group = match.group(groupNum)))
# Note: for Python 2.7 compatibility, use ur"" to prefix the regex and u"" to prefix the test string and substitution.
In python you can use the Regex module (re):
import re # import the RE module
import Bio
from Bio import SeqIO
seqs = SeqIO.parse(X, 'fasta')
for sequence in seqs:
line = sequence.se
RE = r'B[A-OQ-Z]C|M[A-OQ-Z]D'
# [A-OQ-Z] : Match from A to O and from Q to Z (exl. P)
# | : is an operator OR = either the left or right part should match
# The r before the string specify that the string is regex: r"regex"
results = re.findall(RE, line)
# The function findall will return a list of all non-overlapping matches.
# To iterate over each result :
for res in results:
print(res)
Then you can also modify the Regular expression to match any other rule you would like to match.
More information about the findall function here : re.findall(...)
The following website can help you build a regex :
https://regex101.com/
Use a regular expression with an exception assertion "^".
import re
string = 'VATLDSCBACSKVNDNVKNKVKVKNVKMLDHHHV'
re.findall(r"B[^P]C|M[^P]D", string)
Output:
['BAC', 'MLD']
This question already has answers here:
Retrieving parameters from a URL
(20 answers)
Closed 3 years ago.
I'm using Python 3.7. I want to extract the portion of a url between the "q=...&" part of a query string. I have this code
href = span.a['href']
print("href:" + href)
matchObj = re.match( r'q=(.*?)\&', href, re.M|re.I)
if matchObj:
criteria = matchObj.group(1)
but despite the fact that my href is this
href:/search?hl=en-US&q=bet+i+won+t+get+one+share&tbm=isch&tbs=simg:CAQSkwEJyapBtj9kKiIahwELEKjU2AQaAAwLELCMpwgaYgpgCAMSKMILxAufFcsLnBWeFZsVnRWABMcPsCKgLaMtoi2hLZ0tqziiI6w4uSQaMG01mL5LQ62s4q5ZMf-Wetz68lCkHfrFOOKs2CELzQJlPjHIMzmlp2Ny-a5t7hZbiCAEDAsQjq7-CBoKCggIARIEXLNODAw&sa=X&ved=0ahUKEwjThcCx59ziAhWKHLkGHfWjDs4Q2A4ILCgB
the "matchObj" is always NoneType and the subsequent lines aren't evaluated. What else do I need to do to fix my regex?
You can use the urllib module
Ex:
import urllib.parse as urlparse
url = "href:/search?hl=en-US&q=bet+i+won+t+get+one+share&tbm=isch&tbs=simg:CAQSkwEJyapBtj9kKiIahwELEKjU2AQaAAwLELCMpwgaYgpgCAMSKMILxAufFcsLnBWeFZsVnRWABMcPsCKgLaMtoi2hLZ0tqziiI6w4uSQaMG01mL5LQ62s4q5ZMf-Wetz68lCkHfrFOOKs2CELzQJlPjHIMzmlp2Ny-a5t7hZbiCAEDAsQjq7-CBoKCggIARIEXLNODAw&sa=X&ved=0ahUKEwjThcCx59ziAhWKHLkGHfWjDs4Q2A4ILCgB"
data = urlparse.urlparse(url)
print(urlparse.parse_qs(data.query)['q'][0])
Output:
bet i won t get one share
You're using the wrong function if you wish to match in the middle of the string.
re.match only matches from start of the string
If zero or more characters at the beginning of string match the
regular expression pattern, return a corresponding match object.
Here use re.search instead.
import re
href = 'href:/search?hl=en-US&q=bet+i+won+t+get+one+share&tbm=isch&tbs=simg:CAQSkwEJyapBtj9kKiIahwELEKjU2AQaAAwLELCMpwgaYgpgCAMSKMILxAufFcsLnBWeFZsVnRWABMcPsCKgLaMtoi2hLZ0tqziiI6w4uSQaMG01mL5LQ62s4q5ZMf-Wetz68lCkHfrFOOKs2CELzQJlPjHIMzmlp2Ny-a5t7hZbiCAEDAsQjq7-CBoKCggIARIEXLNODAw&sa=X&ved=0ahUKEwjThcCx59ziAhWKHLkGHfWjDs4Q2A4ILCgB'
print("href:" + href)
matchObj = re.search( r'q=(.*?)\&', href, re.M|re.I)
if matchObj:
criteria = matchObj.group(1)
print(criteria)
'bet+i+won+t+get+one+share'
Here, we would apply a simple expression with left and right boundaries such as:
&q=(.+?)&
Demo
# coding=utf8
# the above tag defines encoding for this document and is for Python 2.x compatibility
import re
regex = r"&q=(.+?)&"
test_str = "href:/search?hl=en-US&q=bet+i+won+t+get+one+share&tbm=isch&tbs=simg:CAQSkwEJyapBtj9kKiIahwELEKjU2AQaAAwLELCMpwgaYgpgCAMSKMILxAufFcsLnBWeFZsVnRWABMcPsCKgLaMtoi2hLZ0tqziiI6w4uSQaMG01mL5LQ62s4q5ZMf-Wetz68lCkHfrFOOKs2CELzQJlPjHIMzmlp2Ny-a5t7hZbiCAEDAsQjq7-CBoKCggIARIEXLNODAw&sa=X&ved=0ahUKEwjThcCx59ziAhWKHLkGHfWjDs4Q2A4ILCgB
"
matches = re.finditer(regex, test_str, re.MULTILINE)
for matchNum, match in enumerate(matches, start=1):
print ("Match {matchNum} was found at {start}-{end}: {match}".format(matchNum = matchNum, start = match.start(), end = match.end(), match = match.group()))
for groupNum in range(0, len(match.groups())):
groupNum = groupNum + 1
print ("Group {groupNum} found at {start}-{end}: {group}".format(groupNum = groupNum, start = match.start(groupNum), end = match.end(groupNum), group = match.group(groupNum)))
# Note: for Python 2.7 compatibility, use ur"" to prefix the regex and u"" to prefix the test string and substitution.
RegEx Circuit
jex.im visualizes regular expressions:
I need to parse a line of text and separate in parts and add it to a list, thing that i was able to do with the help of re.parse('regexp'). The thing is that i get some text that i dont want that match on this, but i need to know where is it, and how to detect it and of course what is it, to show an error.
the code matches and filters out all perfectly, the thing is i need to filter out the 12 and the 32 that are not matching the regexp
import re
str = '12 32 455c 2v 12tv v 0.5b -3b -b+b-3li b-0.5b 3 c -3 ltr'
a=re.compile(r'[+-]?[0-9]*\.[0-9]+\s*[a-z]+|[+-]?[0-9]*\s*[a-z]+')
r=a.findall(str)
print (r)
Initial String:
str= '12 32 455c 2v 12tv v 0.5b -3b -b+b-3li b-0.5b 1 3 c -3 ltr'
list parsed, correctly
['455c', '2v', '12tv', ' v', '0.5b', '-3b', '-b', '+b', '-3li', ' b', '-0.5b', '3 c', '-3 ltr']
list that i need as well and any other string not matched ie: (/%&$%)
[12, 32, 1]
My guess is that if we might not want to collect the digits only, then we would be starting with a simple expression:
\b([\d]{1,}\s)\b|([\w+-.]+)
with two parts:
\b([\d]{1,}\s)\b
are our undesired digits, and
([\w+-.]+)
has our desired outputs.
Test
# coding=utf8
# the above tag defines encoding for this document and is for Python 2.x compatibility
import re
regex = r"\b([\d]{1,}\s)\b|([\w+-.]+)"
test_str = "12 32 455c 2v 12tv v 0.5b -3b -b+b-3li b-0.5b 3 c -3 ltr"
matches = re.finditer(regex, test_str, re.MULTILINE)
for matchNum, match in enumerate(matches, start=1):
print ("Match {matchNum} was found at {start}-{end}: {match}".format(matchNum = matchNum, start = match.start(), end = match.end(), match = match.group()))
for groupNum in range(0, len(match.groups())):
groupNum = groupNum + 1
print ("Group {groupNum} found at {start}-{end}: {group}".format(groupNum = groupNum, start = match.start(groupNum), end = match.end(groupNum), group = match.group(groupNum)))
# Note: for Python 2.7 compatibility, use ur"" to prefix the regex and u"" to prefix the test string and substitution.
Demo
RegEx
If this expression wasn't desired and you wish to modify it, please visit this link at regex101.com.
RegEx Circuit
jex.im visualizes regular expressions:
I've solved this by myself by replacing the correctly parsed on the initial string, so i get the difference then split to get it as a list
str = '12 32 455c 2v 12tv v 0.5b -3b -b+b-3li b-0.5b 1 3 c -3 ltr'
a=re.compile(r'[+-]?[0-9]*\.[0-9]+\s*[a-z]+|[+-]?[0-9]*\s*[a-z]+')
r=a.findall(str)
print (r)
errors = str
for t in r:
errors = errors.replace(t, '', 1)
errors = errors.split()
print(errors)
I have the following shape of string: PW[Yasui Chitetsu]; and would like to get only the name inside the brackets: Yasui Chitetsu. I'm trying something like
[^(PW\[)](.*)[^\]]
as a regular expression, but the last bracket is still in it. How do I unselect it? I don't think I need anything fancy like look behinds, etc, for this case.
The Problems with What You've Tried
There are a few problems with what you've tried:
It will omit the first and last characters of your match from the group, giving you something like asui Chitets.
It will have even more errors on strings that start with P or W. For example, in PW[Paul McCartney], you would match only ul McCartne with the group and ul McCartney with the full match.
The Regex
You want something like this:
(?<=\[)([^]]+)(?=\])
Here's a regex101 demo.
Explanation
(?<=\[) means that the match must be preceded by [
([^]]+) matches 1 or more characters that are not ]
(?=\])means that the match must be followed by ]
Sample Code
Here's some sample code (from the above regex101 link):
# coding=utf8
# the above tag defines encoding for this document and is for Python 2.x compatibility
import re
regex = r"(?<=\[)([^]]+)(?=\])"
test_str = "PW[Yasui Chitetsu]"
matches = re.finditer(regex, test_str, re.MULTILINE)
for matchNum, match in enumerate(matches):
matchNum = matchNum + 1
print ("Match {matchNum} was found at {start}-{end}: {match}".format(matchNum = matchNum, start = match.start(), end = match.end(), match = match.group()))
for groupNum in range(0, len(match.groups())):
groupNum = groupNum + 1
print ("Group {groupNum} found at {start}-{end}: {group}".format(groupNum = groupNum, start = match.start(groupNum), end = match.end(groupNum), group = match.group(groupNum)))
# Note: for Python 2.7 compatibility, use ur"" to prefix the regex and u"" to prefix the test string and substitution.
Semicolons
In your title, you mentioned finding text between semicolons. The same logic would work for that, giving you this regex:
(?<=;)([^;]+)(?=;)
I have a string like:
(A\2009_2009-01-04:0.2,(A\name2\human\2007_2007:0.3,A\chicken\ird16\2016_20016:0.4)A\name3\epi66321\2001_2001-04-04:0.5)A\name_with_space\2014_2014:0.1)A\name4\66036-8a\2004_2004-12-05;
In this tree, names are enclosed on the left by either an open bracket "(", a closing bracket ")", or a comma, and enclosed on the right with a colon ':'. That is, the substrings "A\2009_2009-01-04", "A\name2\human\2007_2007", "A\name3\epi66321\2001_2001-04-04", are names. (this is actually a tree in newick format).
I'd like to find a regex pattern which finds all names, with as little restriction on namespace as possible. Think of names as variables, like this example from Wikipedia:
(A:0.1,B:0.2,(C:0.3,D:0.4)E:0.5)F;
Where A, B, C etc. can be any string. The only restriction on namespace is that names cannot contain rounded or square brackets, '&', ',' or ':', because these are special characters that define the tree format, the same way that the comma defines a csv format.
Bonus: sometimes, internal nodes within the tree aren't labelled:
(A:0.1,B:0.2,(C:0.3,D:0.4):0.5);
In which case, a regex that correctly returns a string of length zero would be great.
It seems you want to extract substrings that start with 1+ (, ) or , and then contain 1+ non-whitespace characters other than : and ;, as many as possible, but stop at the word boundary.
Use
r'[(),]+([^;:]+)\b'
See the regex demo.
Pattern details
[(),]+ - one or more characters in the character class: (, ) or ,
([^;:]+) - Group 1: one or more chars other than ; and :, as many as possible
\b - a word boundary
Python demo:
import re
rx = r'[(),]+([^;:]+)\b'
s = "(A:0.1,B:0.2,(C:0.3,D:0.4)E:0.5)F;((A\\2009_2009-01-04:0.2,(A\\name2\\human\\2007_2007:0.3,A\\chicken\\ird16\\2016_20016:0.4)A\\name3\\epi66321\\2001_2001-04-04:0.5)A\\name_with_space\\2014_2014:0.1)A\\name4\\66036-8a\\2004_2004-12-05;"
res = re.findall(rx, s)
for val in res:
print(val)
Output:
A
B
C
D
E
F
A\2009_2009-01-04
A\name2\human\2007_2007
A\chicken\ird16\2016_20016
A\name3\epi66321\2001_2001-04-04
A\name_with_space\2014_2014
A\name4\66036-8a\2004_2004-12-05
you can use the regex
(\w+)(?=:|;)
see the sample code
import re
regex = r"(\w+)(?=:|;)"
test_str = "((B:0.2,(C:0.3,D:0.4)E:0.5)F:0.1)A;"
matches = re.finditer(regex, test_str)
for matchNum, match in enumerate(matches):
matchNum = matchNum + 1
print ("Match {matchNum} was found at {start}-{end}: {match}".format(matchNum = matchNum, start = match.start(), end = match.end(), match = match.group()))
The output is
Match 1 was found at 2-3: B
Match 2 was found at 9-10: C
Match 3 was found at 15-16: D
Match 4 was found at 21-22: E
Match 5 was found at 27-28: F
Match 6 was found at 33-34: A
A working solution:
[(),]([A-E])(?!;)
See live demo. One mistake you made was escaping characters inside the character class; but inside it they don't have special meaning.
I also took care of selecting against a trailing semicolon.
pattern = re.compile(r'[(),]A/[\S]*?:')
Not the most elegant, because I made use of the fact that all my names start with "A/". This will not be true for future use cases, just this current one. Will leave this question open if someone can find a more generalizable solution.