I am given a string which is of this pattern:
[blah blah blah] [more blah] some text
I want to split the string into three parts: blah blah blah, more blah and some text.
A crude way to do it is to use mystr.split('] '), and then removes the lead [ from the first two elements. Is there a better and performant way (need to do this for thousands of strings very quickly).
You can use a regular expression to extract the text, if you know that it will be in that form. For efficiency, you can precompile the regex and then repeatedly use it when matching.
prog = re.compile('\[([^\]]*)\]\s*\[([^\]]*)\]\s*(.*)')
for mystr in string_list:
result = prog.match(mystr)
groups = result.groups()
If you'd like an explanation on the regex itself, you can get one using this tool.
You can use a regular expression to split where you want to leave out characters:
>>> import re
>>> s = '[...] [...] ...'
>>> re.split(r'\[|\] *\[?', s)[1:]
['...', '...', '...']
Related
I have a text file need to be analyzed here, what I am interested is only the whole word with the first letter capitalized,
For example: test string: Everyday HOLDS the poSSibility Of A Miracle
I want to capture: Everyday Of A Miracle
I am currently trying to build my regular expression in Python, strangely, my regex only can capture the first whole word that is captalized.
Test String: Everyday HOLDS the poSSibility Of A Miracle
My regex: ^([A-Z])?([a-z])+
Capture: Everyday
What am I missing here ?
Instead of anchoring the regex at the beginning of the string, utilize boundary checking:
import re
s = 'Everyday HOLDS the poSSibility Of A Miracle'
new_s = ' '.join(re.findall(r'\b[A-Z][a-z]+|\b[A-Z]\b', s))
Output:
'Everyday Of A Miracle'
Without regex (only if words are delimited by whitespaces):
>>> s='Everyday HOLDS the poSSibility Of A Miracle'
>>> [x for x in s.split() if x.title()==x]
['Everyday', 'Of', 'A', 'Miracle']
Note that you can also use re.split to split on any non-letter characters.
I have a relatively complex string that contains a bunch of data. I am trying to extract the relevant pieces of the string using a regex command. The portions I am interested in are contained in square brackets, like this:
s = '"data":["value":3.44}] lol haha "data":["value":55.34}]
"data":["value":2.44}] lol haha "data":["value":56.34}]'
And the regex expression I have built is as follows:
l = re.findall(r'\"data\"\:.*(\[.*\])', s)
I was expecting this to return
['["value":3.44}]', '["value":55.34}]', '["value":2.44}]', '["value":56.34}]']
But instead all I get is the last one, i.e.,
['["value":56.34}]']
How can I catch 'em all?
It's because quantifiers are greedy by default. So .* will match everything between the first "data": and the last [, so there's only one [...] left to match.
Use non-greedy quantifiers by adding ?.
l = re.findall(r'\"data\"\:.*?(\[.*?\])', s)
You can also use finditer to extract the relevant content iteratively:
import re
s = '"data":["value":3.44}] lol haha "data":["value":55.34}] "data":["value":2.44}] lol haha "data":["value":56.34}]'
for m in re.finditer(r'(\[.*?\])', s):
print m.group(1)
OUTPUT
["value":3.44}]
["value":55.34}]
["value":2.44}]
["value":56.34}]
I have strings that are of the form below:
<p>The is a string.</p>
<em>This is another string.</em>
They are read in from a text file one line at a time. I want to separate these into words. For that I am just splitting the string using split().
Now I have a set of words but the first word will be <p>The rather than The. Same for the other words that have <> next to them. I want to remove the <..> from the words.
I'd like to do this in one line. What I mean is I want to pass as a parameter something of the form <*> like I would on the command line. I was thinking of using the replace() function to try to do this, but I am not sure how the replace() function parameter would look like.
For example, how could I change <..> below in a way that it will mean that I want to include anything that is between < and >:
x = x.replace("<..>", "")
Unfortunately, str.replace does not support Regex patterns. You need to use re.sub for this:
>>> from re import sub
>>> sub("<[^>]*>", "", "<p>The is a string.</p>")
'The is a string.'
>>> sub("<[^>]*>", "", "<em>This is another string.</em>")
'This is another string.'
>>>
[^>]* matches zero or more characters that are not >.
No Need for a 2-Step Solution
You don't need to 1. Split then 2. Replace. The two solutions below show you how to do it with one single step.
Option 1: Match All Instead of Splitting
Match All and Split are Two Sides of the Same Coin, and in this case it is safer to match all:
<[^>]+>|(\w+)
The words will be in Group 1.
Use it like this:
subject = '<p>The is a string.</p><em>This is another string.</em>'
regex = re.compile(r'<[^>]+>|(\w+)')
matches = [group for group in re.findall(regex, subject) if group]
print(matches)
Output
['The', 'is', 'a', 'string', 'This', 'is', 'another', 'string']
Discussion
This problem is a classic case of the technique explained in this question to "regex-match a pattern, excluding..."
The left side of the alternation | matches complete <tags>. We will ignore these matches. The right side matches and captures words to Group 1, and we know they are the right ones because they were not matched by the expression on the left.
Reference
How to match (or replace) a pattern except in situations s1, s2, s3...
Article about matching a pattern unless...
Option 2: One Single Split
<[^>]+>|[ .]
On the left side of the |, we use <complete tags> as a split delimiter. On the right side, we use a space character or a period.
Output
This
is
a
string
I have a string:
This is #lame
Here I want to extract lame. But here is the issue, the above string can be
This is lame
Here I dont extract anything. And then this string can be:
This is #lame but that is #not
Here i extract lame and not
So, output I am expecting in each case is:
[lame]
[]
[lame,not]
How do I extract these in robust way in python?
Use re.findall() to find multiple patterns; in this case for anything that is preceded by #, consisting of word characters:
re.findall(r'(?<=#)\w+', inputtext)
The (?<=..) construct is a positive lookbehind assertion; it only matches if the current position is preceded by a # character. So the above pattern matches 1 or more word characters (the \w character class) only if those characters were preceded by an # symbol.
Demo:
>>> import re
>>> re.findall(r'(?<=#)\w+', 'This is #lame')
['lame']
>>> re.findall(r'(?<=#)\w+', 'This is lame')
[]
>>> re.findall(r'(?<=#)\w+', 'This is #lame but that is #not')
['lame', 'not']
If you plan on reusing the pattern, do compile the expression first, then use the .findall() method on the compiled regular expression object:
at_words = re.compile(r'(?<=#)\w+')
at_words.findall(inputtext)
This saves you a cache lookup every time you call .findall().
You should use re lib here is an example:
import re
test case = "This is #lame but that is #not"
regular = re.compile("#[\w]*")
lst= regular.findall(test case)
This will give the output you requested:
import re
regex = re.compile(r'(?<=#)\w+')
print regex.findall('This is #lame')
print regex.findall('This is lame')
print regex.findall('This is #lame but that is #not')
I have a regular expression that looks for a url in some text like:
my_urlfinder = re.compile(r'\shttp:\/\/(\S+.|)blah.com/users/(\d+)(\/|)')
text = "blah blah http://blah.com/users/123 blah blah http://blah.com/users/353"
for match in my_urlfinder.findall(text):
print match #prints an array with all the individual parts of the regex
How do I get the entire url? Currently match just prints out the matched parts (which I need for other things)...but I also want the full url.
You should make your groups non-capturing:
my_urlfinder = re.compile(r'\shttp:\/\/(?:\S+.|)blah.com/users/(?:\d+)(?:\/|)')
findall() changes behaviour when there are capturing groups. With groups, it'll only return the groups, without capturing groups, the whole matched text is returned instead.
Demo:
>>> text = "blah blah http://blah.com/users/123 blah blah http://blah.com/users/353"
>>> my_urlfinder = re.compile(r'\shttp:\/\/(?:\S+.|)blah.com/users/(?:\d+)(?:\/|)')
>>> for match in my_urlfinder.findall(text):
... print match
...
http://blah.com/users/123
http://blah.com/users/353
An alternative to not using any capturing groups would be to add another one around everything:
my_urlfinder = re.compile(r'\s(http:\/\/(\S+.|)blah.com/users/(\d+)(\/|))')
This will allow you to keep the inner capturing groups while still having the whole result.
For the demo text it would yield these results:
('http://blah.com/users/123', '', '123', '')
('http://blah.com/users/353', '', '353', '')
As a side note beware that the current expression requires a whitespace in front of the URL, so if the text started with one that would not be matched.