Related
I am trying to extract some number value from a text. Skipping is done based on a matching text.
For example :
Input Text -
ABC Company Export Items 4 Bought by XYZ Amount 400.00 with GST# 36479 GST percentage is 20%.
OR
ABC Company Export Items 4 Bought by XYZ Amount 400.00 with GST Reg No. 36479 GST% is 20%.
OR
ABC Company Export Items 4 Bought by XYZ Amount 400.00 with GST Reg# 36479 GST% is 20%.
Output Text -
Amount 400.00
GST 36479
GST 20%
Main point is input text can be in any format but output text should be same. One thing that will be same is GST Number will be non-decimal number, GST percentage will be number followed by "%" symbol and amount will be in decimal form.
I tried but not able to skip the non-numeric value after GST. Please help.
What I tried :
pattern = re.compile(r"\b(?<=GST).\D(\d+)")
You can use
\bAmount\s*(?P<amount>\d+(?:\.\d+)?).*?\bGST\D*(?P<gst_id>\d+(?:\.\d+)?).*?\bGST\D*(?P<gst_prcnt>\d+(?:\.\d+)?%)
See the regex demo. Details:
\bAmount\s* - a whole word Amount and zero or more whitespaces
(?P<amount>\d+(?:\.\d+)?) - Group "amount": one or more digits and then an optional sequence of . and one or more digits
.*? - some text (excluding whitespace)
\bGST - a word GST
\D* - zero or more chars other than digits
(?P<gst_id>\d+(?:\.\d+)?) - Group "gst_id": one or more digits and then an optional sequence of . and one or more digits
.*? - some text (excluding whitespace)
\bGST\D* - a word GST and then zero or more chars other than digits
(?P<gst_prcnt>\d+(?:\.\d+)?%) - Group "gst_prcnt": one or more digits and then an optional sequence of . and one or more digits, and then a % char.
See the Python demo:
import re
pattern = r"\bAmount\s*(?P<amount>\d+(?:\.\d+)?).*?\bGST\D*(?P<gst_id>\d+(?:\.\d+)?).*?\bGST\D*(?P<gst_prcnt>\d+(?:\.\d+)?%)"
texts = ["ABC Company Export Items 4 Bought by XYZ Amount 400.00 with GST# 36479 GST percentage is 20%.",
"ABC Company Export Items 4 Bought by XYZ Amount 400.00 with GST Reg No. 36479 GST% is 20%.",
"ABC Company Export Items 4 Bought by XYZ Amount 400.00 with GST Reg# 36479 GST% is 20%."]
for text in texts:
m = re.search(pattern, text)
if m:
print(m.groupdict())
Output:
{'amount': '400.00', 'gst_id': '36479', 'gst_prcnt': '20%'}
{'amount': '400.00', 'gst_id': '36479', 'gst_prcnt': '20%'}
{'amount': '400.00', 'gst_id': '36479', 'gst_prcnt': '20%'}
I'm working on project which require to extract all the case number from the given string. Can anyone please help me to create a regex to match the pattern for all the case numbers.
Pattern is like: alphanumeric must followed with / alphanumeric must followed with / alphanumeric
*Housekeeping Services For the period( 1‐03‐2020 to 31‐03‐2020) ‐ HDC ‐5i
SL.NO HSN/SAC
Code UOM
Facility
Approved
HC
Total Billing
Hours
Actual Manpower
HC
Unit Rate Per
Month Taxable Value
1 HK Supervisor 9985 HR 4 832 4.00 18,644.00 7 4,576.00*
Case no.**MH20/00285/VAS**
Case no. **MH20/00294/GVN1**
Case no. **MH20/000026/MUMR**
Case no. **KA20/00346/BN**
Case no. **DL20/0024/DLH39**
Case no. **MH20/003B30/GUR2**
Case no. **GJ20/001A75/GJ**
Case no. **GJ20/001A77/GJ**
Case no. **MH20/002CK89/GVN1**
*3,15,962.69
2 8,436.64
2 8,436.64
3,72,836.00
AMOUNT IN WORDS:‐ Rupees Three Lakhs Seventy Two Thousand Eight Hundred Thirty Six Only*
This one should do the Job
[\d\w]{4}/[\d\w]+/[\d\w]+
I have a nested list_3 which looks like:
[['Company OverviewCompany: HowSector: SoftwareYear Founded: 2010One Sentence Pitch: Easily give and request low-quality feedback with your team to achieve more togetherUniversity Affiliation(s): Duke$ Raised: $240,000Investors: Friends & familyTraction to Date: 10% of monthly active users (MAU) are also active weekly'], [['Company OverviewCompany: GrubSector: SoftwareYear Founded: 2018One Sentence Pitch: Find food you likeUniversity Affiliation(s): Stanford$ Raised: $340,000Investors: Friends & familyTraction to Date: 40% of monthly active users (MAU) are also active weekly']]]
I would like to use regex to add a comma followed by a single space between each joined word ie(HowSector:, SoftwareYear, 2010One), So far I have tried to write a re.sub code to do, by selecting all the characters without whitespace and replacing this, but have run into some issues:
for i, list in enumerate(list_3):
list_3[i] = [re.sub('r\s\s+', ', ', word) for word in list]
list_33.append(list_3[i])
print(list_33)
error:
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
I would like the output to be:
[['Company Overview, Company: How, Sector: Software, Year Founded: 2010, One Sentence Pitch: Easily give and request low-quality feedback with your team to achieve more together University, Affiliation(s): Duke, $ Raised: $240,000, Investors: Friends & family, Traction to Date: 10% of monthly active users (MAU) are also active weekly'],[...]]
Any ideas how I can use regex to do this?
The main problem is that your nested list has no constant levels. Sometimes it has 2 levels and sometimes it has 3 levels. This is why you are getting the above error. In the case the list has 3 levels, re.sub receives a list as the third argument instead of a string.
The second problem is that the regex you are using is not the correct regex. The most naive regex we can use here should (at the very least) be able to find a non-whitespace charcter followed by a capital letter.
In the below example code, I'm using re.compile (since the same regex will be used over and over again, we might as well pre-compile it and gain some performance boost) and I'm just printing the output. You'll need to figure out a way to get the output in the format you want.
regex = re.compile(r'(\S)([A-Z])')
replacement = r'\1, \2'
for inner_list in nested_list:
for string_or_list in inner_list:
if isinstance(string_or_list, str):
print(regex.sub(replacement, string_or_list))
else:
for string in string_or_list:
print(regex.sub(replacement, string))
Outputs
Company Overview, Company: How, Sector: Software, Year Founded: 2010, One Sentence Pitch: Easily give and request low-quality feedback with your team to achieve more together, University Affiliation(s): Duke$ Raised: $240,000, Investors: Friends & family, Traction to Date: 10% of monthly active users (, MA, U) are also active weekly
Company Overview, Company: Grub, Sector: Software, Year Founded: 2018, One Sentence Pitch: Find food you like, University Affiliation(s): Stanford$ Raised: $340,000, Investors: Friends & family, Traction to Date: 40% of monthly active users (, MA, U) are also active weekly
I believe you can use the following Python code.
rgx = r'(?<=[a-z\d])([A-Z$][A-Za-z]*(?: +\S+?)*)*:'
rep = r', \1:'
re.sub(rgx, rep, s)
where s is the string.
Start your engine! | Python code
Python's regex engine performs the following operations when matching.
(?<= : begin positive lookbehind
[a-z\d] : match a letter or digit
) : end positive lookbehind
( : begin capture group 1
[A-Z$] : match a capital letter or '$'
[A-Za-z]* : match 0+ letters
(?: +\S+?) : match 1+ spaces greedily, 1+ non-spaces
non-greedily in a non-capture group
* : execute non-capture group 0+ times
) : end capture group
: : match ':'
Note that the positive lookbehind and permissible characters for each token in the capture group may need to be adjusted to suit requirements.
The regular expression employed to construct replacement strings (, \1:) creates the string ', ' followed by the contents of capture group 1 followed by a colon.
If your list of lists is arbitrary deep, you can recursively traverse it and process (with THIS regex) the strings and yield the same structure:
import re
from collections.abc import Iterable
def process(l):
for el in l:
if isinstance(el, Iterable) and not isinstance(el, (str, bytes)):
yield type(el)(process(el))
else:
yield ', '.join(re.split(r'(?<=[a-z])(?=[A-Z])', el))
Given your example as LoL here is the result:
>>> list(process(LoL))
[['Company Overview, Company: How, Sector: Software, Year Founded: 2010One Sentence Pitch: Easily give and request low-quality feedback with your team to achieve more together, University Affiliation(s): Duke$ Raised: $240,000Investors: Friends & family, Traction to Date: 10% of monthly active users (MAU) are also active weekly'], [['Company Overview, Company: Grub, Sector: Software, Year Founded: 2018One Sentence Pitch: Find food you like, University Affiliation(s): Stanford$ Raised: $340,000Investors: Friends & family, Traction to Date: 40% of monthly active users (MAU) are also active weekly']]]
I've written a script in python to get certain from a text container. I used re module to do the job. However, it is giving me unnecesary output along with the required ones.
How can I modify my expression to be stick to the lines I wanna grab?
This is my try:
import re
content = """
A Gross exaggeration,
-- Gross 5 90,630,08,
Gross 4 13,360,023,
Gross 2 70,940,02,
Luke gross is an actor
"""
for item in re.finditer(r'Gross(?:[\d\s,]*)',content):
print(item.group().strip())
Output I'm having:
Gross
Gross 5 90,630,08,
Gross 4 13,360,023,
Gross 2 70,940,02,
Output I wish to have:
Gross 4 13,360,023
Gross 2 70,940,02
I changed the regex string to r'(?:^\s*?)Gross[\d\s,]*?(?=,$)' and added multiline flag (online regex here):
import re
content = """
A Gross exaggeration,
-- Gross 5 90,630,08,
Gross 4 13,360,023,
Gross 2 70,940,02,
Luke gross is an actor
"""
for item in re.finditer(r'(?:^\s*?)Gross[\d\s,]*?(?=,$)',content, flags=re.M):
print(item.group().strip())
Output is:
Gross 4 13,360,023
Gross 2 70,940,02
^\s*Gross[\d ,]*(?=,) Will capture what you want.
I just tacked on ^ to signal the start of the line, used \s* to indicate optional whitespace before "gross" and trimmed the , from the end. I also removed your \s from your character class because it worked with new lines. I replaced it with a blank space.
Demo
I am relatively new to regex (always struggled with it for some reason)...
I have text that is of this form:
David Meredith, Financial Director sold post-exercise 15,000 shares in the company on YYYY-mm-dd at a price of 1044.00p. The Director now holds 6,290 shares representing 0.01% of the...
Mark Brookes, Non Executive Director bought 811 shares in the company on YYYY-mm-dd at a price of 76.75p. The Director now holds 189,952 shares representing 0.38% of the shares in...
Albert Ellis, CEO bought 262 shares in the company on YYYY-mm-dd at a price of 52.00p. The Director now holds 465,085 shares. NOTE: Purchased through Co's SIP Story provided by...
Parsing the text reveals the following structure:
Two or more words beginning the sentence, and before the first comma, is the name of the person involved in the transaction
One or more words before ('sold'|'bought'|'exercised'|'sold post-exercise') is the title of the person
Presence of either one of these: ('sold'|'bought'|'exercised'|'sold post-exercise') AFTER the title, identifies the transaction type
first numeric string following the transaction type ('sold'|'bought'|'exercised'|'sold post-exercise') denotes the size of the transaction
'price of ' preceeds a numeric string, which specifies the price at which the deal was struck.
My question is:
How can I use this knowledge (and regex), to write a function that parses similar text to return the variables of interest (listed 1 - 5 above)?
Pseudo code for the function I want to write ..
def grok_directors_dealings_text(text_input):
name, title, transaction_type, lot_size, price = (None, None, None, None, None)
....
name = ...
title = ...
transaction_type = ...
lot_size = ...
price = ...
pass
How would I use regex to implement the functions to return the variables of interest when passed in text that conforms to the structure I have identified above?
[[Edit]]
For some reason, I have seemed to struggle with regex for a while, if I am to learn from the correct answer here on S.O, it will be much better, if an explanation is offered as to why the magical expression (sorry, regexpr) actually works.
I want to actually learn this stuff instead of copy pasting expressions ...
You can use the following regex:
(.*?),\s(.*)\s(sold(?: post-exercise)?|bought|exercised)\s*([\d,]*).*price of\s*(\d*.\d+?p)
DEMO
Python:
import re
financialData = """
David Meredith, Financial Director sold post-exercise 15,000 shares in the company on YYYY-mm-dd at a price of 1044.00p. The Director now holds 6,290 shares representing 0.01% of the...
Mark Brookes, Non Executive Director bought 811 shares in the company on YYYY-mm-dd at a price of 76.75p. The Director now holds 189,952 shares representing 0.38% of the shares in...
Albert Ellis, CEO bought 262 shares in the company on YYYY-mm-dd at a price of 52.00p. The Director now holds 465,085 shares. NOTE: Purchased through Co's SIP Story provided by...
"""
print(re.findall('(.*?),\s(.*)\s(sold(?: post-exercise)?|bought|exercised)\s*([\d,]*).*price of\s*(\d*.\d+?p)',financialData))
Output:
[('David Meredith', 'Financial Director', 'sold post-exercise', '15,000', '1044.00p'), ('Mark Brookes', 'Non Executive Director', 'bought', '811', '76.75p'), ('Albert Ellis', 'CEO', 'bought', '262', '52.00p')]
EDIT 1
To understand how and what they mean, follow the DEMO link,on top right you can find a block explaining what each and every character means as follows:
Also Debuggex helps you simulate the string by showing what group matches which characters!
Here's a debuggex demo for your particular case:
(.*?),\s(.*)\s(sold(?: post-exercise)?|bought|exercised)\s*([\d,]*).*price of\s*(\d*.\d+?p)
Debuggex Demo
I came up with this regex:
([\w ]+), ([\w ]+) (sold post-exercise|sold|bought|exercised) ([\d,\.]+).*price of ([\d\.,]+)p
Debuggex Demo
Basically, we are using the parenthesis to capture the important info you want so let's check it out each one:
([\w ]+): \w matches any word character [a-zA-Z0-9_] one or more times, this will give us the name of the person;
([\w ]+)Another one of these after a space and comma to get the title;
(sold post-exercise|sold|bought|exercised) then we search for our transaction types. Notice I put the post-exercise before the post so that it tries to match the bigger word first;
([\d,\.]+) Then we try to find the numbers, which are made of digits (\d), a comma and probbably a dot may appear as well;
([\d\.,]+) Then we need to get to the price which is basically the same as the size of the transaction.
The regex that connects each group are pretty basic as well.
If you try it on regex101 it provides some explanation about the regex and generates this code in python to use:
import re
p = re.compile(ur'([\w ]+), ([\w ]+) (sold post-exercise|sold|bought|exercised) ([\d,\.]+).*price of ([\d\.,]+)p')
test_str = u"David Meredith, Financial Director sold post-exercise 15,000 shares in the company on YYYY-mm-dd at a price of 1044.00p. The Director now holds 6,290 shares representing 0.01% of the...\n\nMark Brookes, Non Executive Director bought 811 shares in the company on YYYY-mm-dd at a price of 76.75p. The Director now holds 189,952 shares representing 0.38% of the shares in...\n\nAlbert Ellis, CEO bought 262 shares in the company on YYYY-mm-dd at a price of 52.00p. The Director now holds 465,085 shares. NOTE: Purchased through Co's SIP Story provided by..."
re.findall(p, test_str)
You can use the following regex that just looks for characters surrounding the delimiters:
(.*?), (.*?) (sold post-exercise|bought|exercised|sold) (.*?) shares .*? price of (.*?)p
The parts in parentheses will be captured as groups.
>>> import re
>>> l = ['''David Meredith, Financial Director sold post-exercise 15,000 shares in the company on YYYY-mm-dd at a price of 1044.00p. The Director now holds 6,290 shares representing 0.01% of the...''', '''Mark Brookes, Non Executive Director bought 811 shares in the company on YYYY-mm-dd at a price of 76.75p. The Director now holds 189,952 shares representing 0.38% of the shares in...''', '''Albert Ellis, CEO bought 262 shares in the company on YYYY-mm-dd at a price of 52.00p. The Director now holds 465,085 shares. NOTE: Purchased through Co's SIP Story provided by...''']
>>> for s in l:
... print(re.findall(r'(.*?), (.*?) (sold post-exercise|bought|exercised|sold) (.*?) shares .*? price of (.*?)p', s))
...
[('David Meredith', 'Financial Director', 'sold post-exercise', '15,000', '1044.00')]
[('Mark Brookes', 'Non Executive Director', 'bought', '811', '76.75')]
[('Albert Ellis', 'CEO', 'bought', '262', '52.00')]
this is the regex that will do it
(.*?),(.*?)(sold post-exercise|sold|bought|exercised).*?([\d|,]+).*?price of ([\d|\.]+)
you use it like this
import re
def get_data(line):
pattern = r"(.*?),(.*?)(sold post-exercise|sold|bought|exercised).*?([\d|,]+).*?price of ([\d|\.]+)"
m = re.match(pattern, line)
return m.groups()
for the first line this will return
('David Meredith', ' Financial Director ', 'sold post-exercise', '15,000', '1044.00')
EDIT:
adding explanation
this regex works as follows
the first characters (.*?), mean - take the string until the next match(witch is the ,)
. means every character
the * means that it can be many times (many characters and not just 1)
? means dont be greedy, that means that it will use the first ',' and another one (if there are many ',')
after that there is this again (.*?)
again take the characters until the next thing to match (with is the constant words)
after that there is (sold post-exercise|sold|bought|exercised) witch means - find one of the words (sperated by | )
after that there is a .*? witch again means take all text until next match (this time it is not surounded by () so it wont be selected as a group and wont be part of the output)
([\d|,]+) means take a digit (\d) or a comma. the + stands for one or more times
again .*? like before
'price of ' finds the actual string 'price of '
and last ([\d|.]+) means again take a digit or a dot (escaped because the character . is used by regex for 'any character') one or more times