I've got a file that has a ton of text in it. Some of it looks like this:
X-DSPAM-Processed: Fri Jan 4 18:10:48 2008
X-DSPAM-Confidence: 0.6178
X-DSPAM-Probability: 0.0000
Details: http://source.sakaiproject.org/viewsvn/?view=rev&rev=39771
Author: louis#media.berkeley.edu
Date: 2008-01-04 18:08:50 -0500 (Fri, 04 Jan 2008)
New Revision: 39771
Modified:
bspace/site-manage/sakai_2-4-x/site-manage-tool/tool/src/bundle/sitesetupgeneric.properties
bspace/site-manage/sakai_2-4-x/site-manage-tool/tool/src/java/org/sakaiproject/site/tool/SiteAction.java
Log:
BSP-1415 New (Guest) user Notification
I need to pull out only dates that follow this pattern:
2008-01-04 18:08:50 -0500
Here's what I tried:
import re
text = open('mbox-short.txt')
for line in text:
dates = re.compile('\d{4}(?P<sep>[-/])\d{2}(?P=sep)\d{2}\s\d{2}:\d{2}:]\d{2}\s[-/]\d{4}')
print(dates)
text.close()
The return I got was hundreds of:
\d{4}(?P<sep>[-/])\d{2}(?P=sep)\d{2}\s\d{2}:\d{2}:]\d{2}\s[-/]\d{4}
Two things:
First, the regex itself:
regex = re.compile(r'\b\d{4}[-/]\d{2}[-/]\d{2}\s\d{2}:\d{2}:\d{2}\s[-+]\d{4}\b')
Secondly, you need to call regex.findall(file) where file is a string:
>>> regex.findall(file)
['2008-01-04 18:08:50 -0500']
re.compile() produces a compiled regular expression object. findall is one of several methods of this object that let you do the actual searching/matching/finding.
Lastly: you're currently using named capturing groups. ((?P<sep>[-/])) From your question, "I need to pull out only dates that follow this pattern," it doesn't seem like you need these. You want to extract the entire expression, not capture the "separators," which is what capturing groups are designed for.
Full code block:
>>> import re
>>> regex = re.compile(r'\b\d{4}[-/]\d{2}[-/]\d{2}\s\d{2}:\d{2}:\d{2}\s[-+]\d{4}\b')
>>> with open('mbox-short.txt') as f:
... print(regex.findall(f.read()))
...
['2008-01-04 18:08:50 -0500']
Here's another solution.
import re
numberExtractRegex = re.compile(r'(\d\d\d\d[-]\d\d[-]\d\d\s\d\d[:]\d\d[:]\d\d\s[-]\d\d\d\d)')
print(numberExtractRegex.findall('Date: 2008-01-04 18:08:50 -0500 (Fri, 04 Jan 2008), Date: 2010-01-04 18:08:50 -0500 (Fri, 04 Jan 2010)'))
Related
I have a string showing as
text = "isLogIn=true; Max-Age=400; Path=/; Expires=Wed, 25 Sep 2019 17:46:30 GMT, value=qasde% , client=retail"
I need to extract the string showing for value, ie qasde%. I am using Python scripting
What i tried was
text = text.split('value', 1)[1]
text = text.split('client',1)[0]
It gives me the correct value, but is there any simple way to achieve this ?
https://regex101.com/r/u9gpm5/2
import re
text = 'isLogIn=true; Max-Age=400; Path=/; Expires=Wed, 25 Sep 2019 17:46:30 GMT, value=qasde% , client=retail'
m = re.search('value=(.+?) , client=', text)
if m:
found = m.group(1)
Credits to https://stackoverflow.com/a/4667014/11063448
This expression might simply return that:
import re
expression = r'(?:value|client)\s*=\s*([^,\r\n]*?)(?=\s*,|$)'
string = """
isLogIn=true; Max-Age=400; Path=/; Expires=Wed, 25 Sep 2019 17:46:30 GMT, value=qasde% , client=retail
"""
print(re.findall(expression, string, re.M))
Output
['qasde%', 'retail']
If you wish to simplify/modify/explore the expression, it's been explained on the top right panel of regex101.com. If you'd like, you can also watch in this link, how it would match against some sample inputs.
RegEx Circuit
jex.im visualizes regular expressions:
You didn't provide enough information to fully answer the question. Does value always appear before client? Does it always appear after the time and date?
Anyway - this answer, or Quicksilver's answer should work if value does always appear after date and time and before client.
However, you might also just want to search for whatever appears after value= and before ,, in which case this should work: https://regex101.com/r/a4d7Bx/1
import re
text = 'isLogIn=true; Max-Age=400; Path=/; Expires=Wed, 25 Sep 2019 17:46:30 GMT, value=qasde% , client=retail'
m = re.search('value=(.+?) ,', text)
if m:
found = m.group(1)
print(found)
Another way to do this by slightly editing your code:
text = 'isLogIn=true; Max-Age=400; Path=/; Expires=Wed, 25 Sep 2019 17:46:30 GMT, value=qasde% , client=retail'
text = text.split('value=', 1)[1]
text = text.split(' , client',1)[0]
print(text)
You asked for the simpler way. Arguably, both of these methods are pretty simple, so I'd opt for the faster one (unless you want the more general solution, in which case use the first option). I timed them using the following code:
import re
import cProfile
text = 'isLogIn=true; Max-Age=400; Path=/; Expires=Wed, 25 Sep 2019 17:46:30 GMT, value=qasde% , client=retail'
def option1(text):
for i in range(10000):
re.search('value=(.+?) ,', text)
def option2(text):
for i in range(10000):
temp = text.split('value=', 1)[1]
temp = temp.split(' , client',1)[0]
cProfile.run("option1(text)")
print("\n\n\n\n\n")
cProfile.run("option2(text)")
And the first option took 0.076 seconds while the second took 0.027 seconds, so the faster option is using split.
I have a series of url
www.domain.com/calendar.php?month=may.2019
www.domain.com/calendar.php?month=april.2019
www.domain.com/calendar.php?month=march.2019
www.domain.com/calendar.php?month=feb.2019
...
...
...
www.domain.com/calendar.php?month=feb.2007
I wanted to extract the year after month.
What I'm looking for
2019
2019
...
...
2007
and save them into another columns
Here's what I have:
data["urls"].str.extract('(?<=month=).*$')
Fix your code
df["urls"].str.extract('(?<=month=).*\.(\d{4})$')
If you can trust that all do have the same pattern, then these should work.
split
df["urls"].str.rsplit('.', 1).str[-1]
slice
df["urls"].str[-4:]
Here, we can also use simple expression without look-arounds, such as:
.+month=.+\.([0-9]{4})
or:
month=.+\.([0-9]{4})
Demo 1
or:
.+month=.+\.(.+)
or:
month=.+\.(.+)
Demo 2
I have the string like this :
str = '4 167213860 Mar 7 2017 10:37:42 +00:00 c7600rsp72043-advipservicesk9-mz-obs_v151_3_s1_RLS10_ES5'
I want to recover only one part of this word (c7600rsp72043-advipservicesk9-mz-obs_v151_3_s1_RLS10_ES5)
I looking for the regex pattern, but I can't find. I do something like that in python :
import re
str = '4 167213860 Mar 7 2017 10:37:42 +00:00 c7600rsp72043-advipservicesk9-mz-obs_v151_3_s1_RLS10_ES5'
output = re.findall(r'[a-z0-9]rsp[a-zA-Z0-9_-]+$',string)
This return me []
If some one of you can help me I will be very happy.
Use a regex that gets all adjacent non whitespace at the end of the string: \S+$
string = '4 167213860 Mar 7 2017 10:37:42 +00:00 c7600rsp72043-advipservicesk9-mz-obs_v151_3_s1_RLS10_ES5'
output = re.findall(r'\S+$',string)
Working example: https://regex101.com/r/lXFRNT/1
#Ruzhim's answer is good, but if you want to keep on doing it the way you thought about it you could just replace the "rsp" bit with a \w+
output = re.findall(r'[a-z0-9]\w+[a-zA-Z0-9_-]+$', str)
>>>['c7600rsp72043-advipservicesk9-mz-obs_v151_3_s1_RLS10_ES5']
I am trying to extract immunization records of this form:
Immunization: Tetanus
Other: Booster
Method: Injection
Date Received: 07 Jan 2013
and also of this form:
Immunization: TETANUS DIPTHERIA (TD-ADULT)
Date Received: 07 Dec 2012 # 1155
Location: PORTLAND (OR) VAMC
Reaction:* None Reported
Comments: 1234567
Here is my pattern string:
"Immunization:(.*?)\n[.\n*?]*?Date Received:(.*?)\n"
This is identifying the second pattern and extracting vaccination name and date but not the first pattern. I thought that [.\n*?]*? would take care of the two possibilities (that there are other fields between vaccination name and vaccination date...or not...but this doesn't seem to be doing the trick. What is wrong with my regex and how cna I fix it?
You can use:
import re
matches = re.findall(r"Immunization:\s+(.*?)\s+.*?Date Received:\s+(.*?)$", subject, re.IGNORECASE | re.DOTALL | re.MULTILINE)
Regex Demo | Python Demo
Regex Explanation:
Tested this on pythex with MULTILINE and DOTALL:
Input
Immunization: Tetanus
Other: Booster
Method: Injection
Date Received: 07 Jan 2013
Immunization: TETANUS DIPTHERIA (TD-ADULT)
Date Received: 07 Dec 2012 # 1155
Location: PORTLAND (OR) VAMC
Reaction:* None Reported
Comments: 1234567
Pattern: Immunization:\s+(\w+).*?Date Received:\s+([^\n]+)
Match 1
Tetanus
07 Jan 2013
Match 2
TETANUS
07 Dec 2012 # 1155
Pythex
Pythex with different grouping
The . in [.\n] is taken as a literal '.', not as a symbol for any-character. This is why the date line immediately following the immunisation is accepted but you fail to jump across a character that is not a newline or a dot.
(.*\n)* comes to mind to help you out in the closest way to what you already have. However, it is a bit unfortunate to have so many nested * since this means a long breath for parsing the record and as a human I also find it more difficult to understand. It may be preferable to start every loop with a literal to help the decision making if a loop shall be entered/continued at all.
If I did not mess it up then
Immunization:(.*?)(\n.*)*\nDate Received:(.*)\n
would do without left recursion and "Date Received" would only be detected at the beginning of the line.
I am fairly new to Python. An external simulation software I use gives me reports which include data in the following format:
1 29 Jan 2013 07:33:19.273 29 Jan 2013 09:58:10.460 8691.186
I am looking to split the above data into four strings namely;
'1', '29 Jan 2013 07:33:19.273', '29 Jan 2013 09:58:10.460', '8691.186'
I cannot use str.split since it splits out the date into multiple strings. There appears to be four white spaces between 1 and the first date and between the first and second dates. I don't know if this is four white spaces or tabs.
Using '\t' as a delimiter on split doesn't do much. If I specify ' ' (4 spaces) as a delimiter, I get the first three strings. I also then get an empty string and leading spaces in the final string. There are 10 spaces between the second date and the number.
Any suggestions on how to deal with this would be much helpful!
Thanks!
You can split on more than one space with a simple regular expression:
import re
multispace = re.compile(r'\s{2,}') # 2 or more whitespace characters
fields = multispace.split(inputline)
Demonstration:
>>> import re
>>> multispace = re.compile(r'\s{2,}') # 2 or more whitespace characters
>>> multispace.split('1 29 Jan 2013 07:33:19.273 29 Jan 2013 09:58:10.460 8691.186')
['1', '29 Jan 2013 07:33:19.273', '29 Jan 2013 09:58:10.460', '8691.186']
If the data is fixed width you can use character addressing in the string
n=str[0]
d1=str[2:26]
d2=str[27:51]
l=str[52:]
However, if Jan 02 is shown as Jan 2 this may not work as the width of the string may be variable