I need to create a script that will run every 5 minutes via cron and parse syslog, but only match the entries within the last 5 minutes of the execution time.
If there a module I can leverage for part of this? If so, will it work when running at midnight when the date is one day ahead? The date format of my log is:
Jun 20 14:04:47 <log msg details>
You can use datetime's strptime function:
>>> datetime.strptime("Jun 20 14:04:47", "%b %d %H:%M:%S")
# => datetime.datetime(1900, 6, 20, 14, 4, 47)
You can then use simple operators like < to compare datetimes. (You'll have to be aware that if a year is not specified, it will default to 1900.)
I had to write a script that parses the syslog continuously (think "tail -f"), so the case is slightly different, but maybe this helps anyway. I parsed each line using this regex. depending your your syslog implementation you'll have to adapt of course.
pattern=re.compile("^(?P<month>[a-zA-Z]{3})\s+(?P<day>\d\d?)\s(?P<hour>\d\d)\:(?P<minute>\d\d):(?P<second>\d\d)(?:\s(?P<suppliedhost>[a-zA-Z0-9_-]+))?\s(?P<host>[a-zA-Z0-9_-]+)\s(?P<process>[a-zA-Z0-9\/_-]+)(\[(?P<pid>\d+)\])?:\s(?P<message>.+)$")
then I could convert it into a datetime object
minute=int(m.group('minute'))
hour=int(m.group('hour'))
day=int(m.group('day'))
second=int(m.group('second'))
month=monthindex[m.group('month')]
now=datetime.datetime.utcnow()
year=now.year
ts=datetime.datetime(year,month,day,hour,minute,second)
you could then check for each line if that object is within the last 5 minutes.
your problem will be that you'll have to parse the whole syslog in each run unless you store the last file position somewhere.
Related
I am trying to run a job at dawn and dusk everyday using crontab (which seems like the easiest way), as part of the dawn job I want to calculate dawn/dusk for the next day and modify the existing dawn and dusk jobs. I am using the astral module to do the calculation and get the time strings out into an array successfully but then getting them into a format that the crontab accepts is proving challenging.
Below is the code I am using:
today = datetime.date.today()
tomorrow = today + datetime.timedelta(days=1)
s = sun(city.observer, tomorrow)
dawn = s["dawn"]
dusk = s["dusk"]
My_cron_user = CronTab(user='root')
job = My_cron_user.new(command='python3 /home/DoorControl/DoorOpen.py', comment='DoorOpen')
job.setall(datetime(dawn[0:4], dawn[5:7], dawn[8:10], dawn[11:13], dawn[14:16]))
My_cron_user.write()
I have tried several methods to translate the dawn element (which comes out like this 2021-12-17 07:20:37.364901+00:00) of s into a format that job.setall will accept from splitting the string up (as above) through to using datetime, but cannot seem to get it right and hit several different types of errors. Any suggestions would be appreciated, thanks.
#karthiknair, so the below command seems to work and allows me to create the job.
#job.setall(datetime(2021, 12, 15, 7, 00))
The 7 cannot be 07 as that throws an error.
Matt
If I knew the format in which a string represents date-time information, then I can easily use datetime.datetime.strptime(s, fmt). However, without knowing the format of the string beforehand, would it be possible to determine whether a given string contains something that could be parsed as a datetime object with the right format string?
Obviously, generating every possible format string to do an exhaustive search is not a feasible idea. I also don't really want to write one function with many format strings hardcoded into it.
Does anyone have any thoughts on how this can be accomplished (perhaps some sort of regex?)?
What about fuzzyparsers:
Sample inputs:
jan 12, 2003
jan 5
2004-3-5
+34 -- 34 days in the future (relative to todays date)
-4 -- 4 days in the past (relative to todays date)
Example usage:
>>> from fuzzyparsers import parse_date
>>> parse_date('jun 17 2010') # my youngest son's birthday
datetime.date(2010, 6, 17)
Install with:
$ pip install fuzzyparsers
You can use parser from dateutil
Example usage:
from dateutil import parser
dt = parser.parse("Aug 28 1999 12:00AM")
How do I get the current time and date of the operating system (the one in the clock). I tried to use datetime.now(). But it returns different value.
As suggested by mcalex I've rechecked the time and date setting and this has always been like this:
Use time.localtime().
From https://docs.python.org/2/library/time.html#time.localtime
Like gmtime() but converts to local time. If secs is not provided or
None, the current time as returned by time() is used. The dst flag is
set to 1 when DST applies to the given time.
You can use the Python time module for various time-related functions. It appears you are requesting the following format:
Day Month Hour:Min:Sec Year
If so, you can use the following:
>>> import time
>>> time.asctime(time.localtime())
'Mon Jun 30 22:19:34 2014'
To convert to a specific format, you can use the following function:
time.strftime(format[, t])
This converts a struct_time object tm representing a time as returned by gmtime() or localtime() to a string. See the following link for more info on the format codes:
https://docs.python.org/2/library/time.html#time.localtime
Source: Beazley, D. M., "Python Essential Reference", 4th. Ed.
I'm astounded by some code I wrote some time ago. For not entering in much detail i have a method that runs through some objects, wich have a date parameter. If the date parameter is equal to today's date, goes on.
I have set this in my local machine for test and have like 695 objects all with the same date, today, but when the action is run nothing happens, so i debug it to find that my expression date.today() returns datetime.date(2014, 3, 19).
This is is incorrect, as the date of my computer from the date command is Tue Mar 18 20:56:09 AST 2014.
I used from datetime import date. This is one of the more cryptic errors i have ever got. Any experience someone can share here? Thanks a lot.
The method is not timezone aware and there's no platform-independent way to make it so. What is generally done is incorporate something like pytz and call .today() as:
datetime.utcnow().replace(tzinfo = pytz.utc).strftime('%Y-%m-%d')
I have a datetime object created from which I subtract 13 days as follow:
(date.today()-timedelta(days=13)).strftime('%Y-%m-%d')
The strangeness occurs when I execute the code at 6AM and 8:30AM. At 6AM, the resulting string is returned as (if today is 2012-02-29):
2012-02-15
which is 14 days before the current! However, running the same line at 8:30AM, the resulting string is returned as:
2012-02-16
Which then correct. So far I have not been able to figure out what the difference is between the small period of time. I use timezone naive datetime objects, if that is important. I would like to know what could cause this change in the resulting string date.
Many thanks.
EDIT: (based on eumiro's suggestion below)
datetime.datetime.now() returns:
>>> datetime.datetime(2012, 2, 29, 10, 46, 20, 659862)
And the timezone is Europe/Vienna on the server and in the django application that runs the line of code.
I also tried running a similar line to the one you suggested:
(pytz.timezone(settings.TIME_ZONE).localize(datetime.now(), is_dst=True) - \
timedelta(days=13)).strftime('%Y-%m-%d')
But with the same results... which is why I think I don't think it has much to do with timezones also. But at the same time not sure where else to look.
You live somewhere in America? This is the place where the timezones are around 6-8 hours behind the UTC and that's the time of UTC midnight.
What does datetime.datetime.now() return?
If you want to get the real local time, use this (replace "America/New_York" with your timezone):
from datetime import datetime, timedelta
import pytz
now = datetime.datetime.now(pytz.timezone("America/New_York"))
dt = (now - timedelta(days=13)).strftime('%Y-%m-%d')
and it should return the same correct values from midnight until midnight.
Unfortunately DST is poorly supported in Python.
Even pytz is not perfect, but can be made to work with hacks.
You have to decide what it means to subtract 2 days from 10th, 1p.m., either 2 calendar days or 48 hours or 172800 seconds.