RFC 1123 Date Representation in Python? - python

Is there a fairly easy way to convert a datetime object into an RFC 1123 (HTTP/1.1) date/time string, i.e. a string with the format
Sun, 06 Nov 1994 08:49:37 GMT
Using strftime does not work, since the strings are locale-dependant. Do I have to build the string by hand?


You can use wsgiref.handlers.format_date_time from the stdlib which does not rely on locale settings
from wsgiref.handlers import format_date_time
from datetime import datetime
from time import mktime
now = datetime.now()
stamp = mktime(now.timetuple())
print format_date_time(stamp) #--> Wed, 22 Oct 2008 10:52:40 GMT
You can use email.utils.formatdate from the stdlib which does not rely on locale settings
from email.utils import formatdate
from datetime import datetime
from time import mktime
now = datetime.now()
stamp = mktime(now.timetuple())
print formatdate(
timeval = stamp,
localtime = False,
usegmt = True
) #--> Wed, 22 Oct 2008 10:55:46 GMT
If you can set the locale process wide then you can do:
import locale, datetime
locale.setlocale(locale.LC_TIME, 'en_US')
datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')
If you don't want to set the locale process wide you could use Babel date formating
from datetime import datetime
from babel.dates import format_datetime
now = datetime.utcnow()
format = 'EEE, dd LLL yyyy hh:mm:ss'
print format_datetime(now, format, locale='en') + ' GMT'
A manual way to format it which is identical with wsgiref.handlers.format_date_time is:
def httpdate(dt):
"""Return a string representation of a date according to RFC 1123
(HTTP/1.1).
The supplied date must be in UTC.
"""
weekday = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"][dt.weekday()]
month = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep",
"Oct", "Nov", "Dec"][dt.month - 1]
return "%s, %02d %s %04d %02d:%02d:%02d GMT" % (weekday, dt.day, month,
dt.year, dt.hour, dt.minute, dt.second)


You can use the formatdate() function from the Python standard email module:
from email.utils import formatdate
print formatdate(timeval=None, localtime=False, usegmt=True)
Gives the current time in the desired format:
Wed, 22 Oct 2008 10:32:33 GMT
In fact, this function does it "by hand" without using strftime()


If anybody reading this is working on a Django project, Django provides a function django.utils.http.http_date(epoch_seconds).
from django.utils.http import http_date
some_datetime = some_object.last_update
response['Last-Modified'] = http_date(some_datetime.timestamp())
This function uses email.utils.formatdate, this is equivalent to:
from email.utils import formatdate
response['Last-Modified'] = formatdate(some_datetime.timestamp(), usegmt=True)


You can set LC_TIME to force stftime() to use a specific locale:
>>> locale.setlocale(locale.LC_TIME, 'en_US')
'en_US'
>>> datetime.datetime.now().strftime(locale.nl_langinfo(locale.D_T_FMT))
'Wed 22 Oct 2008 06:05:39 AM '


Well, here is a manual function to format it:
def httpdate(dt):
"""Return a string representation of a date according to RFC 1123
(HTTP/1.1).
The supplied date must be in UTC.
"""
weekday = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"][dt.weekday()]
month = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep",
"Oct", "Nov", "Dec"][dt.month - 1]
return "%s, %02d %s %04d %02d:%02d:%02d GMT" % (weekday, dt.day, month,
dt.year, dt.hour, dt.minute, dt.second)

Related

How to validate if a date indicated as a string belongs to an interval of 2 dates indicated in another string?

import os, datetime
content = os.listdir("when")
print(content)
#this print...
#['2022_-_12_-_29 12pp33 am _--_ 2023_-_01_-_25 19pp13 pm.txt', '2023-02-05 00pp00 am.txt']
for i in range(len(content)):
content[i] = content[i].replace("_-_", "-").replace("pp", ":")
print(content) #I prepare the input to use it to search
#this print...
#['2022-12-29 12:33 am _--_ 2023-01-25 19:13 pm.txt', '2023-02-05 00:00 am.txt']
input_to_search_in_folder = "2022_-_01_-_05 12:33 am" #file data to find in the 'when' folder
I have changed the : to pp (referring to point-point) because you cannot place : in folders or/and files, at least not in Windows
2022_-_12_-_29 12pp33 am _--_ 2023_-_01_-_25 19pp13 pm
initial date _--_ final date
In this case input_to_search_in_folder = "2022_-_01_-_05 12:33 am" does not match a file with a specific date name. But if it belongs to the interval of days indicated in the file name '2022_-_12_-_29 12pp33 am _--_ 2023_-_01_-_25 19pp13 pm.txt'
How could I validate that this date "2022_-_01_-_05 12:33 am" does belong to that time interval '2022_-_12_-_29 12pp33 am _--_ 2023_-_01_-_25 19pp13 pm' or if it's this date '2023-02-05 00:00 am'?
If the validation is successful, the program should print the content inside that .txt (in this case inside the 2022_-_12_-_29 12pp33 am _--_ 2023_-_01_-_25 19pp13 pm.txt )
text_file = open("when/" + , "r")
data_inside_this_file = text_file.read()
text_file.close()
#And finally prints the content of the .txt file that matches the date specified in the 'input_to_search_in_folder' variable
print(repr(data_inside_this_file))

I would clean the strings fully, convert them to datetime objects (because these can be compared to each other), then compare then and you have the result and can do whatever with it:
import os
from datetime import datetime
content = os.listdir("when")
print(content)
#['2022_-_12_-_29 12pp33 am _--_ 2023_-_01_-_25 19pp13 pm.txt', '2023-02-05 00pp00 am.txt']
for i in range(len(content)):
content[i] = content[i].replace("_-_", "-").replace("pp", ":")
#['2022-12-29 12:33 am _--_ 2023-01-25 19:13 pm.txt', '2023-02-05 00:00 am.txt']
cleaned_filename = os.path.splitext(content[0])[0] #="2022-12-29 12:33 am _--_ 2023-01-25 19:13 pm"
start_dt = datetime.strptime(content[0].split(" _--_ ")[0], "%Y-%m-%d %H:%M")
#="2022-12-29 12:33 am" = datetime(2022, 12, 29, 12, 33)
last_dt = datetime.strptime(content[0].split(" _--_ ")[1], "%Y-%m-%d %H:%M")
#="2023-01-25 19:13 pm"
third_dt = datetime.strptime(os.path.splitext(content[1])[0], "%Y-%m-%d %H:%M")
#="2023-02-05 00:00 am"
input_to_search = "2022_-_01_-_05 12:33 am".replace("_-_", "-")
#"2022-01-05 12:33 am".
input_dt = datetime.strptime(input_to_search, "%Y-%m-%d %H:%M")
#="datetime(2022, 01, 05, 12, 33)"
if start_dt <= input_dt <= last_dt:
print("in between")
elif input_dt == third_dt:
print("Match")
else:
print("No!")

A way is to extract the dates using regex and then convert them as date like mrblue6's answer:
#!/usr/bin/python3
from datetime import datetime
import re
# Let's assume this is one of the directory entries
direntry='2022_-_12_-_29 12pp33 am _--_ 2023_-_01_-_25 19pp13 pm.txt'
# We exclude in the regex the AM/PM part since the format is 24-hour clock
datePattern = '(\d{4}_-_\d{2}_-_\d{2} \d{2}pp\d{2}) [ap]m'
dirPattern = f'{datePattern} _--_ {datePattern}.txt'
# Let's extract the "milestone" dates
matches = re.search(dirPattern, direntry)
extractedDate1 = matches.group(1)
extractedDate2 = matches.group(2)
# Let's extract the date to check
matches = re.search(datePattern, "2022_-_01_-_05 12pp33 am")
extractedDateToCheck = matches.group(1)
# Let's convert them as date time
readDateFormat = '%Y_-_%m_-_%d %Hpp%M'
date1 = datetime.strptime(extractedDate1, readDateFormat)
date2 = datetime.strptime(extractedDate2, readDateFormat)
dateToCheck = datetime.strptime(extractedDateToCheck, readDateFormat)
# Let's compare them
print (f"Date 1 : {date1}")
print (f"Date 2 : {date2}")
print (f"Date to check: {dateToCheck}")
print (f"Check: {date1 <= dateToCheck <= date2}" )
Output:
Date 1 : 2022-12-29 12:33:00
Date 2 : 2023-01-25 19:13:00
Date to check: 2022-01-05 12:33:00
Check: False

Date format conversion 'Wed Oct 20 16:42:04 +0000 2021' on Python

I wrote a script for receiving data from Twitter, now I'm doing column splitting, I want the date and time to be in separate columns.
I get a date like: Wed Oct 20 16:42:04 +0000 2021
I do it with the following code:
filtered_data['date'] = tweet['created_at']
Next, I want to convert this date at the time of receipt into two fields using datetime
date_formats = '%d-%m-%Y'
time_formats = '%H:%M:%S'
At the time of application:
filtered_data['date'] = datetime.strptime(tweet['created_at'], date_formats)
I get the following error:
time data 'Wed Oct 20 16:42:04 +0000 2021' does not match format '%d-%m-%Y'
Tell me how I can do this transformation and is it possible to do it at all

If you want a datetime object:
given
tweet['created_at'] = 'Wed Oct 20 16:42:04 +0000 2021'
use
input_format = '%a %b %d %H:%M:%S %z %Y'
filtered_data['date'] = datetime.strptime( \
tweet['created_at'], \
input_format)
results in
>>> filtered_data['date']
datetime(2021, 10, 20, 16, 42, 4, tzinfo=datetime.timezone.utc)
If you want a formatted string as '%d-%m-%Y':
given
tweet['created_at'] = 'Wed Oct 20 16:42:04 +0000 2021'
use
input_format = '%a %b %d %H:%M:%S %z %Y'
output_format = '%d-%m-%Y'
filtered_data['date'] = datetime.strptime( \
tweet['created_at'], \
input_format).strftime(output_format)
results in
>>> filtered_data['date']
'20-10-2021'
Reference
See python datetime.strptime and datetime.strftime

You need to use the current format first. See an example:
from datetime import datetime
my_date = 'Wed Oct 20 16:42:04 +0000 2021'
initial_format = '%a %b %d %H:%M:%S %z %Y'
final_format = '%d-%m-%Y'
new_date = datetime.strptime(my_date, initial_format).strftime(final_format)
print(new_date)
Output:
20-10-2021
So in your case, try:
filtered_data['date'] = datetime.strptime(tweet['created_at'], '%a %b %d %H:%M:%S %z %Y').strftime(date_formats)

Get the format in dateutil.parse

Is there a way to get the "format" after parsing a date in dateutil. For example something like:
>>> x = parse("2014-01-01 00:12:12")
datetime.datetime(2014, 1, 1, 0, 12, 12)
x.get_original_string_format()
YYYY-MM-DD HH:MM:SS # %Y-%m-%d %H:%M:%S
# Or, passing the date-string directly
get_original_string_format("2014-01-01 00:12:12")
YYYY-MM-DD HH:MM:SS # %Y-%m-%d %H:%M:%S
Update: I'd like to add a bounty to this question to see if someone could add an answer that would do the equivalent on getting the string-format of a common date-string passed. It can use dateutil if you want, but it doesn't have to. Hopefully we'll get some creative solutions here.

Is there a way to get the "format" after parsing a date in dateutil?
Not possible with dateutil. The problem is that dateutil never has the format as an intermediate result any time during the parsing as it detects separate components of the datetime separately - take a look at this not quite easy to read source code.

I don't know of a way that you can return the parsed format from dateutil (or any other python timestamp parser that I know of).
Implementing your own timestamp parsing function that returns a list of possible formats and related datetime objects is fairly trivial using datetime.strptime() but doing it efficiently against a broadly useful list of possible timestamp formats is not.
The following example utilizes a list of just over 100 formats. It does not even scratch the surface of the wide variety of formats parsed by dateutil. It tests each format in sequence until it exhausts all formats in the list (likely much less efficient than the dateutil approach of locating the various datetime parts independently as noted in the answer from #alecxe).
In addition, I have included some example timestamp formats that include time zone names (instead of offsets). If you run the example function below against those particular datetime strings, you may find that it does not return the expected matches even though I have included matching formats using the %Z directive. Some explanation for the challenges with using %Z to handle time zone names can be found in issue 22377 at bugs.python.org (just to highlight another non-trivial aspect of implementing your own datetime parsing function).
With all of those caveats, if you are dealing with a manageable set of potential formats, implementing something simple like the below may get you what you need.
Example function that attempts to match a datetime string against a list of formats and return a dict that includes the original datestring and a list of matches, each a dict that includes a datetime object along with the matched format:
from datetime import datetime
def parse_timestamp(datestring, formats):
results = {'datestring': datestring, 'matches': []}
for f in formats:
try:
d = datetime.strptime(datestring, f)
except:
continue
results['matches'].append({'datetime': d, 'format': f})
return results
Example formats and datetime strings:
formats = ['%A, %B %d, %Y', '%A, %B %d, %Y %I:%M:%S %p %Z', '%A, %d %B %Y', '%B %d %Y', '%B %d, %Y', '%H:%M:%S', '%H:%M:%S,%f', '%H:%M:%S.%f', '%Y %b %d %H:%M:%S.%f', '%Y %b %d %H:%M:%S.%f %Z', '%Y %b %d %H:%M:%S.%f*%Z', '%Y%m%d %H:%M:%S.%f', '%Y-%m-%d %H:%M:%S %z', '%Y-%m-%d %H:%M:%S%z', '%Y-%m-%d %H:%M:%S,%f', '%Y-%m-%d %H:%M:%S,%f%z', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M:%S.%f%z', '%Y-%m-%d %I:%M %p', '%Y-%m-%d %I:%M:%S %p', '%Y-%m-%d*%H:%M:%S', '%Y-%m-%d*%H:%M:%S:%f', '%Y-%m-%dT%H:%M:%S', '%Y-%m-%dT%H:%M:%S%Z', '%Y-%m-%dT%H:%M:%S%z', '%Y-%m-%dT%H:%M:%S*%f%z', '%Y-%m-%dT%H:%M:%S.%f', '%Y-%m-%dT%H:%M:%S.%f%z', '%Y/%m/%d', '%Y/%m/%d*%H:%M:%S', '%a %b %d %H:%M:%S %Z %Y', '%a, %d %b %Y %H:%M:%S %z', '%b %d %H:%M:%S', '%b %d %H:%M:%S %Y', '%b %d %H:%M:%S %z', '%b %d %H:%M:%S %z %Y', '%b %d %Y', '%b %d %Y %H:%M:%S', '%b %d, %Y', '%b %d, %Y %I:%M:%S %p', '%b.%d.%Y', '%d %B %Y', '%d %B %Y %H:%M:%S %Z', '%d %b %Y %H:%M:%S', '%d %b %Y %H:%M:%S %z', '%d %b %Y %H:%M:%S*%f', '%d%m_%H:%M:%S', '%d%m_%H:%M:%S.%f', '%d-%b-%Y', '%d-%b-%Y %H:%M:%S', '%d-%b-%Y %H:%M:%S.%f', '%d-%b-%Y %I:%M:%S %p', '%d-%m-%Y', '%d-%m-%Y %I:%M %p', '%d-%m-%Y %I:%M:%S %p', '%d-%m-%y', '%d-%m-%y %I:%M %p', '%d-%m-%y %I:%M:%S %p', '%d/%b %H:%M:%S,%f', '%d/%b/%Y %H:%M:%S', '%d/%b/%Y %I:%M %p', '%d/%b/%Y:%H:%M:%S', '%d/%b/%Y:%H:%M:%S %z', '%d/%m/%Y', '%d/%m/%Y %H:%M:%S %z', '%d/%m/%Y %I:%M %p', '%d/%m/%Y %I:%M:%S %p', '%d/%m/%Y %I:%M:%S %p:%f', '%d/%m/%Y*%H:%M:%S', '%d/%m/%Y*%H:%M:%S*%f', '%d/%m/%y', '%d/%m/%y %H:%M:%S', '%d/%m/%y %H:%M:%S %z', '%d/%m/%y %I:%M %p', '%d/%m/%y %I:%M:%S %p', '%d/%m/%y*%H:%M:%S', '%m%d_%H:%M:%S', '%m%d_%H:%M:%S.%f', '%m-%d-%Y', '%m-%d-%Y %I:%M %p', '%m-%d-%Y %I:%M:%S %p', '%m-%d-%y', '%m-%d-%y %I:%M %p', '%m-%d-%y %I:%M:%S %p', '%m/%d/%Y', '%m/%d/%Y %H:%M:%S %z', '%m/%d/%Y %I:%M %p', '%m/%d/%Y %I:%M:%S %p', '%m/%d/%Y %I:%M:%S %p:%f', '%m/%d/%Y*%H:%M:%S', '%m/%d/%Y*%H:%M:%S*%f', '%m/%d/%y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S %z', '%m/%d/%y %I:%M %p', '%m/%d/%y %I:%M:%S %p', '%m/%d/%y*%H:%M:%S', '%y%m%d %H:%M:%S', '%y-%m-%d %H:%M:%S', '%y-%m-%d %H:%M:%S,%f', '%y-%m-%d %H:%M:%S,%f %z', '%y/%m/%d %H:%M:%S']
datestrings = ['03-11-1999', '03-12-1999 5:06 AM', '03-12-1999 5:06:07 AM', '03-12-99 5:06 AM', '03-12-99 5:06:07 AM', '03/12/1999', '03/12/1999 5:06 AM', '03/12/1999 5:06:07 AM', '03/12/99 5:06 AM', '03/12/99 5:06:07', '03/12/99 5:06:07 AM', '04/23/17 04:34:22 +0000', '0423_11:42:35', '0423_11:42:35.883', '05/09/2017*08:22:14*612', '06/01/22 04:11:05', '08/10/11*13:33:56', '10-04-19 12:00:17', '10-06-26 02:31:29,573', '10/03/2017 07:29:46 -0700', '11-02-11 16:47:35,985 +0000', '11/22/2017*05:13:11', '11:42:35', '11:42:35,173', '11:42:35.173', '12/03/1999', '12/03/1999 5:06 AM', '12/03/99 5:06 AM', '12/3/1999', '12/3/1999 5:06 AM', '12/3/1999 5:06:07 AM', '150423 11:42:35', '19/Apr/2017:06:36:15 -0700', '1999-03-12 05:06:07.0', '1999-03-12 5:06 AM', '1999-03-12 5:06:07 AM', '1999-03-12+01:00', '1999-3-12 5:06 AM', '1999-3-12 5:06:07 AM', '1999/3/12', '20150423 11:42:35.173', '2017 Mar 03 05:12:41.211 PDT', '2017 Mar 10 01:44:20.392', '2017-02-11T18:31:44', '2017-03-10 14:30:12,655+0000', '2017-03-12 13:11:34.222-0700', '2017-03-12T17:56:22-0700', '2017-06-26 02:31:29,573', '2017-07-01T14:59:55.711+0000', '2017-07-04*13:23:55', '2017-07-22T16:28:55.444', '2017-08-19 12:17:55 -0400', '2017-08-19 12:17:55-0400', '2017-09-08T03:13:10', '2017-10-14T22:11:20+0000', '2017-10-30*02:47:33:899', '2017-11-22T10:10:15.455', '2017/04/12*19:37:50', '2018 Apr 13 22:08:13.211*PDT', '2018-02-27 15:35:20.311', '2018-08-20T13:20:10*633+0000', '22 Mar 1999 05:06:07 +0100', '22 March 1999', '22 March 1999 05:06:07 CET', '22-Mar-1999', '22-Mar-1999 05:06:07', '22-Mar-1999 5:06:07 AM', '22/03/1999 5:06:07 AM', '22/Mar/1999 5:06:07 +0100', '22/Mar/99 5:06 AM', '23 Apr 2017 10:32:35*311', '23 Apr 2017 11:42:35', '23-Apr-2017 11:42:35', '23-Apr-2017 11:42:35.883', '23/Apr 11:42:35,173', '23/Apr/2017 11:42:35', '23/Apr/2017:11:42:35', '3-11-1999', '3-12-1999 5:06 AM', '3-12-99 5:06 AM', '3-12-99 5:06:07 AM', '3-22-1999 5:06:07 AM', '3/12/1999', '3/12/1999 5:06 AM', '3/12/1999 5:06:07 AM', '3/12/99 5:06 AM', '3/12/99 5:06:07', '8/5/2011 3:31:18 AM:234', '9/28/2011 2:23:15 PM', 'Apr 20 00:00:35 2010', 'Dec 2, 2017 2:39:58 AM', 'Jan 21 18:20:11 +0000 2017', 'Jun 09 2018 15:28:14', 'Mar 16 08:12:04', 'Mar 22 1999', 'Mar 22, 1999', 'Mar 22, 1999 5:06:07 AM', 'Mar.22.1999', 'March 22 1999', 'March 22, 1999', 'Mon Mar 22 05:06:07 CET 1999', 'Mon, 22 Mar 1999 05:06:07 +0100', 'Monday, 22 March 1999', 'Monday, March 22, 1999', 'Monday, March 22, 1999 5:06:07 AM CET', 'Sep 28 19:00:00 +0000']
Example usage:
print(parse_timestamp('2018-08-20T13:20:10*633+0000', formats))
# OUTPUT
# {'datestring': '2018-08-20T13:20:10*633+0000', 'matches': [{'datetime': datetime.datetime(2018, 8, 20, 13, 20, 10, 633000, tzinfo=datetime.timezone.utc), 'format': '%Y-%m-%dT%H:%M:%S*%f%z'}]}

My idea was to:
Create an object that has a list of candidate specifiers you think might be in the date pattern (the more you add, the more possibilities you will get out the other end)
Parse the date string
Create a list of possible specifiers for each element in the string, based on the date and the list of candidates you supplied.
Recombine them to produce a list of 'possibles'.
If you get only a single candidate, you can be pretty sure is it the right format. But you will often get many possibilities (especially with dates, months, minutes and hours all in the 0-10 range).
Example class:
import re
from itertools import product
from dateutil.parser import parse
from collections import defaultdict, Counter
COMMON_SPECIFIERS = [
'%a', '%A', '%d', '%b', '%B', '%m',
'%Y', '%H', '%p', '%M', '%S', '%Z',
]
class FormatFinder:
def __init__(self,
valid_specifiers=COMMON_SPECIFIERS,
date_element=r'([\w]+)',
delimiter_element=r'([\W]+)',
ignore_case=False):
self.specifiers = valid_specifiers
joined = (r'' + date_element + r"|" + delimiter_element)
self.pattern = re.compile(joined)
self.ignore_case = ignore_case
def find_candidate_patterns(self, date_string):
date = parse(date_string)
tokens = self.pattern.findall(date_string)
candidate_specifiers = defaultdict(list)
for specifier in self.specifiers:
token = date.strftime(specifier)
candidate_specifiers[token].append(specifier)
if self.ignore_case:
candidate_specifiers[token.
upper()] = candidate_specifiers[token]
candidate_specifiers[token.
lower()] = candidate_specifiers[token]
options_for_each_element = []
for (token, delimiter) in tokens:
if token:
if token not in candidate_specifiers:
options_for_each_element.append(
[token]) # just use this verbatim?
else:
options_for_each_element.append(
candidate_specifiers[token])
else:
options_for_each_element.append([delimiter])
for parts in product(*options_for_each_element):
counts = Counter(parts)
max_count = max(counts[specifier] for specifier in self.specifiers)
if max_count > 1:
# this is a candidate with the same item used more than once
continue
yield "".join(parts)
And some sample tests:
def test_it_returns_value_from_question_1():
s = "2014-01-01 00:12:12"
candidates = FormatFinder().find_candidate_patterns(s)
sut = FormatFinder()
candidates = sut.find_candidate_patterns(s)
assert "%Y-%m-%d %H:%M:%S" in candidates
def test_it_returns_value_from_question_2():
s = 'Jan. 04, 2017'
sut = FormatFinder()
candidates = sut.find_candidate_patterns(s)
candidates = list(candidates)
assert "%b. %d, %Y" in candidates
assert len(candidates) == 1
def test_it_can_ignore_case():
# NB: apparently the 'AM/PM' is meant to be capitalised in my locale!
# News to me!
s = "JANUARY 12, 2018 02:12 am"
sut = FormatFinder(ignore_case=True)
candidates = sut.find_candidate_patterns(s)
assert "%B %d, %Y %H:%M %p" in candidates
def test_it_returns_parts_that_have_no_date_component_verbatim():
# In this string, the 'at' is considered as a 'date' element,
# but there is no specifier that produces a candidate for it
s = "January 12, 2018 at 02:12 AM"
sut = FormatFinder()
candidates = sut.find_candidate_patterns(s)
assert "%B %d, %Y at %H:%M %p" in candidates
To make it a bit clearer, here's some example of using this code in an iPython shell:
In [2]: ff = FormatFinder()
In [3]: list(ff.find_candidate_patterns("2014-01-01 00:12:12"))
Out[3]:
['%Y-%d-%m %H:%M:%S',
'%Y-%d-%m %H:%S:%M',
'%Y-%m-%d %H:%M:%S',
'%Y-%m-%d %H:%S:%M']
In [4]: list(ff.find_candidate_patterns("Jan. 04, 2017"))
Out[4]: ['%b. %d, %Y']
In [5]: list(ff.find_candidate_patterns("January 12, 2018 at 02:12 AM"))
Out[5]: ['%B %d, %Y at %H:%M %p', '%B %M, %Y at %H:%d %p']
In [6]: ff_without_case = FormatFinder(ignore_case=True)
In [7]: list(ff_without_case.find_candidate_patterns("JANUARY 12, 2018 02:12 am"))
Out[7]: ['%B %d, %Y %H:%M %p', '%B %M, %Y %H:%d %p']

Idea:
Inspect the user input date string, and build possible date format set
Loop over the format set, use datetime.strptime parse the date string with individual possible date format.
Format the date from step 2 with datetime.strftime, if the result equal to the origin date string, then this format is a possible date format.
Algorithm implementation
from datetime import datetime
import itertools
import re
FORMAT_CODES = (
r'%a', r'%A', r'%w', r'%d', r'%b', r'%B', r'%m', r'%y', r'%Y',
r'%H', r'%I', r'%p', r'%M', r'%S', r'%f', r'%z', r'%Z', r'%j',
r'%U', r'%W',
)
TWO_LETTERS_FORMATS = (
r'%p',
)
THREE_LETTERS_FORMATS = (
r'%a', r'%b'
)
LONG_LETTERS_FORMATS = (
r'%A', r'%B', r'%z', r'%Z',
)
SINGLE_DIGITS_FORMATS = (
r'w',
)
TWO_DIGITS_FORMATS = (
r'%d', r'%m', r'%y', r'%H', r'%I', r'%M', r'%S', r'%U', r'%W',
)
THREE_DIGITS_FORMATS = (
r'%j',
)
FOUR_DIGITS_FORMATS = (
r'%Y',
)
LONG_DIGITS_FORMATS = (
r'%f',
)
# Non format code symbols
SYMBOLS = (
'-',
':',
'+',
'Z',
',',
' ',
)
if __name__ == '__main__':
date_str = input('Please input a date: ')
# Split with non format code symbols
pattern = r'[^{}]+'.format(''.join(SYMBOLS))
components = re.findall(pattern, date_str)
# Create a format placeholder, eg. '{}-{}-{} {}:{}:{}+{}'
placeholder = re.sub(pattern, '{}', date_str)
formats = []
for comp in components:
if re.match(r'^\d{1}$', comp):
formats.append(SINGLE_DIGITS_FORMATS)
elif re.match(r'^\d{2}$', comp):
formats.append(TWO_DIGITS_FORMATS)
elif re.match(r'^\d{3}$', comp):
formats.append(THREE_DIGITS_FORMATS)
elif re.match(r'^\d{4}$', comp):
formats.append(FOUR_DIGITS_FORMATS)
elif re.match(r'^\d{5,}$', comp):
formats.append(LONG_DIGITS_FORMATS)
elif re.match(r'^[a-zA-Z]{2}$', comp):
formats.append(TWO_LETTERS_FORMATS)
elif re.match(r'^[a-zA-Z]{3}$', comp):
formats.append(THREE_LETTERS_FORMATS)
elif re.match(r'^[a-zA-Z]{4,}$', comp):
formats.append(LONG_LETTERS_FORMATS)
else:
formats.append(FORMAT_CODES)
# Create a possible format set
possible_set = itertools.product(*formats)
found = 0
for possible_format in possible_set:
# Create a format with possible format combination
dt_format = placeholder.format(*possible_format)
try:
dt = datetime.strptime(date_str, dt_format)
# Use the format to parse the date, and format the
# date back to string and compare with the origin one
if dt.strftime(dt_format) == date_str:
print('Possible result: {}'.format(dt_format))
found += 1
except Exception:
continue
if found == 0:
print('No pattern found')
Usage:
$ python3 reverse.py
Please input a date: 2018-12-31 10:26 PM
Possible result: %Y-%d-%M %I:%S %p
Possible result: %Y-%d-%S %I:%M %p
Possible result: %Y-%m-%d %I:%M %p
Possible result: %Y-%m-%d %I:%S %p
Possible result: %Y-%m-%M %I:%d %p
Possible result: %Y-%m-%M %I:%S %p
Possible result: %Y-%m-%S %I:%d %p
Possible result: %Y-%m-%S %I:%M %p
Possible result: %Y-%H-%d %m:%M %p
Possible result: %Y-%H-%d %m:%S %p
Possible result: %Y-%H-%d %M:%S %p
Possible result: %Y-%H-%d %S:%M %p
Possible result: %Y-%H-%M %d:%S %p
Possible result: %Y-%H-%M %m:%d %p
Possible result: %Y-%H-%M %m:%S %p
Possible result: %Y-%H-%M %S:%d %p
Possible result: %Y-%H-%S %d:%M %p
Possible result: %Y-%H-%S %m:%d %p
Possible result: %Y-%H-%S %m:%M %p
Possible result: %Y-%H-%S %M:%d %p
Possible result: %Y-%I-%d %m:%M %p
Possible result: %Y-%I-%d %m:%S %p
Possible result: %Y-%I-%d %M:%S %p
Possible result: %Y-%I-%d %S:%M %p
Possible result: %Y-%I-%M %d:%S %p
Possible result: %Y-%I-%M %m:%d %p
Possible result: %Y-%I-%M %m:%S %p
Possible result: %Y-%I-%M %S:%d %p
Possible result: %Y-%I-%S %d:%M %p
Possible result: %Y-%I-%S %m:%d %p
Possible result: %Y-%I-%S %m:%M %p
Possible result: %Y-%I-%S %M:%d %p
Possible result: %Y-%M-%d %I:%S %p
Possible result: %Y-%M-%S %I:%d %p
Possible result: %Y-%S-%d %I:%M %p
Possible result: %Y-%S-%M %I:%d %p

My idea was to create a class something like this, might not be accurate
from datetime import datetime
import re
class DateTime(object):
dateFormat = {"%d": "dd", "%Y": "YYYY", "%a": "Day", "%A": "DAY", "%w": "ww", "%b": "Mon", "%B": "MON", "%m": "mm",
"%H": "HH", "%I": "II", "%p": "pp", "%M": "MM", "%S": "SS"} # wil contain all format equivalent
def __init__(self, date_str, format):
self.dateobj = datetime.strptime(date_str, format)
self.format = format
def parse_format(self):
output=None
reg = re.compile("%[A-Z a-z]")
fmts = None
if self.format is not None:
fmts = re.findall(reg, self.format)
if fmts is not None:
output = self.format
for f in fmts:
output = output.replace(f, DateTime.dateFormat[f])
return output
nDate = DateTime("12 January, 2018", "%d %B, %Y")
print(nDate.parse_format())

You can wrap the function to store the arguments along with the result any time you call the wrapped version:
from dateutil.parser import parse
from functools import wraps
def parse_wrapper(function):
#wraps(function)
def wrapper(*args):
return {'datetime': function(*args), 'args': args}
return wrapper
wrapped_parse = parse_wrapper(parse)
x = wrapped_parse("2014-01-01 00:12:12")
# {'datetime': datetime.datetime(2014, 1, 1, 0, 12, 12),
# 'args': ('2014-01-01 00:12:12',)}

Python dateformat does not match Portugese format locale

I'm trying to parse Portugese date into datetime. Below is what I'm trying:
import locale
from datetime import datetime
locale.setlocale(locale.LC_ALL, 'pt_PT.iso88591')
date_format = '%A, %d %B %Y, %H:%M'
date_str = 'sexta-feira, 8 de setembro de 2017, 20:08'
datetime.strptime(date_str, date_format)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/lib/python2.7/_strptime.py", line 332, in _strptime
(data_string, format))
ValueError: time data 'sexta-feira, 8 de setembro de 2017, 20:08' does not
match format '%A, %d %B %Y, %H:%M'
Also tried below date_str but still getting the same error:
date_str = 'sexta-feira, 8 setembro 2017, 20:08'
What am I doing wrong here?

POSIX date and time format has limited support for different date formats. I suggest you take a look at PyICU:
from datetime import datetime
from icu import Locale, DateFormat, ICUtzinfo, TimeZone
locale = Locale('pt_PT')
tz = ICUtzinfo.getInstance('Portugal')
now = datetime.now(tz)
df = DateFormat.createDateTimeInstance(DateFormat.kFull, DateFormat.kFull, locale)
s = df.format(now)
print s
now2 = df.parse(s)
print now2
s2 = df.format(now2)
print s2
Output:
sexta-feira, 8 de setembro de 2017 às 23:26:20 Hora de verão da Europa Central
1504905980.0
sexta-feira, 8 de setembro de 2017 às 23:26:20 Hora de verão da Europa Central
I'm scraping a website so I need to convert a date string (sexta-feira, 8 de setembro de 2017, 20:08) into datetime so I can save it into database. How would I do that using PyICU?
This would require custom CLDR pattern:
df = SimpleDateFormat("EEEE, d 'de' MMMM 'de' yyy, HH:mm", locale)
print df.parse('sexta-feira, 8 de setembro de 2017, 20:08')

You can easily test the reverse conversion with strftime method. For your example it looks like this (and is pretty obvious, why your strptime doesn't work):
In [1]: import locale
In [2]: from datetime import datetime
In [3]: locale.setlocale(locale.LC_ALL, 'pt_PT.ISO-8859-1')
Out[3]: 'pt_PT.ISO-8859-1'
In [4]: date_format = '%A, %d %B %Y, %H:%M'
In [5]: datetime.now().strftime(date_format)
Out[5]: 'S\xe1bado, 09 Setembro 2017, 00:02'

Python time.strptime() gives wrong result?

I'm trying to parse a mbox format email spool.
I have code that does this:
if string.find(line, 'Date: ') == 0:
try:
when = time.mktime(time.strptime(line[6:30], "%a, %d %b %Y %H:%M:%S"))
Usually it seems to work OK, except that when line = 'Date: Sat, 17 Apr 2004 22:29:37 -0400\n'
it seems to give the wrong result (22:29:03 instead of 22:29:37).
Here's my pdb trace:
(Pdb) p line
'Date: Sat, 17 Apr 2004 22:29:37 -0400\n'
(Pdb) p time.strptime(line[6:30], "%a, %d %b %Y %H:%M:%S")
time.struct_time(tm_year=2004, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=29, tm_sec=3, tm_wday=5, tm_yday=108, tm_isdst=-1)
(Pdb)
The result seems to be off by 34 seconds. What am I doing wrong?

You are slicing your line too short; the second value is exclusive, not inclusive:
>>> line[6:30]
'Sat, 17 Apr 2004 22:29:3'
>>> line[6:31]
'Sat, 17 Apr 2004 22:29:37'
>>> time.strptime(line[6:31], "%a, %d %b %Y %H:%M:%S")
time.struct_time(tm_year=2004, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=29, tm_sec=37, tm_wday=5, tm_yday=108, tm_isdst=-1)

Categories