Convert Chrome history date/time stamp to readable format - python

I originally posted this question looking for an answer with using python, got some good help, but have still not been able to find a solution. I have a script running on OS X 10.5 client machines that captures internet browsing history (required as part of my sys admin duties in a US public school). Firefox 3.x stores history in a sqlite db, and I have figured out how to get that info out using python/sqlite3. Firefox 3.x uses a conventional unixtimestamp to mark visits and that is not difficult to convert... Chrome also stores browser history in a sqlite db, but its timestamp is formatted as the number of microseconds since January, 1601. I'd like to figure this out using python, but as far as I know, the sqlite3 module doesn't support that UTC format. Is there another tool out there to convert Chrome timestamps to a human readable format?

Use the datetime module. For example, if the number of microseconds in questions is 10**16:
>>> datetime.datetime(1601, 1, 1) + datetime.timedelta(microseconds=1e16)
datetime.datetime(1917, 11, 21, 17, 46, 40)
>>> _.isoformat()
'1917-11-21T17:46:40'
this tells you it was just past a quarter to 6pm of November 21, 1917. You can format datetime objects in any way you want thanks to their strftime method, of course. If you also need to apply timezones (other than the UTC you start with), look at third-party module pytz.

Bash
$ date -ud #$[13315808702856828/10**6-11644473600] +"%F %T %Z"
2022-12-18 03:45:02 UTC
$ printf '%(%FT %T %z)T\n' $[13315808702856828/10**6-11644473600]
2022-12-17 T19:45:02 -0800
Perl
$ echo ".. 13315808702856828 .." |\
perl -MPOSIX -pe 's!\b(1\d{16})\b!strftime(q/%F/,gmtime($1/1e6-11644473600))!e'
.. 2022-12-17 ..

Related

Python: date from plaintext (foreign language) weekday

I am looking to retrieve the next possible date for a weekday contained in a string. Complexity being that this weekday will be in foreign language (sv_SE).
In bash I can solve this using `dateround´:
startdate=$(dateround --from-locale=sv_SE -z CET today $startday)
Highly appreciate your guidance on how to solve this in Python.
Thank you very much!
Dateparser has support for quite a few languages. You could parse the weekday to a datetime object then determine the next possible date available.
-- Edit --
from dateparser import parse
parse('Onsdag').isoweekday() # 3
Now that you have the iso weekday, you can find the next possible date. You can refer to this to see how.
It seems locale aliases are platform specific and case sensitive. I've windows. So locale will be sv_SE.
You can use babel for date/time conversion and is much more comprehensive than native locale module.
Babel is an integrated collection of utilities that assist in internationalizing and localizing Python applications, with an emphasis on web-based applications.
Which can be installed as:
pip install Babel
Once installed, we can use format_date , format_datetime , format_time utilities to format one language date , time to other.
You can use these utilities to convert date/time data between English and Swedish.
>>>import datetime
>>>from babel.dates import format_date, format_datetime, format_time
#Here we get current date time in an datetime object
>>> now = datetime.datetime.now()
>>> now
datetime.datetime(2017, 10, 31, 9, 46, 32, 650000)
#We format datetime object to english using babel
>>> format_date(now, locale='en')
u'Oct 31, 2017'
#We format datetime object to sweedish using babel
>>> format_date(now, locale='sv_SE')
u'31 okt. 2017'
>>>

Python pytz timezone function returns a timezone that is off by 9 minutes

For some reason which I haven't been able to figure out yet, from the the following code:
>>> from pytz import timezone
>>> timezone('America/Chicago')
I get:
<DstTzInfo 'America/Chicago' LMT-1 day, 18:09:00 STD>
When, I assume, I should get:
<DstTzInfo 'America/Chicago' LMT-1 day, 18:00:00 STD>
...since I don't think that my timezone is 6 hours and 9 minutes away from UTC.
I have looked at the source code for pytz but I will admit that I haven't exactly been able to figure out what is going wrong.
I have passed other values to the timezone() function, and the values it returns appear to be correct. For some reason though, the information relevant to my timezone is not correct.
Finally, my co-worker in the cube next to me has confirmed that the function returns the correct timezone info on his machine.
Does anyone have any idea why my timezone ('America/Chicago') would be off by 9 minutes? I am running version 2015.7 of pytz installed using pip. Thank you!
Answer based on the answer by Carl Meyer in Google Groups Answer
The reason for this difference, is that this is NOT the right way of converting a timezone agnostic datetime object to a timezone aware object.
The explanation being:
"A pytz timezone class does not represent a single offset from UTC, it
represents a geographical area which, over the course of history, has
probably gone through several different UTC offsets. The oldest offset
for a given zone, representing the offset from before time zones were
standardized (in the late 1800s, most places) is usually called "LMT"
(Local Mean Time), and it is often offset from UTC by an odd number of
minutes."
(quote from the cited answer in Google Groups)
Basically, you should do:
from datetime import datetime
import pytz
my_datetime = datetime(2015, 6, 11, 13, 30)
my_tz = pytz.timezone('America/Chicago')
good_dt = my_tz.localize(my_datetime)
print(good_dt)
out: 2015-06-11 13:30:00-05:00
Unless your local timezone has a fixed UTC offset then it is pointless to talk about its specific value without providing a specific date/time.
If you provide the time e.g., the current time then you'll see that pytz produces the expected UTC offset:
>>> from datetime import datetime
>>> import pytz
>>> datetime.now(pytz.timezone('America/Chicago')).strftime('%Z%z')
'CST-0600'
See
Datetime Timezone conversion using pytz
pytz localize vs datetime replace
If you don't provide a specific date/time then pytz may return an arbitrary utc offset from the set of available utc offsets for the given timezone. The recent pytz versions return utc offsets that correspond to the earliest time (LMT as a rule) but you should not rely on it. You and your friend may use different pytz versions that may explain the difference in results.
Just because my curiosity wasn't exactly satisfied, I did a little more digging into this problem recently.
Initially, it seemed that the difference stemmed from different versions of pytz. However, after downgrading my version of pytz to a version where I had confirmed that I got a different result from that on my machine, I found that this wasn't the root of the issue: even with the same version of pytz my machine seemed to be using a UTC offset based on LMT, while the other machines were using one based off CDT or CST.
Based on my conversation with #J.F.Sebastian, I assumed that the only other likely possibility was a system level difference. I dug into the pytz source code a little bit more, and found that the file where pytz gets at least some of it's timezone information from is in /usr/share/zoneinfo/. So I looked at the file /usr/share/zoneinfo/America/Chicago and although it is a binary file, part of it is readable. Half way through the file there is a list of timezones: LMTCDTCSTESTCWTCPT. As you can see, LMT is the first name in the list, and as #J.F.Sebastian suggested, that seems to be the one that pytz uses in the situation described in my original question.
That is how the list looks in Ubuntu 15.10. However, in earlier versions of Ubuntu (e.g., Trusty and Precise) where I was getting the result -600 instead of -609 result, the same list is CDTCSTESTCWTCPT.
I will admit that this comes from a lot of blind exploring and half understanding, but it seems like this is what accounts for the differences I was seeing across machines. As far as why the zoneinfo files differ across versions, and what these differences mean for Ubuntu, I have no idea, but I thought I would share my findings for those who are similarly curious, and to potentially receive insightful corrections/supplemental information from the community.
As you mention there are some differences in the original file into the pytz module: (in my case using the Central time)
xxxx......lib/python2.7/site-packages/pytz/zoneinfo/US/Central
In [66]: start = start.replace(tzinfo=central)
In [67]: start.isoformat()
Out[67]: '2018-02-26T00:00:00-05:51'
if you use the standard file of the OS (I tested in mac, ubuntu and centos)
/usr/share/zoneinfo/US/Central
mv xxxx...../lib/python2.7/site-packages/pytz/zoneinfo/US/Central xxxx...../lib/python2.7/site-packages/pytz/zoneinfo/US/Central-bak
ln -s /usr/share/zoneinfo/US/Central xxxx...../lib/python2.7/site-packages/pytz/zoneinfo/US/Central
The problem is resolved
In [7]: central = timezone('US/Central')
In [8]: central
Out[8]: <DstTzInfo 'US/Central' CST-1 day, 18:00:00 STD>
In [10]: start = start.replace(tzinfo=central)
In [11]: start.isoformat()
Out[11]: '2018-02-27T00:00:00-06:00'

Python date.today shows the next day

I'm astounded by some code I wrote some time ago. For not entering in much detail i have a method that runs through some objects, wich have a date parameter. If the date parameter is equal to today's date, goes on.
I have set this in my local machine for test and have like 695 objects all with the same date, today, but when the action is run nothing happens, so i debug it to find that my expression date.today() returns datetime.date(2014, 3, 19).
This is is incorrect, as the date of my computer from the date command is Tue Mar 18 20:56:09 AST 2014.
I used from datetime import date. This is one of the more cryptic errors i have ever got. Any experience someone can share here? Thanks a lot.
The method is not timezone aware and there's no platform-independent way to make it so. What is generally done is incorporate something like pytz and call .today() as:
datetime.utcnow().replace(tzinfo = pytz.utc).strftime('%Y-%m-%d')

getting datetime from python string when tzinfo is present

I have found answers to question like this one helpful but not complete for my problem.
I have a form where the user automatically produces a date. I would like to store that as a date time.
I don't need any of the information after the seconds, but I cannot find a datetime.datetime.strptime code to translate the remaining stuff. So I would either like a strptime code that works for python2.7 on google app engine, or a string editing trick for removing the extra information that is not needed.
date-from-user='2012-09-22 07:36:36.333373-05:00'
You can slice your string to only select the first 19 characters:
>>> datefromuser='2012-09-22 07:36:36.333373-05:00'
>>> datefromuser[:19]
'2012-09-22 07:36:36'
This let's you parse the date without having to bother with the microseconds and timezone.
Do note that you probably do want to parse the timezone too though. You can use the iso8601 module to handle the whole format, without the need to slice:
>>> import iso8601
>>> iso8601.parse_date(datefromuser)
datetime.datetime(2012, 9, 22, 7, 36, 36, 333373, tzinfo=<FixedOffset '-05:00'>)
The iso8601 module is written in pure python and works without problems on the Google App Engine.
Python Docs would be a good place to start. strptime() would be your best option.
import datetime
datefromuser = '2012-09-22 07:36:36.333373-05:00'
datetime.datetime.strptime(datefromuser.split(".")[0], "%Y-%m-%d %H:%M:%S")
2012-09-22 07:36:36
http://docs.python.org/library/datetime.html#strftime-and-strptime-behavior

Python - Trying to get a different timezone

Totally new to python, I'm actually working on an ex-colleague's script. in looking at it it seems fairly straight-forward. Here's the situation:
The script looks at current localtime (UTC) and renders a time-based table that scrolls/changes throughout the day as the hours pass so there's always a rolling 8 hour table.
The problem is that now we'd like to deploy a copy of this tool (on the same server) in CST ('America/Chicago') (meaning I need to change the UTC time to CST) so I'm just trying to find a way to modify what he has to make the 'current_time' variable == GMT -6.
He used strftime() to get the first hour:
current_time = int(strftime("%H"))
if current_time <19:
temp_page.write(...)
elif current_time == 19:
temp_page.write(...)
etc.
So - from my php knowledge, I'd love to be able to do something like:
current_time = int(strftime("%H"), (localtime() -6 hours))
(yes, I realize that's not real php code, but hopefully you get my meaning ;-))
In my research, I've come across pytz, but this is not installed on the webserver, though I can probably get it if that's the best/easiest way too implement it.
Any help is greatly appreciated.
Yes, try to install pytz, it will help you a lot when working with different timezones (and UTC).
The current UTC time (independent from the timezone of your computer) can be obtained with:
import pytz
from datetime import datetime
now = datetime.now(pytz.utc)
now is now datetime.datetime(2011, 11, 30, 14, 26, 30, 628014, tzinfo=<UTC>) and you can use it to calculate the current UTC hour with now.hour (returns 14)
You could probably use the datetime module (it's part of the standard library, so it's installed if a standard python is on the system).
In particular, datetime objects can have an optional tzinfo attribute, used during timezone conversions. Here's a blog post that explains step-by-step how to use those.

Categories