not able to convert file read() to an integer - python

I have a simple file which contains exactly one integer. This integer is an epoch timestamp value.
ts_f = open('latest_ts','r')
pattern = '%a %b %d %H:%M:%S NZDT %Y'
tmp = ts_f.read()
//Do some processing to update the timestamp value.
ts_f.close()
ts_f = open('latest_ts','w+')
ts_f.write(latest_ts_epoch)
ts_f.close()
Since both of these are integer values and read returns a string, I tried to convert tmp to an interger as int(tmp).It does not allow me to convert tmp to an integer and gives the error
ValueError: invalid literal for int() with base 10:

See added line below, your file starts with a BOM which needs to be decoded first.
ts_f = open('latest_ts','r')
pattern = '%a %b %d %H:%M:%S NZDT %Y'
tmp = ts_f.read()
tmp = tmp.decode("utf-8-sig")
ts_f.close()
ts_f = open('latest_ts','w+')
ts_f.write(latest_ts_epoch)
ts_f.close()

Related

Time data does not match format '%c'

This is very unexpected behavior...
I create a time string using the '%c' directive.
%c is the Locale’s appropriate date and time representation.
Then I try to parse the resulting time string, specifying the same '%c' as the string's format.
However this does not work as you can see from the error below. What am I missing?
I need to be able to store the time in a human-readable localized string, and then convert the string back into a struct_time so I can extract information from it.
(It is extremely important that the string be localized, and I of course don't want to write parsing algorithms for all locales around the world!)
# Ensure the locale is set.
import locale
locale.setlocale(locale.LC_ALL, '')
'en_US.UTF-8'
# 1. Create a localized time string using the '%c' directive.
import datetime
time_stamp = datetime.datetime.now().strftime('%c')
time_stamp
'Mon 21 Dec 2020 03:47:55 PM '
# 2. Try to parse the string using the same directive used to create it.
import time
time.strptime(time_stamp, '%c')
# 3. Unexpected error...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/_strptime.py", line 562, in _strptime_time
tt = _strptime(data_string, format)[0]
File "/usr/lib/python3.8/_strptime.py", line 349, in _strptime
raise ValueError("time data %r does not match format %r" %
ValueError: time data 'Mon 21 Dec 2020 03:47:55 PM ' does not match format '%c'
Your locale is probably not configuring .strftime("%c") the way you expect and .strptime is objecting to the postfixed %p (PM)
Use locale.nl_langinfo(locale.D_T_FMT) to build your format instead!
>>> locale.nl_langinfo(locale.D_T_FMT)
'%a %b %e %H:%M:%S %Y'
>>> locale.setlocale(locale.LC_ALL, '')
'en_US.UTF-8'
>>> locale.nl_langinfo(locale.D_T_FMT)
'%a %b %e %X %Y'
However, if you
.. know the exact structure of the output, filter exact matches with a regex and then parse
.. can control the format, don't bother to format it and directly use time.time()
.. or always work in UTC and format as ISO 8601, deriving a tz-aware object and reading back with a custom parser (refer to the Caution on .fromisoformat)
>>> datetime.datetime.now(tz=datetime.timezone.utc)
datetime.datetime(2020, 12, 22, 0, 4, 29, 537007, tzinfo=datetime.timezone.utc)
use pytz, which is much "smarter" than the datetime builtin lib and properly supports a huge variety of locales
Instead of using %c, you can specify how you want to format the date using %a, %b and other directives. For example:
import locale
locale.setlocale(locale.LC_ALL, 'en_US.utf-8')
import datetime
fmt = '%a %b %d %Y %H:%M:%S'
time_stamp = datetime.datetime.now().strftime(fmt)
print(time_stamp)
import time
print(time.strptime(time_stamp, fmt))
This produces an output that you are looking for:
Output:
Mon Dec 21 2020 21:27:50
time.struct_time(tm_year=2020, tm_mon=12, tm_mday=21, tm_hour=21, tm_min=27, tm_sec=50, tm_wday=0, tm_yday=356, tm_isdst=-1)

Error: time data "b'YYYY/MM/DD" does not match format '%Y/%m/%d' but it does

I'm trying to parse dates from a textfile, but executing the scripts throws incorrect data format, when the format is correct.
The file is a .txt file with the following structure
2018/02/15 05:00:13 - somestring - anotherstring
2018/02/15 05:00:14 - somestring - anotherstring
2018/02/15 05:00:15 - somestring - anotherstring
... etc
The script gets the file divided in lines, and each line is divided on fields, of which one field is a date and time. I divided the date and the time in two separate fields, the time gets converted ok so the problem is in the date.
This is what I get on execution:
ValueError: time data "b'2018/02/15" does not match format '%Y/%m/%d'
I noticed it prints the string with a "b" in front of it, which if I'm not mistaken it means it's a byte literal. I've tried using "decode("utf-8")" on it, but it throw's exception as "string" has no method decode.
#the file is in one long string as I get it from a 'cat' bash command via ssh
file = str(stdout.read()) #reads the cat into a long string
strings = file.split("\\n") #splits the string into lines
for string in strings:
fields = string.split(" - ")
if len(fields) >= 3:
#dates.append(datetime.strptime(campos[0],"%Y/%m/%d %H:%M:%S")) #Wrong format
datentime = fields[0].split()
dates.append(datetime.strptime(datentime[0],"%Y/%m/%d")) #Wrong format
print(datentime[1])
dates.append(datetime.strptime(datentime[1],"%H:%M:%S")) #WORKS
I can't figure out why that is happening to you with the code you gave so I can't offer a fix for that but I tried testing on it and this worked for me:
datetime.strptime(str(datentime[0])[2,:-1], "%Y/%m/%d")
It removes the B and ' from the string, if you still have problems with that, please post how you got that string, maybe there was some error on the way.
use try and except:
import datetime
def convertDate(d):
strptime = datetime.datetime.strptime
try:
return strptime(d, "%Y/%m/%d")
except TypeError:
return strptime(d.decode("utf-8"), "%Y/%m/%d")
print(convertDate(b'2018/02/15'))
print(convertDate('2018/02/15'))

How to convert a downloaded string to datetime format?

I am trying to check if today's date < date downloaded from text file online. Here is my code :
import datetime
import requests
URL = "http://directlinktotextfile.com/text.txt"
result = requests.get(URL)
today = datetime.datetime.now().date()
Url_date = result.text
Url_date.strip()
Url_date = datetime.date(Url_date)
if today < Url_date :
print "Today is less than future date"
raw_input()
else:
print "Today is greater than or = to future date"
raw_input()
The result that comes back is just this : 2018,02,14. I use .strip() in case there might be blank spaces or extra lines. I've printed out result.text after strip() and it shows the correct details. Why is it that I can't check if today < Url_date. It works fine if I enter manually a date into datetime.date(2018,02,14), but when I'm downloading the string it won't work. Any suggestions?
You pass string to datetime.date() which should be each an integer.
Url_list = []
Url_list = Url_date.split(",")
yr = int(Url_list[0])
mn = int(Url_list[1])
d = int(Url_list[2])
Now pass these integers to datetime.date
Url_date = datetime.date(yr, mn, d)
The arguments you pass to datetime.date(arg1, arg2, arg3) are not strings as a whole. When you pass it from url, what you are actually doing is
datetime.date("2018,2,14")
Note that you are passing only one string argument and not 3 different integers. You should split the date string using comma and then convert each into integers and then pass them as arguments to datetime.date.
Here is what your code is trying to do :
Url_date = datetime.date("2018,02,14")
But he wants to have:
Url_date = datetime.date(2018,02,14)
Do
Url_date.split(',') # Result: ['2018','02','14']
And then convert all the string in the array in integers
It should be ok :)
Use strptime:
import datetime
today = datetime.datetime.now().date()
parsed = datetime.datetime.strptime("2018,02,14", "%Y,%m,%d").date()
print(today < parsed) # True

python strptime from UTC string to datetime regex re.sub

im using the following that works awesome for converting a UTC string to date times -- except when the milliseconds are LONGER than 6 digits. than it blows. urgh*&^*&
format = '%Y-%m-%dT%H:%M:%S'
if '.' in value:
format = format + '.%f'
if value.endswith('Z'):
format = format + 'Z'
return datetime.strptime(value, format)
here is the stacktrace
File "/usr/lib64/python2.6/_strptime.py", line 325, in _strptime
(data_string, format))
ValueError: time data '2013-07-19T13:02:53.8241899Z' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'
whats a method to restrict the milliseconds to 6 digits in length. ive used using '{:%Y-%m-%dT%H:%M:%S.6%f}'.format( datetime ) but that helps nonce/
so trying re.sub( ) to limit the decimals (if any) following the '.'character in the provided string. but am not very regex savvy.
im using python 2.6.* on the machine.
Sounds like a job for dateutil.
$ pip install python-dateutil
$ python
>>> import dateutil.parser
>>> d = dateutil.parser.parse('2013-07-19T13:02:53.8241899Z')
>>> print d
2013-07-19 13:02:53.824189+00:00
You could use .partition() method to parse the string:
from datetime import datetime
sz = '2013-07-19T13:02:53.8241899Z' # utc time in rfc3339 format (without offset)
sz = sz.rstrip('Z') # remove ending 'Z' if present
timestr, _, digits = sz.partition('.')
utc_dt = datetime.strptime(timestr, '%Y-%m-%dT%H:%M:%S')
if digits:
microseconds = int(digits, 10) * 10**(6 - len(digits))
utc_dt = utc_dt.replace(microsecond=int(microseconds + .5))
If you want to use re.sub and don't mind truncating the microseconds:
import re
from datetime import datetime
sz = '2013-07-19T13:02:53.8241899Z'
if '.' in sz:
sz = re.sub(r'(\.\d{,6})\d*Z?$', r'\1Z', sz) # truncate to 6 digits +Z
else: # no fractions
sz = sz.rstrip('Z') + '.0Z'
utc_dt = datetime.strptime(sz, '%Y-%m-%dT%H:%M:%S.%fZ')
If you wanted to do it with re.sub, here's a short example of one way to do that.
import re
from datetime import datetime
value = '2013-07-19T13:02:53.8241899Z'
regex = r"(\d{6,}\d+)"
if re.search(regex, value):
repl = lambda x: x.group(0)[:6]
value = re.sub(pattern=regex, repl=repl, string=value)
format = '%Y-%m-%dT%H:%M:%S'
if '.' in value:
format = format + '.%f'
if value.endswith('Z'):
format = format + 'Z'
print datetime.strptime(value, format)

Using a Unicode format for Python's `time.strftime()`

I am trying to call Python's time.strftime() function using a Unicode format string:
u'%d\u200f/%m\u200f/%Y %H:%M:%S'
(\u200f is the "Right-To-Left Mark" (RLM).)
However, I am getting an exception that the RLM character cannot be encoded into ascii:
UnicodeEncodeError: 'ascii' codec can't encode character u'\u200f' in position 2: ordinal not in range(128)
I have tried searching for an alternative but could not find a reasonable one. Is there an alternative to this function, or a way to make it work with Unicode characters?
Many standard library functions still don't support Unicode the way they should. You can use this workaround:
import time
my_format = u'%d\u200f/%m\u200f/%Y %H:%M:%S'
my_time = time.localtime()
time.strftime(my_format.encode('utf-8'), my_time).decode('utf-8')
You can format string through utf-8 encoding:
time.strftime(u'%d\u200f/%m\u200f/%Y %H:%M:%S'.encode('utf-8'), t).decode('utf-8')
You should read from a file as Unicode and then convert it to Date-time format.
from datetime import datetime
f = open(LogFilePath, 'r', encoding='utf-8')
# Read first line of log file and remove '\n' from end of it
Log_DateTime = f.readline()[:-1]
You can define Date-time format like this:
fmt = "%Y-%m-%d %H:%M:%S.%f"
But some programming language like C# doesn't support it easily, so you can change it to:
fmt = "%Y-%m-%d %H:%M:%S"
Or you can use like following way (to satisfy .%f):
Log_DateTime = Log_DateTime + '.000000'
If you have an unrecognized symbol (an Unicode symbol) then you should remove it too.
# Removing an unrecognized symbol at the first of line (first character)
Log_DateTime = Log_DateTime[1:] + '.000000'
At the end, you should convert string date-time to real Date-time format:
Log_DateTime = datetime.datetime.strptime(Log_DateTime, fmt)
Current_Datetime = datetime.datetime.now() # Default format is '%Y-%m-%d %H:%M:%S.%f'
# Calculate different between that two datetime and do suitable actions
Current_Log_Diff = (Current_Datetime - Log_DateTime).total_seconds()

Categories