I'm writing a program that checks an excel file and if today's date is in the excel file's date column, I parse it
I'm using:
cur_date = datetime.today()
for today's date. I'm checking if today is in the column with:
bool_val = cur_date in df['date'] #evaluates to false
I do know for a fact that today's date is in the file in question. The dtype of the series is datetime64[ns]
Also, I am only checking the date itself and not the timestamp afterwards, if that matters. I'm doing this to make the timestamp 00:00:00:
cur_date = datetime.strptime(cur_date.strftime('%Y_%m_%d'), '%Y_%m_%d')
And the type of that object after printing is datetime as well
For anyone who also stumbled across this when comparing a dataframe date to a variable date, and this did not exactly answer your question; you can use the code below.
Instead of:
self.df["date"] = pd.to_datetime(self.df["date"])
You can import datetime and then add .dt.date to the end like:
self.df["date"] = pd.to_datetime(self.df["date"]).dt.date
You can use
pd.Timestamp('today')
or
pd.to_datetime('today')
But both of those give the date and time for 'now'.
Try this instead:
pd.Timestamp('today').floor('D')
or
pd.to_datetime('today').floor('D')
You could have also passed the datetime object to pandas.to_datetime but I like the other option mroe.
pd.to_datetime(datetime.datetime.today()).floor('D')
Pandas also has a Timedelta object
pd.Timestamp('now').floor('D') + pd.Timedelta(-3, unit='D')
Or you can use the offsets module
pd.Timestamp('now').floor('D') + pd.offsets.Day(-3)
To check for membership, try one of these
cur_date in df['date'].tolist()
Or
df['date'].eq(cur_date).any()
When converting datetime64 type using pd.Timestamp() it is important to note that you should compare it to another timestamp type. (not a datetime.date type)
Convert a date to numpy.datetime64
date = '2022-11-20 00:00:00'
date64 = np.datetime64(date)
Seven days ago - timestamp type
sevenDaysAgoTs = (pd.to_datetime('today')-timedelta(days=7))
convert date64 to Timestamp and see if it was in the last 7 days
print(pd.Timestamp(pd.to_datetime(date64)) >= sevenDaysAgoTs)
Related
i have a variable consisting of 300k records with dates and the date look like
2015-02-21 12:08:51
from that date i want to remove time
type of date variable is pandas.core.series.series
This is the way i tried
from datetime import datetime,date
date_str = textdata['vfreceiveddate']
format_string = "%Y-%m-%d"
then = datetime.strftime(date_str,format_string)
some Random ERROR
In the above code textdata is my datasetname and vfreceived date is a variable consisting of dates
How can i write the code to remove the time from the datetime.
Assuming all your datetime strings are in a similar format then just convert them to datetime using to_datetime and then call the dt.date attribute to get just the date portion:
In [37]:
df = pd.DataFrame({'date':['2015-02-21 12:08:51']})
df
Out[37]:
date
0 2015-02-21 12:08:51
In [39]:
df['date'] = pd.to_datetime(df['date']).dt.date
df
Out[39]:
date
0 2015-02-21
EDIT
If you just want to change the display and not the dtype then you can call dt.normalize:
In[10]:
df['date'] = pd.to_datetime(df['date']).dt.normalize()
df
Out[10]:
date
0 2015-02-21
You can see that the dtype remains as datetime:
In[11]:
df.dtypes
Out[11]:
date datetime64[ns]
dtype: object
You're calling datetime.datetime.strftime, which requires as its first argument a datetime.datetime instance, because it's an unbound method; but you're passing it a string instead of a datetime instance, whence the obvious error.
You can work purely at a string level if that's the result you want; with the data you give as an example, date_str.split()[0] for example would be exactly the 2015-02-21 string you appear to require.
Or, you can use datetime, but then you need to parse the string first, not format it -- hence, strptime, not strftime:
dt = datetime.strptime(date_str, '%Y-%m-%d %H:%M:%S')
date = dt.date()
if it's a datetime.date object you want (but if all you want is the string form of the date, such an approach might be "overkill":-).
simply writing
date.strftime("%d-%m-%Y") will remove the Hour min & sec
My data 'df' shows data 'Date' as 1970-01-01 00:00:00.019990103 when this is formatted to date_to using pandas. How do I show the date as 01/03/1999?
consider LoneWanderer's comment for next time and show some of the code that you have tried.
I would try this:
from datetime import datetime
now = datetime.now()
print(now.strftime('%d/%m/%Y'))
You can print now to see that is in the same format that you have and after that is formatted to the format required.
I see that the actual date is in last 10 chars of your source string.
To convert such strings to a Timestamp (ignoring the starting part), run:
df.Date = df.Date.apply(lambda src: pd.to_datetime(src[-8:]))
It is worth to consider to keep this date just as Timestamp, as it
simplifies operations on date / time and apply your formatting only in printouts.
But if you want to have this date as a string in "your" format, in the
"original" column, perform the second conversion (Timestamp to string):
df.Date = df.Date.dt.strftime('%m/%d/%Y')
I have a date string formatted like this: "2017-05-31T06:44:13Z".
I need to check whether this date is within a one year span from today's date.
Which is the best method to do it: convert it into a timestamp and check, or convert into a date format?
Convert the timestamp to a datetime object so it can be compared with other datetime objects using <, >, =.
from datetime import datetime
from dateutil.relativedelta import relativedelta
# NOTE this format basically ignores the timezone. This may or may not be what you want
date_to_check = datetime.strptime('2017-05-31T06:44:13Z', '%Y-%m-%dT%H:%M:%SZ')
today = datetime.today()
one_year_from_now = today + relativedelta(years=1)
if today <= date_to_check <= one_year_from_now:
# do whatever
Use the datetime package together with timedelta:
import datetime
then = datetime.datetime.strptime("2017-05-31T06:44:13Z".replace('T',' ')[:-1],'%Y-%m-%d %H:%M:%S')
now = datetime.datetime.now()
d = datetime.timedelta(days = 365)
and simply check if now-d > then.
I have an column in excel which has dates in the format ''17-12-2015 19:35". How can I extract the first 2 digits as integers and append it to a list? In this case I need to extract 17 and append it to a list. Can it be done using pandas also?
Code thus far:
import pandas as pd
Location = r'F:\Analytics Materials\files\paymenttransactions.csv'
df = pd.read_csv(Location)
time = df['Creation Date'].tolist()
print (time)
You could extract the day of each timestamp like
from datetime import datetime
import pandas as pd
location = r'F:\Analytics Materials\files\paymenttransactions.csv'
df = pd.read_csv(location)
timestamps = df['Creation Date'].tolist()
dates = [datetime.strptime(timestamp, '%d-%m-%Y %H:%M') for timestamp in timestamps]
days = [date.strftime('%d') for date in dates]
print(days)
The '%d-%m-%Y %H:%M'and '%d' bits are format specififers, that describe how your timestamp is formatted. See e.g. here for a complete list of directives.
datetime.strptime parses a string into a datetimeobject using such a specifier. dateswill thus hold a list of datetime instances instead of strings.
datetime.strftime does the opposite: It turns a datetime object into string, again using a format specifier. %d simply instructs strftime to only output the day of a date.
How do I convert a datetime.datetime object (e.g., the return value of datetime.datetime.now()) to a datetime.date object in Python?
Use the date() method:
datetime.datetime.now().date()
From the documentation:
datetime.datetime.date()
Return date object with same year, month and day.
You use the datetime.datetime.date() method:
datetime.datetime.now().date()
Obviously, the expression above can (and should IMHO :) be written as:
datetime.date.today()
You can convert a datetime object to a date with the date() method of the date time object, as follows:
<datetime_object>.date()
Answer updated to Python 3.7 and more
Here is how you can turn a date-and-time object
(aka datetime.datetime object, the one that is stored inside models.DateTimeField django model field)
into a date object (aka datetime.date object):
from datetime import datetime
#your date-and-time object
# let's supposed it is defined as
datetime_element = datetime(2020, 7, 10, 12, 56, 54, 324893)
# where
# datetime_element = datetime(year, month, day, hour, minute, second, milliseconds)
# WHAT YOU WANT: your date-only object
date_element = datetime_element.date()
And just to be clear, if you print those elements, here is the output :
print(datetime_element)
2020-07-10 12:56:54.324893
print(date_element)
2020-07-10
you could enter this code form for (today date & Names of the Day & hour) :
datetime.datetime.now().strftime('%y-%m-%d %a %H:%M:%S')
'19-09-09 Mon 17:37:56'
and enter this code for (today date simply):
datetime.date.today().strftime('%y-%m-%d')
'19-09-10'
for object :
datetime.datetime.now().date()
datetime.datetime.today().date()
datetime.datetime.utcnow().date()
datetime.datetime.today().time()
datetime.datetime.utcnow().date()
datetime.datetime.utcnow().time()
import time
import datetime
# use mktime to step by one day
# end - the last day, numdays - count of days to step back
def gen_dates_list(end, numdays):
start = end - datetime.timedelta(days=numdays+1)
end = int(time.mktime(end.timetuple()))
start = int(time.mktime(start.timetuple()))
# 86400 s = 1 day
return xrange(start, end, 86400)
# if you need reverse the list of dates
for dt in reversed(gen_dates_list(datetime.datetime.today(), 100)):
print datetime.datetime.fromtimestamp(dt).date()
I use data.strftime('%y-%m-%d') with lambda to transfer column to date
Solved: AttributeError: 'Series' object has no attribute 'date'
You can use as below,
df["date"] = pd.to_datetime(df["date"]).dt.date
where in above code date contains both date and time (2020-09-21 22:32:00), using above code we can get only date as (2020-09-21)
If you are using pandas then this can solve your problem:
Lets say that you have a variable called start_time of type datetime64 in your dataframe then you can get the date part like this:
df.start_time.dt.date