I need the number (as integer) of users that signed up (User.date_joined) between '2016-01-01'(string) and '2016-04-01' (string) (both dates fully included)
The below queries didn't give me accurate results, since the date_joined is datetime field
User.objects.filter(date_joined__gte='2016-01-01',date_joined_lte='2016-04-01').count()
User.objects.filter(date_joined__range('2016-01-01 00:00:00','2016-04-01 12:59:59')).count()
I am new to django and python , want to know how to pass the string date values to datetime field and use of range function
dt1 = datetime.datetime(2016, 1, 1) # midnight
dt2 = dt1 + datetime.timedelta(days=1)
User.objects.filter(date_joined__range=(dt1, dt2)).count()
Related
I'm currently using the filter "created_at__range" to specify the first and last day of the month, but this code doesn't reflect the data registered today.
this_month = datetime.datetime.today()
first_day = datetime.datetime(this_month.year, this_month.month, 1)
this_month = this_month.strftime('%Y-%m-%d')
first_day = first_day.strftime('%Y-%m-%d')
time = obj.filter(created_at__range=(first_day, this_month)).aggregate(
time=Sum('time'))['time']
Currently, I'm using timedelta(days=1) to add a day, but if I do this, for example, if the date is 3/31, it will be 4/1 and the tally will be wrong.
this_month = datetime.datetime.today() + timedelta(days=1)
Why is this happening?
If anyone knows how to improve it, I'd appreciate it if you could let me know.
I assume that your field created_at is a DateTimeField. Quoting the warning from Django's documentation
Warning
Filtering a DateTimeField with dates won’t include items on the
last day, because the bounds are interpreted as “0am on the given
date”. If pub_date was a DateTimeField, the above expression
would be turned into this SQL:
SELECT ... WHERE pub_date BETWEEN '2005-01-01 00:00:00' and '2005-03-31 00:00:00';
Generally speaking, you can’t mix dates and datetimes.
I would like to add why even convert the datetime to a string simply use the object in your query:
this_month = datetime.datetime.today()
first_day = datetime.datetime(this_month.year, this_month.month, 1)
time = obj.filter(
created_at__range=(first_day, this_month)
).aggregate(time=Sum('time'))['time']
Edit: In fact to make this a little easier for yourself and if there is no object that would have a datetime in the future, just let the ORM and database do a little more work if needed by just comparing the day and the year:
today = datetime.datetime.today()
time = obj.filter(
created_at__year=today.year,
created_at__month=today.month,
).aggregate(time=Sum('time'))['time']
I have a date string formatted like this: "2017-05-31T06:44:13Z".
I need to check whether this date is within a one year span from today's date.
Which is the best method to do it: convert it into a timestamp and check, or convert into a date format?
Convert the timestamp to a datetime object so it can be compared with other datetime objects using <, >, =.
from datetime import datetime
from dateutil.relativedelta import relativedelta
# NOTE this format basically ignores the timezone. This may or may not be what you want
date_to_check = datetime.strptime('2017-05-31T06:44:13Z', '%Y-%m-%dT%H:%M:%SZ')
today = datetime.today()
one_year_from_now = today + relativedelta(years=1)
if today <= date_to_check <= one_year_from_now:
# do whatever
Use the datetime package together with timedelta:
import datetime
then = datetime.datetime.strptime("2017-05-31T06:44:13Z".replace('T',' ')[:-1],'%Y-%m-%d %H:%M:%S')
now = datetime.datetime.now()
d = datetime.timedelta(days = 365)
and simply check if now-d > then.
I have a timestamp such 1474398821633L that I think is in utc. I want to compare it to datetime.datetime.now() to verify if it is expired.
I am using python 2.7
from datetime import datetime
timestamp = 1474398821633L
now = datetime.now()
if datetime.utcfromtimestamp(timestamp) < now:
print "timestamp expired"
However I got this error when trying to create a datetime object from the timestamp: ValueError: timestamp out of range for platform localtime()/gmtime() function
What can I do?
It looks like your timestamp is in milliseconds. Python uses timestamps in seconds:
>>> datetime.datetime.utcfromtimestamp(1474398821.633)
datetime.datetime(2016, 9, 20, 19, 13, 41, 633000)
In other words, you might need to divide your timestamp by 1000. in order to get it in the proper range.
Also, you'll probably want to compare datetime.utcnow() instead of datetime.now() to make sure that you're handling timezones correctly :-).
As #mgilson pointed out your input is likely "milliseconds", not "seconds since epoch".
Use time.time() instead of datetime.now():
import time
if time.time() > (timestamp_in_millis * 1e-3):
print("expired")
If you need datetime then use datetime.utcnow() instead of datetime.now(). Do not compare .now() that returns local time as a naive datetime object with utcfromtimestamp() that returns UTC time also as a naive datetime object (it is like comparing celsius and fahrenheit directly: you should convert to the same unit first).
from datetime import datetime
now = datetime.utcnow()
then = datetime.utcfromtimestamp(timestamp_in_millis * 1e-3)
if now > then:
print("expired")
See more details in Find if 24 hrs have passed between datetimes - Python.
I have the following dateTime text type variable in Postgres table
"2016-05-12T23:59:11+00:00"
"2016-05-13T11:00:11+00:00"
"2016-05-13T23:59:11+00:00"
"2016-05-15T10:10:11+00:00"
"2016-05-16T10:10:11+00:00"
"2016-05-17T10:10:11+00:00"
I have to write a Python function to extract the data for a few variables between two dates
def fn(dateTime):
df1=pd.DataFrame()
query = """ SELECT "recordId" from "Table" where "dateTime" BETWEEN %s AND %s """ %(dStart,dEnd)
df1=pd.read_sql_query(query1,con=engine)
return df1
I need to create dStart and dEnd variables and use them as function parameters as below
fn('2016-05-12','2016-05-15')
I tried using to_char("dateTime", 'YYYY-MM-DD') Postgres function but didn't work out. Please let me know how to solve this
When working with sql, you should always use your sql library to substitute parameters into the query, instead of using Python's string operators. This avoids the risk of malformed queries or sql injection attacks. See e.g., this page. Right now your code won't run because it directly inserts dStart and dEnd without any quoting, so they are interpreted as mathematical expressions (2016 - 5 - 12 = 1999).
There's also a secondary problem that your query will exclude dateTime values on the end date, because endDate will be treated as having a time value of 00:00:00 when it is compared to dateTime. And if you use to_char() or some other function to extract just the date from the dateTime column to do the comparison, it will prevent your query from using indexes, making it very inefficient.
Here is some revised code that may work for you:
def fn(dStart, dEnd):
query = """
SELECT "recordId"
FROM "Table"
WHERE "dateTime" >= %(start)s AND "dateTime" < %(end)s + interval '1 day'
"""
query_params = {'start': dStart, 'end': dEnd}
df1 = pd.read_sql_query(query1, con=engine, params=query_params)
return df1
This code relies on a few assumptions (welcome to the wonderful world of datetime querying!):
you will pass dStart and dEnd to fn(), instead of just a single dateTime,
the dateTime column is type timestamp with timezone (not text),
the timezones in the dateTime column are correct, and
the dates given by dStart and dEnd are in the server's timezone or you have used SET TIMEZONE ... with your engine object to select the right time zone to use for this session.
Notes
Different database engines use different placeholders for the parameters, so you will need to check your database driver's documentation to decide what placeholders to use. The code above should work fine for postgresql.
With the code above, dStart and dEnd will be inserted into the query as strings, and postgresql automatically convert them into timestamps when it runs the query. This should work fine for the example dates you gave, but if you need more direct control, you have two options:
call fn() with Python date or datetime values for dStart and dEnd, and the code above will insert them into the query as postgresql dates or timestamps; or
explicitly convert the dStart and dEnd strings into postgresql dates by replacing %(start)s and %(end)s with something like this: to_date(%(start)s, 'YYYY-MM-DD').
I'm not familiar with postgresql, but you can convert the strings to the struct_time class which is part of the built in time package in Python and simply make comparisons between them.
import time
time_data = ["2016-05-12T23:59:11+00:00",
"2016-05-13T11:00:11+00:00",
"2016-05-13T23:59:11+00:00",
"2016-05-15T10:10:11+00:00",
"2016-05-16T10:10:11+00:00",
"2016-05-17T10:10:11+00:00"]
def fn(t_init, t_fin, t_all):
# Convert string inputs to struct_time using time.strptime()
t_init, t_fin = [time.strptime(x, '%Y-%m-%d') for x in [t_init, t_fin]]
t_all = [time.strptime(x, '%Y-%m-%dT%H:%M:%S+00:00') for x in time_all]
out = []
for jj in range(len(t_all)):
if t_init < t_all[jj] < t_fin:
out.append(jj)
return out
out = fn('2016-05-12','2016-05-15', time_data)
print(out)
# [0, 1, 2]
The time.strptime routine uses a format specifiers to specify which parts of the string correspond to different time components.
%Y Year with century as a decimal number.
%m Month as a decimal number [01,12].
%d Day of the month as a decimal number [01,31].
%H Hour (24-hour clock) as a decimal number [00,23].
%M Minute as a decimal number [00,59].
%S Second as a decimal number [00,61].
%z Time zone offset from UTC.
%a Locale's abbreviated weekday name.
%A Locale's full weekday name.
%b Locale's abbreviated month name.
%B Locale's full month name.
%c Locale's appropriate date and time representation.
%I Hour (12-hour clock) as a decimal number [01,12].
%p Locale's equivalent of either AM or PM.
How do I convert a datetime.datetime object (e.g., the return value of datetime.datetime.now()) to a datetime.date object in Python?
Use the date() method:
datetime.datetime.now().date()
From the documentation:
datetime.datetime.date()
Return date object with same year, month and day.
You use the datetime.datetime.date() method:
datetime.datetime.now().date()
Obviously, the expression above can (and should IMHO :) be written as:
datetime.date.today()
You can convert a datetime object to a date with the date() method of the date time object, as follows:
<datetime_object>.date()
Answer updated to Python 3.7 and more
Here is how you can turn a date-and-time object
(aka datetime.datetime object, the one that is stored inside models.DateTimeField django model field)
into a date object (aka datetime.date object):
from datetime import datetime
#your date-and-time object
# let's supposed it is defined as
datetime_element = datetime(2020, 7, 10, 12, 56, 54, 324893)
# where
# datetime_element = datetime(year, month, day, hour, minute, second, milliseconds)
# WHAT YOU WANT: your date-only object
date_element = datetime_element.date()
And just to be clear, if you print those elements, here is the output :
print(datetime_element)
2020-07-10 12:56:54.324893
print(date_element)
2020-07-10
you could enter this code form for (today date & Names of the Day & hour) :
datetime.datetime.now().strftime('%y-%m-%d %a %H:%M:%S')
'19-09-09 Mon 17:37:56'
and enter this code for (today date simply):
datetime.date.today().strftime('%y-%m-%d')
'19-09-10'
for object :
datetime.datetime.now().date()
datetime.datetime.today().date()
datetime.datetime.utcnow().date()
datetime.datetime.today().time()
datetime.datetime.utcnow().date()
datetime.datetime.utcnow().time()
import time
import datetime
# use mktime to step by one day
# end - the last day, numdays - count of days to step back
def gen_dates_list(end, numdays):
start = end - datetime.timedelta(days=numdays+1)
end = int(time.mktime(end.timetuple()))
start = int(time.mktime(start.timetuple()))
# 86400 s = 1 day
return xrange(start, end, 86400)
# if you need reverse the list of dates
for dt in reversed(gen_dates_list(datetime.datetime.today(), 100)):
print datetime.datetime.fromtimestamp(dt).date()
I use data.strftime('%y-%m-%d') with lambda to transfer column to date
Solved: AttributeError: 'Series' object has no attribute 'date'
You can use as below,
df["date"] = pd.to_datetime(df["date"]).dt.date
where in above code date contains both date and time (2020-09-21 22:32:00), using above code we can get only date as (2020-09-21)
If you are using pandas then this can solve your problem:
Lets say that you have a variable called start_time of type datetime64 in your dataframe then you can get the date part like this:
df.start_time.dt.date