Aggregate function for count using MongoDB PyMongo - python

[{'_id': '5ebe39e41e1729d90de',
'modelId': '5ebe3536c289711579',
'lastAt': datetime.datetime(2020, 5, 15, 6, 42, 44, 79000),
'proId': '5ebe3536c2897115793dccfb',
'genId': '5ebe355ac2897115793dcd04',
'count':'ab'},
{'_id': '5ebe3a0d94fcb800fa474310',
'modelId': '5ebe3536c289711579',
'proId': '5ebe3536c2897115793d',
'genId': '5ebe355ac2897115793',
'lastAt': datetime.datetime(2020, 5, 15, 6, 43, 25, 105000),'count':'cd'}]
i've count in the above collection documents which is in encrypted form, how can i extract the counts for each day for a week and total count for a week and for a month.
Note: need to extract the sum of the counts

Related

Finding time taken to complete task in a specified hour

new = {('E_1E_2', 1.0235591568933329): [datetime.datetime(2019, 6, 25, 4, 38, 47)],
('E_1E_2', 0.9023917716540885): [datetime.datetime(2019, 6, 25, 4, 38, 51)],
('E_12E_2', 1.4927612857980848): [datetime.datetime(2019, 6, 25, 3, 18, 42)],
('E12_1E_2', 1.0235591568933329): [datetime.datetime(2019, 6, 25, 3, 30, 17)],
('E_1E_2', 0.9023917716540885): [datetime.datetime(2019, 6, 25, 3, 13, 51)],
('E_1E_2', 1.4927612857980848): [datetime.datetime(2019, 6, 25, 3, 8, 42)]}
This is how my data input looks like. I need to display the time taken for the students to complete the task in hourly basis. The dictionary keys consist of the student pair doing the task and the distance between the pair. The value consists of the timestamp at which they were close to each other.
tasktime=0
prev=datetime.datetime(1,1,1)
for i,j in new.items():
#print(k[0],k[1])
print(i[0],"-------------------------------------------------")
for k,l in new.items():
if i[0]==k[0]:
if k[1] < 1 and len(l)==1 and j[0].hour==l[0].hour:
print("distance=======",k[1])
print(l[0].hour,i[0],l)
if (k[0],l[0]) not in sample.keys():
sample[k[0],l[0].hour]=0
if prev==datetime.datetime(1,1,1):
prev=l[0]
else:
diff=(l[0]-prev).seconds
tasktime=tasktime+diff
prev=l[0]
elif loitime!=0 and k[1] > 1:
print(k[0],"loitered for ",tasktime/60,"minutes")
print("timestamp ++++++++++++++++",l[0])
I get the time they took for the task for the whole data and not on hourly basis.

Formatting dates for annotating count in Django + Python 3

I'm currently trying to annotate and count some dates, based on the number of times they appear.
visits = Subs.objects.filter(camp=campdata, timestamp__lte=datetime.datetime.today(), timestamp__gt=datetime.datetime.today()-datetime.timedelta(days=30)).\
values('timestamp').annotate(count=Count('timestamp'))
If I print this in a for loop like,
for a in visits:
print(a)
I would get back the following in Json.
{'timestamp': datetime.datetime(2018, 10, 5, 15, 16, 25, 130966, tzinfo=<UTC>), 'count': 1}
{'timestamp': datetime.datetime(2018, 10, 5, 15, 16, 45, 639464, tzinfo=<UTC>), 'count': 1}
{'timestamp': datetime.datetime(2018, 10, 6, 8, 43, 24, 721050, tzinfo=<UTC>), 'count': 1}
{'timestamp': datetime.datetime(2018, 10, 7, 4, 54, 59, tzinfo=<UTC>), 'count': 1}
This is kinda the right direction, however, it's counting to the second.. I just need to days, so that the event that happened on 2018, 10, 5 would be count: 2 for example.
Can anyone lead me into the right direction?
Additionally, whats the most "django" way of converting the dates into something more json / api friendly?
My ideal json return would be something like
{'timestamp': 2018-10-5, 'count': 2}
Thanks!
You can use the TruncDate annotation to achieve this:
visits = Subs.objects.annotate(date=TruncDate('timestamp')).filter(
camp=campdata,
date__lte=datetime.datetime.today(),
date__gt=datetime.datetime.today() - datetime.timedelta(days=30)
).values('date').annotate(count=Count('date'))
As for your question about serializing dates for JSON, Django provides the DjangoJSONEncoder to help with just that:
import json
from django.core.serializers.json import DjangoJSONEncoder
json.dumps(list(visits), cls=DjangoJSONEncoder)

How to use Django __time lookup in a queryset

I have a model with 2 datetime fields which looks like this:
class Booking(models.Model):
start_date = models.DateTimeField()
end_date = models.DateTimeField()
...
As test data I have 2 bookings with start_date before 17:30 and 2 bookings after 17:45, all on the same day (8 May 2018). I am trying to filter the bookings with the __time lookup to find all the bookings before (and including) 17:30. My queryset is:
bookings = Booking.objects.filter(date__time__lte=datetime.time())
Where datetime.time prints as
datetime.time(17, 30)
and where the date part of the datetime is the same as the bookings dates. The above query is returning an empty queryset but if I use the same query except filtering for times greater than datetime.time() i.e.
bookings = Booking.objects.filter(date__time__gte=datetime.time())
The queryset returns all the bookings (where it should only return the 2 bookings with start_date after 17:30). Can someone please explain to me how the __time lookup is meant to be used?
EDIT
I updated the filter to
bookings = Booking.objects.filter(start_date__time__lte=datetime.time())
and the result is the same. When I print the values of the bookings, the values are:
print Booking.objects.all().values('date', 'end_date')
[
{'start_date': datetime.datetime(2018, 5, 8, 16, 30, tzinfo=<DstTzInfo 'Africa/Harare' CAT+2:00:00 STD>), 'end_date': datetime.datetime(2018, 5, 8, 17, 0, tzinfo=<DstTzInfo 'Africa/Harare' CAT+2:00:00 STD>)},
{'start_date': datetime.datetime(2018, 5, 8, 17, 0, tzinfo=<DstTzInfo 'Africa/Harare' CAT+2:00:00 STD>), 'end_date': datetime.datetime(2018, 5, 8, 17, 30, tzinfo=<DstTzInfo 'Africa/Harare' CAT+2:00:00 STD>)},
{'start_date': datetime.datetime(2018, 5, 8, 17, 45, tzinfo=<DstTzInfo 'Africa/Harare' CAT+2:00:00 STD>), 'end_date': datetime.datetime(2018, 5, 8, 18, 15, tzinfo=<DstTzInfo 'Africa/Harare' CAT+2:00:00 STD>)},
{'start_date': datetime.datetime(2018, 5, 8, 17, 45, tzinfo=<DstTzInfo 'Africa/Harare' CAT+2:00:00 STD>), 'end_date': datetime.datetime(2018, 5, 8, 18, 15, tzinfo=<DstTzInfo 'Africa/Harare' CAT+2:00:00 STD>)}
]
EDIT 2
I forgot to mention I need to get the bookings that are on the same date. As siddhant0905 suggested I filtered the queryset with datetimes instead and added an extra filter to make sure it was on the same date. The following worked for me:
bookings = Booking.objects.filter(Q(start_date__date=datetime.date()) & Q(start_date__lte=datetime))
I think you should compare the complete datetime object rather than just comparing the time part.
Debug the 'Type' of the time that the query returns and the type of time you are providing it to compare with. Both should be same.
Django shell can be of great help.

Iterable Datetime - How to get continuos datetime objects with day name in Python?

I want to create a mapping of the everyday of the week with its datetime object. So my dictionary should be having keys as "Monday", "Tuesday", .. (so on) so that I can get a datetime object for every day on the next(!) week.
At the moment I have a dictionary with these values:
DAYS_DATETIME_RELATIONS = {
"today": datetime.datetime.now(),
"tomorrow": datetime.datetime.now() + datetime.timedelta(days=1),
"after_tomorrow": datetime.datetime.now() + datetime.timedelta(days=2)
}
Unfortunately I cannot find any algorithmic solution for this and hope anyone of you could help me.
This can be achieved by using 2 dictionaries in the following manner:
import calendar
import datetime
days = {i: calendar.day_name[i-1] for i in range(7)}
today = datetime.datetime.now()
# using i % 7 so an arbitrary range can be used, for example
# range(7, 15) to get the week after the next week
next_week = {days[i % 7]: (today + datetime.timedelta(days=i)).date()
for i in range(7)}
print(next_week)
# {'Tuesday': datetime.date(2018, 1, 9), 'Sunday': datetime.date(2018, 1, 7),
# 'Monday': datetime.date(2018, 1, 8), 'Thursday': datetime.date(2018, 1, 11),
# 'Wednesday': datetime.date(2018, 1, 10), 'Friday': datetime.date(2018, 1, 12),
# 'Saturday': datetime.date(2018, 1, 13)}
print(next_week['Saturday'])
# 2018-01-13
Here is another way to solve your question using datetime and timedelta from datetime module:
from datetime import datetime, timedelta
def generate_dict_relation(_time, _days=0):
keys = {'Yesterday': -1, 'Today': 0, 'Tomorow': 1, 'After_tomorrow': 2}
if not _days:
return {key: _time + timedelta(days=keys.get(key, 0)) for key in keys}
else:
return {(_time + timedelta(days=_days+k)).strftime('%A'): _time + timedelta(days=_days+k)
for k in range(0, 7)}
_date_now = datetime.now()
DAYS_DATETIME_RELATIONS = {}
# get dates: yesterday, today, tomorrow and after tomorrow
DAYS_DATETIME_RELATIONS.update(generate_dict_relation(_date_now, 0))
# get dates after 7 days = 1 week
DAYS_DATETIME_RELATIONS.update(generate_dict_relation(_date_now, 7))
next_tuesday = DAYS_DATETIME_RELATIONS.get('Tuesday')
next_monday = DAYS_DATETIME_RELATIONS.get('Monday')
yesterday = DAYS_DATETIME_RELATIONS.get('Yesterday')
print('{0:%d/%m/%Y %H:%M:%S:%s} \t {1}'.format(next_tuesday, repr(next_tuesday)))
print('{0:%d/%m/%Y %H:%M:%S:%s} \t {1}'.format(next_monday, repr(next_monday)))
print('{0:%d/%m/%Y %H:%M:%S:%s} \t {1}'.format(yesterday, repr(yesterday)))
Output:
16/01/2018 10:56:26:1516096586 datetime.datetime(2018, 1, 16, 10, 56, 26, 659949)
15/01/2018 10:56:26:1516010186 datetime.datetime(2018, 1, 15, 10, 56, 26, 659949)
06/01/2018 10:56:26:1515232586 datetime.datetime(2018, 1, 6, 10, 56, 26, 659949)
One very generic way will be to create a custom iterator to return you the continuos datetime objects as:
from datetime import datetime, timedelta
class RepetetiveDate(object):
def __init__(self, day_range=7, datetime_obj=datetime.now(), jump_days=1):
self.day_range = day_range
self.day_counter = 0
self.datetime_obj = datetime_obj
self.jump_days = jump_days
self.time_deltadiff = timedelta(days=self.jump_days)
def __iter__(self):
return self
# If you are on Python 2.7
# define this function as `next(self)`
def __next__(self):
if self.day_counter >= self.day_range:
raise StopIteration
if self.day_counter != 0: # don't update for the first iteration
self.datetime_obj += self.time_deltadiff
self.day_counter += 1
return self.datetime_obj
Here, this iterator returns continuos datetime object starting from the datetime object you'll initially pass (default starts from current date).
It is using 3 optional params which you may customize as per your need:
day_range: Maximum allowed iteration for the RepetetiveDate iterator. Default value is 7.
jump_days: Integer value for jumping the number of days for the datetime object in next iteration. That means, if jump_days is equal to "2", will return datetime objects of every alternate date. To get the datetime objects of past, pass this value as negative. Default value is 1.
datetime_obj: Accepts the datetime from which date you want to start your iteration. Default value is current date.
If you are new to iterators, take a look at:
What exactly are Python's iterator, iterable, and iteration protocols?
Difference between Python's Generators and Iterators
Sample Run for upcoming dates:
>>> x = RepetetiveDate()
>>> next(x)
datetime.datetime(2018, 1, 8, 15, 55, 39, 124654)
>>> next(x)
datetime.datetime(2018, 1, 9, 15, 55, 39, 124654)
>>> next(x)
datetime.datetime(2018, 1, 10, 15, 55, 39, 124654)
Sample Run for previous dates:
>>> x = RepetetiveDate(jump_days=-1)
>>> next(x)
datetime.datetime(2018, 1, 6, 15, 55, 39, 124654)
>>> next(x)
datetime.datetime(2018, 1, 5, 15, 55, 39, 124654)
>>> next(x)
datetime.datetime(2018, 1, 4, 15, 55, 39, 124654)
How to get your desired dictionary?
Using this, you may create your dictionary using the dict comprehension as:
Dictionary of all days of week
>>> {d.strftime("%A"): d for d in RepetetiveDate(day_range=7)}
{
'Monday': datetime.datetime(2018, 1, 8, 15, 23, 16, 926364),
'Tuesday': datetime.datetime(2018, 1, 9, 15, 23, 16, 926364),
'Wednesday': datetime.datetime(2018, 1, 10, 15, 23, 16, 926364),
'Thursday': datetime.datetime(2018, 1, 11, 15, 23, 16, 926364),
'Friday': datetime.datetime(2018, 1, 12, 15, 23, 16, 926364),
'Saturday': datetime.datetime(2018, 1, 13, 15, 23, 16, 926364),
'Sunday': datetime.datetime(2018, 1, 14, 15, 23, 16, 926364)
}
Here I am using d.strftime("%A") to extract day name from the datetime object.
List of current days for next 4 weeks
>>> [d for d in RepetetiveDate(jump_days=7, day_range=4))]
[
datetime.datetime(2018, 1, 7, 16, 17, 45, 45005),
datetime.datetime(2018, 1, 14, 16, 17, 45, 45005),
datetime.datetime(2018, 1, 21, 16, 17, 45, 45005),
datetime.datetime(2018, 1, 28, 16, 17, 45, 45005)
]
One very clean way to implement this is using rrule from the dateutil library. For example:
>>> from dateutil.rrule import rrule, DAILY
>>> from datetime import datetime
>>> start_date = datetime.now()
>>> {d.strftime("%A"): d for d in rrule(freq=DAILY, count=7, dtstart=start_date)}
which will return your desired dict object:
{
'Sunday': datetime.datetime(2018, 1, 7, 17, 2, 30),
'Monday': datetime.datetime(2018, 1, 8, 17, 2, 30),
'Tuesday': datetime.datetime(2018, 1, 9, 17, 2, 30),
'Wednesday': datetime.datetime(2018, 1, 10, 17, 2, 30),
'Thursday': datetime.datetime(2018, 1, 11, 17, 2, 30),
'Friday': datetime.datetime(2018, 1, 12, 17, 2, 30),
'Saturday': datetime.datetime(2018, 1, 13, 17, 2, 30)
}
(Special thanks to Jon Clements for telling me about rrule)

PyMongo: Print length and items of arrays inside MongoDB Documents

I have such documents in my MongoDB collection:
{u'date': datetime.datetime(2014, 9, 2, 12, 27, 51, 184000),
u'domains': [u'domain1', u'domain2']}
{u'date': datetime.datetime(2014, 9, 3, 11, 23, 51, 184000),
u'domains': [u'domain3', u'domain4', u'domain5']}
I want output like this printed out by python:
2-9-2014
domain1
domain2
total: 2 domains
3-9-2014
domain3
domain4
domain5
total: 3 domains
import datetime
cursor = [{u'date': datetime.datetime(2014, 9, 2, 12, 27, 51, 184000),
u'domains': [u'domain1', u'domain2']},
{u'date': datetime.datetime(2014, 9, 3, 11, 23, 51, 184000),
u'domains': [u'domain3', u'domain4', u'domain5']}]
for node in cursor:
print node['date'].strftime("%d-%m-%Y")
for domain in node['domains']:
print domain
print "total: {0} domains\n".format(len(node['domains']))
output
02-09-2014
domain1
domain2
total: 2 domains
03-09-2014
domain3
domain4
domain5
total: 3 domains

Categories