We Keep Coding

How I can find closest date by single request using pymongo? Date should be closest to Current Date

Dataset :
{"_id":{"$oid":"61e038a052124accf41cb5e4"},"event_date":{"$date":{"$numberLong":"1642204800000"}},
"name":"UFC Fight Night","event_url":"https://www.tapology.com/fightcenter/events/82805-ufc-fight-night","location":"Las Vegas, NV"}
{"_id":{"$oid":"61e038a252124accf41cb5e5"},"event_date":{"$date":{"$numberLong":"1642809600000"}},"name":"UFC 270","event_url":"https://www.tapology.com/fightcenter/events/82993-ufc-270","location":"Anaheim, CA"}
{"_id":{"$oid":"61e038a252124accf41cb5e6"},"event_date":{"$date":{"$numberLong":"1644019200000"}},"name":"UFC Fight Night","event_url":"https://www.tapology.com/fightcenter/events/83125-ufc-fight-night","location":"Las Vegas, NV"}
I'm using python, and that means I have no way to use $commands in my code for Mongo DB to find collection I need. Question is how can find object which should have datetime value closest to current date. As I understand I have to use python's datetime.now() to set current date and compare it ($currentDate doesn't work for Python). But but in order to compare values I have to deserialize the object and this looks very heavy. By default Mongo uses ISO datetime type.
Can you help me? At least direction to put me on a right way?

Here is a solution with an aggregate
db.collection.aggregate([
{
"$addFields": {
"dateDiff": {
"$abs": {
"$subtract": [
"$event_date",
{
"$literal": new Date()
}
]
}
}
}
},
{
"$sort": {
"dateDiff": 1
}
}
])
I use $subtract to have a difference between event_date and today.
$abs is to have the absolute value. To have the closest in the future or in the past.
And then we just have to sort by dateDiff
In your code you have to replace new Date()
Try it here

Questions on mongoDB query exact year

I am building a html form where it will query the MongoDB and retrieve the entire record based on the year.
How can I query only the year?
Something along the line in SQL:
select * from database where date == '2021'
How to do the equivalent in MongoDB?

If you are storing the dates as Date you have two options:
First one is to create dates and compare (not tested in python but should work):
import datetime
start = datetime.datetime(2021, 1, 1)
end = datetime.datetime(2022,1,1)
db.find({date: {$gte: start, $lt: end}});
Note the trick here is to query dates between the desired year, like this example
The other way is using aggregation like this:
Here you are getting the year using $year and comparing with your desired value.
db.collection.aggregate([
{
"$match": {
"$expr": {
"$eq": [
{
"$year": "$date"
},
2021
]
}
}
}
])
Example here

How to compare datetime strings inside a dict of a list with nested dicts?

I'm writing a function that takes a vendorID and a date_time string that should return if a vendorID can deliver if time/date doesn't overlap for a delivery to be done.
I'm trying to compare datetime strings inside a dict that has a list of nested dict with various elements including the datetime string. I want to compare each datetime string from each nested dict inside the list and check if the date is different and then compare if a certain amount of minutes have been passed.
Tried to loop through dict and items and use datetime.strptime() to parse the datetime string but I'm not sure how to compare the dates inside the list of dicts when iterating through the dict items.
dict = {
"results": [
{
"vendor_id": 1,
"client_id": 10,
"datetime": "2017-01-01 13:30:00"
},
{
"vendor_id": 1,
"client_id": 40,
"datetime": "2017-01-01 14:30:00"
}

Hope this helps you. Using dict as you have advised;
dict = {
"results": [
{
"vendor_id": 1,
"client_id": 10,
"datetime": "2017-01-01 13:30:00"
},
{
"vendor_id": 1,
"client_id": 40,
"datetime": "2017-01-01 14:30:00"
}]}
Having some date times, use the for loop and test if;
somedatetime1 = '2017-01-01 14:00:00'
somedatetime2 = '2017-01-01 15:00:00'
for d in dict['results']:
if d['datetime'] < somedatetime1:
print('somedatetime1 :: Vendor {} Client {} DateTime {}'.format(d['vendor_id'], d['client_id'], d['datetime']))
if d['datetime'] < somedatetime2:
print('somedatetime2 :: Vendor {} Client {} DateTime {}'.format(d['vendor_id'], d['client_id'], d['datetime']))
Returns;
somedatetime1 :: Vendor 1 Client 10 DateTime 2017-01-01 13:30:00
somedatetime2 :: Vendor 1 Client 10 DateTime 2017-01-01 13:30:00
somedatetime2 :: Vendor 1 Client 40 DateTime 2017-01-01 14:30:00

I would think it would be easier to do this in pandas as you can groupby vendor ID and do operations only for that vendor.
from sklearn.metrics.pairwise import euclidean_distances
import pandas as pd
import numpy as np
df = pd.DataFrame() #Initiate
grp = df.from_dict(dict['results']).groupby('vendor_id') #Convert into df and groupby vendor id, should be useful when u have more than one vendor
dct = {}
for group in grp.groups:
df_vid = grp.get_group(group) # Obtain data for that vendor
df_vid['datetime'] = pd.to_datetime(df_vid['datetime'])
ab = np.array(df_vid['datetime'])
dist = euclidean_distances(ab.reshape(-1, 1), ab.reshape(-1, 1)) # Find distance matrix
dct[group] = dist # Assign it to dict by vendor ID as key

You can change time of string type to datetime type.
And, Just you use -.
After that, You can use timedelta what is returned by time_diff function if you handle the time more.
If you want to get minutes, use seconds attribute.
from datetime import datetime
def time_diff (a, b):
return a - b
dict = {
"results": [
{
"vendor_id": 1,
"client_id": 10,
"datetime": "2017-01-01 13:30:00"
},
{
"vendor_id": 1,
"client_id": 40,
"datetime": "2017-01-01 14:30:00"
}
]
}
for r in dict['results']:
r['datetime'] = datetime.strptime(r['datetime'], '%Y-%m-%d %H:%M:%S')
print (int(time_diff(dict['results'][1]['datetime'], dict['results'][0]['datetime']).seconds/60))

Update timestamps in a Json file with Python script

I would like your help with updating a Json file times.json with a Python script that will update each of the following time stamps:
For Id1: CurrentTS-9days, Id2: CurrentTS-7days, Id3: CurrentTS-5days.. etc
I tried to use datetime.date.today() but I just couldn't get to a full script that works.
[{
"creationTime": 1543647600000,
"id":1
},
{
"creationTime": 1543647600000,
"id":2
},
{
"creationTime": 1543647600000,
"id":3
}]

In your code, I assume the field "creationTime" is the date converted to seconds, so I based my implementation on that. Here is a quick code to update the time stamps given the requirement:
from datetime import datetime, timedelta
data = [{"creationTime": 1543647600000,"id":1},
{"creationTime": 1543647600000,"id":2},
{"creationTime": 1543647600000,"id":3}]
day_start = 9
for tuple in data:
print('Previous: ' , tuple['creationTime'])
tuple['creationTime'] -= int(timedelta(days = day_start).total_seconds())
day_start -= 2
print('After: ', tuple['creationTime'])
This is what I understood from the question, if something is not the way you intended it to be, please comment and I will try to look.

How to convert and over write MongoDB date string into ISODate using Python PyMongo?

I'm trying to convert a number of document strings in the format "YYYY-MM-DD" into ISODates in MongoDB and have successfully written this for the console which has the desired effect:
db.address.find({ "date" : { $type : 2 } } ).forEach(function(element){ element.date = ISODate(element.date); db.address.save(element);})
I'm trying to do the same in Python something like this:
client = MongoClient(my_mongodb_ip)
db = client.address
result = db.address.find( { "date" : { "$type" : 2 } } );
for r in result:
print(r['date'])
r["date"] = datetime.strptime(r["date"], "%Y-%m-%d")
print(r['date'])
db.address.update_one({"company": r['company']},
{"$set": {"date" : r['date']}})
Which I'd like to deliver this:
"date": {
"$date": "2017-06-28T00:00:00.000Z"
},
I don't get any updates to the DB.

Try to use:
import dateutil.parser
dateutil.parser.parse(<your time String>)

You can achieve this goal using the arrow module in python. All You need to do is, just create a small function that can take your date as parameter and convert it into the ISO format.
This is how you can do it:
import arrow
def convert_to_ISO_Format(self, value):
date = arrow.get(value)
date = date.format("YYYY-MM-DDTHH:mm:ss")
date = date + "Z"
self.converted_date_iso = date
Or If you know the region of the state and you want to convert it accordingly then you can do like this;
def convert_to_ISO_Region(self,value):
date = arrow.get(value)
date = date.to("Asia/Singapore")
date = date.format("YYYY-MM-DD HH:mm:ss")
date = date + "Z"
self.converted_date_iso = date
OR
If you want to convert the present date and time into ISO, it's pretty simple like the below one statement.
arrow.utcnow().datetime
I hope this can be helpful to your problem.

I figured out what was going wrong here. No need to import arrow or parser, datetime works just fine:
newest_date = datetime.strptime(r["date"], "%Y-%m-%d")
This creates the new datetime object from my "date" item in my collection assuming the format YYYY-MM-DD. Once I figured that out I had also been incorrectly putting .isoformat() to the end of this creating a string again, no idea why I did that.
With the incorrect isoformat() removed I can now run:
db.address.update_one({"address": "1234", {"$set": {"date" : newest_date}})
And the address will update correctly to type 9, i.e. a date object not a string. I checked this with:
db.address.find({ "date" : { $type : 9 } } )