I am building a html form where it will query the MongoDB and retrieve the entire record based on the year.
How can I query only the year?
Something along the line in SQL:
select * from database where date == '2021'
How to do the equivalent in MongoDB?
If you are storing the dates as Date you have two options:
First one is to create dates and compare (not tested in python but should work):
import datetime
start = datetime.datetime(2021, 1, 1)
end = datetime.datetime(2022,1,1)
db.find({date: {$gte: start, $lt: end}});
Note the trick here is to query dates between the desired year, like this example
The other way is using aggregation like this:
Here you are getting the year using $year and comparing with your desired value.
db.collection.aggregate([
{
"$match": {
"$expr": {
"$eq": [
{
"$year": "$date"
},
2021
]
}
}
}
])
Example here
Related
Dataset :
{"_id":{"$oid":"61e038a052124accf41cb5e4"},"event_date":{"$date":{"$numberLong":"1642204800000"}},
"name":"UFC Fight Night","event_url":"https://www.tapology.com/fightcenter/events/82805-ufc-fight-night","location":"Las Vegas, NV"}
{"_id":{"$oid":"61e038a252124accf41cb5e5"},"event_date":{"$date":{"$numberLong":"1642809600000"}},"name":"UFC 270","event_url":"https://www.tapology.com/fightcenter/events/82993-ufc-270","location":"Anaheim, CA"}
{"_id":{"$oid":"61e038a252124accf41cb5e6"},"event_date":{"$date":{"$numberLong":"1644019200000"}},"name":"UFC Fight Night","event_url":"https://www.tapology.com/fightcenter/events/83125-ufc-fight-night","location":"Las Vegas, NV"}
I'm using python, and that means I have no way to use $commands in my code for Mongo DB to find collection I need. Question is how can find object which should have datetime value closest to current date. As I understand I have to use python's datetime.now() to set current date and compare it ($currentDate doesn't work for Python). But but in order to compare values I have to deserialize the object and this looks very heavy. By default Mongo uses ISO datetime type.
Can you help me? At least direction to put me on a right way?
Here is a solution with an aggregate
db.collection.aggregate([
{
"$addFields": {
"dateDiff": {
"$abs": {
"$subtract": [
"$event_date",
{
"$literal": new Date()
}
]
}
}
}
},
{
"$sort": {
"dateDiff": 1
}
}
])
I use $subtract to have a difference between event_date and today.
$abs is to have the absolute value. To have the closest in the future or in the past.
And then we just have to sort by dateDiff
In your code you have to replace new Date()
Try it here
I would like your help with updating a Json file times.json with a Python script that will update each of the following time stamps:
For Id1: CurrentTS-9days, Id2: CurrentTS-7days, Id3: CurrentTS-5days.. etc
I tried to use datetime.date.today() but I just couldn't get to a full script that works.
[{
"creationTime": 1543647600000,
"id":1
},
{
"creationTime": 1543647600000,
"id":2
},
{
"creationTime": 1543647600000,
"id":3
}]
In your code, I assume the field "creationTime" is the date converted to seconds, so I based my implementation on that. Here is a quick code to update the time stamps given the requirement:
from datetime import datetime, timedelta
data = [{"creationTime": 1543647600000,"id":1},
{"creationTime": 1543647600000,"id":2},
{"creationTime": 1543647600000,"id":3}]
day_start = 9
for tuple in data:
print('Previous: ' , tuple['creationTime'])
tuple['creationTime'] -= int(timedelta(days = day_start).total_seconds())
day_start -= 2
print('After: ', tuple['creationTime'])
This is what I understood from the question, if something is not the way you intended it to be, please comment and I will try to look.
Ok so I'm looking for advice and suggestions on the best way to comb through json data to look for today's date/time and return the right value.
Here is a sample of what the json data looks like:
[
{
"startDateTime": "2018-04-11T14:17:00-05:00",
"endDateTime": "2018-04-11T14:18:00-05:00",
"oncallMember": [
"username1"
],
"shiftType": "historical"
},
{
"startDateTime": "2018-04-11T14:18:00-05:00",
"endDateTime": "2018-04-16T08:00:00-05:00",
"oncallMember": [
"username2"
],
"shiftType": null
},
{
"startDateTime": "2018-04-16T08:00:00-05:00",
"endDateTime": "2018-04-23T08:00:00-05:00",
"oncallMember": [
"username1"
],
"shiftType": null
},
{
"startDateTime": "2018-04-23T08:00:00-05:00",
"endDateTime": "2018-04-30T08:00:00-05:00",
"oncallMember": [
"username2"
],
"shiftType": null
},
......continues on for the year
The start/end dates are set to be weekly rotations between the members, however when exceptions are set or changed, the start/end dates could vary daily or any other amount of time. What I want to do is check for today's date and find the current "oncallMember". I'm not sure how to search between the start and end times for today's date.
Any help is appreciated.
the module arrow maybe helpful.
first,get the today's timestamp range
import arrow
today = arrow.now()
(day_start,day_end) = today.span('day')
day_start_timestamp = day_start.timestamp
day_end_timestamp = day_end.timestamp
and then you need parse detail data into timestamp,but your raw data looks like a duration of time like "2018-04-16T08:00:00-05:00",maybe you need slice part of it like "2018-04-16T08:00:00",and using arrow parse it into timestamp,like
raw = "2018-04-16T08:00:00"
FORMAT = "YYYY-MM-DDTHH:mm:SS"
obj = arrow.get(raw,FORMAT)
obj_ts = obj.timestamp
and then you need judge whether obj_ts is in range between day_start_timestamp and day_end_timestamp
but if you need your code running for days,the timestamp range need to be changed everyday
json and datetime libraries
Usejson library for reading json by json.loads and converting it into dictionary
and for str to datetime conversion use datetime and dateutil.parser.parse
import json
from dateutil.parser import parse
from datetime import datetime
dict_ = json.loads(json_str)
# json str is the json you mentioned
startDate = dict_[0]['startDateTime']
# '2018-04-11T14:17:00-05:00'
date = parse(startDate)
# datetime.datetime(2018, 4, 11, 14, 17, tzinfo=tzoffset(None, -18000))
Once you get the date-time do further coding for start end date of today's comparison and return oncallMember
I'm trying to convert a number of document strings in the format "YYYY-MM-DD" into ISODates in MongoDB and have successfully written this for the console which has the desired effect:
db.address.find({ "date" : { $type : 2 } } ).forEach(function(element){ element.date = ISODate(element.date); db.address.save(element);})
I'm trying to do the same in Python something like this:
client = MongoClient(my_mongodb_ip)
db = client.address
result = db.address.find( { "date" : { "$type" : 2 } } );
for r in result:
print(r['date'])
r["date"] = datetime.strptime(r["date"], "%Y-%m-%d")
print(r['date'])
db.address.update_one({"company": r['company']},
{"$set": {"date" : r['date']}})
Which I'd like to deliver this:
"date": {
"$date": "2017-06-28T00:00:00.000Z"
},
I don't get any updates to the DB.
Try to use:
import dateutil.parser
dateutil.parser.parse(<your time String>)
You can achieve this goal using the arrow module in python. All You need to do is, just create a small function that can take your date as parameter and convert it into the ISO format.
This is how you can do it:
import arrow
def convert_to_ISO_Format(self, value):
date = arrow.get(value)
date = date.format("YYYY-MM-DDTHH:mm:ss")
date = date + "Z"
self.converted_date_iso = date
Or If you know the region of the state and you want to convert it accordingly then you can do like this;
def convert_to_ISO_Region(self,value):
date = arrow.get(value)
date = date.to("Asia/Singapore")
date = date.format("YYYY-MM-DD HH:mm:ss")
date = date + "Z"
self.converted_date_iso = date
OR
If you want to convert the present date and time into ISO, it's pretty simple like the below one statement.
arrow.utcnow().datetime
I hope this can be helpful to your problem.
I figured out what was going wrong here. No need to import arrow or parser, datetime works just fine:
newest_date = datetime.strptime(r["date"], "%Y-%m-%d")
This creates the new datetime object from my "date" item in my collection assuming the format YYYY-MM-DD. Once I figured that out I had also been incorrectly putting .isoformat() to the end of this creating a string again, no idea why I did that.
With the incorrect isoformat() removed I can now run:
db.address.update_one({"address": "1234", {"$set": {"date" : newest_date}})
And the address will update correctly to type 9, i.e. a date object not a string. I checked this with:
db.address.find({ "date" : { $type : 9 } } )
Im struggling with localized datetimes.
All the dates stored in mongo are converted to UTC automatically, so we have to localized them after retrieving them. Im fine with that, but..
In the case where I make a query to group records by date, meaning YYY-MM-DD, problems arise. Since my local time is GMT-3, any record with time above 21:00 will be stored in mongo as 00:00, thus corresponding to the following day. When grouping by date in the query i'd be getting records in the wrong day, and wont be able to recover from that because i lose the hour details.
Is there a way of localizing the dates in the groupby command in a pymongo query?
Here's the code:
def records_by_date():
pipeline = []
pipeline.append({"$group": {
"_id": {
"$concat": [
{"$substr": [{"$year": "$date"}, 0, 4]},
"-",
{"$substr": [{"$month": "$date"}, 0, 2]},
"-",
{"$substr": [{"$dayOfMonth": "$date"}, 0, 2]}
]},
"record_id": {"$push": "$_id"},
"count": {"$sum": 1}
}})
pipeline.append({"$project": {
"_id": 0,
"date": "$_id",
"record_id": 1,
"count": 1
}})
pipeline.append({"$sort": {"date": 1}})
return self.collection.aggregate(pipeline)['result']
If I add the hour details, I could verify the records after that, but then I wouldn't be grouping by date.
Any ideas?