I want to insert date and time into mongo ,using pymongo.
However, I can insert datetime but not just date or time .
here is the example code :
now = datetime.datetime.now()
log_date = now.date()
log_time = now.time()
self.logs['test'].insert({'log_date_time': now, 'log_date':log_date, 'log_time':log_time})
it show errors :
bson.errors.InvalidDocument: Cannot encode object: datetime.time(9, 12, 39, 535769)
in fact , i don't know how to insert just date or time in mongo shell too.
i know insert datetime is new Date(), but I just want the date or time filed.
You are experiencing the defined behavior. MongoDB has a single datetime type (datetime). There are no separate, discrete types of just date or just time.
Workarounds: Plenty, but food for thought:
Storing just date is straightforward: assume Z time, use a time component of 00:00:00, and ignore the time offset upon retrieval.
Storing just time is trickier but doable: establish a base date like the epoch and only vary the time component, and ignore the date component upon retrieval.
Related
Some code I am writing in Python takes in a date from a server in the struct_time format (with the 9 args).
How can I store this date in an SQL database, and be able to read back this date as a struct_time while keeping the timezone and all additional information coming from struct_time?
I tried putting the struct_time directly in the SQL
struct_date = time.struct_time(tm_year=2020, tm_mon=9, tm_mday=10, tm_hour=22, tm_min=31, tm_sec=4, tm_wday=3, tm_yday=254, tm_isdst=0)
cursor.execute("UPDATE dbo.RSS_Links SET last_update=? WHERE link=?;", struct_date, links)
> "A TVP's rows must be Sequence objects.", 'HY000'
I can put the time in the database using the below, but I don't see where the timezone is kept when converting to strftime.
date_to_store = time.strftime("%Y-%m-%d %H:%M:%S", struct_date)
I'd highly suggest doing one of these (in this specific order):
Use built-in DATETIME data type and store all dates in UTC
Use LONG/BIGINT type to store date in epoch
Use built-in DATETIME format that can store time zone information
Don't store dates as strings, don't couple it with struct_time or any other struct/class, you'll regret it later :)
Your application should have a data layer, which would handle data serialization/deserialization.
I generate and insert a time and date into my SQLite3 database in Python with the following command:
connection.execute("INSERT INTO ActiveTable (IDPK, time) VALUES (NULL, datetime('now', 'localtime'))")
The time is stored into the database in an INTEGER field with the format:
2016-07-18 08:58:48
I think that the value is actually stored as a TEXT since Sqlite3 works with type affinity, INTEGER is just left over code that I will probably change.
My goal is to query my database to get this value from a SELECT operation, and then find the difference compared to the current time using python's native time and date modules. My final goal is to have time difference from original insert to now represented similarly to this:
2d:1h:5m
To be inserted into the Database in a new field called TimeDiff.
Is this possible using time, date, or datetime in python? I'm having trouble with the syntax for these modules.
from datetime import datetime
day1 = datetime.strptime("2016-07-18 08:58:48","%Y-%m-%d %H:%M:%S")
day2 = datetime.strptime("2016-02-18 07:58:48","%Y-%m-%d %H:%M:%S")
print(day1 -day2)
Output:
151 days, 1:00:00
I'm using the Proficy Historian SDK with python27. I can create a data record object and add the query criteria attributes (sample type, start time, end time, sample interval - in milliseconds) and use datarecord.QueryRecordset() to execute a query.
The problem I'm facing is that method QueryRecordset seems to only work for returning a small number of data sets (a few hundred records at most) i.e. a small date range, otherwise it returns no results for any of the SCADA tags. I can sometimes get it to return more (a few thousand) records by slowly incriminating the date range, but it seems unreliable. So, is there a way to fix this or a different way to do the query or set it up? Most of my queries contain multiple tags. Otherwise, I guess I'll just have to successively execute the query/slide the date range and pull a few hundred records at a time.
Update:
I'm preforming the query using the following steps:
from win32com.client.gencache import EnsureDispatch
from win32com.client import constants as c
import datetime
ihApp = EnsureDispatch('iHistorian_SDK.Server')
drecord = ihApp.Data.NewRecordset()
drecord.Criteria.Tags = ['Tag_1', 'Tag_2', 'Tag_3']
drecord.Criteria.SamplingMode = c.Calculated
drecord.Criteria.CalculationMode = c.Average
drecord.Criteria.Direction = c.Forward
drecord.Criteria.NumberOfSamples = 0 # This is the default value
drecord.Criteria.SamplingInterval = 30*60*1000 # 30 min interval in ms
# I've tried using the win32com pytime type instead of datetime, but it
# doesn't make a difference
drecord.Criteria.StartTime = datetime.datetime(2015, 11, 1)
drecord.Criteria.EndTime = datetime.datetime(2015, 11, 10)
# Run the query
drecord.Fields.Clear()
drecord.Fields.AllFields()
drecord.QueryRecordset()
One problem that may be happening is the use of dates/times in the dd/mm/yyyy hh:mm format. When I create a pytime or datetime object the individual attributes e.g. year, day, month, hour, minute are all correct before and after assignment to drecord.Criteria.StartTime and drecord.Criteria.EndTime, but when I print the variable it always comes out in mm/dd/yyyy hh:mm format, but this is probably due to the object's str or repr method.
So, it turns out there were two properties that could be adjusted to increase the number of samples returned and time before a timeout occurred. Both properties are set on the server object (ihApp):
ihApp.MaximumQueryIntervals = MaximumQueryIntervals # Default is 50000
ihApp.MaximumQueryTime = MaximumQueryTime # Default is 60 (seconds)
Increasing both these values seemed to fix my problems. Some tags definitely seem to take longer to query than others (over the same time period and same sampling method), so increasing the max. query time helped make returning query data more reliable.
When QueryRecordset() completes it returns False if there was an error and doesn't populate any of the data records. The error type can be show using:
drecord.LastError
I'm trying to figure out what's going on with a timezone conversion that's happening in Django.
My view code is as below, it filters on a date range and groups on the day of creation:
def stats_ad(request):
start_date = datetime.datetime.strptime(request.GET.get('start'), '%d/%m/%Y %H:%M:%S')
end_date = datetime.datetime.strptime(request.GET.get('end'), '%d/%m/%Y %H:%M:%S')
fads = Ad.objects.filter(created__range=[start_date, end_date]).extra(select={'created_date': 'created::date'}).values('created_date').annotate(total=Count('id')).order_by("created_date")
The SQL query that is produced by django when I set the get variable of start to "01/05/2013 00:00:00" and the request end variable to "11/05/2013 23:59:00":
SELECT (created::date) AS "created_date", COUNT("ads_ad"."id") AS "total" FROM "ads_ad" WHERE "ads_ad"."created" BETWEEN E'2013-05-01 00:00:00+10:00' and E'2013-05-11 23:59:59+10:00' GROUP BY created::date, (created::date) ORDER BY "created_date" ASC
If I manually run that on my Postgresql database, it's all good, finds the following:
created_date total
2013-05-10 22
2013-05-11 1
However If I do the following:
for a in fads:
recent_ads.append({"dates": a['created_date'].strftime('%d/%m/%Y'), 'ads': a['total']})
It gives me the following output:
[{"dates": "09/05/2013", "ads": 1}, {"dates": "10/05/2013", "ads": 22}]
I'm at a loss at why it's changed the dates?
Anyone got any ideas?
Cheers,
Ben
Just a through on this. As of Django 1.4, Django now supports timezone aware dates and times. Perhaps it's possible that a conversion between your local timezone and the timezone that the data is stored in (possibly GMT) is taking place at some point. Perhaps that difference crosses the international date line, in which case the dates may show up differently.
Django has an interesting section describing the new timezone support feature.
https://docs.djangoproject.com/en/1.4/topics/i18n/timezones/
That's what came to mind, anyway, when you described your problem. Hope this helps.
Python datetime from the standard python library is a mess.
Probably you are creating naive datetime instances (instances that lack timezone information).
# naive
now = datetime.datetime.now()
# TZ aware
from django.utils.timezone import utc
now = datetime.datetime.utcnow().replace(tzinfo=utc)
In recent Django, datetime storage is always offset-aware, so you better convert naive datetimes - otherwise an automatic (and sometimes wrong) conversion will take place.
Take a look at the docs about Django Time Zones.
I am trying to find records in a Django app where a datetime field is older than the current datetime.
I have something like this:
records = Record.objects.filter(my_field__lte=datetime.now())
However, it only finds records where the date is older than today (if there are records older than now but on today's date, they are not returned.)
Edit: Just to clarify. If the field is set to 19:59:00 12-10-2011 and the current time is 20:00:00 12-10-2011, then I would want this record to be returned in the queryset. However, it's not actually being returned until the query is run on the following day. (It's run every five minutes on a cronscript and is used to send emails based on this datetime field)
Hope that clears it up.
BTW, I am using MySQL
Can anyone help?
Any advice appreciated.
Thanks
check in database if you can see time attributes, next check if you have models.DateTimeField in your models and no models.DateField by error
Just test for midnight tomorrow instead of now:
now = datetime.now()
midnight = datetime(now.year, now.month, now.day+1, 0, 0, 0)
records = Record.objects.filter(my_field__lt=midnight)
Or you can use date.today and timedelta to get the same effect:
from datetime import date, timedelta
tomorrow = date.today() + timedelta(days=1)
records = Record.objects.filter(my_field__lt=tomorrow)