Automatically parsing Unix/Epoch timestamp to datetime in Django Model? - python

I am writing/learning my first Django REST API. The database I am working on has all times are stored as an integers representing the seconds since Jan 01 1970 - aka Unix/Epoch time.
Right now I simply have two function that convert between this integer and a normal timestamp. So user input through parameters is converted to integers, and JSON responses are converted to timestamps.
I was wondering if there was a way to simply have Django "know" this conversion. So in my model:
class User(models.Model):
createDate = models.IntegerField(blank=True, null=True, db_column='createdate')
I could change to models.DateTimeField and createDate would be a time stamp. Correspondingly if I call:
data = Subgroup.objects.filter(createDate_gt='2016-30-04')
In my view Django knows to convert this to an integer and filter by that, rather than the string interpretation.
I know it's a pretty pedantic request but I've been impressed with Django so far and was curious :)

Related

save Jalali(Hijri shamsi) datetime in database in django

I have a Django project, and I want to save created_at datetime in the database. I generate datetime.now with jdatetime (or Khayyam) python package and try to save this in DateTimeField. But sometimes it raises error because the Gregorian(miladi) date of the entry does not exist. what can I do about this?
In my idea, you can save two model fields.
One is DateTimeField contains gregorian datetime, and
another one, CharField contains converted Jalali to a String value and save it.
The DateTimeField for functionality, e.g., filter between to datetime.
The StringField for representing in response(without overload).

Django datetime.date.min and Mysql DATE minimum value is different

Actual Question
How can i deal with different minimum date in Django DateField and Mysql DATE type. What can be done to deal with the issue?
TL:DR
datetime.date.min has lower value than minimum supported in Mysql for DATE type. Saving datetime.date.min to Mysql is possible but it is not guaranteed to work.
Context
I'm working with Django (v1.11) model:
class MyModel(model.Model):
begin_date = models.DateField(null=True, blank=True)
end_date = models.DateField(null=True, blank=True)
Currently the fields with NULL value designate the lower and higher date range limits respectively.
Example:
Date range from beginning of time to today will be:
mymodel.begin_date = None
mymodel.end_date = datetime.date(2019, 11, 20)
mymodel.save()
The usage of None/NULL creates a need to convert it to datetime.date.min/datetime.date.max on any sorting operations etc. Simply put: its inconvenient and adds unnecessary complexity.
My though was to do necessary migrations and start putting datetime.date.min/datetime.date.max instead of None/`NULL'
But there is a problem:
Mysql has different min/max date range
According to the mysql docs:
The DATE type is used for values with a date part but no time part.
MySQL retrieves and displays DATE values in 'YYYY-MM-DD' format. The
supported range is '1000-01-01' to '9999-12-31'.
According to python docs:
The smallest year number allowed in a date or datetime object. MINYEAR is 1.
The largest year number allowed in a date or datetime object. MAXYEAR is 9999.
The earliest representable date, date(MINYEAR, 1, 1).
The latest representable date, date(MAXYEAR, 12, 31).
It turns out that in Django you can still use datetime.date.min and it will be saved to the database. But it is not guranteed to work.
Because of:
For the DATE and DATETIME range descriptions, “supported” means that although earlier values might work, there is no guarantee.
My idea
I'm thinking about doing all necessary migrations, clean the code from None convertions and just use Field.default. Set it to default=datetime(1000, 1, 1) (minimum mysql supported date)
I think your idea will work fine.
Note that it's database-specific, though. The more ungainly approach of using None does have the virtue of working across databases.
Also note that apparently MySQL let's you store dates below that ostensible minimum value. Therefore, there could be dates in the database that will not be caught by your filter. So be sure to validate dates before you store them to make sure they are at or above the minimum.

Default value format on Django DateField

I'm trying to specify a default date in a Django model, for example:
from datetime import date
class A(models.Model):
date = models.DateField(default=date.today())
This works and I can see the default date in a ModelForm, but when I change the form input_format to %d-%m-%Y, the default date never appears in the field.
I've also tried:
from datetime import date
class A(models.Model):
date = models.DateField(default=date.today().strftime('%d-%m-%Y'))
This doesn't work either. Can anyone help me?
There are two problems here:
the DateField(default=today.today()) will not work, since then the function will be evaluated eagerly, and then the default value is thus the result of that function call. As a result the default value is not calculated when constructing a new object, and hence eventually will be different; and
the representation of a DateField. Now the model layer does not specify a representation, it only specifies how to store values in the database, and defines functions on a model.
We can solve the first problem by passing a reference to the today function, like:
from datetime import date
class A(models.Model):
date = models.DateField(default=date.today) # no ()
As for the representation, you should specify the format in the template, for example with the date template filter [Django-doc], like:
<!-- template.html -->
{{ some_a.date|date:'d-m-Y' }}
Or in a form with:
# app/forms.py
class AForm(ModelForm):
date = DateField(input_formats=['%d-%m-%Y'])
class Meta:
model = A
You can use the DATE_INPUT_FORMATS setting in your Django project settings, this will allow you to make date.today().strftime('%d-%m-%Y') be accepted by your model field; however, the DateField is stored in your database as the native column of the same type and not as a string with a specific format. There is a difference between storing the data and representing it in your forms, templates or DRF serializers. I really recommend keeping the default format for the database and present the data in the format you want by using the DATE_FORMAT setting to d-m-Y that will take care of presenting your dates in that format as long as the USE_L10N setting is False.
if you want to use the same format for display and input, you have to specify input_formats and format(in widget) respectively. for example
class OrderForm(forms.ModelForm):
deadline = forms.DateTimeField(
input_formats=['%d/%m/%Y %I:%M %p', ], # input format
widget=forms.DateTimeInput(format="%d/%m/%Y %I:%M %p"), # initial display format
)

Unable to save differenent date format in django

I am working on a form and have a date field.
I want to save different date format for date field instead django used.
I am getting "01-jan-2016" date and want to save as it is in my database.when i am trying to save this same format is raise an error
[u"'07-Dec-2016' value has an invalid date format. It must be in YYYY-MM-DD format."].
I know this type of question asked already but they do not solve my problem.
views.py
post_date = request.POST['date']
lead_obj = CustomerLeads.objects.create(posting_date = post_date)
my models.py
class Leads(models.Model):
customer_name = models.CharField(max_length=50,null=True, blank=True)
title = models.CharField(max_length=100, null=True, blank=True)
posting_date = models.DateField()
Mysql
The DATE type is used for values with a date part but no time part.
MySQL retrieves and displays DATE values in 'YYYY-MM-DD' format. The
supported range is '1000-01-01' to '9999-12-31'.
Postgresql
Date and time input is accepted in almost any reasonable format,
including ISO 8601, SQL-compatible, traditional POSTGRES, and others.
For some formats, ordering of day, month, and year in date input is
ambiguous and there is support for specifying the expected ordering of
these fields. Set the DateStyle parameter to MDY to select
month-day-year interpretation, DMY to select day-month-year
interpretation, or YMD to select year-month-day interpretation.
The table at the link shows that postgresql accept the format that you are looking for. But how is it stored? Let's try an insert
INSERT INTO stackoverflow_heatwatchlist(next_date_from, next_date_to)
VALUES('1999 Jan 05','2001 Jun 06');
And when you select, what you get is '2001-06-06' and '1999-01-05'
SQlite
Here you can insert in any format you want and it will be saved exactly as you entered, that's because sqlite does not enforce any strict type checking. Types in sqlite are purely cosmetic. but django has a different opinion on the matter and will not let you divert too much from the standard formats.
SQL Server
A lot of flexibility in how the data is accepted for insert. Refer to the table at the link. But how is it stored? '1999-01-05'
In conclusion
How the date is stored in the table is totally irrelevent, all that matters is what formats are accepted for input and how it's displayed. And that's where django's and python's excellent date and time formats come into play.
If you are not happy with them, you have two choices. The first is to store dates and times as unix timestamps and do all the formatting yourself, all you need is a little math really - a fairly common practice.
The second is to use CharField for storing your data but this is not very usefull. Not only do you have to do all the formatting and input validation yourself, any calculation would involve a lot of string processing and casting.
Try to add '%d-%b-%Y' to DATE_INPUT_FORMATS in your project settings.
In your views.py try this:
from dateutil import parser
d = parser.parse("07-December-2016")
print d
d = d.strftime("%d-%b-%Y")
print d
Output:
2016-12-07 00:00:00
07-Dec-2016
It will handle various formats.

Python Date Time with JSON API

I have a Python 2.7 API that queries a SQL db and delivers a JSON list of dictionaries that is then used in a bootstrap/Django site.
Dates in the DB are strings in the format '2017-04-20 00:00:00', but sometimes the time of the source data instead has a decimal, which causes trouble with strptime, so I'm removing the seconds by keeping only the first 10 characters of the string.
import datetime
dict_list = response['my_list_of_dicts']
for dt_to_cmpr in dict_list:
dt_to_cmpr['date_key'] = dt_to_cmpr['date_key'][:10]
Before I can compare date ranges, the dates need to be date time not strings. (Note: For production, I plan to account for exceptions such as null values.)
dt_to_cmpr['date_key'] = datetime.datetime.strptime(dt_to_cmpr['date_key'],
'%Y-%m-%d')
I want to know things about dictionaries where date_key is roughly no more than 90 days from today. (i.e. the total number in the time frame, or the sum of every dictionary's price_key.)
under_days = datetime.timedelta(days=-1)
over_days = datetime.timedelta(days=91)
now = datetime.datetime.now()
ttl_within_90days = sum(1 for d in response['my_list_of_dicts'] if (under_days <
(d.get('date_key')-now) < over_days))
One problem is now that I've converted my dates, the are not JSON serializable. So, now I have to put them back into a string again
for dt_to_cmpr in dict_list:
dt_to_cmpr['date_key'] = dt_to_cmpr['date_key'].strftime("%Y-%m-%d")
I cleaned up the above for simplicity, but that should all work. When it gets to Django, the view is going to covert them all back to date time again for use in a template.
Can I have Python just treat my date strings as time for the 90 day comparison, but leave them alone. Or, maybe have JSON use the Python date times? That much iteration every page load is slow, and can't be the best way.
The main problem is the way you're storing the datetimes. You should probably be storing them as actual datetimes in your database, not strings. You can't do date queries on string fields. Instead, you have to use the inefficient method of querying all the records and then filtering all of them in python after the fact. Database data types were created for a reason, use them.
There's no reason to convert datetimes to strings except at the very last moment when you need to format it for json or html, and the only bit of code that should need to do that is the Django app. That means:
Your Django app should almost entirely be using datetimes. It only coverts to strings when it needs to render out html or json.
Your API should only use python datetimes.
Your database should only use datetimes as well.
If you don't control the database, the best case is going to be 2 conversions
string -> datetime when pulling data out of the database.
datetime -> string when serializing to html or json.
If you can fix the database, then you only need to do the 2nd conversion.

Categories