We Keep Coding

How to cast integer (epoch) column to datetime in sqlalchemy?

I'm using sqlalchemy to query a database and have a Table.c.Field object from inspect, where Field is of type integer (e.g., 1596657600). How do I cast that field to a datetime? In the query I want to cast Field as datetime, then later extract and groupby dayofweek (or some other aspect of datetime). But first I need to cast Field as a date or datetime.
I have tried a few ways, but all fail. In the code below I first import sqlalchemy as sa.
I tried
sa.func.to_timestamp(Table.c.Field / 1.0).cast(sa.Date)
as suggested here. But it produces the error: no such function: to_timestamp.
I tried
Table.c.Field.cast(sa.DateTime)
as suggested here. But as in that link it produces the error: Couldn't parse datetime string '1596657600' - value is not a string. If it helps, the database I am querying is also Sqlite, but I would like my solution to work for any integer field from any database.
I also tried
sa.cast(Table.c.Field, SQA.Interval)
but it produced the same error: Couldn't parse datetime string '1596657600' - value is not a string.
If the solution is to create some special function (e.g., Fx) to do the conversion, to be used as
(Table.c.Field).Fx()
what would that function look like? Or what kind of function would do the trick? I cannot write to the database, and need to do all calculations (casting, grouping, filtering, etc.) via a single sqlalchemy query.

I found a solution in the docs, but I wasn't sure at first how to apply it in my case.
First, create a new type that accepts integers, then simply cast Field to that type. I first import sqlalchemy as SQA and import tzinfo and datetime and timedelta from datetime.
class MyEpochType(SQA.types.TypeDecorator):
impl = SQA.types.Integer
epoch = datetime(1970, 1, 1, 0, 0, 0, tzinfo=pytz.timezone('UTC'))
def process_bind_param(self, value, dialect):
pass # not needed in my use case
def process_result_value(self, value, dialect):
# either of the two returns below, or similar ones
#return self.epoch + timedelta(seconds=value )
return datetime.fromtimestamp(value).isoformat()
Table.c.Field.cast(MyEpochType)
The above solution will not allow for a group_by on the transformed values (e.g., a datetime or string). For this, a better solution is to use the native functions of the underlying database. In my case it is Sqlite, and would be called similar to that below:
SQA.func.strftime('%H%M',
SQA.func.datetime(Table.c.Field, 'unixepoch')).label('someLabel')
Here as an example I am calling two different Sqlite functions and giving the result a label. Each database has its own set of particular functions that can be called.

How to apply __contains to formatted date with django ORM?

I want to implement a database search and I need to filter queryset by date that contains search request. The problem is that users will see formatted date (like dd.mm.yyyy) and will expect search to see it the same way, but default __contains consider date in yyyy-mm-dd format. How can I apply filter to DateField before using __contains? I need a way using django orm because otherwise it will be too long (not using sql).
I tried to set DATE_FORMAT = 'd.m.Y' to satisfying format but it has no effect (even setting USE_L10N = False)
Also thought about creating custom lookup but not sure how to implement it
I need to apply __contains to all model fields, so I can't just reformat search value

You can Annotate the date attribute to a CharField and execute your filter on Annotated attribute.
from django.db.models import Func, F, Value, CharField
queryset.annotate(string_date=Func(
F('YOUR_DATE_FIELD'),
Value('YOUR OUTPUT FORMAT'),
output_field=CharField(),
function='to_char')
).filter(string_date__contains=INPUT_DATE)

Modify column output for sqlform.grid() in Web2py

I have started using web2py for a web application and try to use SQLFORM.grid(...) to display a paginated listing of one of my db-table's data like in the following minimal example.
grid=SQLFORM.grid(query,
links=links,
fields=[db.example.date,db.example.foo, db.example.bar])
The db.example.date field contains a Python datetime.datetime object in UTC. At the moment it is displayed just plainly like that. However, I want to have more control about the actual output in a way that I can set the local timezone and modify the output string to have something like "2 hours ago".
As seen in another question[0] I can use the links to insert new columns. Unfortunately I can't seem to sort the rows by a field I have inserted in such way. Also, they are inserted on the right instead of actually replacing my first column. So that does not seem to be a solution.
To sum it up: How do I gain control about the way db.example.date is printed out in the end?
[0] Calculated Fields in web2py sqlgrid

You can achieve your goal when you define the table in your model. The represent parameter in the Field constructor that you used in define_table will be recognized by the SQLFORM.grid. For example, if you wanted to just print the date with the month name you could put the following in your model.
Field('a_date', type='date', represent=lambda x, row: x.strftime("%B %d, %Y")),
your function could also convert to local time.

You need to use prettydate to change the datetime arid format in a humanized string, and call it in the represent parameter of your Field() descriptor. For example :
from gluon.tools import prettydate
db.example.date.represent = lambda v,r: prettydate(r.date)
That way, any display of the db.example.date would be displayed humanized, including through SQLFORM.grid

If you don't want to have the date always represented in this way as per David Nehme's answer. Just before your grid creation, you can set the db.table.field.represent in the controller.
db.example.date.represent = lambda value, row: value.strftime("%B %d, %Y")
followed by.
grid = SQLFORM.grid(query,....
I use this often when I join tables. If there is a row.field in the represent from the model file it breaks because it then must be more specific, row.table.field.

Manipulating the DateTime object in Google app engine

I am making a blog and store the publishing date of a blog post in the datastore. It looks like this:
post.date = datetime.datetime.now()
It now displays like: 2010-10-04 07:30:15.204352 But I want the datetime to be displayed differently. How (and where) can I set that how the date is displayed? I'd like to set the date format like in UNIX date function (like %Y/%m etc). I tried to add some parameters in my templates but that returned errors.
Thanks in advance!
-skazhy

I think strftime is the method you're looking for.
From the link:
>>> d.strftime("%d/%m/%y")
'11/03/02'
If you pass in the result of the strftime in your 'template_values' or similar (the dictionary you use to pass parameters to the template) instead of the actual date it will be displayed instead.

You can use .strftime() on a datetime object to do the formatting. See the relevant python documentation for details.

Store django forms.MultipleChoiceField in Models directly

Say I have choices defined as follows:
choices = (('1','a'),
('2','b'),
('3','c'))
And a form that renders and inputs these values in a MultipleChoiceField,
class Form1(forms.Form):
field = forms.MultipleChoiceField(choices=choices)
What is the right way to store field in a model.
I can of course loop through the forms.cleaned_data['field'] and obtain a value that fits in models.CommaSeperatedIntegerField.
Again, each time I retrieve these values, I will have to loop and convert into options.
I think there is a better way to do so, as in this way, I am in a way re-implementing the function that CommaSeperateIntegerField is supposed to do.

The first thing I would consider is better normalization of your database schema; if a single instance of your model can have multiple values for this field, the field perhaps should be a linked model with a ForeignKey instead.
If you're using Postgres, you could also use an ARRAY field; Django now has built-in support.
If you can't do either of those, then you do basically need to reimplement a (better) version of CommaSeparatedIntegerField. The reason is that CommaSeparatedIntegerField is nothing but a plain CharField whose default formfield representation is a regex-validated text input. In other words, it doesn't do anything that's useful to you.
What you need to write is a custom ListField or MultipleValuesField that expects a Python list and returns a Python list, but internally converts that list to/from a comma-separated string for insertion in the database. Read the documentation on custom model fields; I think in your case you'll want a subclass of CharField with two methods overridden: to_python (convert CSV string to Python list) and get_db_prep_value (convert Python list to CSV string).

I just had this same problem and the solution (to me) because as Carl Meyer put it. I don't want a normalized version of this "list of strings" is to just have a CharField in the model. This way your model will store the normalized list of items. In my case this is countries.
So the model declaration is just
countries = CharField(max_lenght=XXX)
where XXX is a precalculated value of 2x my country list. Because it's simpler for us to apply a check to see if the current country is in this list rather than do it as a M2M to a Country table.