We Keep Coding

Transform continious date string (20190327200000000W) in date time

I'm doing an application which parse a XML from http request and one of the attributes is a date.
The problem is that the format is a string without separation, for example: '20190327200000000W' and I need to transform it into a datetime format to send it to a database.
All the information I have found is with some kind of separation char (2019-03-23 ...). Can you help me?
Thanks!!!

Maybe this? (in jupypter notebook)
from datetime import datetime
datetime_object = datetime.strptime('20190327200000000W', '%Y%m%d%H%M%S%fW')
datetime_object

Well I have solved this, at first I did that Xenobiologist said, but I had a format problem, so I decided to delete the last character (the X of %X)...and I realized that I hadn't a string, I had a list, so I transformed to string and did the operations. My code (I'll put only the inside for loop part, without the parsing part):
for parse in tree.iter(aaa):
a = parse.get(m)
respon = a.split(' ')
if m == 'Fh':
x = str(respon[0])
x2 = len(x)
x3 = x[:x2-1]
print (x3)
y = time.strptime(x3, "%Y%m%d%H%M%S%f")

Sqlalchemy get row in timeslot

I have a model called Appointment which has the columns datetime which is a DateTime field and duration which is an Integer field and represents duration in minutes. Now I want to check if func.now() is between the datetime of the appointment and the sum of the datetime and duration
I am currently to try to do it this way, but I need a solution that will work for both PostgreSQL and SQLite.
current_appointment = Appointment.query.filter(
Appointment.datetime.between(
func.now(),
func.timestampadd(
'MINUTES', Appointment.duration, func.now()
)
)
).limit(1).one_or_none()

I don't think you'll be able to do this directly in the ORM for both sqlite and postgres, but sqlalchemy lets you extend it in a cross-dialect way with Custom SQL Constructs and Compilation Extension.
This snippet might not be exactly right because I hacked at it with some different models and translated it over for this, but I got something very close to render the postgres SQL correctly:
from sqlalchemy import func
from sqlalchemy.sql import expression
from sqlalchemy.types import DateTime
from sqlalchemy.ext.compiler import compiles
class durationnow(expression.FunctionElement):
type = DateTime()
name = 'durationnow'
#compiles(durationnow, 'sqlite')
def sl_durationnow(element, compiler, **kw):
return compiler.process(
func.timestampadd('MINUTES', element.clauses, func.now())
)
#compiles(durationnow, 'postgresql')
def pg_durationnow(element, compiler, **kw):
return compiler.process(
func.now() + func.make_interval(0, 0, 0, 0, 0, element.clauses)
)
# Or alternatively...
# return "now() - make_interval(0, 0, 0, 0, 0, {})".format(compiler.process(element.clauses))
# which is more in-line with how the documentation uses 'compiles'
With something like that set up you should be able to turn your original query into a cross-dialect one that renders to SQL directly instead of doing the duration computation in Python:
current_appointment = Appointment.query.filter(
Appointment.datetime.between(
func.now(),
durationnow(Appointment.duration)
).limit(1).one_or_none()

Disclaimer 1: First of all, think if it is not "cheaper" to actually use postgresql instead of sqlite everywhere. I assume you have development/production differences, which you should avoid. Installation of postgresql on any modern OS is quite trivial.
Assuming above is not an option/desired, let's continue.
Disclaimer 2: The solution with the custom SQL construct (as per #Josh's answer) is really the only reasonable way to achieve this.
Unfortunately, the proposed solution does not actually work for sqlite, and could not be fixed with just few lines, hence a separate answer.
Solution:
Assuming you have the following model:
class Appointment(Base):
__tablename__ = 'appointment'
id = Column(Integer, primary_key=True)
name = Column(String(255))
datetime = Column(DateTime) # #note: should be better named `start_date`?
duration = Column(Integer)
sqlite is really tricky dealing with dates operations, especially adding/subtracting intervals from dates. Therefore, let's approach it somewhat differently and create custom functions to get an interval between two dates in minutes:
class diff_minutes(expression.FunctionElement):
type = Integer()
name = 'diff_minutes'
#compiles(diff_minutes, 'sqlite')
def sqlite_diff_minutes(element, compiler, **kw):
dt1, dt2 = list(element.clauses)
return compiler.process(
(func.strftime('%s', dt1) - func.strftime('%s', dt2)) / 60
)
#compiles(diff_minutes, 'postgresql')
def postgres_diff_minutes(element, compiler, **kw):
dt1, dt2 = list(element.clauses)
return compiler.process(func.extract('epoch', dt1 - dt2) / 60)
You can already implement your check using following query (i am not adding limit(1).one_or_none in my examples, which you can obviously do when you need it):
q = (
session
.query(Appointment)
.filter(Appointment.datetime <= func.now())
.filter(diff_minutes(func.now(), Appointment.datetime) <= Appointment.duration)
)
But now you are not limited by current time (func.now()), and you can check (and unit test) your data against any time:
# at_time = func.now()
at_time = datetime.datetime(2017, 11, 11, 17, 50, 0)
q = (
session
.query(Appointment)
.filter(Appointment.datetime <= at_time)
.filter(diff_minutes(at_time, Appointment.datetime) <= Appointment.duration)
)
Basically, problem is solved here, and the solution should work for both database engines you use.
BONUS:
You can hide the implementation of checking if the event is current using Hybrid Methods.
Lets add following to the Appointment class:
#hybrid_method
def is_current(self, at_time=None):
if at_time is None:
at_time = datetime.datetime.now()
return self.datetime <= at_time <= self.datetime + datetime.timedelta(minutes=self.duration)
#is_current.expression
def is_current(cls, at_time=None):
if at_time is None:
at_time = datetime.datetime.now()
stime = cls.datetime
diffm = diff_minutes(at_time, cls.datetime)
return and_(diffm >= 0, cls.duration >= diffm).label('is_current')
The first one allows you to run the check in memory (on python, not on SQL side):
print(my_appointment.is_current())
The second one allows you to construct query like below:
q = session.query(Appointment).filter(Appointment.is_current(at_time))
where if at_time it not specified, current time will be used. You can, of course then modify the query as you wish:
current_appointment = session.query(Appointment).filter(Appointment.is_current()).limit(1).one_or_none()

If I'm understanding the question correctly...
Something like this?
def check_for_current_appt(appt_epoch, appt_duration):
'''INPUT : appt_timestamp (int (epoch time)): start time for appointment
appt_duration (int): duration of appointment in seconds
OUTPUT : appt_underway (bool): True if appointment is currently underway'''
now = time.time()
appt_underway = 0 < (now - appt_epoch) < appt_duration
return appt_underway
I'll leave it to you to convert to epoch time and seconds for the duration

From what I understand of it, PostgreSQL uses unix timestamps while Sqlite uses iso-8601 timestamps stored as strings. So if you change the overall structure of your database to use the Sqlite format it should give you the functionality you want, you can convert datetime with the .isoformat() function. Unfortunately if you are not working with only test data you will have to iterate over all the rows to change them. Not sure if this is acceptable to you but is an easy way to do it.
Based on the datetime section of http://docs.sqlalchemy.org/en/latest/core/type_basics.html

How to trim spaces within timestamps using 'm/d/yy' format

I have a Python script that generates .csv files from other data sources.
Currently, an error happens when the user manually adds a space to a date by accident. Instead of inputting the date as "1/13/17", a space may be added at the front (" 1/13/17") so that there's a space in front of the month.
I've included the relevant part of my Python script below:
def processDateStamp(sourceStamp):
matchObj = re.match(r'^(\d+)/(\d+)/(\d+)\s', sourceStamp)
(month, day, year) = (matchObj.group(1), matchObj.group(2), matchObj.group(3))
return "%s/%s/%s" % (month, day, year)
How do I trim the space issue in front of month and possibly on other components of the date (the day and year) as well for the future?
Thanks in advance.

Since you're dealing with dates, it might be more appropriate to use datetime.strptime than regex here. There are two advantages of this approach:
It makes it slightly clearer to anyone reading that you're trying to parse dates.
Your code will be more prone to throw exceptions when trying to parse data that doesn't represent dates, or represent dates in an incorrect format - this is good because it helps you catch and address issues that might otherwise go unnoticed.
Here's the code:
from datetime import datetime
def processDateStamp(sourceStamp):
date = datetime.strptime(sourceStamp.replace(' ', ''), '%M/%d/%y')
return '{}/{}/{}'.format(date.month, date.day, date.year)
if __name__ == '__main__':
print(processDateStamp('1/13/17')) # 1/13/17
print(processDateStamp(' 1/13/17')) # 1/13/17
print(processDateStamp(' 1 /13 /17')) # 1/13/17

You also can use parser from python-dateutil library. The main benefit you will get - it can recognize the datetime format for you (sometimes it may be useful):
from dateutil import parser
from datetime import datetime
def processDateTimeStamp(sourceStamp):
dt = parser.parse(sourceStamp)
return dt.strftime("%m/%d/%y")
processDateTimeStamp(" 1 /13 / 17") # returns 01/13/17
processDateTimeStamp(" jan / 13 / 17")
processDateTimeStamp(" 1 - 13 - 17")
processDateTimeStamp(" 1 .13 .17")

Once again, a perfect opportunity to use split, strip, and join:
def remove_spaces(date_string):
date_list = date_string.split('/')
result = '/'.join(x.strip() for x in date_list)
return result
Examples
In [7]: remove_spaces('1/13/17')
Out[7]: '1/13/17'
In [8]: remove_spaces(' 1/13/17')
Out[8]: '1/13/17'
In [9]: remove_spaces(' 1/ 13/17')
Out[9]: '1/13/17'

Next upcoming date model object ignoring past dates

I'm trying to utilise latest() on a django model queryset to return the next upcoming date in a model.
I've tried a few different things, using __lte and __gte lookups on a filter and to no avail.
The filter option would work for me, if there was a way to effectively utilise a model method within an exclude() but without writing a custom manager that's not going to be an option.
There must be an easier way?
class RaidSession(models.Model):
scheduled = models.DateTimeField()
duration = models.DurationField()
def is_expired(self):
duration_to_date = self.scheduled + self.duration
return True if duration_to_date < timezone.now() else False

Since I'm a little old school, it usually helps me to think of such problems as an SQL query. In your case this would be
SELECT * FROM app_raidsession rs
WHERE rs.scheduled >= now()
ORDER BY rs.scheduled
LIMIT 1
This gives you the next scheduled raid.
In django ORM, you should be able to translate this more or less straightforward to:
from django.utils.timezone import now
# first() returns None if the result is empty
next_raid = models.RaidSession.objects \
.filter(scheduled__gte=now()) \
.order_by('scheduled') \
.first()
If the duration is relevant, you will need an F-expression:
from django.db.models import F
next_raid = models.RaidSession.objects \
.filter(scheduled__gte=now() - F('duration')) \
.order_by('scheduled') \
.first()

Django ORM calculate number of days between two date attributes

Scenario
I have a table student. it has following attributes
name,
age,
school_passout_date,
college_start_date
I need a report to know what is the avg number of days student get free between the passing the school and starting college.
Current approach
Currently i am irritating over the range of values finding days for each student and getting its avg.
Problem
That is highly inefficient when the record set gets bigger.
Question
Is there any ability in the Django ORM that gives me totals days between the two dates?
Possibility
I am looking for something like this.
Students.objects.filter(school_passed=True, started_college=True).annotate(total_days_between=Count('school_passout_date', 'college_start_date'), Avg_days=Avg('school_passout_date', 'college_start_date'))

You can do this like so:
Model.objects.annotate(age=Cast(ExtractDay(TruncDate(Now()) - TruncDate(F('created'))), IntegerField()))
This lets you work with the integer value, eg you could then do something like this:
from django.db.models import IntegerField, F
from django.db.models.functions import Cast, ExtractDay, TruncDate
qs = (
Model
.objects
.annotate(age=Cast(ExtractDay(TruncDate(Now()) - TruncDate(F('created'))), IntegerField()))
.annotate(age_bucket=Case(
When(age__lt=30, then=Value('new')),
When(age__lt=60, then=Value('current')),
default=Value('aged'),
output_field=CharField(),
))
)

This question is very old but Django ORM is much more advanced now.
It's possible to do this using F() functions.
from django.db.models import Avg, F
college_students = Students.objects.filter(school_passed=True, started_college=True)
duration = college_students.annotate(avg_no_of_days=Avg( F('college_start_date') - F('school_passout_date') )

Mathematically, according to the (expected) fact that the pass out date is allway later than the start date, you can just get an average off all your start date, and all your pass out date, and make the difference.
This gives you a solution like that one
from django.db.models import Avg
avg_start_date = Students.objects.filter(school_passed=True, started_college=True).aggregate(Avg('school_start_date'))
avg_passout_date = Students.objects.filter(school_passed=True, started_college=True).aggregate(Avg('school_passout_date'))
avg_time_at_college = avg_passout_date - avg_start_date

Django currently only accept aggregation for 4 function : Max, Min, Count, et Average, so this is a little tricky to do.
Then the solution is using the method extra . That way:
Students.objects.
extra(select={'difference': 'school_passout_date' - 'college_start_date'}).
filter('school_passed=True, started_college=True)
But then, you still have to do the average on the server side