I have a SQL Server table that contains some datetime data. I'm using a Python script with pymssql to connect to the database and execute queries.
The problem I'm having is that when my queries return datetime data, the dates are shifted and are no longer accurate. For example, one data entry in my table is for the date '2012-03-27', but when I retrieve it and store it in my script as a datetime object in Python, the date is stored as '2012-01-03'.
Any ideas on how to fix this? Should I be using a library like pytz?
Thanks!
I am having the same issue with pymssql version 1.02. Although its not the cleanest aproach, I am using the workaround suggested by the OP and converting the string value to a python datetime object with the correct values using the dateutil.parser module. Here is the code I am using, provided that iso_datetime_string contains the string with the datetime in ISO format, as returned by the workaround:
import dateutil.parser as ps
correct_datetime_obj = ps.parse(iso_datetime_string)
I'm still not sure what is causing this problem, but I found a solution: convert the datetime data to type varchar when the query is performed:
select convert(varchar(20),event_time,120) from event_detail;
Previously I was performing this query:
select event_time from event_detail;
where event_time is of type datetime
Related
I am using python 3.5
I have query like 31-Jul-03.
Now I would like to change this to 2003-07-31 or something available for mysql Date object.
Of course I can parse this query step by step, but, it is a bit trouble some and Some have the same problem as well.
Is there any good library or idea to handle this Date object??
I think the datetime module is what you're looking for. It can be used as such to solve your problem:
>>> import datetime
>>> datetime.datetime.strptime('31-Jul-03', '%d-%b-%y').strftime('%Y-%m-%d')
'2003-07-31'
Source: http://www.tutorialspoint.com/python/time_strptime.htm
In MySQL you could use to convert string to DATE type:
SELECT STR_TO_DATE('31-Jul-03','%d-%b-%y');
Rextester Demo
I am having some issues with finding documents with a today's date function through python.
I use the following function:
datetime.datetime.now().date().strftime('%Y-%m-%d')
which gives the following value: 2017-05-21
However, the values in my documents (mongo) have double strings attached around it like the following: "2017-05-21"
So literally filling in the above string ("2017-05-21") exactly in my pymongo query works like a charm. However, I need the dynamicness of the datetime function, but unfortunately, that doesn't match with the double quoted date strings which I need for the query.
Does anyone know any workaround? I have already tried a replace function etc. It either creates double quotes within the single quotes or doesn't do anything.
It sounds like your MongoDB documents were inserted incorrectly, using text for date fields instead of BSON datetimes.
PyMongo automatically converts between Python datetimes and BSON datetimes, so you can insert a document that contains a BSON datetime like this:
dt = datetime.datetime.utcnow()
collection.insert_one({'myDate': dt})
Prove that date comparisons work now like so:
# There is a document with myDate in the past, now.
print(collection.find_one({'myDate': {'$lt': datetime.datetime.utcnow()}}))
# No document with myDate in the future.
print(collection.find_one({'myDate': {'$gt': datetime.datetime.utcnow()}}))
I am moving some of my code onto sqlalchemy from using raw MySQL queries.
The current issue I am having is that the datetime was saved in a string format by a C# tool. Unfortunately, the representation does not match up with Python's (as well as that it has an extra set of single quotes), thus making filtering somewhat cumbersome.
Here is an example of the format that the date was saved in:
'2016-07-01T17:27:01'
Which I was able to convert to a usable datetime using the following MySQL command:
STR_TO_DATE(T.PredicationGeneratedTime, \"'%%Y-%%m-%%dT%%H:%%i:%%s'\")
However, I cannot find any documentation that describes how to invoke built-in functions such as STR_TO_DATE when filtering with sqlalchemy
The following Python code:
session.query(Train.Model).filter(cast(Train.Model.PredicationGeneratedTime, date) < start)
is giving me:
TypeError: Required argument 'year' (pos 1) not found
There does not seem to be a way to specify the format for the conversion.
Note: I realize the solution is to fix the way the datetime is stored, but in the mean time I'd like to run queries against the existing data.
You can try to use func.str_to_date(COLUMN, FORMAT_STRING) instead of cast
In the cast() you should be using sqlalchemy.DateTime, not (what I assume is) a datetime.date - that is the cause of the exception.
However, fixing that will not really help because of the embedded single quotes.
You are fortunate that the dates stored in your table are in ISO format. That means that lexicographic comparisons will work on the date strings themselves, without casting. As long as you use a string for start with the surrounding single quotes, it will work.
from datetime import datetime
start = "'{}'".format(datetime.now().isoformat())
session.query(Train.Model).filter(Train.Model.PredicationGeneratedTime < start)
Code is pretty straight forward:
import Quandl
import sqlite3
myData = Quandl.get("DMDRN/AAPL_ALLFINANCIALRATIOS")
cnx = sqlite3.connect("APPL.db")
myData.to_sql('AAPL', cnx)
I make a call to Quandl API. It gives me a pandas dataframe.
When I try to commit the data to a SQL table I get this error
sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type.
The index is a Timestamp.
I have tried this
1- How to write Pandas dataframe to sqlite with Index
2- Set the index to an other value + Convert numpy.datetime64 to string object in python
For the first one I still get an error binding parameter 1 and 2 isn't working.
What I should do (or what's the best way) if I want to commit the dataframe to a sqlite table and keep the date as the index.
See pandas to_sql method gives error with date column
The problem is that writing datetime64 values is not yet supported with a sqlite connection. With upcoming pandas 0.15, this bug will be fixed.
Writing the index is already supported (from pandas 0.14), and controllable with the index keyword (default of True).
You some options to solve this:
use sqlalchemy to make the connection (you need at least 0.14 for this), as this already supports writing datetime values:
import sqlalchemy
engine = sqlalchemy.create_engine('sqlite:///APPL.db')
myData.to_sql('AAPL', engine, index=True)
convert the datetime index to strings (and then you can keep using sqlite connection directly). You can do this with:
myData.index = myData.index.map(lambda x: x.strftime('%Y-%m-%d %H:%M:%S'))
use pandas development version (https://github.com/pydata/pandas)
I am trying to store a Python datetime object in an ORACLE column of type date.
so far, I have used,
rpt_time = time.strftime('%Y-%m-%d %H:%M:%S') or
rpt_time = str(datetime.datetime.now())
but all are giving ORA-01843: not a valid month
I am really confused how to insert ORACLE date type python datetime object
cx_Oracle supports passing objects of class datetime.datetime. Because of this when you already have object of this class (for example when calling datetime.datetime.now()) you should not try to format it and pass as a string but instead pass it directly. This way you prevent all errors caused by wrong format of date and/or time.
Example:
cursor.execute("INSERT INTO SomeTable VALUES(:now)", {'now': datetime.datetime.now()})
Be aware that you have to take additional steps if you want to prevent truncation of fractional part of seconds. For details please read Mastering Oracle+Python, Part 2: Working with Times and Dates article by Przemysław Piotrowski.
As far as my search shows, ORACLE can be picky on dates so this might be what you need to do.
Firstly, check the format of date you have. For example, if you have something like, 2010/01/26:11:00:00AM, then you might want to do following on your cursor execute:
insert into x
values(99, to_date('2010/01/26:11:00:00AM', 'yyyy/mm/dd:hh:mi:ssam'));
You have to convert date from python to oracle by setting nls_date_format in you session
>>> rpt_time = time.strftime('%Y-%m-%d %H:%M:%S')
>>> rpt_time
'2014-05-12 21:06:40'
Then before inserting into oracle, do the following
cursor.execute("ALTER SESSION SET NLS_DATE_FORMAT = 'YYYY-MM-DD HH24:MI:SS'")
datetime.now() in python gives you milliseconds and have to get rid of it before sucessfully writing to Oracle.
from datetime import datetime
....
cursor.execute("INSERT INTO myTable VALUES(to_date('" + str(datetime.now().replace(microsecond=0)) + "','yyyy-mm-dd hh24:mi:ss'))")
....