Python: Deleting mySQL rows that are x many days old - python
Im using a raspberry pi with the rasbpian wheezy distribution running headless. I currently have a cronjob that runs a python script to put the current temperature and datetime in a mySQL database
(table: tempLog, attributes: datetime, temperature float(5,2)).
I want to delete rows that are say 5 days (num days is arbitrary) old and i'm having trouble accomplishing this in python. Here is the code, its not long.
import os
import time
import datetime
import glob
import MySQLdb
from time import strftime
from datetime import timedelta
from datetime import date
# Variables for MySQL
db = MySQLdb.connect(host="localhost", user="root",passwd="password", db="temp_database")
cur = db.cursor()
del_basedate = datetime.datetime.today() - timedelta(1)
# DATE_SUB(NOW() , INTERVAL 1 DAY)
try:
cur.execute("DELETE FROM tempLog WHERE datetime.date.day = del_basedate")
print "Delete successful"
except:
print "An error occured in: deleteRows.py"
finally:
cur.close()
db.close()
I had to do sever import from, because it kept throwing errors of objects not existing.
You need to learn about coding context:
del_basedate = datetime.datetime.today() - timedelta(1)
^^----python variable
cur.execute("DELETE FROM tempLog WHERE datetime.date.day = del_basedate")
text string with the letters "d", "e", "l", "_", etc... --- ^^^
Python isn't magical, and will NOT rummage around in a string to see if any of the text in that string LOOKS like a python variable. So you're effectively telling your DB to compare datetime.date.day against some unknown/undefined del_basedate field.
Try
cur.execute("DELETE FROM tempLog WHERE datetime.date.day = '" + del_basedate + "'")
^^^^^^^^^^^^^^^^^^
Note the extra quotes within the string. Without them, you'd be comparing against = 2016-02-18, which is a math operation, and is parsed/executed by the database as = 1996
Related
Python query to run sql query
I have a Python script i managed to piece together to query a database and provide back a csv file. Looking for help to set a beginning date (2022 Mar 15) and range (65 days forward) to iterate through. I need to run the query for each day and write the results separately to their own CSV file with the date in the file name. So, I believe I need to set the initial date and how many days to increment through and pass the variable to the SQL statement and to the CSV export. I've been playing around with it but not making progress.
import mysql.connector
import sys
import csv
mydb = mysql.connector.connect(
host="my ip",
user="db username",
password="db pwd"
)
mycursor = mydb.cursor()
from datetime import timedelta, datetime
date = datetime(2022,3,15)
for i in range(65):
date += timedelta(days=1)
print(date)
mycursor.execute ("SELECT * FROM myDB WHERE Division like 'ABC' AND ContactDate LIKE date")
rows = mycursor.fetchall()
headers = [col[0] for col in mycursor.description]
rows.insert(0, tuple(headers))
fp = open('Calls-date.csv', 'w+', newline="")
myFile = csv.writer(fp)
myFile.writerows(rows)
fp.close()
mydb.close()
sqlite3 function inside python program
I'm using a function inside a python program that doesn't work as expected.
I would like to call a sqlite3 function that give me the last record registrered every 2 seconds
It works fine until midnight, then it continues reading the valuew of the same day, it doesn't change when a new day arrives.
the function is(data is today, ora is actual hour):
import sqlite3
from sqlite3 import Error
import time
def leggi_tmp():
try:
time.sleep(2)
conn = sqlite3.connect('DB.db')
cursor = conn.cursor()
cursor.execute('''SELECT * FROM tmp_hr WHERE data = date('now') ORDER BY ora DESC LIMIT 1''')
#Fetching 1st row from the table
result = cursor.fetchone()
tmpe = result[0]
print(result)
#Closing the connection
conn.close()
except Error as e:
print(e)
return tmpe
when I do:
while data.tm_hour in fase_1 and func2_letture.leggi_tmp() <= temp_min_giorno :
func2_letture.leggi_tmp() only reads the day when it is called the first time(but works as expected during the day), it doesn't read the new date when new day arrives
I can't understand where my mistake is...
I suspect that this is a timezone problem.
Add the 'localtime' modifier to the function date():
WHERE data = date('now', 'localtime')
Postgresql and Python : Selecting data (datetime) from SQL
I'm making Car parking system, and I have some difficulities with SQL database.
I'm selecting data from SQL database, but I need to get the time correctly that I could use it for further calculations. So for example I need to get the time that was inserted to database as VARCHAR, maybe the bad thing is that I needed to use other method as TIME, but that's not the case. The thing I need is to use this line Started_Parking = row [3]. This should get the time from database and after that, I should be able to see the time difference from the start when car was registered and current time. By doing that I should be able to calculate the sum which the "User" should pay for parking.
So by short I just need to somehow get the time from database and use it for calculations. Here's my code, I also get errors when compiling :
Error while fetching data from PostgreSQL unsupported operand type(s)
for -: 'datetime.datetime' and 'str'
try:
connection = psycopg2.connect(user="postgres",
password="Dziugas420",
host="127.0.0.1",
port="5432",
database="postgres")
cursor = connection.cursor()
postgreSQL_select_Query = "select * from vartotojai WHERE carnum=('%s')" % car_numb
cursor.execute(postgreSQL_select_Query) # PALEIST KOMANDA
vartotoju_data = cursor.fetchall() # READ DATA
print(" CAR DETAILS: ")
for row in vartotoju_data:
print("Current ID: ", row[0])
print("Car Number: ", row[1])
print("Parked on: ", row[3], "\n")
Pay_Time = datetime.datetime.now()
Started_Parking = row [3]
Prastovetas_Laikas = Pay_Time - Started_Parking
print(Prastovetas_Laikas)
# NOW LET'S CHECK IF THE TIME DIFFERENCE IS WORKING, LET'S SEE THE DIFFERENCE AFTER 20SECS.
time.sleep(20)
Pay_Time2 = datetime.datetime.now()
Prastovetas_Laikas2 = Pay_Time2 - Started_Parking
print(Prastovetas_Laikas2)`
**EDIT
Here's the code I use to import this time into database:
Car_Reg_Time = datetime.datetime.now().strftime("%Y-%m-%d %H:%M")
postgres_insert_query = """ INSERT INTO vartotojai (CARNUM, TIME, LAIKAS) VALUES (%s,%s, %s)"""
record_to_insert = (car_numb, Reg_Tikslus_Laikas, Car_Reg_Time)
And here's the table of my database:
! laikas in database is when car was registered, the time in database is the time when the injection was made.
Prastovetas_Laikas = Pay_Time - Started_Parking will not work since Pay_Time is datetime.datetime and Started_Parking is str you need to try to use datetime.strptime() to convert Started_Parking to correct type and you want to store them as str in your DB using str(mydate)
CSV - MYSQL Using Python
After reading several inputs I still can't get this to work.
Most likely I'm doing it all wrong but I've tried several different approaches
What I'm trying to do is extract data from a CSV and add it into my newly created database/table
My csv input look like this
NodeName,NeId,Object,Time,Interval,Direction,NeAlias,NeType,Position,AVG,MAX,MIN,percent_0-5,percent_5-10,percent_10-15,percent_15-20,percent_20-25,percent_25-30,percent_30-35,percent_35-40,percent_40-45,percent_45-50,percent_50-55,percent_55-60,percent_60-65,percent_65-70,percent_70-75,percent_75-80,percent_80-85,percent_85-90,percent_90-95,percent_95-100,IdLogNum,FailureDescription
X13146PAZ,5002,1/11/100,2016-05-16 00:00:00,24,Near End,GE0097-TN01.1,AMM 20PB,-,69684,217287,772,10563,8055,10644,15147,16821,13610,7658,2943,784,152,20,3,0,0,0,0,0,0,0,0,0,-
...
X13146PAZ,5002,1/11/102,2016-05-16 00:00:00,24,Near End,GE0097-TN01.1,AMM 20PB,-,3056,28315,215,86310,90,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,-
...
X13146PAZ,5002,1/11/103,2016-05-16 00:00:00,24,Near End,GE0097-TN01.1,AMM 20PB,-,769,7195,11,86400,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,-
The mysql table is created but possibly that might be the issue as some ar varchar columns and some are integer columns
My server is a Ubuntu if that is of any use
My Code
# -*- coding: utf-8 -*-
#Imports
from datetime import date, timedelta
import sys
import MySQLdb as mdb
import csv
import os
#Vars
Yesterday = date.today() - timedelta(1)
#Opening document
RX_Document = open('./reports/X13146PAZ_TN_WAN_ETH_BAND_RX_' + Yesterday.strftime("%Y%m%d") + "_231500.csv" , 'r')
RX_Document_Str = './reports/X13146PAZ_TN_WAN_ETH_BAND_RX_' + Yesterday.strftime("%Y%m%d") + "_231500.csv"
csv_data = csv.reader(file(RX_Document_Str))
con = mdb.connect('localhost', 'username', 'password','tn_rx_utilization');
counter = 0
for row in csv_data:
if counter == 0:
print row
continue
counter = 1
if counter == 1:
cur = con.cursor()
cur.execute('INSERT INTO RX_UTIL(NodeName, NeId, Object, Time, Interval1,Direction,NeAlias,NeType,Position,AVG,MAX,MIN,percent_5-10,percent_10-15,percent_15-20,percent_20-25,percent_25-30,percent_30-35,percent_35-40,percent_40-45,percent_45-50,percent_50-55,percent_55-60,percent_60-65,percent_65-70,percent_70-75,percent_75-80,percent_80-85,percent_85-90,percent_90-95,percent_95-100,IdLogNum,FailureDescription)' 'VALUES("%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s","%s")',tuple(row[:34]))
con.commit()
#cur.execute("SELECT VERSION()")
#ver = cur.fetchone()
con.commit()
con.close()
You should not put the placeholder %s in quotes ":
cur.execute('''INSERT INTO RX_UTIL(NodeName, NeId, Object, Time, Interval1,Direction,
NeAlias,NeType,Position,AVG,MAX,MIN,"percent_5-10","percent_10-15",
"percent_15-20","percent_20-25","percent_25-30","percent_30-35",
"percent_35-40","percent_40-45","percent_45-50","percent_50-55",
"percent_55-60","percent_60-65","percent_65-70","percent_70-75",
"percent_75-80","percent_80-85","percent_85-90","percent_90-95",
"percent_95-100",IdLogNum,FailureDescription)
VALUES(%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,
%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)''', tuple(row[:33]))
You are missing Percent_0-5 from your Insert
Remove the quotes from the %s references, this needs to be in String format, but the underlying data type will be passed.
There may be issues with datatype resulting from the csv reader. Have Python eval() the csv data to alter type as an INT. Here is some more information from another post:
Read data from csv-file and transform to correct data-type
cur.execute('INSERT INTO RX_UTIL(NodeName, NeId, Object, Time, Interval1,Direction,NeAlias,NeType,Position,AVG,MAX,MIN,percent_0-5,percent_5-10,percent_10-15,percent_15-20,percent_20-25,percent_25-30,percent_30-35,percent_35-40,percent_40-45,percent_45-50,percent_50-55,percent_55-60,percent_60-65,percent_65-70,percent_70-75,percent_75-80,percent_80-85,percent_85-90,percent_90-95,percent_95-100,IdLogNum,FailureDescription)' 'VALUES(%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)',tuple(row[:34]))
SQLite received a naive datetime while time zone support is active
I'm getting the following error while running my sql in python /usr/lib/python2.6/site-packages/django/db/backends/mysql/base.py:64: RuntimeWarning: SQLite received a naive datetime (2012-06-22 15:53:43) while time zone support is active. my query also returns the wrong data. if i change the time up 2 hours (17:53:43 instead of 15:53:43), my timezone atm is gmt +2 so i think the problem is in the time zone. how do i change my query to make the sql execute in the way i intend it to? sql: sqlQuery = """SELECT w.id, w.serial, w.finishdate, w.weighingtype_id, w.netto, w.bruto, w.deleted FROM weighing w LEFT JOIN weighing w1 ON w1.id = w.parent_id WHERE w.user_id = %(userid)s""" if date: sqlQuery = sqlQuery + " AND (w.created = %(date)s OR w.modified > %(date)s)" edit: added my code for transforming the datetime data = request.GET.copy() if 'date' in data: try: data['date'] = datetime.datetime.strptime(data['date'], "%Y-%m-%dT%H:%M:%S") except: raise error(311)