I am running into an issue building a random project for myself. I am trying to record entries into a mysql database that the user types in. I am storing them in a dictionary. The error message is
while self.nextset():
AttributeError: 'str' object has no attribute 'nextset'
I have googled and searched for this issue, but I only find issues kind of like it but not the same error.
My table headers in mysql db match the dictionary keys. I do realize I have issues with selection (2) but my error and what I am troubleshooting now is just when I select option (1).
import mysql
import pymysql
from datetime import date, datetime, timedelta
cursor = pymysql.cursors.Cursor
# Function for adding a new entry
def new_entry(name, date, task, time, notes):
# Build dictionary with new entry information
myDict = {
'Employee': name, # Name of employee
'Date': date, # Date of worked task
'Task': task, # Title of Task
'Time': time, # Time spent on task
'Notes': notes # Notes on the task
}
table = ('timesheet')
placeholders = ', '.join(['%s'] * len(myDict))
columns = ', '.join(myDict.keys())
sql = "INSERT INTO %s ( %s ) VALUES ( %s )" % (table, columns,
placeholders)
pymysql.cursors.Cursor.execute(sql, myDict)
#list all entries for a particular employee
def previous_entries(emp_name):
pymysql.cursors.Cursor.execute(
"SELECT * from user_data WHERE Name = %s", (emp_name,))
#list all entries that match a date or search term
#def search_entries():
# return null
#Print a report of this information to the screen, including the date, title
#of task, time spent, employee, and general notes.
if __name__ == '__main__':
#cnx = mysql.connect(user='root', database='me10_mig')
cnx = pymysql.connect(user='root', password='password',
database='me10_mig')
print("Please enter (1), (2), or (3)")
begin = input("Would you like to (1) enter a new entry or (2) display
all
previous entries or (3) display entries that match a date or search
term? ")
if begin == '1':
name = input("Your Name: ")
date = input("Date of Time Worked: ")
task = input("Title of Task: ")
time = input("Time Spent on Task: ")
notes = input("Notes on Time Worked: ")
new_entry(name, date, task, time, notes)
if begin == '2':
name = input("What is the employee name: ")
previous_entries(name)
#if begin == '3':
The error I get says:
Traceback (most recent call last):
File "C:/Users/a089673/Desktop/Python/TeamTreeHouse/Part 4/timesheet.py", line 61, in <module>
new_entry(name, date, task, time, notes)
File "C:/Users/a089673/Desktop/Python/TeamTreeHouse/Part 4/timesheet.py", line 27, in new_entry
pymysql.cursors.Cursor.execute(sql, myDict)
File "C:\Users\a089673\AppData\Roaming\Python\Python36\site packages\pymysql\cursors.py", line 165, in execute
while self.nextset():
AttributeError: 'str' object has no attribute 'nextset'
Process finished with exit code 1
Any suggestions?
I suspect your might stem from using a dict to hold the arguments to .execute(), but not using named string patterns in the SQL statement.
The docs suggest using %s when passing a list or tuple, but rather use %(name)s when passing a dict.
I suggest you try this code:
def new_entry(name, date, task, time, notes):
# Build dictionary with new entry information
myDict = {
'Employee': name, # Name of employee
'Date': date, # Date of worked task
'Task': task, # Title of Task
'Time': time, # Time spent on task
'Notes': notes # Notes on the task
}
table = ('timesheet')
column_list = []
placeholder_list = []
for k in myDict:
column_list.append(k)
placeholder_list.append('%(' + k + ')s')
sql = "INSERT INTO %s ( %s ) VALUES ( %s )" % (
table,
', '.join(column_list),
', '.join(placeholder_list))
pymysql.cursors.Cursor.execute(sql, myDict)
This will also ensure that the column names and the placeholders are in the same order. Your original code did not consider that (remember, iterating over dicts multimple time is not garanteed to give the same order each time).
I totally overlooked the part where you establish a connection to the database. You need to pass that connection as a parameter to new_entry() and use it.
Try this:
def new_entry(cnx, name, date, task, time, notes):
sql = "INSERT INTO timesheet (Employee, Date, Task, Time, Notes) VALUES (%s, %s, %s, %s, %s)"
values = (name, date, task, time, notes)
with cnx.cursor() as cursor:
cursor.execute(sql, values)
cnx = pymysql.connect(user='root', password='password', database='me10_mig')
new_entry(cnx, ...)
Related
Been playing with this for 14 hours (I am a beginner)
Data is pulled from one database table to search on yahoo for all the data on that ticker and then its "meant" to upload it.
I orginally had it as panda df but got "ambiguous error" so I have now put it as [] again. New error. I rack my brains :( However, it does work if I leave it blank.
from __future__ import print_function
import yfinance as yf
import pandas as pd
import datetime
import warnings
import MySQLdb as mdb
import requests
import numpy as np
import MySQLdb as mdb
import requests
# Obtain a database connection to the MySQL instance
con = mdb.connect("localhost","sec_user","","securities_master")
def obtain_list_of_db_tickers():
"""
Obtains a list of the ticker symbols in the database.
"""
with con:
cur = con.cursor()
cur.execute("SELECT id, ticker FROM symbol")
data = cur.fetchall()
print(data)
return [(d[0], d[1]) for d in data]
def get_daily_historic_data_yahoo(ticker):
blow = yf.download(ticker)
data = []
data.append(yf.download(ticker).reset_index())
return data
def insert_daily_data_into_db(data_vendor_id, symbol_id, daily_data):
'''
Takes a list of tuples of daily data and adds it to the MySQL database.
Appends the vendor ID and symbol ID to the data.
daily_data: List of tuples of the OHLC data (with adj_close and volume)
'''
# Create the time now
now = datetime.datetime.utcnow()
df = pd.DataFrame(data=daily_data[0])
df.insert(0, 'data_vendor_id', data_vendor_id)
df.insert(1, 'symbol_id', symbol_id)
df.insert(3, 'created_date', now)
df.insert(4, 'last_updated_date', now)
daily_data = []
daily_data.append(df)
#df = daily_data
# Amend the data to include the vendor ID and symbol ID
# Connect to the MySQL instance
db_host = 'localhost'
db_user = ''
db_pass = ''
db_name = 'securities_master'
con = mdb.connect("localhost", "sec_user", "", "securities_master"
# host=db_host, user=db_user, passwd=db_pass, db=db_name
)
try:
mdb.connect
# If connection is not successful
except:
print("Can't connect to database")
return 0
# If Connection Is Successful
print("Connected")
final_str = """INSERT INTO daily_price (data_vendor_id, symbol_id, price_date, created_date,
last_updated_date, open_price, high_price, low_price, close_price, volume, adj_close_price) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)"""
with con:
cur = con.cursor()
cur.executemany(final_str, daily_data)
con.commit()
if __name__ == "__main__":
# This ignores the warnings regarding Data Truncation
# from the Yahoo precision to Decimal(19,4) datatypes
warnings.filterwarnings('ignore')
# Loop over the tickers and insert the daily historical
# data into the database
tickers = obtain_list_of_db_tickers()
lentickers = len(tickers)
for i, t in enumerate(tickers):
print(
"Adding data for %s: %s out of %s" %
(t[1], i+1, lentickers)
)
yf_data = get_daily_historic_data_yahoo(t[1])
insert_daily_data_into_db('1', t[0], yf_data)
print("Successfully added Yahoo Finance pricing data to DB.")
Errors
Traceback (most recent call last):
File "/home/quant/price_retrieval.py", line 106, in <module>
insert_daily_data_into_db('1', t[0], yf_data)
File "/home/quant/price_retrieval.py", line 88, in insert_daily_data_into_db
cur.executemany(final_str, daily_data)
File "/home/quant/.local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 230, in executemany
return self._do_execute_many(
File "/home/quant/.local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 255, in _do_execute_many
v = values % escape(next(args), conn)
TypeError: not enough arguments for format string
I'm no data scientist so there's probably a more elegant way to fix it directly with pandas. But the way I usually work with MySQL (and really any SQL drivers) is to give it lists of python tuples.
If you parse each row of the pandas data frame with for row in df.itertuples(): and craft each tuple carefully - making sure the types match the SQL table, all should work ;)
Example:
def insert_daily_data_into_db(data_vendor_id, symbol_id, daily_data):
'''
Takes a list of tuples of daily data and adds it to the MySQL database.
Appends the vendor ID and symbol ID to the data.
daily_data: List of tuples of the OHLC data (with adj_close and volume)
'''
# Create the time now
now = datetime.datetime.utcnow()
df = pd.DataFrame(data=daily_data[0])
daily_data = []
created_date = now
last_updated_date = now
for row in df.itertuples():
_index = row[0] # discard
date = row[1]
open = row[2]
high = row[3]
low = row[4]
close = row[5]
adj_close_price = row[6]
volume = row[7]
daily_data.append((int(data_vendor_id), symbol_id, date, created_date, last_updated_date, open, high, low, close, volume, adj_close_price))
# Connect to the MySQL instance
con = mdb.connect(host="localhost", user="user", password="yourpassword",
db="yourdbname", port=3306)
final_str = """
INSERT INTO daily_price (data_vendor_id, symbol_id, price_date, created_date,
last_updated_date, open_price, high_price, low_price, close_price, volume, adj_close_price)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
"""
with con:
cur = con.cursor()
cur.executemany(final_str, daily_data)
con.commit()
I've tried not to tamper with your existing code too much. Just enough to make it work.
I think what was happening there for you was that you're technically passing it a list of pandas dataframes with only a single pandas dataframe in the list. Instead what you want is a list of tuples with 11 fields to unpack per tuple.
Maybe you mean to pass the dataframe directly i.e. not contained inside of a list but I still don't think that would be right because 1) there's an "Index" column in the dataframe which would give erroneous results 2) you'd need to call some methods on the dataframe to retrieve only the values (not the headers to the columns) and transform it to the correct list of tuples. It's probably very doable but I will leave that to you to find out.
I am also assuming your table schema is something like this:
CREATE TABLE IF NOT EXISTS daily_price (
data_vendor_id INT,
symbol_id INT,
price_date DATETIME,
created_date DATETIME,
last_updated_date TIMESTAMP,
open_price VARCHAR(256),
high_price VARCHAR(256),
low_price VARCHAR(256),
close_price VARCHAR(256),
volume INT,
adj_close_price VARCHAR(256)
);
I have tried doing some sql connectivity and faced a problem recently, i could'nt take isert dates into a table. I have given the code below. what should i add in this?
import mysql.connector as con
db = con.connect(host = "localhost", user = "root", password = "root", database = "vish")
if db.is_connected():
print("success")
a = int(input("roll no. : "))
b = input("name : ")
c = int(input("phone : "))
cursor = db.cursor()
q = ("insert into student values({}, '{}', {})".format(a,b,c))
cursor.execute(q)
db.commit()
how do i take date and insert it into a table
If you have a table t and a column c and you wish to insert a date literal, the format for that literal is 'YYYY-MM-DD', so for example:
INSERT INTO t(c) VALUES('2020-01-11')
But whenever you are getting input from the "outside", you are leaving yourself open to SQL Injection attacks and should be using prepared statements. If you have parsed the input date and built a string literal such as '2020-01-11' from the input, then you are safe. But if you have isolated the year, month and day and you are using a prepared statement anyway because there are other values you have not so rigorously validated, you can also use a '?' parameter for the date thus:
from datetime import date
today = date(2020, 1, 11)
cursor = db.cursor()
cursor.execute("INSERT INTO t(c) VALUES(?)", (today,))
cursor.commit()
I have the following code. When you understand the code, you can look at the two comments with
the capital letters. I could test if there are new items in the channel with insert or ignore but
I'm trying the better mechanism with utilization feed.updated_parsed attribute. Why doesn't it work
as expected?
from __future__ import unicode_literals
import feedparser
from sqlite3 import dbapi2 as sqlite
import sys, os
from datetime import datetime
from time import mktime
from daeutils import *
import re
import random
import optparse
import curses
import socket
def getActiveChannels():
"""Returns a list of active RSS channels"""
con = sqlite.connect(connectionString)
cur = con.cursor()
cur.execute("select id, title, xmlurl, updated from channels")
channels = cur.fetchall()
cur.close()
con.close()
return channels
def getItemsForChannel(xmlUrl, lastUpdate):
socket.setdefaulttimeout(60)
feedparserDictionary = feedparser.parse(xmlUrl)
updatedTime = datetime.fromtimestamp(mktime(feedparserDictionary.feed.updated_parsed))
lst = datetime.strptime(lastUpdate, "%Y-%m-%dT%H:%M:%S.%f")
if updatedTime < lst:
return [] # HERE NOT BEHAVING CORRECTLY, WHEN I COMMENT THIS LINE, THERE MAY BE A FEW ITEMS
items = feedparserDictionary.entries
print "There are new %d items" % len(items)
return items
def setChannelUpdateTime(xmlUrl, tm):
con = sqlite.connect(connectionString)
cur = con.cursor()
cur.execute("update channels set updated = :tm where xmlurl = :xmlUrl", locals())
con.commit()
print "updated successfully"
cur.close()
con.close()
if __name__ == "_main__":
con = sqlite.connect(connectionString)
for channel in getActiveChannels():
channelId, channelTitle, channelXmlUrl, lastChannelUpdate = channel
countOfNewItems = 0
items = getItemsForChannel(channelXmlUrl, lastChannelUpdate)
for item in items:
title, link, description, priority, updated = item
cur = con.cursor()
cur.execute("insert or ignore into feeds \
(title, link, description, read, updated, channelid) \
values (?, ?, ?, ?, ?, ?)", \
(title, link, description, 0, updated, channelId))
countOfNewItems += cur.rowcount # WHICH ARE INSERTED HERE
con.commit()
cur.close()
if countOfNewItems:
print "Found new items"
now = datetime.now().isoformat()
if "." not in now:
now = now + ".000000"
setChannelUpdateTime(channelXmlUrl, now)
Here are the two tables in sqlite:
CREATE TABLE channels (id integer primary key, title string, text string, description string, type string, xmlurl string unique, htmlurl string, priority integer, active integer, deactivated integer, updated text);
CREATE TABLE feeds (id integer primary key, title string, link string unique, description string, read integer, priority integer, updated string, channelid integer, foreign key (channelid) references channels(id));
I think the possible error is that you are trying to compare updated field on the feed, the feeds could be not well supported by the feed creator. Or timezone formatting because of using isoformat or etc.
Anyway, I believe that it is much better to compare PER ENTRY updated properties rather than comparing the feed property which is mostly used for invalidating feed cache.
Here is a working example, where I return only new entries from the function.
import socket
from datetime import datetime, timedelta
from time import mktime
import feedparser
from pprint import pprint
def getItemsForChannel(xmlUrl, lastUpdate):
lst = datetime.fromisoformat(lastUpdate)
socket.setdefaulttimeout(60)
parsed = feedparser.parse(xmlUrl)
items = [entry for entry in parsed.entries if
datetime.fromtimestamp(mktime(entry.updated_parsed)) > lst]
print("There are new {} items".format(len(items)))
return items
pprint(getItemsForChannel(
'http://serverfault.com/feeds/tag/+or+linux+or+ubuntu+or+vim+or+rsync+or+gnome',
(datetime.now() - timedelta(hours=3)).isoformat()
))
It uses from/to iso formatting for the last parsed date in your database value and compares entries per entry instead of global comparison based on the feed updated property.
I'm making Car parking system, and I have some difficulities with SQL database.
I'm selecting data from SQL database, but I need to get the time correctly that I could use it for further calculations. So for example I need to get the time that was inserted to database as VARCHAR, maybe the bad thing is that I needed to use other method as TIME, but that's not the case. The thing I need is to use this line Started_Parking = row [3]. This should get the time from database and after that, I should be able to see the time difference from the start when car was registered and current time. By doing that I should be able to calculate the sum which the "User" should pay for parking.
So by short I just need to somehow get the time from database and use it for calculations. Here's my code, I also get errors when compiling :
Error while fetching data from PostgreSQL unsupported operand type(s)
for -: 'datetime.datetime' and 'str'
try:
connection = psycopg2.connect(user="postgres",
password="Dziugas420",
host="127.0.0.1",
port="5432",
database="postgres")
cursor = connection.cursor()
postgreSQL_select_Query = "select * from vartotojai WHERE carnum=('%s')" % car_numb
cursor.execute(postgreSQL_select_Query) # PALEIST KOMANDA
vartotoju_data = cursor.fetchall() # READ DATA
print(" CAR DETAILS: ")
for row in vartotoju_data:
print("Current ID: ", row[0])
print("Car Number: ", row[1])
print("Parked on: ", row[3], "\n")
Pay_Time = datetime.datetime.now()
Started_Parking = row [3]
Prastovetas_Laikas = Pay_Time - Started_Parking
print(Prastovetas_Laikas)
# NOW LET'S CHECK IF THE TIME DIFFERENCE IS WORKING, LET'S SEE THE DIFFERENCE AFTER 20SECS.
time.sleep(20)
Pay_Time2 = datetime.datetime.now()
Prastovetas_Laikas2 = Pay_Time2 - Started_Parking
print(Prastovetas_Laikas2)`
**EDIT
Here's the code I use to import this time into database:
Car_Reg_Time = datetime.datetime.now().strftime("%Y-%m-%d %H:%M")
postgres_insert_query = """ INSERT INTO vartotojai (CARNUM, TIME, LAIKAS) VALUES (%s,%s, %s)"""
record_to_insert = (car_numb, Reg_Tikslus_Laikas, Car_Reg_Time)
And here's the table of my database:
! laikas in database is when car was registered, the time in database is the time when the injection was made.
Prastovetas_Laikas = Pay_Time - Started_Parking
will not work
since Pay_Time is datetime.datetime and Started_Parking is str
you need to try to use datetime.strptime() to convert Started_Parking to correct type
and you want to store them as str in your DB using str(mydate)
I want a structure like
var={userid:{bookid:rating,book2:rating,.....}}
I am not getting how to query it.
I am getting a structure like this:-
example:-
var={1:{101:2}}{1:{102:1}}
but I want
var={1:{101:2,102:1}}
I am fetching all the values of userid,bookid,ratings from database.
And also I would like to know how to access those records
Actually I am creating a recommender engine so I need to compare each user with every other user.Please if anyone can help me in this
Thankyou
To answer your first question on how to create such a structure. You can create a dictionary like this one by following lines:
a = {}
a[1] = {}
a[1][101] = 2
a[1][102] = 1
print a # output: {1: {101: 2, 102: 1}}
(It's actually a dictionary in a dictionary)
To answer your question how to access the results:
Here is a code example for accessing the results of your query.
import datetime
import mysql.connector
cnx = mysql.connector.connect(user='scott', database='employees')
cursor = cnx.cursor()
query = ("SELECT first_name, last_name, hire_date FROM employees "
"WHERE hire_date BETWEEN %s AND %s")
hire_start = datetime.date(1999, 1, 1)
hire_end = datetime.date(1999, 12, 31)
cursor.execute(query, (hire_start, hire_end))
for (first_name, last_name, hire_date) in cursor:
print("{}, {} was hired on {:%d %b %Y}".format(
last_name, first_name, hire_date))
cursor.close()
cnx.close()
(This example is taken from this page: https://dev.mysql.com/doc/connector-python/en/connector-python-example-cursor-select.html)
This could be helpful as well: https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html