conn = pymysql.connect(host='localhost', port=3306, user='root', passwd='123456', db='jd', charset='utf8')
cur = conn.cursor()
sql = "select * from user where username = 'XXX'"
cur.execute(sql)
After this piece of code has been executed,the cursor should pointing to the record I've selected.
What I want to do is get one record,handle something with the data in the record,and then detele the record.
I know there's cur.fetchone(),cur.fetchall() methods in Python.I my opnion,since the cursor can fetchone record,why there is no cur.deteleOne() or cur.delete_previousOne() function?
A cursor points at a row of data returned by a SELECT statement, which is not the same thing as an actual row in a table.
A SELECT statement can perform many manipulations on its results, such as joining two or more tables together, ordering the results by a specific field, gathering only distinct values for a specific field, etc.
In the course of performing these manipulations, it is not always possible to maintain a link back to specific table rows. You can even use the SELECT AS statement to fetch data that isn't part of any table; deleting such a "row" would clearly be impossible.
Is there a reason you don't want to use sql to delete the rows?
As mentioned above I need to query several databases with all the same schema using pandas.read_sql_query. I've tried to create loop over all the dbs and run on the fly the sql statement that would result in something like this:
USE db_test_1; SELECT * from test
That's what I've done so far:
cursor = conn.cursor()
cursor.execute("SHOW DATABASES LIKE '%test_%'")
cursor.close()
dbs = [v.replace("'", "") for (v, ) in cursor]
for db in dbs[:100]:
temp = "USE " + db + ";"
fd = open('my_query.sql')
query = fd.read()
fd.close
sql = temp + query
data = pd.read_sql_query(sql, conn)
print(data)
Gives an error saying that the mysql syntax is wrong. Do you have any idea how to handle it or point me to the error?
Many thanks
Your problem lies with your my_query.sql file.
SELECT (SELECT * from tab1), (SELECT * from tab2)
The above is not valid SQL; a subselect can only return a single column. To do this, you would need to join the two subselects in the FROM clause. Which columns you do this on will be entirely dependent on your schema and the needed relation.
Update:
Okay, so the problem here seems to be more about how you're dealing with the query. Your cursor object is connected to a single database. Not the entire database server.
That means that your cursor object cannot use the use keyword here. You need to create a new connection and cursor object for each database you want to connect to.
Question: Is it possible to use a variable as your table name without having to use string constructors to do so?
Info:
I'm working on a project right now that catalogs data from a star simulation of mine. To do so I'm loading all the data into a sqlite database. It's working pretty well, but I've decided to add a lot more flexibility, efficiency, and usability to my db. I plan on later adding planetoids to the simulation, and wanted to have a table for each star. This way I wouldn't have to query a table of 20m some planetoids for the 1-4k in each solar system.
I've been told using string constructors is bad because it leaves me vulnerable to a SQL injection attack. While that isn't a big deal here as I'm the only person with access to these dbs, I would like to follow best practices. And also this way if I do a project with a similar situation where it is open to the public, I know what to do.
Currently I'm doing this:
cursor.execute("CREATE TABLE StarFrame"+self.name+" (etc etc)")
This works, but I would like to do something more like:
cursor.execute("CREATE TABLE StarFrame(?) (etc etc)",self.name)
though I understand that this would probably be impossible. though I would settle for something like
cursor.execute("CREATE TABLE (?) (etc etc)",self.name)
If this is not at all possible, I'll accept that answer, but if anyone knows a way to do this, do tell. :)
I'm coding in python.
Unfortunately, tables can't be the target of parameter substitution (I didn't find any definitive source, but I have seen it on a few web forums).
If you are worried about injection (you probably should be), you can write a function that cleans the string before passing it. Since you are looking for just a table name, you should be safe just accepting alphanumerics, stripping out all punctuation, such as )(][;, and whitespace. Basically, just keep A-Z a-z 0-9.
def scrub(table_name):
return ''.join( chr for chr in table_name if chr.isalnum() )
scrub('); drop tables --') # returns 'droptables'
For people searching for a way to make the table as a variable, I got this from another reply to same question here:
It said the following and it works. It's all quoted from mhawke:
You can't use parameter substitution for the table name. You need to add the table name to the query string yourself. Something like this:
query = 'SELECT * FROM {}'.format(table)
c.execute(query)
One thing to be mindful of is the source of the value for the table name. If that comes from an untrusted source, e.g. a user, then you need to validate the table name to avoid potential SQL injection attacks. One way might be to construct a parameterised query that looks up the table name from the DB catalogue:
import sqlite3
def exists_table(db, name):
query = "SELECT 1 FROM sqlite_master WHERE type='table' and name = ?"
return db.execute(query, (name,)).fetchone() is not None
I wouldn't separate the data into more than one table. If you create an index on the star column, you won't have any problem efficiently accessing the data.
Try with string formatting:
sql_cmd = '''CREATE TABLE {}(id, column1, column2, column2)'''.format(
'table_name')
db.execute(sql_cmd)
Replace 'table_name' with your desire.
To avoid hard-coding table names, I've used:
table = "sometable"
c = conn.cursor()
c.execute('''CREATE TABLE IF NOT EXISTS {} (
importantdate DATE,
somename VARCHAR,
)'''.format(table))
c.execute('''INSERT INTO {} VALUES (?, ?)'''.format(table),
(datetime.strftime(datetime.today(), "%Y-%m-%d"),
myname))
As has been said in the other answers, "tables can't be the target of parameter substitution" but if you find yourself in a bind where you have no option, here is a method of testing if the table name supplied is valid.
Note: I have made the table name a real pig in an attempt to cover all of the bases.
import sys
import sqlite3
def delim(s):
delims="\"'`"
use_delim = []
for d in delims:
if d not in s:
use_delim.append(d)
return use_delim
db_name = "some.db"
db = sqlite3.connect(db_name)
mycursor = db.cursor()
table = 'so""m ][ `etable'
delimiters = delim(table)
if len(delimiters) < 1:
print "The name of the database will not allow this!"
sys.exit()
use_delimiter = delimiters[0]
print "Using delimiter ", use_delimiter
mycursor.execute('SELECT name FROM sqlite_master where (name = ?)', [table])
row = mycursor.fetchall()
valid_table = False
if row:
print (table,"table name verified")
valid_table = True
else:
print (table,"Table name not in database", db_name)
if valid_table:
try:
mycursor.execute('insert into ' +use_delimiter+ table +use_delimiter+ ' (my_data,my_column_name) values (?,?) ',(1,"Name"));
db.commit()
except Exception as e:
print "Error:", str(e)
try:
mycursor.execute('UPDATE ' +use_delimiter+ table +use_delimiter+ ' set my_column_name = ? where my_data = ?', ["ReNamed",1])
db.commit()
except Exception as e:
print "Error:", str(e)
db.close()
you can use something like this
conn = sqlite3.connect()
createTable = '''CREATE TABLE %s (# );''' %dateNow)
conn.execute(createTable)
basically, if we want to separate the data into several tables according to the date right now, for example, you want to monitor a system based on the date.
createTable = '''CREATE TABLE %s (# );''' %dateNow) means that you create a table with variable dateNow which according to your coding language, you can define dateNow as a variable to retrieve the current date from your coding language.
You can save your query in a .sql or txt file and use the open().replace() method to use variables in any part of your query. Long time reader but first time poster so I apologize if anything is off here.
```SQL in yoursql.sql```
Sel *
From yourdbschema.tablenm
```SQL to run```
tablenm = 'yourtablename'
cur = connect.cursor()
query = cur.execute(open(file = yoursql.sql).read().replace('tablenm',tablenm))
You can pass a string as the SQL command:
import sqlite3
conn = sqlite3.connect('db.db')
c = conn.cursor()
tablename, field_data = 'some_table','some_data'
query = 'SELECT * FROM '+tablename+' WHERE column1=\"'+field_data+"\""
c.execute(query)
I'm having some trouble altering tables in my postgres database. I'm using psycopg2 and working out of Python. I tried to add a serial primary key. It took a long time (large table), and threw no error, so it did something, but when I went to check, the new column wasn't there.
I'm hoping this is something silly that I've missed, but right now I'm at a total loss.
import psycopg2
username = *****
password = *****
conn = psycopg2.connect(database='mydb',user=username,password=password)
query = "ALTER TABLE mytable ADD COLUMN sid serial PRIMARY KEY"
cur = conn.cursor()
cur.execute(query)
conn.close()
Other things I've tried while debugging:
It doesn't work when I remove PRIMARY KEY.
It doesn't work when try a different data type.
You need to add a commit statement in order for your changes to reflect in the table. Add this before you close the connection.
conn.commit()
I would like to check if a database table exists or not, but I don't know how to do.
I wrote (for example with SQLite, although I use MySQL mainly),
import sqlite3
table_name = "some_table"
connection = sqlite3.connect(db)
cursor = connection.cursor()
table_check = "SELECT name FROM sqlite_master WHERE type='table' AND name={};".format(table_name)
if not cursor.execute(table_check).fetchone(): # if the table doesn't exist
# OR if cursor.execute(table_check).fetchone() == "":
create_table()
else:
update_table()
But, an Error occured and I cannot proceed.
sqlite3.OperationalError: no such column: some_table
I read several Q&A here, but I couldn't get those.
Any advice can help me.
Thank you.
Python 3.5.1
The answer is depending on what rdbms product (mysql, sqlite, ms sql, etc.) you use.
You are getting this particular error in your above query because you do not enclose the value of table_name variable in single quotes.
In mysql you can use information_schema.tables table to query if a table exists.