We Keep Coding

retrieving a single value from sql table

I wish to retrieve a single value from this database I have created. For example, The user will select a Name from a drop down box (these names correspond to the name column in the database). The name chosen will be stored in a variable called name_value. I would like to know how to search the database for the name in name_value AND return ONLY the other text in the next column called Scientific, into another variable called new_name. I hope I explained that well?
connection = sqlite3.connect("Cw.db")
crsr = connection.cursor()
crsr.execute("""CREATE TABLE Names(
Name text,
Scientific text)""")
Inserting these values: (There is more but its unnecessary to add them all)
connection = sqlite3.connect("Cw.db")
crsr = connection.cursor()
crsr.execute("""INSERT INTO Names (Name, Scientific)
VALUES
('Human', 'Homo Sapien');""")

The SELECT statement in SQL can be used to query for rows with specific values, and to specify the columns to be returned.
In your case, the code would look something like this
stmt = """\
SELECT Scientific
FROM Names
WHERE Name = ?
LIMIT 1
"""
name = 'Human'
crsr.execute(stmt, (name,))
new_name = crsr.fetchone()[0]
A few points to note:
we use a ? in the SELECT statement as a placeholder for the value that we are querying for
we set LIMIT 1 in the SELECT statement to ensure that at most only one row is returned, since you want to assign the result to a single variable.
the value(s) passed to crsr.execute must be a tuple, even if there is only one value
the return value of crsr.fetchone is a tuple, even though we are only fetching one column.

How can I get a MySQL database to insert a default value if there's an attempt to insert a null value with Python?

I've read answers that do something similar but not exactly what I'm looking for, which is: attempting to insert a row with a NULL value in a column will result instead in that column's DEFAULT value being inserted.
I'm trying to process a large number of inserts in the mySQL Python connector with a large number of column values that I don't want to deal with individually, and none of the typical alternatives work here. Here is a sketch of my code:
qry = "INSERT INTO table (col1, col2, ...) VALUES (%s, %s, ...)"
row_data_dict = defaultdict(lambda : None, {...})
params = []
for col in [col1, col2, ...]:
params.append(row_data_dict[col])
cursor.execute(qry, tuple(params))
My main problem is that setting None as the default in the dictionary results in either NULL being inserted or an error if I specify the row as NOT NULL. I have a large number of columns that might change in the future so I'd want to avoid setting different 'default' values for different entries if at all possible.
I can't do the typical way of inserting DEFAULT by skipping over columns on the insert because while those columns might have the DEFAULT value, I can't guarantee it and considering I'm doing a large number of inserts I don't want to change the query string each time I insert depending on if it's default or not.
The other way of inserting DEFAULT seems to be to have DEFAULT as one of the parameters (e.g. INSERT INTO table (col1,...) VALUES (DEFAULT,...)) but in my case setting the default in the dictionary to 'DEFAULT' results in error (mySQL complains about it being an incorrect integer value on trying to insert into an integer column, making it seem like it's interpreting the default as a string and not a keyword).
This seems like it would be a relatively common use case, so it kind of shocks me that I can't figure out a way to do this. I'd appreciate any way to do this or get around it that I haven't already listed here.
EDIT: All the of the relevant columns are already labeled with a DEFAULT value, it doesn't seem to actually replace NULL (or python's None) when it's inserted.
EDIT 2: The reason why I want to avoid NULL so badly is because NULL != NULL and I want to have unique rows, so that if there's one row (1, 2, 3, 'Unknown'), INSERT IGNORE'ing a row (1, 2, 3, 'Unknown') won't insert it. With NULL you end up with a bunch of copies of the same record because one of the values is unknown.

You can use the DEFAULT() function in the VALUES list to specify that default value for the column should be used. And you can put this in an IFNULL() call so it will be used when the supplied value is NULL.
qry = """INSERT INTO table (col1, col2, ...)
VALUES (IFNULL(%s, DEFAULT(col1)), IFNULL(%s, DEFAULT(col2)), ...)"""

Welcome to Stackoverflow. What you need to do is in your database add a default value for the column you want to have the default value. When you create your table just use DEFAULT and then the value after you create the column in the table, like this:
CREATE TABLE `yourTable` (`id` INT DEFAULT 0, .....)
if you have already created the table and you need to alter the existing column, you would do something like this:
ALTER TABLE `yourTable` MODIFY `id` INT DEFAULT 0
so in your insert statement coming from python, as long as you pass in either NULL or Nothing for the value of that column then when the row is inserted into your database, the default value will be populated for that column
Another thing to keep in mind is that you have to pass in the proper number of values when you have a default set up for a column. Say you have a table with 3 columns, we'll call them colA, colB and colC.
if you want to insert a row with colA_value for colA, nothing for colB so it will use it's default value and colC_value for colC then you need to still pass in 3 values that will be used for your insert. If you just passed in colA_value and colC_value, then colA will get colA_value and colB will get colC_value and colC will be null. you need to pass in values that will be interpreted by MySQL like this:
INSERT INTO `yourTable` (`colA`, `colB`, `colC`)
VALUES
('colA_value', null, 'colC_value')
even though you are not passing in anything for colB you need to pass a null value from your python program by either passing null or None to MySQL for the value for colB in order to get colB to be populated with it's default value
if you only pass in 2 values to MySQL to insert a row in your table, the insert statement under the hood will look like this:
INSERT INTO `yourTable` (`colA`, `colB`, `colC`)
VALUES
('colA_value', 'colC_value')
which would result in colA getting set to colA_value, colB getting set to colC_value and colC being left as null
if you are passing in the right number of values to be inserted into MySQL (that would mean you need to include null or None for the value to be inserted into the column with the default value) than that is another story. Please let me know if you are passing in the right number of values so I can help you troubleshoot further if needed.

Python SQLite3 auto_increment properly increase even when deleting the max ID

Gday,
I'm starting to work with an SQL Database in python and I want to have multiple tables, where I can reference rows in each other's table by ID. Therefore, I'm using a column "testID integer PRIMARY KEY".
The Column increases in value as desired, but if I delete the row with the maximum ID and then add another entry, it will receive the ID which was set already before. That works because deleting the most recent row leads to a lower maximum ID in the column, which makes sense to me.
I was wondering now, is there a way to have the database memorize every ID that was set earlier and not set the same ID twice, even when the maximum ID is deleted from the database?
MWE to maybe make that clearer:
conn = sqlite3.connect("db_problem.db")
c = conn.cursor()
with conn:
c.execute("CREATE TABLE test ("
"testID integer PRIMARY KEY, "
"col1 integer)")
c.execute("INSERT INTO test (col1) VALUES (1)")
c.execute("INSERT INTO test (col1) VALUES (2)")
c.execute("DELETE FROM test WHERE col1 LIKE 1")
c.execute("SELECT testID FROM test WHERE col1 LIKE 2")
print(c.fetchall()) # this stays 2, even tho there is only one row left, which works fine
c.execute("INSERT INTO test (col1) VALUES (3)")
c.execute("DELETE FROM test WHERE col1 LIKE 3")
c.execute("INSERT INTO test (col1) VALUES (4)")
c.execute("SELECT testID FROM test WHERE col1 LIKE 4")
print(c.fetchall()) # Here the autoincrement was set to 3 although it is the fourth entry made

From SQLite Autoincrement:
If the AUTOINCREMENT keyword appears after INTEGER PRIMARY KEY, that
changes the automatic ROWID assignment algorithm to prevent the reuse
of ROWIDs over the lifetime of the database. In other words, the
purpose of AUTOINCREMENT is to prevent the reuse of ROWIDs from
previously deleted rows.
So create the table with this statement:
c.execute("CREATE TABLE test ("
"testID integer PRIMARY KEY AUTOINCREMENT , "
"col1 integer)")
But there is another part of the documentation which you must consider:
The AUTOINCREMENT keyword imposes extra CPU, memory, disk space, and
disk I/O overhead and should be avoided if not strictly needed.
The choice is yours.

Replacing NULLS by some value in a SQL table

I have a table in SQLite3 database (using Python), Tweet Table (TwTbl) that has some values in the column geo_id. Most of the values in this column are NULL\None. I want to replace/update all NULLS in the geo_id column of TwTbl by a number 999. I am not sure about the syntax. I am trying the following query, but I am getting an error ("No such Column: None")
c.execute("update TwTbl SET geo_id = 999 where geo_id = None").fetchall()
I even tried using Null instead of None, that did not give any errors but did not do any update.
Any help will be appreciated.

As an answer, so that you can accept it if you're inclined.
You need Is Null instead of = Null. Null is a special value that's indeterminate, and neither equal nor non-equal in most database implementations.

Saving Tuples as blob data types in Sqlite3 in Python

I have a dictionary in python. They keys are tuples with varying size containing unicode characters and the values are just a single int number. I want to insert this dictionary into sqlite db with a 2 column table.
The first column is for the key values and the second column should have the corresponding int value. Why do I want to do this? well I have a very large dictionary and I used cPickle, even setting the protocol to 2. The size is still big and saving and loading this file takes a lot of time. So I decided to save it in db. This dictionary only loads once into memory at the beginning of program, so there is no extra operation.
Now the problem is that I want to save the tuples exactly as tuples (not strings), so whenever I load my table into memory, I can immediately build my dictionary with no problem.
Does anyone know how I can do this?

A couple of things. First, SQLite doesn't let you store Python data-structures directly. Second, I'm guessing you want to ability to query the value by the tuple key on demand, so you don't want to pickle and unpickle and then search the keys in the dict.
The problem is, you can't query with tuple and you can't break the tuple entries into their own columns because they are of varying sizes. If you must use SQLite, you pretty much have to concatenate the unicode characters in the tuple, possibly with a delimiter that is not 1 of the characters in the tuple values. Use that as a key, and store it into a column in SQLite as a primary key column.
def tuple2key(t, delimiter=u':'):
return delimiter.join(t)
import sqlite3
conn = sqlite3.connect('/path/to/your/db')
cur = conn.cursor()
cur.execute('''create table tab (k text primary key, value integer)''')
# store the dict into a table
for k, v in my_dict.iteritems():
cur.execute('''insert into tab values (?, ?)''', (tuple2key(k), v))
cur.commit()
# query the values
v = cur.execute(''' select value from tab where key = ? ''', tuple2key((u'a',u'b'))).fetchone()

It is possible to store tuples into sqlite db and to create indices on tuples. It needs some extra code to get it done.
Whether the storing of tuples into db an appropriate solution in this particular case is another issue (probably a two-key solution is better suited).
import sqlite3
import pickle
def adapt_tuple(tuple):
return pickle.dumps(tuple)
sqlite3.register_adapter(tuple, adapt_tuple) #cannot use pickle.dumps directly because of inadequate argument signature
sqlite3.register_converter("tuple", pickle.loads)
def collate_tuple(string1, string2):
return cmp(pickle.loads(string1), pickle.loads(string2))
con = sqlite3.connect(":memory:", detect_types=sqlite3.PARSE_DECLTYPES)
con.create_collation("cmptuple", collate_tuple)
cur = con.cursor()
cur.execute("create table test(p tuple unique collate cmptuple) ")
cur.execute("create index tuple_collated_index on test(p collate cmptuple)")
#insert
p = (1,2,3)
p1 = (1,2)
cur.execute("insert into test(p) values (?)", (p,))
cur.execute("insert into test(p) values (?)", (p1,))
#ordered select
cur.execute("select p from test order by p collate cmptuple")

I think it is better to create 3 columns in your table - key1, key2 and value.
If you prefer to save the key as a tuple, you can still use pickle but apply to the key only. Then you can save it as blob.
>>> pickle.dumps((u"\u20AC",u"\u20AC"))
'(V\\u20ac\np0\ng0\ntp1\n.'
>>> pickle.loads(_)
(u'\u20ac', u'\u20ac')
>>>