We Keep Coding

MySQL query inside python program returning unexpected 'b' inside the tuple fetched from the query execution

I'm writing a DBMS and am validating user inputs ( for tables in the database ) using the lengths and data_types stored in the MySQL database's INFORMATION_SCHEMA.COLUMNS for the particular table they are entering data into. Im using Python 3.8.4 ( 64 bit ) with all the needed mysql connector modules installed etc and i have the mysql server running on a local host on the same machine.
After executing the following query = "SELECT COLUMN_TYPE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = ''my_schema' AND TABLE_NAME = 'the_specific_table';"
with the following python code:
NOTE: the particular table im using in this query is set out like below
user_id - INT - PRIMARY KEY- NOT NULL
first_name - VARCHAR(45) - NOT NULL
last_name - VARCHAR(45) - NOT NULL
import mysql.connector
connection = mysql.connector.connect(
host = host,
user = user,
passwd = password,
database = db_name)
cursor = connection.cursor()
cursor.execute("SELECT COLUMN_TYPE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = 'SCHEMA_HERE' AND TABLE_NAME = 'TABLE_HERE';")
result = cursor.fetchall()
print(result)
The result i get from this query is the following
-- >
[(b'int',), (b'varchar(45)',), (b'varchar(45)',)]. This is completely unchanged from how it is returned when i print the cursor content out
Looping through the list gives each tuple as you would expect,however when indexing those tuples, instead of giving a value such as - string = 'hello' - print(string[0]) outputing of course 'h' - a seemingly random number is outputed instead.This means at the moment i cant work out how to validate inputs that need to be validated against the length and datatype of a column
This weird 'b' inclusion only happens on the DATA_TYPE column of the query, so if the query
cursor.execute("""SELECT TABLE_NAME,COLUMN_NAME, DATA_TYPE, CHARACTER_MAXIMUM_LENGTH, IS_NULLABLE
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'my_table'""")
is executed the result is
[('tblusers', 'user_id', b'int', None, 'NO'), ('tblusers', 'first_name', b'varchar', 45, 'NO'), ('tblusers', 'last_name', b'varchar', 45, 'NO')]
NOTE this is my first time posting a question on here so do forgive me if i have left out any crucial pieces of infomation needed for someone to help, just let me know and i will try to add that.
Any help is hugely appreciated :)

Python and SQLite3 - Select Where not working inside script but works in console

As the title says, I'm having problems retrieving data from a SQLite DB when using WHERE statement.
Here is the piece of code that tries to get a row where an ID is given:
def check_attendance(self, cred):
query = """SELECT * FROM clients WHERE dni=?"""
self.conn.cursor().execute(query, (cred,))
record = self.conn.cursor().fetchone()
The var cred is already inside a tuple as specified by SQLite API for Python. Sadly, the query returns None when executed here.
If I do the same but using sqlite.exe, then I do get the right row back. In fact, this is the only query I cannot execute properly from my python script, everything else return rows normally.
Here it is executing from the Python script
And here is in sqlite.exe
Here is the piece that stores values in the DB:
def new_client(self, *args):
success = False
# Check if all inputs are filled
if self.dialog.content_cls.ids.user_name.text and self.dialog.content_cls.ids.user_surname.text and len(self.dialog.content_cls.ids.user_dni.text) == 8 and self.dialog.content_cls.ids.user_date.text:
# Convert str date to a datetime obj in order to use it with timedelta
paid_date = datetime.strptime(self.dialog.content_cls.ids.user_date.text, "%d-%m-%Y")
# paid_date is now YYYY-MM-DD HH-MM-SS format
# Add 30 days to paid_date
exp_date = paid_date + timedelta(days=30)
# Convert YYYY-MM-DD HH-MM-DD to string YYYY-MM-DD as we don't need clock
paid_date = datetime.strptime(str(paid_date), "%Y-%m-%d %H:%M:%S").strftime("%Y-%m-%d")
exp_date = datetime.strptime(str(exp_date), "%Y-%m-%d %H:%M:%S").strftime("%Y-%m-%d")
# Create query blueprint and try executing
query = """INSERT INTO clients (name, surname, dni, membership_date, expiration_date) VALUES (?,?,?,?,?)"""
try:
self.conn.execute(query, (self.dialog.content_cls.ids.user_name.text,
self.dialog.content_cls.ids.user_surname.text,
self.dialog.content_cls.ids.user_dni.text,
paid_date,
exp_date
)
)
success = True
except sqlite3.IntegrityError:
pass
if success:
self.conn.commit()
The try/except was used for other reasons. Adding to the database from the Python script works fine as shown in the second screenshot.
And the table clients is as follows:
c.execute(''' CREATE TABLE clients (id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
surname TEXT NOT NULL,
dni INTEGER NOT NULL UNIQUE,
membership_date date NOT NULL,
expiration_date date NOT NULL); ''')
Using Python v3.7.7 32bit.
Thanks!

In your code, the cursor is initialized two times (cursor()).
You should either get the results from the same cursor you used to execute the SELECT statement:
def check_attendance(self, cred):
query = """SELECT * FROM clients WHERE dni=?"""
cur = self.conn.cursor()
cur.execute(query, (cred,))
record = cur.fetchone()
...or you can avoid the implicit cursor creation by using execute method directly on Connection object:
def check_attendance(self, cred):
query = """SELECT * FROM clients WHERE dni=?"""
record = self.conn.execute(query, (cred,)).fetchone()
You can read more about this approach in the documentation (https://docs.python.org/3/library/sqlite3.html#using-sqlite3-efficiently):
Using the nonstandard execute(), executemany() and executescript() methods of the Connection object, your code can be written more concisely because you don’t have to create the (often superfluous) Cursor objects explicitly. Instead, the Cursor objects are created implicitly and these shortcut methods return the cursor objects.

Python Querying on database with where clause

I am new to Python and want to understand the syntax for connecting to MS sql database.
Have successfully created the connection but want to select a conditional data.
Please suggest how can below be done in python:
import pyodbc as odbc
cursor.execute("SELECT Col1,Col2 FROM Data where date between sysdate and (sysdate-1) ;")
I have already assigned Start and end as :
End = dt.datetime.today()
Start = End - dt.timedelta(days=1)
How can this be incorporated in the query above??
Also, it would be really helpful if you could point me to some documentation where other such related things might be mentioned.

As explained in the pyodbc documentation for .execute, you can use ? as parameter placeholders in your SQL command text and pass the parameter values to .execute like so:
End = dt.datetime.today()
Start = End - dt.timedelta(days=1)
sql = "SELECT [Col1],[Col2] FROM [Data] WHERE [date] BETWEEN ? AND ?"
cursor.execute(sql, Start, End)
rows = cursor.fetchall()

I believe Gord's answer is the correct one but in case this is useful, you can also prepare dynamically you SQL statement within your Python code and provide the SQL statement already updated with the parameters to your cursor.
This should solve your error:
Error: ('HYC00', '[HYC00] [Microsoft][ODBC SQL Server Driver]Optional
feature not implemented (0) (SQLBindParameter)')
end = dt.datetime.today()
start = End - dt.timedelta(days=1)
sql = "SELECT [Col1],[Col2] FROM [Data] WHERE [date] BETWEEN '{}' AND '{}'".format(start, end)
cursor.execute(sql)
rows = cursor.fetchall()

Start = StringVar()
End = StringVar()
End = dt.datetime.today()
Start = End - dt.timedelta(days=1)
sql = "SELECT [Col1],[Col2] FROM [Data] WHERE [date] BETWEEN ? AND ?"
cursor.execute(sql, Start, End)
rows = cursor.fetchall()

CosmosDB and Python3: how to query?

I am using CosmosDB (Azure documentDB) in my project, written in Python 3.
I have been looking for a while now, but I cannot find out how to query my table. I have seen some example code, but I do not see an example of how to query... all I can do is get all documents (not ideal when my DB is > 80GB).
The GitHub repo shows a very tiny set of operations for database and collections: https://github.com/Azure/azure-documentdb-python/blob/master/samples/CollectionManagement/Program.py
And the following SO post shows how to read all documents... but not how to perform querying such as "WHERE = X;"
I'd really appreciate it if someone can point me in the right direction, and possibly supply an example showing how to run queries.

Based on my understanding, I think you want to know how to perform a SQL-like query using Python to retrieve documents on Azure CosmosDB of DocumentDB API, please refer to the code below from here.
A query is performed using SQL
# Query them in SQL
query = { 'query': 'SELECT * FROM server s' }
options = {}
options['enableCrossPartitionQuery'] = True
options['maxItemCount'] = 2
result_iterable = client.QueryDocuments(collection['_self'], query, options)
results = list(result_iterable);
print(results)
The above code is using the method QueryDocuments.
Any concern, please feel free to let me know.
Update: Combine with my sample code for the other SO thread you linked, as below.
from pydocumentdb import document_client
uri = 'https://ronyazrak.documents.azure.com:443/'
key = '<your-primary-key>'
client = document_client.DocumentClient(uri, {'masterKey': key})
db_id = 'test1'
db_query = "select * from r where r.id = '{0}'".format(db_id)
db = list(client.QueryDatabases(db_query))[0]
db_link = db['_self']
coll_id = 'test1'
coll_query = "select * from r where r.id = '{0}'".format(coll_id)
coll = list(client.QueryCollections(db_link, coll_query))[0]
coll_link = coll['_self']
query = { 'query': 'SELECT * FROM server s' }
docs = client.QueryDocuments(coll_link, query)
print list(docs)

query = 'SELECT * FROM c'
docs = list(client.QueryItems(coll_link,query))
QueryDocuments has been replaced with QueryItems.

I have a similar problem recently. You can fetch blocks (not entire query set) by calling fetch_next_block().
query = "select * from c"
options = {'maxItemCount': 1000, 'continuation': True}
q = db_source._client.QueryDocuments(collection_link, query, options)
block1 = q.fetch_next_block()
block2 = q.fetch_next_block()

Getting the id of the last record inserted for Postgresql SERIAL KEY with Python

I am using SQLAlchemy without the ORM, i.e. using hand-crafted SQL statements to directly interact with the backend database. I am using PG as my backend database (psycopg2 as DB driver) in this instance - I don't know if that affects the answer.
I have statements like this,for brevity, assume that conn is a valid connection to the database:
conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)")
Assume also that the user table consists of the columns (id [SERIAL PRIMARY KEY], name, country_id)
How may I obtain the id of the new user, ideally, without hitting the database again?

You might be able to use the RETURNING clause of the INSERT statement like this:
result = conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)
RETURNING *")
If you only want the resulting id:
result = conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)
RETURNING id")
[new_id] = result.fetchone()

User lastrowid
result = conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)")
result.lastrowid

Current SQLAlchemy documentation suggests
result.inserted_primary_key should work!

Python + SQLAlchemy
after commit, you get the primary_key column id (autoincremeted) updated in your object.
db.session.add(new_usr)
db.session.commit() #will insert the new_usr data into database AND retrieve id
idd = new_usr.usrID # usrID is the autoincremented primary_key column.
return jsonify(idd),201 #usrID = 12, correct id from table User in Database.

this question has been asked many times on stackoverflow and no answer I have seen is comprehensive. Googling 'sqlalchemy insert get id of new row' brings up a lot of them.
There are three levels to SQLAlchemy.
Top: the ORM.
Middle: Database abstraction (DBA) with Table classes etc.
Bottom: SQL using the text function.
To an OO programmer the ORM level looks natural, but to a database programmer it looks ugly and the ORM gets in the way. The DBA layer is an OK compromise. The SQL layer looks natural to database programmers and would look alien to an OO-only programmer.
Each level has it own syntax, similar but different enough to be frustrating. On top of this there is almost too much documentation online, very hard to find the answer.
I will describe how to get the inserted id AT THE SQL LAYER for the RDBMS I use.
Table: User(user_id integer primary autoincrement key, user_name string)
conn: Is a Connection obtained within SQLAlchemy to the DBMS you are using.
SQLite
======
insstmt = text(
'''INSERT INTO user (user_name)
VALUES (:usernm) ''' )
# Execute within a transaction (optional)
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = result.lastrowid
txn.commit()
MS SQL Server
=============
insstmt = text(
'''INSERT INTO user (user_name)
OUTPUT inserted.record_id
VALUES (:usernm) ''' )
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = result.fetchone()[0]
txn.commit()
MariaDB/MySQL
=============
insstmt = text(
'''INSERT INTO user (user_name)
VALUES (:usernm) ''' )
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = conn.execute(text('SELECT LAST_INSERT_ID()')).fetchone()[0]
txn.commit()
Postgres
========
insstmt = text(
'''INSERT INTO user (user_name)
VALUES (:usernm)
RETURNING user_id ''' )
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = result.fetchone()[0]
txn.commit()

result.inserted_primary_key
Worked for me. The only thing to note is that this returns a list that contains that last_insert_id.

Make sure you use fetchrow/fetch to receive the returning object
insert_stmt = user.insert().values(name="homer", country_id="123").returning(user.c.id)
row_id = await conn.fetchrow(insert_stmt)

For Postgress inserts from python code is simple to use "RETURNING" keyword with the "col_id" (name of the column which you want to get the last inserted row id) in insert statement at end
syntax -
from sqlalchemy import create_engine
conn_string = "postgresql://USERNAME:PSWD#HOSTNAME/DATABASE_NAME"
db = create_engine(conn_string)
conn = db.connect()
INSERT INTO emp_table (col_id, Name ,Age)
VALUES(3,'xyz',30) RETURNING col_id;
or
(if col_id column is auto increment)
insert_sql = (INSERT INTO emp_table (Name ,Age)
VALUES('xyz',30) RETURNING col_id;)
result = conn.execute(insert_sql)
[last_row_id] = result.fetchone()
print(last_row_id)
#output = 3
ex -