We Keep Coding

2 questions: Importing data from MySQL data base to Python

Q1. My database contains 3 columns: time, value A and value B. The time data is written in the form 00:00:00 and the increment is 1 minute.
When I try to import data ...
cursor.execute (f"SELECT * FROM trffc_int_data.{i};")
instead getting (00:00:00, A, B), I get
(datetime.timedelta(0), 7, 2), (datetime.timedelta(seconds=60), 8, 5), .....
I suppose Python doesn't convert the time right. Any suggestions?
Q2. I have an initial database with the data mentioned above. I need to get the data from the initial database, convert it, and save it to another database.
I'm stuck at a point where data should be saved to a new table.
Here are the sections of the code...
# Creating new DB
NewDB = input(" :: Enter the Database name : ")
sqlsynt = f"CREATE DATABASE IF NOT EXISTS {NewDB}"
cursor.execute(sqlsynt,NewDB)
stdb.commit()
# Creating table and writing the data
cursor.execute (f"USE {NewDB}")
sqlsynt = f"CREATE TABLE {dayinweek} (time TIME, Vehicles INT(3), Pedestrians INT(3))"
cursor.execute (sqlsynt, NewDB, dayinweek)
#stdb.commit()
sqlsyntax = f"INSERT INTO {NewDB}.{dayinweek} (time, Vehicles, Pedestrians) VALUES (%s, %s, %s)"
cursor.executemany(sqlsyntax, temp_list_day)
The program stucks on the last line saying that there is no table 1 in NewDB!
mysql.connector.errors.ProgrammingError: 1146 (42S02): Table 'test001.1' doesn't exist
What's wrong with the code? Maybe the problem is in mixing f and %s formating?
Thanks in advance

If I am followin this correctly, you are creating a table called 1. Digit-only identifiers are not allowed in MySQL, unless the identifier is quoted, as explained in the documentation.
Identifiers may begin with a digit but unless quoted may not consist solely of digits.
Your create table statement did fail, but you did not notice that error until you tried to insert.
You could quote the table name, using backticks:
CREATE TABLE `{dayinweek}` (time TIME, Vehicles INT(3), Pedestrians INT(3))
And then:
INSERT INTO `{NewDB}`.`{dayinweek}` (time, Vehicles, Pedestrians) VALUES (%s, %s, %s)
Quoting the database name may also be a good idea: the same rules apply as for table names (and this is user input to start with).
But overall, changing the table name seems like a better option, as this makes for cleaner code: how about something like table1 for example - or better yet, a table name that is more expressive on what kind of data it contains, such as customer1, or sales1.
Note: your code is open to SQL injection, as you are passing user input directly to the database in a create database statement. Obviously such information cannot be parameterized, however I would still recommend performing a minimal sanity check on application side beforehand.

Inserting arrays into databases

I am trying to write a general function that will insert a line of data into a table in a database but I am trying to write an array of unknown length. I am aiming to just be able to call this function in any programand write a line of data of any length to the table (assuming the table and the array are the same length.
I have tried adding the array like it is a singular peice of data.
import sqlite3
def add2Db(dbName, tableName, data):
connection = sqlite3.connect(dbName)
cur = connection.cursor()
cur.execute("INSERT INTO "+ tableName +" VALUES (?)", (data))
connection.commit()
connection.close()
add2Db("items.db", "allItems", (1, "chair", 5, 4))
This just crashes and gives me an error saying it has 4 columns but only one value was supplied.

SQLite does not support arrays - you have to convert to a TEXT using ','.join() to join your array items into a single string and pass that.
Source: SQLite website
https://www.sqlite.org/datatype3.html

I'm not a Python programmer, but I've been doing SQL a long time. I even wrote my own ORM. My advice is do not write your own SQL query builder. There's a myriad of subtle issues and especially security issues. I elaborate on a few of them below.
Instead, use a well-established SQL Query Builder or ORM. They've already dealt with these issues. Here's an example using SQLAlchemy.
from datetime import date
from sqlalchemy import create_engine, MetaData
# Connect to the database with debugging on.
engine = create_engine('sqlite:///test.sqlite', echo=True)
conn = engine.connect()
# Read the schemas from the database
meta = MetaData()
meta.reflect(bind=engine)
# INSERT INTO users (name, birthday, state, country) VALUES (?, ?, ?, ?)
users = meta.tables['users']
conn.execute(
users.insert().values(name="Yarrow Hock", birthday=date(1977, 1, 23), state="NY", country="US")
)
SQLAlchemy can do the entire range of SQL operations and will work with different SQL variants. You also get type safety.
conn.execute(
users.insert().values(name="Yarrow Hock", birthday="in the past", state="NY", country="US")
)
sqlalchemy.exc.StatementError: (exceptions.TypeError) SQLite Date type only accepts Python date objects as input. [SQL: u'INSERT INTO users (name, birthday, state, country) VALUES (?, ?, ?, ?)']
insert into table values (...) relies on column definition order
This relies on the order columns were defined in the schema. This leaves two problems. First is a readability problem.
add2Db(db, 'some_table', (1, 39, 99, 45, 'papa foxtrot', 0, 42, 0, 6)
What does any of that mean? A reader can't tell. They have to go digging into the schema and count columns to figure out what each value means.
Second is a maintenance problem. If, for any reason, the schema is altered and the column order is not exactly the same, this can lead to some extremely difficult to find bugs. For example...
create table users ( name text, birthday date, state text, country text );
vs
create table users ( name text, birthday date, country text, state text );
add2Db(db, 'users', ('Yarrow Hock', date(1977, 1, 23), 'NY', 'US'));
That insert will silently "work" with either column order.
You can fix this by passing in a dictionary and using the keys for column names.
add2Db(db, 'users', (name="Yarrow Hock", birthday=date(1977, 1, 23), state="NY", country="US"));
Then we'd produce a query like:
insert into users
(name, birthday, state, country)
values (?, ?, ?, ?)
This leads to the next and much bigger problem.
SQL Injection Attack
Now this opens up a new problem. If we simply stick the table and column names into the query that leaves us open to one of the most common security holes, a SQL Injection Attack. That's where someone can craft a value which when naively used in a SQL statement causes the query to do something else. Like Little Bobby Tables.
While the ? protects against SQL Injection for values, it's still possible to inject via the column names. There's no guarantee the column names can be trusted. Maybe they came from the parameters of a web form?
Protecting table and column names is complicated and easy to get wrong.
The more SQL you write the more likely you're vulnerable to an injection attack.
You have to write code for everything else.
Ok, you've done insert. Now update? select? Don't forget about subqueries, group by, unions, joins...
If you want to write a SQL query builder, cool! If, instead, you have a job to do using SQL, writing yet another SQL query builder is not your job.
It's harder for anyone else to understand.
There's a good chance that any given Python programmer knows how SQLAlchemy works, and there's plenty of tutorials and documentation if they don't. There's no chance they know about your home-rolled SQL functions, and you have to write all the tutorials and docs.

You shouldn't try to write your own ORMs without an argumented need. You will have a lot of problems, for example here's quick 25 reasons not to.
Instead use any popular orm that is proven. I recommend using SQLAlchemy as a go to outside of Django. Using it you can map a dict of values to insert it into a model just like insert(schema_name).values(**dict_name) (here's an example of insert/update).

Change your function to this:
def add2Db(dbName, tableName, data):
num_qs = len(data)
qm = ','.join(list('?' * num_qs))
query = """
INSERT INTO {table}
VALUES ({qms})
""".format(table=tableName,
qms=qm)
connection = sqlite3.connect(dbName)
cur = connection.cursor()
cur.execute(query, data)
connection.commit()
connection.close()

Interpolate unescaped string in Python

A psycopg2 cursor takes a string with escaped substrings that represent text/char/varchar values. For instance, to utilize a stored procedures that inserts into a table:
cur.execute('SELECT insert_profile("name", "favorite color example")')
My problem occurs when I try to use string interpolation to insert variable the string that is passed to the execute function.
I've tried to just use double quotes on the variables and single quotes on the string literal, but this just performs the usual operation. Eg:
name = "Harry"
color = "Blue"
cur.execute('SELECT insert_profile({0}, {1})'.format(name, color)
I've also tried to nest the variables, but it inserts the literal nested string into the database:
name = "'Harry'"
color = "'Blue'"
What is the correct approach? Sorry if the answer is elsewhere on SO or the internet. I haven't had success looking around.

From the psycopg web page:
cur.execute(
... """INSERT INTO some_table (an_int, a_date, another_date, a_string)
... VALUES (%(int)s, %(date)s, %(date)s, %(str)s);""",
... {'int': 10, 'str': "O'Reilly", 'date': datetime.date(2005, 11, 18)})
The query string has placeholders that refer to keys in the dictionary, and the driver will interpolate the values with appropriate quoting, escaping etc. which can save you quite a lot of trouble.
More importantly: you should never build queries through string operations like concat operation or format. This leaves a huge security hole due to SQL injection. Let the driver do the hard work!

Given table and column name, how to test if INSERT needs quotes ('') around the values to be inserted?

I have a dictionary of column name / values, to insert into a table. I have a function that generates the INSERT statement. I'm stuck because the function always puts quotes around the values, and some are integers.
e.g. If column 1 is type integer then the statement should be INSERT INTO myTable (col1) VALUES 5; vs
INSERT INTO myTable (col1) VALUES '5'; second one causes an error saying column 5 does not exist.
EDIT: I found the problem (I think). the value was in double quotes not single, so it was "5".
In Python, given a table and column name, how can I test if the INSERT statement needs to have '' around the VALUES ?

This question was tagged with "psycopg2" -- you can prepare the statement using a format string and have psycopg2 infer types for you in many cases.
cur.execute('INSERT INTO myTable (col1, col2) VALUES (%s, %s);', (5, 'abc'))
psycopg2 will deal with it for you, because Python knows that 5 is an integer and 'abc' is a string.
http://initd.org/psycopg/docs/usage.html#passing-parameters-to-sql-queries

You certainly want to use a library function to decide whether or not to quote values you insert. If you are inserting anything input by a user, writing your own quoting function can lead to SQL Injection attacks.
It appears from your tags that you're using psycopg2 - I've found another response that may be able to answer your question, since I'm not familiar with that library. The main gist seems to be that you should use
cursor.execute("query with params %s %s", ("param1", "pa'ram2"))
Which will automatically handle any quoting needed for param1 and param2.

Although I personally don't like the idea, you can use single quotes around integers when you insert in Postgres.
Perhaps your problem is the lack of parentheses:
INSERT INTO myTable(col1)
VALUES('5');
Here is a SQL Fiddle illustrating this code.
As you note in the comments, double quotes do not work in Postgres.

You can put always the single quote (be careful, if the value contents a quote you must double it: insert into example (value_t) values ('O''Hara');
You can decide checking the value that you want to insert regardles of the type of de destination
You can decide checking the type of the target field
As you can see in http://sqlfiddle.com/#!15/8bfbd/3 theres no mater with inserting integers into a text field or string that represents an integer in a numeric field.
To check the field type you can use the information_schema:
select data_type from information_schema.columns
where table_schema='public'
and table_name='example'
and column_name='value_i';
http://sqlfiddle.com/#!15/8bfbd/7

Entering variables into a table

So, when I chose to enter certain data into a table, instead of entering the values that the variables represent, it enters the actual name of the variable, here's an example.
name = Josh
surname = Smith
age = 15
import sqlite3
scores_db=sqlite3.connect('E:\\scores_db.db')
c=scores_db.cursor()
c.execute('''INSERT INTO Scores_1A
VALUES ('name','surname','age')''')
scores_db.commit()
scores_db.close()
When I check the database files, instead of having the information that the variable represents, it instead enters the variable name. I guess there is some kind of syntax to put around the variable name?

The string is passed literally to the database driver which has no idea there is a variable of that name in you Python code.
The seemingly straight forward solution to this problem is to use string formatting to substitute the representation of your variables into the SQL query before sending it to the database.
c.execute("INSERT INTO Scores_1A VALUES ('{}', '{}', '{}')".format(name, surname, age))
However, please don't do that. It makes your code vulnerable to SQL injection if one of the variables expands to some unexpected value. For example consider what would happen if name was the string "Robert'; DROP TABLE Scores_1A; -- ".
Instead, you should use prepared statements:
c.execute("INSERT INTO Scores_1A VALUES (?, ?, ?)", (name, surname, age))
Here, the database driver will receive the unmodified string followed by the serialized tuple of arguments. This is not only faster but also secure against SQL injection.
From the Python documentation:
Usually your SQL operations will need to use values from Python variables. You shouldn't assemble your query using Python's string operations because doing so is insecure; it makes your program vulnerable to an SQL injection attack (see http://xkcd.com/327/ for humorous example of what can go wrong).
Instead, use the DB-API's parameter substitution. Put ? as a placeholder wherever you want to use a value, and then provide a tuple of values as the second argument to the cursor's execute() method.