I have a postgres database that I want to query with peewee in python. If I connect to the database directly (psql or pgadmin) I can do something like
set my.setting='test'
or
select current_setting('my.setting')
How can I do this with peewee? The model I have contains only the tables I have in my database.
Thanks for help!
You can execute raw SQL using the Database method execute_sql(), example:
db = PostgresqlDatabase("test")
db.execute_sql("set my.setting to 'test'")
cur = db.execute_sql("show my.setting")
print(cur.fetchone()[0])
Related
I'm new to Django and SQL. I have this following SQL query. How to implement the same in the Django query?
"SELECT DISTINCT C1.CLASSDESC AS CLASS,C2.CLASSCODE AS CODE, C1.CARDCATEGORY AS CATEGORY, C2.CLASSBENEFITS BENEFITS FROM CARDCLASSMAPPING C1,CARDCLASSMASTER C2 WHERE C1.ISACTIVE = 1 AND C2.ISACTIVE = 1 AND C1.CLASSDESC = C2.CLASSDESC AND C1.SCHEMETYPE = ? AND C1.SCHEMECODE = ? AND C1.GLCODE = ? AND C1.ACCOUNTCATEGORY = ? ORDER BY CLASS";
You can go through this link to understand how queries can be constructed in Django. Django provides ORM API to fetch data from your database easily using queries like this:
CardClassMapping.objects.get(id=1)
In order to use the above line of code, you should have a model named CardClassMapping to get the objects from.
Also, you can use raw SQL like this:
CardClassMapping.objects.raw('SELECT DISTINCT C1.CLASSDESC AS CLASS,C2.CLASSCODE AS CODE, C1.CARDCATEGORY AS CATEGORY, C2.CLASSBENEFITS BENEFITS FROM CARDCLASSMAPPING C1')
For more on raw SQL: https://docs.djangoproject.com/en/3.1/topics/db/sql/
I am looking for a way to get the size of a database with SQL Alchemy. Ideally, it will be agnostic to which underlying type of database is used. Is this possible?
Edit:
By size, I mean total number of bytes that the database uses.
The way I would do is to find out if you can run a SQL query to get the answer. Then, you can just run this query via SQLAlchemy and get the result.
Currently, SQLAlchemy does not provide any convenience function to determine the table size in bytes. But you can execute an SQL statement. Caveat is that you have to use a statement that is specific to your type of SQL (MySQL, Postgres, etc)
Checking this answer, for MySQL you can execute a statement manually like:
from sqlalchemy import create_engine
connection_string = 'mysql+pymysql://...'
engine = create_engine(connection_string)
statement = 'SELECT table_schema, table_name, data_length, index_length FROM information_schema.tables'
with engine.connect() as con:
res = con.execute(statement)
size_of_table = res.fetchall()
For SQLite you can just check the entire database size with the os module:
import os
os.path.getsize('sqlite.db')
For PostgreSQL you can do it like this:
from sqlalchemy.orm import Session
dbsession: Session
engine = dbsession.bind
database_name = engine.url.database
# https://www.a2hosting.com/kb/developer-corner/postgresql/determining-the-size-of-postgresql-databases-and-tables
# Return the database size in bytes
database_size = dbsession.execute(f"""SELECT pg_database_size('{database_name}')""").scalar()
I'm connecting to an Oracle database from sqlalchemy and I want to know when the tables in the database were created. I can access this information through the sql developer application so I know that it is stored somewhere, but I don't know if its possible to get this information from sqlalchemy.
Also if its not possible, how should I be getting it?
SqlAlchemy doesn't provide anything to help you get that information. You have to query the database yourself.
something like:
with engine.begin() as c:
result = c.execute("""
SELECT created
FROM dba_objects
WHERE object_name = <<your table name>>
AND object_type = 'TABLE'
""")
I am trying to select data from our main database (postgres) and insert it into a temporary sqlite database for some comparision, analytics and reporting. Is there an easy way to do this in Python? I am trying to do something like this:
Get data from the main Postgres db:
import psycopg2
postgres_conn = psycopg2.connect(connection_string)
from_cursor = postgres_conn.cursor()
from_cursor.execute("SELECT email, firstname, lastname FROM schemaname.tablename")
Insert into SQLite table:
import sqlite3
sqlite_conn = sqlite3.connect(db_file)
to_cursor = sqlite_conn.cursor()
insert_query = "INSERT INTO sqlite_tablename (email, firstname, lastname) values %s"
to_cursor.some_insert_function(insert_query, from_cursor)
So the question is: is there a some_insert_function that would work for this scenario (either using pyodbc or using sqlite3)?
If yes, how to use it? Would the insert_query above work? or should it be modified?
Any other suggestions/approaches would also be appreciated in case a function like this doesn't exist in Python. Thanks in advance!
You should pass the result of your select query to execute_many.
insert_query = "INSERT INTO smallUsers values (?,?,?)"
to_cursor.executemany(insert_query, from_cursor.fetchall())
You should also use a parameterized query (? marks), as explained here: https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.execute
If you want to avoid loading the entire source database into memory, you can use the following code to process 100 rows at a time:
while True:
current_data = from_cursor.fetchmany(100)
if not current_data:
break
to_cursor.exectutemany(insert_query, current_data)
sqlite_conn.commit()
sqlite_conn.commit()
You can look at executemany from pyodbc or sqlite. If you can build a list of parameters from your select, you can pass the list to executemany.
Depending on the number of records you plan to insert, performance can be a problem as referenced in this open issue. https://github.com/mkleehammer/pyodbc/issues/120
Is there anyway I can get the datatypes of fields returned in a query in mysql. Say I have a query:
SELECT a.*,b.* FROM tbl_name a LEFT JOIN other_tbl b ON a.id=b.first_id
Is there a command I can use in mysql that will return the names of the fields that this query will return and their datatypes. I know I can potentially create a view using this query and then DESCRIBE that view, but is there any other way I can do it on the fly?
I'm using SQLAlchemy to perform this raw query and my tables are dynamically generated. Is there a SQLAlchemy way if not a MySQL way.
You can get the datatypes from a table with this in MySQL
SELECT COLUMN_TYPE
FROM information_schema.COLUMNS
WHERE TABLE_NAME = 'a'