We Keep Coding

FASTAPI testing database not creating database

I'm trying to test my FASTAPI app. Seems to me, all settings are correct.
test_users.py
engine = create_engine(
f"postgresql"
f"://{settings.database_username}"
f":{settings.database_password}"
f"#{settings.database_hostname}"
f":{settings.database_port}"
f"/test_{settings.database_name}"
)
TestingSessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
Base.metadata.create_all(bind=engine)
def override_get_db():
try:
db = TestingSessionLocal()
yield db
finally:
db.close()
app.dependency_overrides[get_db] = override_get_db
client = TestClient(app)
def test_create_user():
response = client.post(
"/users/",
json={"email": "nikita#gmail.com", "password": "password"}
)
new_user = schemas.UserOutput(**response.json())
assert response.status_code == 201
assert new_user.email == "nikita#gmail.com"
When I run pytest, I get this error:
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection to server at "localhost" (::1), port 5432 failed: FATAL: database "test_social_media_api" does not exist
Why is the code not creating the database?

With engine = create_engine("postgresql://...") you define a connection to an existing PostgreSql database.
And with Base.metadata.create_all(bind=engine) you create the tables - according to your models - in the existing database.
So the code that you written does not create a database, it expects that you give it an already existing database.
And that has to do with PostgreSQL itself.
PostgreSQL runs as a server, and a PostgreSQL server can run multiple databases. And each database has to be created explicitly.
Just telling SQLAlchemy the connection string is not enough.
It's possible to create a new database from Python itself by connecting to the PostgreSQL server (see https://www.tutorialspoint.com/python_data_access/python_postgresql_create_database.htm), or alternatively you can create it manually before you run your script. E.g. by running CREATE DATABASE databasename; inside psql (or any other database tool).
However if you want to test using a running database, I would suggest using testcontainers. They will spawn a new PostgreSQL server with an empty database everytime you run the tests.
Notice, that the example from the FastAPI documentation works differently.
They just use
SQLALCHEMY_DATABASE_URL = "sqlite:///./test.db"
engine = create_engine(
SQLALCHEMY_DATABASE_URL, connect_args={"check_same_thread": False}
)
Base.metadata.create_all(bind=engine)
which creates the database.
This works, because SQLite doesn't run as a server. It's just one file that represents the full database, and if the file doesn't exist, the sqlite database adapter will assume that the database is just empty, and create a new file for you. PostgreSQL doesn't work like this though.

How to keep Snowflake connection alive in python

I have stored my database in the Snowflake server and using https://pypi.org/project/snowflake-connector-python/ I am accessing the database and doing database queries from flask server as below:
ctx = snowflake.connector.connect(
user='username',
password='pass',
account='account',
client_session_keep_alive=True
)
cs = ctx.cursor()
try:
cs.execute("SELECT current_version()")
one_row = cs.fetchone()
print("Successfully connected to snowflake version: {}".format(one_row[0]))
cs.close()
except Exception as e:
print("Snowflake connection error: {}".format(e))
cs.close()
I have defined the snowflake connection session as a global ctx variable to be accessible from any flask api function.
After starting the flask server, everything works fine but if no api calls are made for continuously few hours then it throws an error 'snowflake.connector.errors.ProgrammingError: 390114 (08001): Authentication token has expired. The user must authent
icate again.'
As you can see i have kept the 'client_session_keep_alive=True' parameter in the snowflake connect api to keep the session alive but still somehow this fails.
I explored about this issue but did not get any conclusive information.
So i want to know how can i keep the database connection session alive or do I have to create new connection session for each query?
Any suggestions will be very helpfull.

Google App Engine and Cloud SQL: Lost connection to MySQL server at 'reading initial communication packet' SQL 2nd Gen

I'm getting an error similar to other posts in this subject.
I tried switching from 1st gen to 2nd gen SQL server (both on us-central1), but it still doesn't work.
I copied my CLOUDSQL_PROJECT from the url on the top of my project.
I copied my CLOUDSQL_INSTANCE from the proprieties part in the SQL page.
In my main.py, I'm trying to run Google sample code, and it doesn't work (locally it does, of course):
if os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine/'):
db = MySQLdb.connect(
unix_socket='/cloudsql/{}:{}'.format(
CLOUDSQL_PROJECT,
CLOUDSQL_INSTANCE),
user=user,passwd=password)
# When running locally, you can either connect to a local running
# MySQL instance, or connect to your Cloud SQL instance over TCP.
else:
db = MySQLdb.connect(host=host,user=user,passwd=password)
cursor = db.cursor()
cursor.execute('SHOW VARIABLES')
for r in cursor.fetchall():
self.response.write('{}\n'.format(r))

The documentation is slightly outdated. You should be able to always use the "Instance connection name" property from the SQL properties page to construct the unix socket path; just append that value after the "/cloudsql/" prefix.
For second generation, the connection format is project:region:name. In your example, it maps to "hello-world-123:us-central1:sqlsomething3", and the unix socket path is "/cloudsql/hello-world-123:us-central1:sqlsomething3".

Zombie Connection in SQLAlchemy

DBSession = sessionmaker(bind=self.engine)
def add_person(name):
s = DBSession()
s.add(Person(name=name))
s.commit()
Everytime I run add_person() another connection is created with my postgreSQL DB.
Looking at:
SELECT count(*) FROM pg_stat_activity;
I see the count going up, until I get a Remaining connection slots are reserved for non-replication superuser connections error.
How do I kill those connections? Am I wrong in opening a new session everytime I want to add a Person record?

In general, you should keep your Session object (here DBSession) separate from any functions that make changes to the database. So in your case you might try something like this instead:
DBSession = sessionmaker(bind=self.engine)
session = DBSession() # create your session outside of functions that will modify database
def add_person(name):
session.add(Person(name=name))
session.commit()
Now you will not get new connections every time you add a person to the database.

How do I use pymongo to connect to an existing document collection/db?

On the command line, this works:
$ mongo
> show dbs
mydatabase 1.0GB
However, this does not:
$ python
>>> import pymongo
>>> connection = pymongo.MongoClient()
>>> connection.mydatabase.find()
I read through docs here:
http://api.mongodb.org/python/current/tutorial.html
But do not understand how to either...
connect to an existing database (using pymongo)
query what databases exist in the mongodb connection.
Why can't I access my database?

Connect to an existing database
import pymongo
from pymongo import MongoClient
connection = MongoClient()
db = connection.mydatabase
List existing databases
import pymongo
from pymongo import MongoClient
connection = MongoClient()
# connection.database_names() # depreciated
connection.list_database_names()

The question implies user has a local MongoDB. However I found this question trying to connect to a remote MongoDB. I think the tutorial is worth mentioning (no other answer here mentioned how I can specify the host and the port)
The above code will connect on the default host and port. We can also specify the host and port explicitly, as follows:
client = MongoClient('localhost', 27017)
Or use the MongoDB URI format:
client = MongoClient('mongodb://localhost:27017/')

show dbs and find() are totally different commands as such you cannot compare the two.
connection.mydatabase.find()
Will actually do nothing because you cannot find() documents on database level. You are probably looking for:
cursor = connection.mydatabase.mycol.find()
I am no Python programmer but something like that and the foreach the cursor var to get your data.
As an added note you will want to replace mycol with the collection name that contains your documents.
As for querying for a list of databases you can do something like:
databases = connection.mydatabase.command({'listDatabases': 1});
As shown here: http://docs.mongodb.org/manual/reference/command/listDatabases/#listDatabases
However again I am no Python programmer but this should get you started.

On the python command line:
import pymongo
from pymongo import MongoClient
connection = MongoClient() ## connects by default to db at localhost:27017
connection.database_names() ## python binding equivalent to show dbs.
Although there doesn't seem to be a wealth of examples, it appears that the bindings are pretty complete within the Python Driver API Documentation.

database_names() is deprecated. One can use list_database_names() instead.
mongo_db_url will be something like "mongodb://localhost:27017/". 27017 is deafult port number, replace suitably.
from pymongo import MongoClient
client = MongoClient(<mongo_db_url>)
#or client = MongoClient('localhost', 27017)
client.list_database_names()