We Keep Coding

Having Trouble Connecting to Cloud SQL (PostgreSQL) using Python's SQLALCHEMY

I set up the Cloud SQL instance on Google Cloud Platform and followed the official instructions, but don't seem to be able to connect to the Cloud SQL instance. When I try to do a sanity check and access the PostgreSQL db through Cloud Shell, I'm able to connect successfully though.
Could someone please help - I would be much obliged.
Code:
from sqlalchemy import create_engine
engine = create_engine('postgresql+psycopg2://<user>:<pass>#<public IP Address/<table>')
engine.connect()
Error:
Is the server running on host "XX.XX.XXX.XX" and accepting
TCP/IP connections on port XXXX?

I found another way to connect to a PostgreSQL GCP instance without using the Cloud SQL Proxy.
Code:
import sqlalchemy
username = '' # DB username
password = '' # DB password
host = '' # Public IP address for your instance
port = '5432'
database = '' # Name of database ('postgres' by default)
db_url = 'postgresql+psycopg2://{}:{}#{}:{}/{}'.format(
username, password, host, port, database)
engine = sqlalchemy.create_engine(db_url)
conn = engine.connect()
I whitelisted my IP address before trying to connect. (https://cloud.google.com/sql/docs/postgres/connect-external-app#appaccessIP)

Use the Cloud SQL proxy to connect to Cloud SQL from external applications.
In order to achieve this please follow the relevant documentation.
The steps described would consist of:
Enabling the Cloud SQL Admin API on your Cloud Console.
Installing the relevant proxy client according to your OS.
Use any of the available methods to authenticate the Cloud SQL Proxy.
Invoke the proxy with ./cloud_sql_proxy -instances=INSTANCE_CONNECTION_NAME=tcp:5432 & ond your terminal and connect the proxy by changing your code and using SQLALCHEMY:
from sqlalchemy import create_engine
engine = create_engine('postgresql+psycopg2://DATABASE_USER:PASSWORD#localhost:5432/')
NOTE: the code above assumes you are not trying to connect to the proxy in a production environment and are using an authenticated Cloud SDK client in order to connect to the proxy.

This worked to me using the Cloud SQL Proxy on my personal computer and uploading the code to Google App Engine standard.
db_user = os.environ.get('CLOUD_SQL_USERNAME')
db_pass = os.environ.get('CLOUD_SQL_PASSWORD')
db_name = os.environ.get('CLOUD_SQL_DATABASE_NAME')
db_connection_name = os.environ.get('CLOUD_SQL_CONNECTION_NAME')
if os.environ.get('GAE_ENV') == 'standard':
db_uri = f'postgresql+psycopg2://{db_user}:{db_pass}#/{db_name}?host=/cloudsql/{db_connection_name}'
else:
db_uri = f'postgresql+psycopg2://{db_user}:{db_pass}#127.0.0.1:1234/{db_name}'
app = Flask(__name__)
app.config["SQLALCHEMY_DATABASE_URI"] = db_uri
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False

Depending on the database client library, the socket (/cloudsql/INSTANCE_CONNECTION_NAME/.s.PGSQL.5432) needs to be specified.
The docs have this example for SQLAlchemy:
db_user = os.environ["DB_USER"]
db_pass = os.environ["DB_PASS"]
db_name = os.environ["DB_NAME"]
db_socket_dir = os.environ.get("DB_SOCKET_DIR", "/cloudsql")
cloud_sql_connection_name = os.environ["CLOUD_SQL_CONNECTION_NAME"]
pool = sqlalchemy.create_engine(
# Equivalent URL:
# postgresql+pg8000://<db_user>:<db_pass>#/<db_name>
# ?unix_sock=<socket_path>/<cloud_sql_instance_name>/.s.PGSQL.5432
sqlalchemy.engine.url.URL.create(
drivername="postgresql+pg8000",
username=db_user, # e.g. "my-database-user"
password=db_pass, # e.g. "my-database-password"
database=db_name, # e.g. "my-database-name"
query={
"unix_sock": "{}/{}/.s.PGSQL.5432".format(
db_socket_dir, # e.g. "/cloudsql"
cloud_sql_connection_name) # i.e "<PROJECT-NAME>:<INSTANCE-REGION>:<INSTANCE-NAME>"
}
),
**db_config
)
Be aware that this example is with pg8000 that uses unix_sock instead of unix_socket as socket identifier.

Cannot connect to cleardb from flask app running on heroku

I'm learning flask etc. and created a simple app locally. Everything worked fine with a local mysql database in xammp. I exported this and imported into clearDB on heroku and deployed my app. The app runs, but whenever I try to access a page relying on the database in the logs I see this error.
pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on 'localhost' ([Errno 111] Connection refused)")
I ran the app locally, but using the clearDB database, and it was able to retrieve the data no problem. Everything is configured in the file to access based on the URL details from heroku.
Code is below. I'm sure there are a ton of other issues, but as I say I'm learning and got all this from a tutorial.
#instaniation
app = Flask(__name__)
app.secret_key= 'secret123'
#config
app.config['MYSQL_HOST'] = 'us-cdbr-iron-east-02.cleardb.net'
app.config['MYSQL_USER'] = 'redacted'
app.config['MYSQL_PASSWORD'] = 'redacted'
app.config['MYSQL_DB'] = 'heroku_0c6326c59d8b6e9'
app.config['MYSQL_CURSORCLASS'] = 'DictCursor' #a cursor is a connection to let us run methods for queries. We set is as a dictionary
#init MySQL
mysql = MySQL(app,cursorclass=DictCursor)
#url endpoint
#app.route('/')
#function for starting app
def index():
#can directly return HTML
return render_template('index.html')
#app.route('/about')
def about():
return render_template('about.html')
#many articles
#app.route('/articles')
def articles():
#create Cursor
cur = mysql.get_db().cursor() #to execute commands
cur.execute("USE myflaskapp")
#get articles
result = cur.execute("SELECT * FROM articles")
#should retrieve all as a dictionary
articles = cur.fetchall()
if result > 0:
return render_template('articles.HTML', articles=articles)
else:
msg= 'No Articles Found'
return render_template('articles.HTML', msg=msg)
cur.close()

As you can see from the error: pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on 'localhost' ([Errno 111] Connection refused)") MySQL tried to connect to localhost server, which mean, its doesnt load your config variables.
If you are using flask extension from https://flask-mysql.readthedocs.io/en/latest/# to load MySQL then you are probably forgot to init your application (pip install flask-mysql)
mysql.init_app(app)
The head of you code should now look like this
And watch for the config variables names, the extension using slightly different convention
from flask import Flask
from flaskext.mysql import MySQL
#instaniation
app = Flask(__name__)
app.secret_key= 'secret'
#config
app.config['MYSQL_DATABASE_USER'] = 'user'
app.config['MYSQL_DATABASE_PASSWORD'] = 'password'
app.config['MYSQL_DATABASE_DB'] = 'db'
app.config['MYSQL_DATABASE_HOST'] = 'host'
#init MySQL
mysql = MySQL()
mysql.init_app(app)

How to instantiate DB object from configuration?

I instantiate Mongo Client as below. It works fine. However I am trying to read the DB name (primer here) from the configuration. How do I do that?
from pymongo import MongoClient
client = MongoClient()
db = client.primer # want to read "primer" string from a variable
coll = db.dataset

You could do:
db_name = 'primer'
db = getattr(client, db_name)

if you are trying to connect to only one database you can specify the dbname while creating the db object itself
dbname = "primer"
db = MongoClient()[dbname]

How to use multiple Mongodb in flask

I have two mysql database one is localhost and another is in server now, am going to create simple app in python using flask for that application i would like to connect the both mysql DB (local and server).
Any one please suggest how to connect multiple DB into flask.
app = Flask(__name__)
client = MongoClient()
client = MongoClient('localhost', 27017)
db = client.sampleDB1
Sample code if possible.
Thanks

I had the same issue, finally figured it out.
Instead of using
client = MongoClient()
client = MongoClient('localhost', 27017)
db = client.sampleDB1
Delete all that and try this:
mongo1 = PyMongo(app, uri = 'mongodb://localhost:27017/Database1')
mongo2 = PyMongo(app, uri = 'mongodb://localhost:27017/Database2')
Then, when you want to call a particular database you can use:
#app.route('/routenamedb1', methods=['GET'])
def get_data_from_Database1():
Database1 = mongo1.db.CollectionName ##Notice I use mongo1,
#If I wanted to access database2 I would use mongo2
#Walk through the Database for DC to
for s in Database1.find():
#Modifying code
return data

#This technique can be used to connect to multiple databases or database servers:
app = Flask(__name__)
# connect to MongoDB with the defaults
mongo1 = PyMongo(app)
# connect to another MongoDB database on the same host
app.config['MONGO2_DBNAME'] = 'dbname_two'
mongo2 = PyMongo(app, config_prefix='MONGO2')
# connect to another MongoDB server altogether
app.config['MONGO3_HOST'] = 'another.host.example.com'
app.config['MONGO3_PORT'] = 27017
app.config['MONGO3_DBNAME'] = 'dbname_three'
mongo3 = PyMongo(app, config_prefix='MONGO3')

create model.py and separate instances of 2 databases inside it, then in app.py:
app = Flask(__name__)
app.config['MODEL'] = model.my1st_database()
app.config['MODEL2'] = model.my2nd_database()
works for me :)

How do I connect as sysdba for oracle db using SQLALchemy

I am using sqlalchemy with flask
and I want to connect to oracle DB as sysdba
SQLALCHEMY_DATABASE_URI ='oracle+cx_oracle://sys:abc#DBNAME[mode=SYSDBA]'
This doesnt work and gives me a
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config.from_object('config')
db = SQLAlchemy(app)
from app import views,models
and I use this db object later. But I am not able to figure out how to write the
SQLALCHEMY_DATABASE_URI to login as sysdba
I also tried
CONN = cx_Oracle.connect('sys/abc', dsn='DBNAME', mode = cx_Oracle.SYSDBA)
SQLALCHEMY_DATABASE_URI = CONN
But that also doesnt work.
I get ORA-12154: TNS: could not resolve the connect identifier specified” .. also If I remove mode=SYSDBA I get ORA-28009 connection as SYS should be as SYSDBA

Your dsn parameter is wrong. You must also separate the user and password parameters. Try this (it's working for me):
dsn_tns = cx_Oracle.makedsn('host', port, 'sid')
CONN = cx_Oracle.connect('sys', 'abc', dsn_tns, mode=cx_Oracle.SYSDBA)
For more info see cx_Oracle.connect constructor.