I tried to push my Flask app to heroku and my app uses Postgres as db. However, when I tried to migrate db on heroku using this command
heroku run python manage.py db init --app app
It gives this "too many clients" error for both psycopg2 and sqlalchchemy.
Running python manage.py db migrate on ⬢ petscom... up, run.5707 (Hobby)
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2285, in _wrap_pool_connect
return fn()
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 303, in unique_connection
return _ConnectionFairy._checkout(self)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 773, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 492, in checkout
rec = pool._do_get()
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 238, in _do_get
return self._create_connection()
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 657, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
exc_value, with_traceback=exc_tb,
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
raise exception
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 652, in __connect
connection = pool._invoke_creator(self)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 488, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/app/.heroku/python/lib/python3.6/site-packages/psycopg2/__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL: sorry, too many clients already
FATAL: sorry, too many clients already
After a bit googling, I tried:
heroku pg:killall
heroku restart
None of those works.....
And I checked heroku pg, it shows 0/20 connections......
Some said dont put db connection in loop, but I have to check if some data is in db. Here's my code:
try:
connection = psycopg2.connect(user="postgres",
password="pass",
host="127.0.0.1",
port="5432",
database="pets")
cursor = connection.cursor()
select_all = "select * from pets"
cursor.execute(select_all)
# Check if api data in db and save to db
for index in range(len(myData)):
# if not exist in db => save to db
key = myData[index]["animal_id"]
exists = check_exists(key, cursor)
if(exists):
pass
else:
doSomething()
I can connect to my local postgres db but not the db on heroku.
How can I deal with this problem? Any advise would help :) Thanks!
I had a similar problem. I was able to fix it by setting SQLALCHEMY_TRACK_MODIFICATIONS=False in the environment, destroying the database, and creating a new one.
You can destroy it by going to https://data.heroku.com/datastores, opening the database, going to Settings, and selecting 'Destroy Database'.
Then provision a new one as you normally would.
Related
When using an SQLAlchemy engine for GeoPandas read_postgis() method to read data from a PostgreSQL/PostGIS database, there are plenty of busy database slots which stay open for nothing, often showing simple COMMIT or ROLLBACK query statements.
Here's a test.py code snippet which represents this problem:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import os
os.environ['USE_PYGEOS'] = '0'
import geopandas as gpd
from sqlalchemy import create_engine, text
def foo():
engine = create_engine("postgresql://postgres:password#db:5432/postgres")
with engine.begin() as connection:
gdf0 = gpd.read_postgis(
sql = text("WITH cte AS (SELECT * FROM public.cities) SELECT *, ST_Buffer(geom,0.0001) FROM public.cities WHERE name ILIKE 'bog';"),
con=connection,
geom_col='geom',
crs=4326,
)
gdf1 = gpd.read_postgis(
sql = text("WITH cte AS (SELECT * FROM public.cities) SELECT *, ST_Buffer(geom,0.0002) FROM public.cities WHERE name ILIKE 'bra';"),
con=connection,
geom_col='geom',
crs=4326,
)
gdf2 = gpd.read_postgis(
sql = text("WITH cte AS (SELECT * FROM public.cities) SELECT *, ST_Buffer(geom,0.0003) FROM public.cities WHERE country ILIKE 'ven';"),
con=connection,
geom_col='geom',
crs=4326,
)
i=-1
while i < 100:
i+=1
foo()
As you can see, I'm using engine.begin(), which is known as "Connect and Begin Once" in the SQLAlchemy documentation:
A convenient shorthand form for the above “begin once” block is to use the Engine.begin() method at the level of the originating Engine object, rather than performing the two separate steps of Engine.connect() and Connection.begin(); the Engine.begin() method returns a special context manager that internally maintains both the context manager for the Connection as well as the context manager for the Transaction normally returned by the Connection.begin() method:
For convenience, here is also a database initialization script:
psql -U postgres -d postgres -c "CREATE TABLE IF NOT EXISTS cities (
id int PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
name text,
country text,
geom geometry(Point,4326)
);
INSERT INTO public.cities (name, country, geom) VALUES
('Buenos Aires', 'Argentina', ST_SetSRID(ST_Point(-58.66, -34.58), 4326)),
('Brasilia', 'Brazil', ST_SetSRID(ST_Point(-47.91,-15.78), 4326)),
('Santiago', 'Chile', ST_SetSRID(ST_Point(-70.66, -33.45), 4326)),
('Bogota', 'Colombia', ST_SetSRID(ST_Point(-74.08, 4.60), 4326)),
('Caracas', 'Venezuela', ST_SetSRID(ST_Point(-66.86,10.48), 4326));"
I usually run this Python code in multiple parallel instances as a dockerized application.
When using the following SQL snippet while the app is running, e.g. from pgAdmin4, I can see plenty of idle connections which are not properly closed. They stay in an idle state as long as the app is running, showing simple COMMIT queries. (In the whole application, there may also be some idle ROLLBACK queries).
SELECT state, query, *
FROM pg_stat_activity
WHERE application_name NOT ILIKE '%pgAdmin 4%'
Here is the result of the previous query:
If by any chance there are too much of those pending idle connections, I can also face the following error, certainly because there are no more available "slots" in the database for extra transactions to take place:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
self._dbapi_connection = engine.raw_connection()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3269, in raw_connection
return self.pool.connect()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 452, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1255, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 168, in _do_get
with util.safe_reraise():
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 166, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 393, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 678, in __init__
self.__connect()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 902, in __connect
with util.safe_reraise():
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 898, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 640, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 580, in connect
return self.loaded_dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL: sorry, too many clients already
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/test.py", line 42, in <module>
foo()
File "/app/test.py", line 19, in foo
with engine.begin() as connection:
File "/usr/local/lib/python3.10/contextlib.py", line 135, in __enter__
return next(self.gen)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3209, in begin
with self.connect() as conn:
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3245, in connect
return self._connection_cls(self)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 147, in __init__
Connection._handle_dbapi_exception_noconnection(
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2410, in _handle_dbapi_exception_noconnection
raise sqlalchemy_exception.with_traceback(exc_info[4]) from e
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
self._dbapi_connection = engine.raw_connection()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3269, in raw_connection
return self.pool.connect()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 452, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1255, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 168, in _do_get
with util.safe_reraise():
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 166, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 393, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 678, in __init__
self.__connect()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 902, in __connect
with util.safe_reraise():
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 898, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 640, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 580, in connect
return self.loaded_dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: sorry, too many clients already
(Background on this error at: https://sqlalche.me/e/20/e3q8)
This error regularly prevents my application from working correctly with many errors, e.g.: psycopg2.OperationalError: FATAL: sorry, too many clients already.
I conclude that the Python code above is not perfectly written. Certainly because I didn't fully understand how the SQLAlchemy engine actually works.
Hence my question: could you explain what's wrong with this code and, most importantly, how could I properly close these queries?
Version info:
Python 3.9.7
geopandas==0.12.2
psycopg2==2.9.5
SQLAlchemy==2.0.1
If I recall correctly, create_engine creates a connection pool by default.
You can adjust idle connections by adjusting the pool parameters you give to create_engine.
See Connection pooling in SQLAlchemy documentation.
I cannot connect to Hive database with the usage of Flask and SQLAlchemy. Below see my configuration in specific files:
.flaskenv file
DATABASE_URL="hive://<host>:10000/<db_name>"
config.py file
class ProductionConfig(Config):
SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL')
SQLALCHEMY_ENGINE_OPTIONS = {connect_args={'auth': 'KERBEROS', 'kerberos_service_name': 'hive'}}
database.py file
import os
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
db = SQLAlchemy()
migrate = Migrate()
def register_db(app):
db.init_app(app)
migrate.init_app(app, db)
app.db = db
But when I try to run the application with flask db run or when I try to create migration script with the commnad flask db migrate I get the error [1] -> it seems that connection parameters are not sent into the engine correctly.
[1]
...
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 2263, in connect
return self._connection_cls(self, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 104, in __init__
else engine.raw_connection()
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 2370, in raw_connection
self.pool.unique_connection, _connection
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 304, in unique_connection
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 778, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 495, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/impl.py", line 241, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 440, in __init__
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 661, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
with_traceback=exc_tb,
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 656, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 493, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/dist-packages/pyhive/hive.py", line 94, in connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/pyhive/hive.py", line 192, in __init__
self._transport.open()
File "/usr/local/lib/python3.6/dist-packages/thrift_sasl/__init__.py", line 96, in open
message=("Bad status: %d (%s)" % (status, payload)))
thrift.transport.TTransport.TTransportException: Bad status: 3 (b'Unsupported mechanism type PLAIN')
Python package versions:
alembic==1.4.3
Flask==1.1.2
Flask-Migrate==2.5.3
Flask-RESTful==0.3.8
Flask-SQLAlchemy==2.4.4
PyHive==0.6.2
python-dotenv==0.15.0
sasl==0.2.1
SQLAlchemy==1.3.20
ssh-import-id==5.7
thrift==0.13.0
thrift-sasl==0.4.2
Could anybody help me please?
UPDATE:
I re-forked the GoogleCloudPlatform example project and tried again. Suddenly, it's working.
Half of my problem had to do with the fact that my target project uses Flask-SQLAlchemy. For that, I needed to use the MySQLdb dialect as shown in this answer:
https://stackoverflow.com/a/10900826/4455571
I'm still not sure why I couldn't get the GoogleCloudPlatform example to work the first time and why it suddenly started working after I re-forked.
Original post:
I am trying to connect a container running on Cloud Run to Cloud SQL using the following guide:
https://cloud.google.com/sql/docs/mysql/connect-run?hl=en_US
I have made sure to do the following:
Enable the API for my project
Add the "Cloud SQL Client" role to my
service account REDACTED-compute#developer.gserviceaccount.com
However, the connection fails with the following error:
File "/usr/local/lib/python3.8/site-packages/pymysql/connections.py",
line 630, in connect raise exc sqlalchemy.exc.OperationalError:
(pymysql.err.OperationalError) (2003, "Can't connect to MySQL server
on 'localhost' ([Errno 2] No such File or directory)")
I made a second attempt using the sample code from GitHub:
https://github.com/GoogleCloudPlatform/python-docs-samples/tree/master/cloud-sql/mysql/sqlalchemy
These are the steps I followed:
Mirrored the repository and made it private
Edited app.yaml to add my credentials similar to the following (I didn't wrap the values in quotes - should I?):
env_variables:
CLOUD_SQL_CONNECTION_NAME: my-project-270323:us-central1:database
DB_USER: root
DB_PASS: areallygreatpassword
DB_NAME: database
Connected the repo to Cloud Build and triggered a new build
Deployed my container in a new service on Cloud Run making sure to select by database under Connections > Cloud SQL connections
I get the same error. What am I doing wrong?
EDIT
Here's the full error dump:
Traceback (most recent call last):()
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2446, in wsgi_app response = self.full_dispatch_request()()
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1944, in full_dispatch_request self.try_trigger_before_first_request_functions()()
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1992, in try_trigger_before_first_request_functions func()()
File "/app/main.py", line 81, in create_tables with db.connect() as conn:()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2209, in connect return self._connection_cls(self, **kwargs)()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 103, in __init__ else engine.raw_connection()()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2306, in raw_connection return self._wrap_pool_connect(()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2279, in _wrap_pool_connect Connection._handle_dbapi_exception_noconnection(()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1547, in _handle_dbapi_exception_noconnection util.raise_from_cause(sqlalchemy_exception, exc_info)()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause)()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 152, in reraise raise value.with_traceback(tb)()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2276, in _wrap_pool_connect return fn()()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 303, in unique_connection return _ConnectionFairy._checkout(self)()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 773, in _checkout fairy = _ConnectionRecord.checkout(pool)()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 492, in checkout rec = pool._do_get()()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 139, in _do_get self._dec_overflow()()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__ compat.reraise(exc_type, exc_value, exc_tb)()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 153, in reraise raise value()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 136, in _do_get return self._create_connection()()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 308, in _create_connection return _ConnectionRecord(self)()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 437, in __init__ self.__connect(first_connect_check=True)()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 652, in __connect connection = pool._invoke_creator(self)()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect return dialect.connect(*cargs, **cparams)()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 489, in connect return self.dbapi.connect(*cargs, **cparams)()
File "/usr/local/lib/python3.8/site-packages/pymysql/__init__.py", line 94, in Connect return Connection(*args, **kwargs)()
File "/usr/local/lib/python3.8/site-packages/pymysql/connections.py", line 325, in __init__ self.connect()()
File "/usr/local/lib/python3.8/site-packages/pymysql/connections.py", line 630, in connect raise exc sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'localhost' ([Errno 2] No such()
File or directory)")
Also, I did this via the web console, not the CLI. Here are the settings I passed to the example project provided by Google:
drive.google.com/open?id=17nl_rQVTU2ZirCEu64bjfe90zGweg3a6
drive.google.com/open?id=1_Riy1HNSPvZZUGl4tJ0Z8puYuuMbUMPH
The question includes some wide-ranging research, nicely done. Addressing a couple points directly:
The app.yaml file is specific to App Engine, and is a method to pass Environment Variable configuration to your app. In Cloud Run, you use the --update-env-vars flag on deploy.
You may also be missing a flag in your service deploy operation: --add-cloudsql-instances INSTANCE-CONNECTION-NAME
These steps together:
gcloud run deploy SERVICE --image gcr.io/PROJECT/SERVICE \
--add-cloudsql-instances my-project-270323:us-central1:database
--update-env-vars CLOUD_SQL_CONNECTION_NAME=my-project-270323:us-central1:database \
--update-env-vars DB_USER=root \
--update-env-vars DB_PASS=areallygreatpassword \
--update-env-vars DB_NAME=database
Note that database in the INSTANCE_CONNECTION_NAME is the Cloud SQL instance, while database as the DB_NAME is a database created within that instance.
The configuration values need to be used as part of creating your database connection.
You can see more of the connection between these environment variables and creating the connection object in the complete sample code on Github.
I'm trying to create a small app with Python and MySQL.
I managed to connect Python and MySQL using the MySQL connection. But now when I try to run this script to actually connect to the database:
import mysql.connector
mydb = mysql.connector.connect(
passwd="12345",
db="DB",
host="hostname",
user="root"
)
print(mydb)
I get a lot of errors like:
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\network.py", line 485, in open_connection
socket.SOL_TCP)
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\socket.py", line 748, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed
File "c:/Users/Krystian/Desktop/DB/db.py", line 7, in <module>
user="root"
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\__init__.py", line 179, in connect
return MySQLConnection(*args, **kwargs)
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\connection.py", line 94, in __init__
self.connect(**kwargs)
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\abstracts.py", line 722, in connect
self._open_connection()
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\connection.py", line 207, in _open_connection
self._socket.open_connection()
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\network.py", line 501, in open_connection
errno=2003, values=(self.get_address(), _strioerror(err)))
mysql.connector.errors.InterfaceError: 2003: Can't connect to MySQL server on 'hostname:3306' (11001 getaddrinfo failed)
I tried other ways of conecting to Database but they didn't work and I'm not really sure what to do now.
Thanks in advance for any help!
EDIT1
As Juergen mentioned, after fixing the hostname most of the errors above are gone, but new ones showed up:
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\connection.py", line 94, in __init__
self.connect(**kwargs)
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\abstracts.py", line 722, in connect
self._open_connection()
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\connection.py", line 211, in _open_connection
self._ssl)
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\connection.py", line 141, in _do_auth
auth_plugin=self._auth_plugin)
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\protocol.py", line 102, in make_auth
auth_data, ssl_enabled)
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\protocol.py", line 58, in _auth_response
auth = get_auth_plugin(auth_plugin)(
File "C:\Users\Krystian\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\authentication.py", line 191, in get_auth_plugin
"Authentication plugin '{0}' is not supported".format(plugin_name))
mysql.connector.errors.NotSupportedError: Authentication plugin 'caching_sha2_password' is not supporte
I guess, your computer (where the database runs) is not named "hostname":
host="hostname",
If it is the local machine, try localhost. "hostname" is just a placeholder inside example coding. You always have to replace such.
About your edit:
Looks as your db server is misconfigured. Maybe you need that plugin that is named in the error message.
try this :
mysql.connector.connect(
host="localhost",
user="user_name",
passwd="db_password",
database="db_name")
I am trying to run following code snippet in python to connect to oracle, but constantly running into following error. I have tried a lot of combinations but it doesn't seem to work. I understand the error, but don't understand what is incompatible here.
Has anyone come across this issue? How do I fix it?
File "", line 1, in File "/workplace/applications/python2.7/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1613, in execute
connection = self.contextual_connect(close_with_result=True) File "/workplace/applications/python2.7/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1661, in contextual_connect
self.pool.connect(), File "/workplace/applications/python2.7/lib/python2.7/site-packages/sqlalchemy/pool.py", line 326, in connect
return _ConnectionFairy(self).checkout() File "/workplace/applications/python2.7/lib/python2.7/site-packages/sqlalchemy/pool.py", line 485, in __init__
rec = self._connection_record = pool._do_get() File "/workplace/applications/python2.7/lib/python2.7/site-packages/sqlalchemy/pool.py", line 770, in _do_get
return self._create_connection() File "/workplace/applications/python2.7/lib/python2.7/site-packages/sqlalchemy/pool.py", line 279, in _create_connection
return _ConnectionRecord(self) File "/workplace/applications/python2.7/lib/python2.7/site-packages/sqlalchemy/pool.py", line 372, in __init__
self.connection = self.__connect() File "/workplace/applications/python2.7/lib/python2.7/site-packages/sqlalchemy/pool.py", line 433, in __connect
connection = self.__pool._creator() File "/workplace/applications/python2.7/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 80, in connect
return dialect.connect(*cargs, **cparams) File "/workplace/applications/python2.7/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 283, in connect
return self.dbapi.connect(*cargs, **cparams)
TypeError: expecting string, unicode or buffer object
from sqlalchemy.ext.declarative import declarative_base;
from sqlalchemy import create_engine;
engine = create_engine(u'oracle+cx_oracle://localhost:1521/orcl', echo=True)
result = engine.execute(u"select 1 from dual");
Setup:
Python 2.7
SqlAlchemy 0.9.7 and 0.8.7
Cx Oracle (latest version)
Oracle Database 10g Release 10.2.0.2.0
If you are running into this problem, most likely the cause is that you are not passing in arguments required by the underlying dbapi call.
In my case I added additional arguments of user, password and dsn to the create_engine call along with existing ones, which got passed to cx_oracle call and it worked.
something like this should work
create_engine(u'oracle+cx_oracle://localhost:1521/orcl', echo=True, user='<>', password='<>', dsn='<>')