I cannot connect to Hive database with the usage of Flask and SQLAlchemy. Below see my configuration in specific files:
.flaskenv file
DATABASE_URL="hive://<host>:10000/<db_name>"
config.py file
class ProductionConfig(Config):
SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL')
SQLALCHEMY_ENGINE_OPTIONS = {connect_args={'auth': 'KERBEROS', 'kerberos_service_name': 'hive'}}
database.py file
import os
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
db = SQLAlchemy()
migrate = Migrate()
def register_db(app):
db.init_app(app)
migrate.init_app(app, db)
app.db = db
But when I try to run the application with flask db run or when I try to create migration script with the commnad flask db migrate I get the error [1] -> it seems that connection parameters are not sent into the engine correctly.
[1]
...
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 2263, in connect
return self._connection_cls(self, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 104, in __init__
else engine.raw_connection()
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 2370, in raw_connection
self.pool.unique_connection, _connection
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 304, in unique_connection
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 778, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 495, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/impl.py", line 241, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 440, in __init__
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 661, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
with_traceback=exc_tb,
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 656, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 493, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/dist-packages/pyhive/hive.py", line 94, in connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/pyhive/hive.py", line 192, in __init__
self._transport.open()
File "/usr/local/lib/python3.6/dist-packages/thrift_sasl/__init__.py", line 96, in open
message=("Bad status: %d (%s)" % (status, payload)))
thrift.transport.TTransport.TTransportException: Bad status: 3 (b'Unsupported mechanism type PLAIN')
Python package versions:
alembic==1.4.3
Flask==1.1.2
Flask-Migrate==2.5.3
Flask-RESTful==0.3.8
Flask-SQLAlchemy==2.4.4
PyHive==0.6.2
python-dotenv==0.15.0
sasl==0.2.1
SQLAlchemy==1.3.20
ssh-import-id==5.7
thrift==0.13.0
thrift-sasl==0.4.2
Could anybody help me please?
Related
I'm having trouble figuring out the proper parameters for connecting to a Postgres database using Django on Heroku with Python 3.7.10 and sqlalchemy 1.3.23. I know that's not the latest release of sqlalchemy, but this started failing the other week when Python was updated from 3.7 so I'm puzzled as to what's going on. I'm finding sqlalchemy's documentation a little hard to navigate and it doesn't help that this stopped working when just Python was updated.
So basically in Python, the connection is being set in this line:
import records as records_client
DB_URL_FROM_ENV_VAR = 'postgresql+pg8000://user:password#host:5432/db?ssl=True'
db = records_client.Database(DB_URL_FROM_ENV_VAR)
And when I go to the implementation of Database, I get this:
class Database(object):
"""A Database connection."""
def __init__(self, db_url=None, **kwargs):
# If no db_url was provided, fallback to $DATABASE_URL.
self.db_url = db_url or DATABASE_URL
if not self.db_url:
raise ValueError('You must provide a db_url.')
self._engine = create_engine(self.db_url, **kwargs)
# Connect to the database.
self.db = self._engine.connect()
self.open = True
That appears to be a part of sqlalchemy.
The error I get when I run the script is: TypeError: connect() got an unexpected keyword argument 'ssl'.
I've tried changing the parameter to ssl_mode and also sslmode since I've seen SO posts and issues posted with similar questions, but neither of those changes have done anything. I've also tried adding another parameter to the Database call like so: db = records_client.Database(DB_URL_FROM_ENV_VAR, connect_args={'sslmode':'require'}) but all to no avail.
How do I configure the database URL and/or the connection attempt with SSL to connect to the database?
Here's a full stack trace with the following settings:
DB_URL_FROM_ENV_VAR = 'postgresql+pg8000://user:password#host:5432'
db = records_client.Database(DB_URL_FROM_ENV_VAR, connect_args={'sslmode':'require'})
Traceback (most recent call last):
File "appdir/manage.py", line 14, in <module>
execute_from_command_line(sys.argv)
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\django\core\management\__init__.py", line 381, in execute_from_command_line
utility.execute()
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\django\core\management\__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\django\core\management\base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\django\core\management\base.py", line 364, in execute
output = self.handle(*args, **options)
File "C:\Users\user\Github\django-app\appdir\apps\etl\management\commands\load.py", line 118, in handle
step.load(items=ds_items, period=period)
File "C:\Users\user\Github\django-app\appdir\apps\etl\models\job.py", line 89, in load
match_status = self.domain_source.load(*args, **kwargs)
File "C:\Users\user\Github\django-app\appdir\apps\etl\models\domain.py", line 90, in load
return self.load_profile.load(items=items, data=data, **kwargs)
File "C:\Users\user\Github\django-app\appdir\apps\etl\models\load_source.py", line 99, in load
db = records_client.Database(DB_URL_FROM_ENV_VAR, connect_args={'sslmode':'require'})
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\records.py", line 245, in __init__
self.db = self._engine.connect()
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\engine\base.py", line 2263, in connect
return self._connection_cls(self, **kwargs)
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\engine\base.py", line 104, in __init__
else engine.raw_connection()
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\engine\base.py", line 2370, in raw_connection
self.pool.unique_connection, _connection
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\engine\base.py", line 2336, in _wrap_pool_connect
return fn()
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\pool\base.py", line 304, in unique_connection
return _ConnectionFairy._checkout(self)
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\pool\base.py", line 778, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\pool\base.py", line 495, in checkout
rec = pool._do_get()
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\pool\impl.py", line 140, in _do_get
self._dec_overflow()
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\util\langhelpers.py", line 70, in __exit__
with_traceback=exc_tb,
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\util\compat.py", line 182, in raise_
raise exception
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\pool\impl.py", line 137, in _do_get
return self._create_connection()
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\pool\base.py", line 309, in _create_connection
return _ConnectionRecord(self)
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\pool\base.py", line 440, in __init__
self.__connect(first_connect_check=True)
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\pool\base.py", line 661, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\util\langhelpers.py", line 70, in __exit__
with_traceback=exc_tb,
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\util\compat.py", line 182, in raise_
raise exception
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\pool\base.py", line 656, in __connect
connection = pool._invoke_creator(self)
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\engine\strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "C:\Users\user\Github\django-app\venv-dev\lib\site-packages\sqlalchemy\engine\default.py", line 508, in connect
return self.dbapi.connect(*cargs, **cparams)
TypeError: connect() got an unexpected keyword argument 'sslmode'
I am currently trying to create an SQLite database using python.
here is my code
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////c/PStuff/friend.db'
db = SQLAlchemy(app)
class ExampleTable(db.Model):
id = db.Column(db.Integer,primary_key=True)
I have already created a database called 'friend.db' in the PStuff folder. It is currently empty.
In cmd i used this to create the database:sqlite3 friend.db
I then used a python shell:from test3 import db
Lastly when i use:
db.create_all()
i get this error:
Traceback (most recent call last):
File "c:\PStuff\lib\site-packages\sqlalchemy\engine\base.py", line 2285, in _wrap_pool_connect
return fn()
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 363, in connect
return _ConnectionFairy._checkout(self)
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 773, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 492, in checkout
rec = pool._do_get()
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\impl.py", line 238, in _do_get
return self._create_connection()
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 657, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "c:\PStuff\lib\site-packages\sqlalchemy\util\langhelpers.py", line 68, in __exit__
compat.raise_(
File "c:\PStuff\lib\site-packages\sqlalchemy\util\compat.py", line 178, in raise_
raise exception
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 652, in __connect
connection = pool._invoke_creator(self)
File "c:\PStuff\lib\site-packages\sqlalchemy\engine\strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "c:\PStuff\lib\site-packages\sqlalchemy\engine\default.py", line 490, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\PStuff\lib\site-packages\flask_sqlalchemy\__init__.py", line 1033, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "c:\PStuff\lib\site-packages\flask_sqlalchemy\__init__.py", line 1025, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "c:\PStuff\lib\site-packages\sqlalchemy\sql\schema.py", line 4320, in create_all
bind._run_visitor(
File "c:\PStuff\lib\site-packages\sqlalchemy\engine\base.py", line 2057, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "c:\PStuff\lib\contextlib.py", line 113, in __enter__
return next(self.gen)
File "c:\PStuff\lib\site-packages\sqlalchemy\engine\base.py", line 2049, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "c:\PStuff\lib\site-packages\sqlalchemy\engine\base.py", line 2251, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "c:\PStuff\lib\site-packages\sqlalchemy\engine\base.py", line 2288, in _wrap_pool_connect
Connection._handle_dbapi_exception_noconnection(
File "c:\PStuff\lib\site-packages\sqlalchemy\engine\base.py", line 1554, in _handle_dbapi_exception_noconnection
util.raise_(
File "c:\PStuff\lib\site-packages\sqlalchemy\util\compat.py", line 178, in raise_
raise exception
File "c:\PStuff\lib\site-packages\sqlalchemy\engine\base.py", line 2285, in _wrap_pool_connect
return fn()
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 363, in connect
return _ConnectionFairy._checkout(self)
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 773, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 492, in checkout
rec = pool._do_get()
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\impl.py", line 238, in _do_get
return self._create_connection()
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 657, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "c:\PStuff\lib\site-packages\sqlalchemy\util\langhelpers.py", line 68, in __exit__
compat.raise_(
File "c:\PStuff\lib\site-packages\sqlalchemy\util\compat.py", line 178, in raise_
raise exception
File "c:\PStuff\lib\site-packages\sqlalchemy\pool\base.py", line 652, in __connect
connection = pool._invoke_creator(self)
File "c:\PStuff\lib\site-packages\sqlalchemy\engine\strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "c:\PStuff\lib\site-packages\sqlalchemy\engine\default.py", line 490, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
I am very new to SQL, any help would be greatly appreciated.
Try with Replace
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////c/PStuff/friend.db'
with
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///c/PStuff/friend.db'
based on SQLALCHEMY Documentation
sqlite:////db_absolute_path
sqlite:///db_relative_path
for windows try this thing (if you are putting database in same dir)
file_path = os.path.abspath(os.getcwd())+"\\friend.db"
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///'+file_path
I want to add columns to a table in my database dynamically since I don't want to have to specify all columns when I set it up in the class
In order to solve this I am instead using alembic to add columns to a table but I am having problems.
In test.py script I have defined a class as below.
DATABASE_URL_dev = "postgresql+psycopg2://user:password#localhost:5432/testdb"
engine = create_engine(DATABASE_URL_dev)
Base = declarative_base(engine)
class my_class1(Base):
__tablename__ = "test1"
id = Column(Integer, primary_key=True)
list = Column(String)
def loadsession():
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
return session
session = loadsession()
Now I want to add a column to table test1 and I am thereby trying to use Alembic.
I followed the instructions at https://alembic.sqlalchemy.org/en/latest/tutorial.html and did the following:
I installed alembic by running pip install alembic
I created an environment using alembic init alembic
In the .ini file I set sqlalchemy.url = "postgresql+psycopg2://user:password#localhost:5432/testdb"
I created a migration script by running alembic revision -m "add a column "
I then edited the script by adding the following "op.add_column('test1', sa.Column('new_column', sa.String))" in upgrade()
I ran the first migration with "alembic upgrade head" This gave me the error: "sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) (Background on this error at: http://sqlalche.me/e/e3q8)"
Question
Have I set sqlalchemy.url correctly or what is else setting the OperationalError?
Entire traceback
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38-32\lib\runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\user\appdata\local\programs\python\python38-32\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\Scripts\alembic.exe\__main__.py", line 9, in <module>
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\config.py", line 577, in main
CommandLine(prog=prog).main(argv=argv)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\config.py", line 571, in main
self.run_cmd(cfg, options)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\config.py", line 548, in run_cmd
fn(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\command.py", line 298, in upgrade
script.run_env()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\script\base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\util\pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\util\compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "y\env.py", line 77, in <module>
run_migrations_online()
File "y\env.py", line 65, in run_migrations_online
with connectable.connect() as connection:
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2218, in connect
return self._connection_cls(self, **kwargs)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 103, in __init__
else engine.raw_connection()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2317, in raw_connection
return self._wrap_pool_connect(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2288, in _wrap_pool_connect
Connection._handle_dbapi_exception_noconnection(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 1554, in _handle_dbapi_exception_noconnection
util.raise_(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\util\compat.py", line 178, in raise_
raise exception
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2285, in _wrap_pool_connect
return fn()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 303, in unique_connection
return _ConnectionFairy._checkout(self)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 773, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 492, in checkout
rec = pool._do_get()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\impl.py", line 238, in _do_get
return self._create_connection()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 657, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\util\langhelpers.py", line 68, in __exit__
compat.raise_(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\util\compat.py", line 178, in raise_
raise exception
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 652, in __connect
connection = pool._invoke_creator(self)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\default.py", line 488, in connect
return self.dbapi.connect(*cargs, **cparams)
File "c:\users\c\appdata\local\programs\python\python38-32\lib\site-packages\psycopg2\__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError)
(Background on this error at: http://sqlalche.me/e/e3q8)
I am spinning up redash using helm on GKE with master/node version 1.12.10-gke.17 and istio 1.1.15
I use as database a GCP CloudSQL with postgres 9.6
When trying to execute the create_db script from the redash image (7.0.0.b18042) I encounter the following error
Traceback (most recent call last):
File "/app/manage.py", line 9, in <module>
manager()
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 716, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask/cli.py", line 345, in main
return AppGroup.main(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 534, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask/cli.py", line 229, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 534, in invoke
return callback(*args, **kwargs)
File "/app/redash/cli/database.py", line 31, in create_tables
db.create_all()
File "/usr/local/lib/python2.7/dist-packages/flask_sqlalchemy/__init__.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python2.7/dist-packages/flask_sqlalchemy/__init__.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/schema.py", line 4005, in create_all
tables=tables)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1939, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1932, in _optional_conn_ctx_manager
with self.contextual_connect() as conn:
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2123, in contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2162, in _wrap_pool_connect
e, dialect, self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception_noconnection
exc_info
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2158, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 400, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 788, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 529, in checkout
rec = pool._do_get()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1193, in _do_get
self._dec_overflow()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1190, in _do_get
return self._create_connection()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 347, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 474, in __init__
self.__connect(first_connect_check=True)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 671, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 106, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 412, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python2.7/dist-packages/psycopg2/__init__.py", line 130, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
(Background on this error at: http://sqlalche.me/e/e3q8)
I cannot figure out what is causing this, i.e. whether it is the CloudSQL's issue, or some istio misconfiguration or sth else...
The same error (with smaller stack trace) is reproduced when I exec into a container and try to manually connect to CloudSQL
>>> import psycopg2
>>> conn = psycopg2.connect(host="192.44.33.1",database="postgres", user="postgres", password="mypass")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/psycopg2/__init__.py", line 130, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Have you enabled a ServiceEntry within the Istio mesh to allow egress traffic to CloudSQL?
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: www.googleapis.com
spec:
hosts:
- www.googleapis.com
- oauth2.googleapis.com
ports:
- number: 443
name: https
protocol: HTTPS
resolution: DNS
location: MESH_EXTERNAL
Further reading here: https://github.com/istio/istio/issues/6593
I want to access an oracle database through sqlalchemy in flask development. An engine is created by sqlalchemy create_engine()
MemoryError is reported when engine.connect() is called.
If setting a wrong password in the linkword, it would report:
sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-01017: invalid username/password; logon denied.
With the correct password and user name, MemoryError would appear.
It is very strange when i copy the codes into jupyter or a new project (not a flask project)in pycharm, the codes work well. I wonder if it is possible the flask has problems.
from sqlalchemy import create_engine
#test_bp.route('/testtd')
def testtd():
#the linkword has no problem because these codes works in Jupyter or a new project
linkword = 'blablabla'
engine = create_engine(linkword)
#here i got a problem. memoryError was reported.
conn = engine.connect()
ret = conn.execute('select table_name from user_tables').fetchall()
print(ret)
conn.close()
Errors reported by pycharm are as follows:
File "E:\Projects\Python\DAM2\blueprints\test.py", line 19, in testtd
conn = engine.connect()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 2196, in connect
return self._connection_cls(self, **kwargs)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 103, in __init__
else engine.raw_connection()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 2296, in raw_connection
self.pool.unique_connection, _connection
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 2265, in _wrap_pool_connect
return fn()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 303, in unique_connection
return _ConnectionFairy._checkout(self)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 760, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 492, in checkout
rec = pool._do_get()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\impl.py", line 139, in _do_get
self._dec_overflow()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\util\langhelpers.py", line 68, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\util\compat.py", line 153, in reraise
raise value
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\impl.py", line 136, in _do_get
return self._create_connection()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 649, in __connect
).exec_once(self.connection, self)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\event\attr.py", line 287, in exec_once
self(*args, **kw)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\event\attr.py", line 297, in __call__
fn(*args, **kw)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\util\langhelpers.py", line 1443, in go
return once_fn(*arg, **kw)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\strategies.py", line 199, in first_connect
dialect.initialize(c)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\dialects\oracle\cx_oracle.py", line 832, in initialize
super(OracleDialect_cx_oracle, self).initialize(connection)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\dialects\oracle\base.py", line 1140, in initialize
super(OracleDialect, self).initialize(connection)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\default.py", line 297, in initialize
connection
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\dialects\oracle\base.py", line 1239, in _get_default_schema_name
connection.execute("SELECT USER FROM DUAL").scalar()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 982, in execute
return self._execute_text(object_, multiparams, params)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 1155, in _execute_text
parameters,
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 1468, in _handle_dbapi_exception
util.reraise(*exc_info)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\util\compat.py", line 153, in reraise
raise value
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
cursor.execute(statement, parameters)
MemoryError
how can i fix the codes?