Problem applying just created migration (Added db.Model) through Flask-Migrate (SQLAlchemy) for PostgresSQL DB.
The error itself:
sqlalchemy.exc.InternalError: (psycopg2.InternalError) cannot drop table parameter_subtype because other objects depend on it
Full error stack trace is:
INFO [alembic.autogenerate.compare] Detected removed foreign key (event_id) (id) on table stage_has_event
INFO [alembic.autogenerate.compare] Detected removed column 'stage_has_event.event_id'
Generating /app/migrations/versions/e224df1a4818_.py ... done
(venv) $ ./manage db upgrade
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade cadf22871ae0 -> e224df1a4818, empty message
Traceback (most recent call last):
File "/app/venv/bin/flask", line 11, in <module>
sys.exit(main())
File "/app/venv/lib/python2.7/site-packages/flask_cli/cli.py", line 502, in main
cli.main(args=args, prog_name=name)
File "/app/venv/lib/python2.7/site-packages/flask_cli/cli.py", line 369, in main
return AppGroup.main(self, *args, **kwargs)
File "/app/venv/lib/python2.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/app/venv/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/app/venv/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/app/venv/lib/python2.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/app/venv/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/app/venv/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/app/venv/lib/python2.7/site-packages/flask/cli.py", line 257, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/app/venv/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/app/venv/lib/python2.7/site-packages/flask_migrate/cli.py", line 134, in upgrade
_upgrade(directory, revision, sql, tag, x_arg)
File "/app/venv/lib/python2.7/site-packages/flask_migrate/__init__.py", line 259, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File "/app/venv/lib/python2.7/site-packages/alembic/command.py", line 254, in upgrade
script.run_env()
File "/app/venv/lib/python2.7/site-packages/alembic/script/base.py", line 427, in run_env
util.load_python_file(self.dir, 'env.py')
File "/app/venv/lib/python2.7/site-packages/alembic/util/pyfiles.py", line 81, in load_python_file
module = load_module_py(module_id, path)
File "/app/venv/lib/python2.7/site-packages/alembic/util/compat.py", line 141, in load_module_py
mod = imp.load_source(module_id, path, fp)
File "migrations/env.py", line 87, in <module>
run_migrations_online()
File "migrations/env.py", line 80, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/app/venv/lib/python2.7/site-packages/alembic/runtime/environment.py", line 836, in run_migrations
self.get_context().run_migrations(**kw)
File "/app/venv/lib/python2.7/site-packages/alembic/runtime/migration.py", line 330, in run_migrations
step.migration_fn(**kw)
File "/app/migrations/versions/e224df1a4818_.py", line 21, in upgrade
op.drop_table('parameter_subtype')
File "<string>", line 8, in drop_table
File "<string>", line 3, in drop_table
File "/app/venv/lib/python2.7/site-packages/alembic/operations/ops.py", line 1187, in drop_table
operations.invoke(op)
File "/app/venv/lib/python2.7/site-packages/alembic/operations/base.py", line 319, in invoke
return fn(self, operation)
File "/app/venv/lib/python2.7/site-packages/alembic/operations/toimpl.py", line 70, in drop_table
operation.to_table(operations.migration_context)
File "/app/venv/lib/python2.7/site-packages/alembic/ddl/impl.py", line 203, in drop_table
self._exec(schema.DropTable(table))
File "/app/venv/lib/python2.7/site-packages/alembic/ddl/impl.py", line 118, in _exec
return conn.execute(construct, *multiparams, **params)
File "/app/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/app/venv/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/app/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1009, in _execute_ddl
compiled
File "/app/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/app/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/app/venv/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/app/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/app`enter code here`/venv/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.InternalError: (psycopg2.InternalError) cannot drop table parameter_subtype because other objects depend on it
DETAIL: constraint parameter_parameter_subtype_id_fkey on table parameter depends on table parameter_subtype
HINT: Use DROP ... CASCADE to drop the dependent objects too.
[SQL: '\nDROP TABLE parameter_subtype']
This leads to table not created in PostgreSQL db and columns either.
Unsure what may have caused this problem and how to understand this Error message.
I did manually drop the tables and it helped. However how to be with migration.
Any Suggestions?
Flask-Migrate does not read in the database to see the dependencies between the objects. You can achieve a successful migration by reordering the drop_table in your migration file.
What you can do is modify the migration file like so:
op.drop_table("parameter_subtype")
op.drop_table("parameter")
And then run the upgrade command. Normally ordering the drop in this order should solve the problem.
I got the same error and what worked for me was to first drop the table in the database using the command:
$ DROP TABLE table_name;
And then in the migration file in the upgrade() function delete or comment out the line where it says:
op.drop_table('table_name')
Related
I am following the install instructions for the security utility opencve which imports data into a postgresql database:
https://docs.opencve.io/installation/manual/
I have followed all of the instructions including creating the 5G swap file. All tables create, no problems, however, the data import fails at this step:
opencve import-data
I receive the below errors referring to a timeout time and time again. Despite straw clutching which includes disabling the firewall, searching for any postgresql timeout parameters, creating a large swap file and attempting the install on another server, as well as verifying that psycopg2-binary is installed, I still hit the same problem in the same spot.
The full errors displayed are below but I believe the main part is:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 725, in do_executem
cursor.executemany(statement, parameters)
psycopg2.DatabaseError: could not receive data from server: Connection timed out
If I look at that psycopg2.py file, this is the block of code with line 725 being the last line:
def do_executemany(self, cursor, statement, parameters, context=None):
if self.psycopg2_batch_mode:
extras = self._psycopg2_extras()
extras.execute_batch(cursor, statement, parameters)
else:
cursor.executemany(statement, parameters)
If this is a genuine timeout somewhere I can't see where. The connection is good as the tables get created in the earlier part of the import-data command.
This is the section that is repeatedly failing.
#####################
Importing CPE list...
#####################
[*] Downloading https://nvd.nist.gov/feeds/xml/cpe/dictionary/official-cpe-dictionary_v2.3.xml.gz... (done in 1.31s).
[*] Parsing XML elements... (done in 269.254s).
[*] Creating list of mappings... (done in 330.919s).
[*] Inserting Vendors and Products...Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1224, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 725, in do_executem
cursor.executemany(statement, parameters)
psycopg2.DatabaseError: could not receive data from server: Connection timed out
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/opencve", line 11, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/opencve/commands/__init__.py", line 39, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 412, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/opencve/commands/imports/__init__.py", line 42, in import_data
cpe.run(vendors)
File "/usr/local/lib/python3.6/site-packages/opencve/commands/imports/cpe.py", line 63, in run
db.session.bulk_insert_mappings(Vendor, mappings["vendors"].values())
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 162, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2790, in bulk_insert_mappings
render_nulls,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2880, in _bulk_save_mappings
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 129, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2874, in _bulk_save_mappings
render_nulls,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 102, in _bulk_insert
bookkeeping=return_defaults,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 1066, in _emit_insert_statements
c = cached_connections[connection].execute(statement, multiparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
distilled_params,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1224, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 725, in do_executem
cursor.executemany(statement, parameters)
sqlalchemy.exc.DatabaseError: (psycopg2.DatabaseError) could not receive data from server: Connection timed out
[SQL: INSERT INTO vendors (id, created_at, updated_at, name) VALUES (%(id)s, now(), now(), %(name)s)]
[parameters: ({'id': UUID('a53ecce1-d16d-4ae5-a20d-3c93034b8d32'), 'name': 'openbsd'}, {'id': UUID('98176222-30ad-498c5a-4a3981bc068e'), 'name': 'freebsd'}, {'id': UUID('f734d65f-f55d-4a3d-a099-189d120bfa99'), 'name': 'bsdi'}, {'id'UID('eadb7068-c668-4981-9190-5aba1ce58d6c'), 'name': 'redhat'}, {'id': UUID('ec37138d-9dc9-432f-a6a4-a4d2e80770b1'),ame': 'caldera'}, {'id': UUID('523ea159-56bc-48c6-914f-1ebecd0f1120'), 'name': 'tritreal'}, {'id': UUID('287ac9fb-654751-a005-625b43059719'), 'name': 'sun'}, {'id': UUID('167edc67-f559-42fb-a501-f91e4114faed'), 'name': 'hp'} ... diaying 10 of 28136 total bound parameter sets ... {'id': UUID('2121a912-76a3-453f-8963-09c537f4bf78'), 'name': 'zend_feedback_tab_project'}, {'id': UUID('5f742c4f-3e37-486d-b5aa-80a92e9f0559'), 'name': 'zyzel'})]
(Background on this error at: http://sqlalche.me/e/4xp6)
I have opened a case for comment on the opencve git page but there has been no response so, hoping that someone may have some advice here please.
I have got this weird issue I ran into when downgrading my migration versions in Flask using Flask-Migrate.
I have this merged head revision which happened from branching and an alembic merge heads.
"""empty message
Revision ID: 3ad527666180
Revises: ecd1f453d08e, 7112dcc79711
Create Date: 2021-11-11 12:17:31.973196
"""
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = '3ad527666180'
down_revision = ('7112dcc79711', 'ecd1f453d08e')
branch_labels = None
depends_on = None
def upgrade():
pass
def downgrade():
pass
When trying to downgrade to preceding revision which is 7112dcc79711, I get this error
docker exec -it users python manage.py db downgrade
/usr/local/lib/python3.7/site-packages/authlib/integrations/flask_oauth2/authorization_server.py:73: AuthlibDeprecationWarning: Define "get_jwt_config" in OpenID Connect grants
It will be compatible before version 1.0.
deprecate('Define "get_jwt_config" in OpenID Connect grants', '1.0')
/usr/local/lib/python3.7/site-packages/authlib/integrations/flask_oauth2/authorization_server.py:73: AuthlibDeprecationWarning: Define "get_jwt_config" in OpenID Connect grants
It will be compatible before version 1.0.
deprecate('Define "get_jwt_config" in OpenID Connect grants', '1.0')
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 227, in _catch_revision_errors
yield
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 457, in _downgrade_revs
for script in revs
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 454, in <listcomp>
migration.MigrationStep.downgrade_from_script(
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 798, in iterate_revisions
assert_relative_length=assert_relative_length,
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 1295, in _collect_downgrade_revisions
assert_relative_length=assert_relative_length,
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 1154, in _parse_downgrade_target
no_overwalk=assert_relative_length,
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 1069, in _walk
raise RevisionError("Ambiguous walk")
alembic.script.revision.RevisionError: Ambiguous walk
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 7, in <module>
cli()
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 380, in main
return AppGroup.main(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 257, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/flask_migrate/cli.py", line 152, in downgrade
_downgrade(directory, revision, sql, tag, x_arg)
File "/usr/local/lib/python3.7/site-packages/flask_migrate/__init__.py", line 282, in downgrade
command.downgrade(config, revision, sql=sql, tag=tag)
File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 366, in downgrade
script.run_env()
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 563, in run_env
util.load_python_file(self.dir, "env.py")
File "/usr/local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
module = load_module_py(module_id, path)
File "/usr/local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "migrations/env.py", line 87, in <module>
run_migrations_online()
File "migrations/env.py", line 80, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/usr/local/lib/python3.7/site-packages/alembic/runtime/environment.py", line 851, in run_migrations
self.get_context().run_migrations(**kw)
File "/usr/local/lib/python3.7/site-packages/alembic/runtime/migration.py", line 608, in run_migrations
for step in self._migrations_fn(heads, self):
File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 355, in downgrade
return script._downgrade_revs(revision, rev)
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 457, in _downgrade_revs
for script in revs
File "/usr/local/lib/python3.7/contextlib.py", line 130, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 261, in _catch_revision_errors
raise util.CommandError(err.args[0]) from err
alembic.util.exc.CommandError: Ambiguous walk
What is the safe way to get past this issue?
I have tried removing one of the revisions which come from a production branch, leaving the other which tallies with my development branch and it didn't produce the error.
Is there an automatic and more feasible way to just downgrade without this issue?
When I got such an error I tried to find another close revision that will work
In this case:
85c43587349b -> 69914284f0b4,
3a9ce166afbb -> 85c43587349b,
885269456833, 3a9ce166afbb -> a42162b3f858 (mergepoint)
3a9ce166afbb -> 885269456833,
f725b3be6238 -> 3a9ce166afbb (branchpoint)
6c164d07441f -> f725b3be6238
When I run alembic downgrade -1 it failed and also alembic downgrade f725b3be6238
But when I tried another version it worked
alembic downgrade 3a9ce166afbb
I want to add columns to a table in my database dynamically since I don't want to have to specify all columns when I set it up in the class
In order to solve this I am instead using alembic to add columns to a table but I am having problems.
In test.py script I have defined a class as below.
DATABASE_URL_dev = "postgresql+psycopg2://user:password#localhost:5432/testdb"
engine = create_engine(DATABASE_URL_dev)
Base = declarative_base(engine)
class my_class1(Base):
__tablename__ = "test1"
id = Column(Integer, primary_key=True)
list = Column(String)
def loadsession():
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
return session
session = loadsession()
Now I want to add a column to table test1 and I am thereby trying to use Alembic.
I followed the instructions at https://alembic.sqlalchemy.org/en/latest/tutorial.html and did the following:
I installed alembic by running pip install alembic
I created an environment using alembic init alembic
In the .ini file I set sqlalchemy.url = "postgresql+psycopg2://user:password#localhost:5432/testdb"
I created a migration script by running alembic revision -m "add a column "
I then edited the script by adding the following "op.add_column('test1', sa.Column('new_column', sa.String))" in upgrade()
I ran the first migration with "alembic upgrade head" This gave me the error: "sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) (Background on this error at: http://sqlalche.me/e/e3q8)"
Question
Have I set sqlalchemy.url correctly or what is else setting the OperationalError?
Entire traceback
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38-32\lib\runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\user\appdata\local\programs\python\python38-32\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\Scripts\alembic.exe\__main__.py", line 9, in <module>
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\config.py", line 577, in main
CommandLine(prog=prog).main(argv=argv)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\config.py", line 571, in main
self.run_cmd(cfg, options)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\config.py", line 548, in run_cmd
fn(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\command.py", line 298, in upgrade
script.run_env()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\script\base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\util\pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\util\compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "y\env.py", line 77, in <module>
run_migrations_online()
File "y\env.py", line 65, in run_migrations_online
with connectable.connect() as connection:
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2218, in connect
return self._connection_cls(self, **kwargs)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 103, in __init__
else engine.raw_connection()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2317, in raw_connection
return self._wrap_pool_connect(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2288, in _wrap_pool_connect
Connection._handle_dbapi_exception_noconnection(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 1554, in _handle_dbapi_exception_noconnection
util.raise_(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\util\compat.py", line 178, in raise_
raise exception
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2285, in _wrap_pool_connect
return fn()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 303, in unique_connection
return _ConnectionFairy._checkout(self)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 773, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 492, in checkout
rec = pool._do_get()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\impl.py", line 238, in _do_get
return self._create_connection()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 657, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\util\langhelpers.py", line 68, in __exit__
compat.raise_(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\util\compat.py", line 178, in raise_
raise exception
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 652, in __connect
connection = pool._invoke_creator(self)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\default.py", line 488, in connect
return self.dbapi.connect(*cargs, **cparams)
File "c:\users\c\appdata\local\programs\python\python38-32\lib\site-packages\psycopg2\__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError)
(Background on this error at: http://sqlalche.me/e/e3q8)
My app is working on my main PC. I tried to clone the app to my laptop (I tried to initialize it on PC in a different directory and the problem was the same so the problem isn't with the laptop) and when initializing the db with flask db init I got the following error:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: race [SQL: 'SELECT race.id AS race_id, race.name AS race_name \nFROM race'] (Background on this error at: http://sqlalche.me/e/e3q8)
What I'm trying to do is in one of my forms to include a wtforms RadioField with choices based on the race table in db. So for every row in that table I want one choice in the selection. The important part of the form looks like this:
class RegistrationForm(FlaskForm):
race = RadioField('Race', validators=[InputRequired('Choose a race.')],
choices=races, coerce=int)
And the races variable is outside the form like this (before the form itself):
with create_app().app_context():
races = [(race.id, race.name.capitalize()) for race in Race.query.all()]
I figured that this last line is the problem from the call stack:
Traceback (most recent call last):
File "/home/dareon4/workspace/test/venv/bin/flask", line 11, in <module>
sys.exit(main())
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 894, in main
cli.main(args=args, prog_name=name)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 557, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 411, in decorator
with __ctx.ensure_object(ScriptInfo).load_app().app_context():
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 372, in load_app
app = locate_app(self, import_name, name)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 235, in locate_app
__import__(module_name)
File "/home/dareon4/workspace/test/shapes.py", line 5, in <module>
app = create_app()
File "/home/dareon4/workspace/test/app/__init__.py", line 43, in create_app
from app.blueprints.auth import bp as auth_bp
File "/home/dareon4/workspace/test/app/blueprints/auth/__init__.py", line 7, in <module>
from . import routes
File "/home/dareon4/workspace/test/app/blueprints/auth/routes.py", line 13, in <module>
from .forms import LoginForm, RegistrationForm, ResetPasswordRequestForm, \
File "/home/dareon4/workspace/test/app/blueprints/auth/forms.py", line 19, in <module>
races = [(race.id, race.name.capitalize()) for race in Race.query.all()]
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2836, in all
return list(self)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2988, in __iter__
return self._execute_and_instances(context)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3011, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
compiled_sql, distilled_params
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 248, in reraise
raise value.with_traceback(tb)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: race [SQL: 'SELECT race.id AS race_id, race.name AS race_name \nFROM race'] (Background on this error at: http://sqlalche.me/e/e3q8)
So the problem is that I don't have the db initialized when I'm running the flask db init command and that one line needs the db to exist. This probably doesn't have a solution so I'm not asking how to correct this implementation, but how to do it correctly to have form based on rows from the db. I don't have hundreds of races so I could hardcode them in but I wanted to do it this way.
So after some googling I found that you can add the RadioField choices dynamically in the views like this:
#bp.route('/register', methods=['GET', 'POST'])
def register():
form = RegistrationForm()
form.race.choices = [(race.id, race.name.capitalize()) for race in Race.query.all()]
This is exactly what I was looking for.
When i run the dbManager.create_all() command, it runs with out errors but fails to create the tables. When i delete the database and run the create_all() command, i get the no such database as ##### error which i should get but when the database does exist, nothing happens.
Please can anyone see what i'm doing wrong?
from blogconfig import dbManager
class Art(dbManager.Model):
id = dbManager.Column(dbManager.Integer, primary_key = True)
title = dbManager.Column(dbManager.String(64), index = True, unique = True)
content = dbManager.Column(dbManager.Text(5000))
def __repr__(self):
return '<Art %r>' %(self.title)
EDIT
This is the shell command
from blogconfig import dbManager
>>> dbManager.create_all()
import models
>>> a = models.Art(title='des', content='asdfvhbdjbjdn')
>>> dbManager.session.add(a)
>>> dbManager.session.commit()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/scoping.py", line 149, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 721, in commit
self.transaction.commit()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 354, in commit
self._prepare_impl()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 334, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1818, in flush
self._flush(objects)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1936, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 58, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1900, in _flush
flush_context.execute()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute
rec.execute(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 525, in execute
uow
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 64, in save_obj
table, insert)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 569, in _emit_insert_statements
execute(statement, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 662, in execute
params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 761, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 874, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1024, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 195, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 867, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 324, in do_execute
cursor.execute(statement, parameters)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1146, "Table 'blog.art' doesn't exist") 'INSERT INTO art (title, content) VALUES (%s, %s)' ('des', 'asdfvhbdjbjdn')
dbManager will not know about the models you define in other modules unless they are imported before running create_all.
In a real application this shouldn't matter because running the flask app should set up the db and import views/blueprints to register them. Since the views use the models, the models are indirectly imported and are available to the dbManager.
Either import your models in the blogconfig module after creating the dbManager instance, or change the order of you shell commands to be
>>> from blogconfig import dbManager
>>> import models
>>> dbManager.create_all()
SQLAlchemy will only create tables, the database must already exist, which is why you're seeing the other error when you delete the database.