Related
I am trying to write to an in-memory SQLite database in multiple threads. According to SQLAlchemy docs this is possible. However, in practice I cannot reliably get it to work. I have tried with both scoped_session and without, yielding the same results. Strangely enough, if I pass echo=True to create_engine the code works every time, maybe because it forces the engine to synchronously print all statements to the screen. Code sample and exceptions are below.
EDIT: The more I look into this, the more I realize that SQLite is just not very good at concurrency for write operations. The section on High Concurrency says that while an unlimited amount of readers are allowed, there can only be one writer at any given time. A better approach here would be to queue all write operations instead of trying to achieve any level of concurrency. However I am still open to suggestions if anyone has managed to find a solution.
import threading
from sqlalchemy import *
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import create_engine
from sqlalchemy.pool import StaticPool
from sqlalchemy.orm import sessionmaker, scoped_session
in_memory_engine = create_engine('sqlite://', connect_args={'check_same_thread':False}, poolclass=StaticPool)
Base = declarative_base()
class Item(Base):
__tablename__ = 'items'
id = Column(Integer, primary_key=True, autoincrement=True)
data = Column(Integer)
Base.metadata.create_all(bind=in_memory_engine)
session_factory = sessionmaker(bind=in_memory_engine)
Session = scoped_session(session_factory)
def insert_items():
session = Session()
for i in range(1000):
session.add(Item(data=i))
session.commit()
Session.remove()
for i in range(10):
t = threading.Thread(target=insert_items)
t.start()
Exception in thread Thread-6:
Traceback (most recent call last):
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1249, in _execute_context
cursor, statement, parameters, context
File "c:\python\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: cannot start a transaction within a transaction
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "c:\python\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "c:\python\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<ipython-input-1-e1040f59c94e>", line 26, in insert_items
session.commit()
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 1027, in commit
self.transaction.commit()
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 494, in commit
self._prepare_impl()
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 473, in _prepare_impl
self.session.flush()
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 2459, in flush
self._flush(objects)
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 2597, in _flush
transaction.rollback(_capture_exception=True)
File "c:\python\lib\site-packages\sqlalchemy\util\langhelpers.py", line 68, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "c:\python\lib\site-packages\sqlalchemy\util\compat.py", line 153, in reraise
raise value
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 2557, in _flush
flush_context.execute()
File "c:\python\lib\site-packages\sqlalchemy\orm\unitofwork.py", line 422, in execute
rec.execute(self)
File "c:\python\lib\site-packages\sqlalchemy\orm\unitofwork.py", line 589, in execute
uow,
File "c:\python\lib\site-packages\sqlalchemy\orm\persistence.py", line 245, in save_obj
insert,
File "c:\python\lib\site-packages\sqlalchemy\orm\persistence.py", line 1138, in _emit_insert_statements
statement, params
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 988, in execute
return meth(self, multiparams, params)
File "c:\python\lib\site-packages\sqlalchemy\sql\elements.py", line 287, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1107, in _execute_clauseelement
distilled_params,
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1253, in _execute_context
e, statement, parameters, cursor, context
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1473, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "c:\python\lib\site-packages\sqlalchemy\util\compat.py", line 398, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "c:\python\lib\site-packages\sqlalchemy\util\compat.py", line 152, in reraise
raise value.with_traceback(tb)
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1249, in _execute_context
cursor, statement, parameters, context
File "c:\python\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) cannot start a transaction within a transaction
[SQL: INSERT INTO items (data) VALUES (?)]
[parameters: (780,)]
(Background on this error at: http://sqlalche.me/e/e3q8)
Exception in thread Thread-7:
Traceback (most recent call last):
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1249, in _execute_context
cursor, statement, parameters, context
File "c:\python\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type.
import os
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
engine = create_engine("postgres://kali:kali#localhost/mydb")
db = scoped_session(sessionmaker(bind=engine))
def main():
flights = db.execute ("SELECT origin, destination, duration FROM flights;").fetchall()
for flight in flights:
print(f"{flight.origin} to {flight.destination}, {flight.duration} minutes.")
if __name__ == "__main__":
main()
After i execute python , he send for me:
I do not know the reason
Error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/default.py", line 588, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.UndefinedTable: relation "flights" does not exist
LINE 1: SELECT origin, destination, duration FROM flights
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "list.py", line 18, in <module>
main()
File "list.py", line 13, in main
flights = db.execute("SELECT origin, destination, duration FROM flights").fetchall()
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/scoping.py", line 162, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/session.py", line 1278, in execute
clause, params or {}
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 984, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1103, in _execute_clauseelement
distilled_params,
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1288, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1482, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/util/compat.py", line 178, in raise_
raise exception
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/default.py", line 588, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "flights" does not exist
LINE 1: SELECT origin, destination, duration FROM flights
^
[SQL: SELECT origin, destination, duration FROM flights]
(Background on this error at: http://sqlalche.me/e/f405)
root#ubuntu:~/lecture3# python3 list2.py
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/default.py", line 588, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.UndefinedTable: relation "flights" does not exist
LINE 1: SELECT origin, destination, duration FROM flights;
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "list2.py", line 15, in <module>
main()
File "list2.py", line 10, in main
flights = db.execute ("SELECT origin, destination, duration FROM flights;").fetchall()
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/scoping.py", line 162, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/session.py", line 1278, in execute
clause, params or {}
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 984, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1103, in _execute_clauseelement
distilled_params,
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1288, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1482, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/util/compat.py", line 178, in raise_
raise exception
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/default.py", line 588, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "flights" does not exist
LINE 1: SELECT origin, destination, duration FROM flights;
^
[SQL: SELECT origin, destination, duration FROM flights;]
(Background on this error at: http://sqlalche.me/e/f405)
I try change "engine = create_engine("postgres://kali:kali#localhost/mydb")" to "engine = create_engine(os.getenv("postgres://kali:kali#localhost:5432/mydb"))" and:
" AttributeError: 'NoneType' object has no attribute '_instantiate_plugins' "
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "flights" does not exist
Means that you don't have a table by the name of(flights) in your database.
Create the table like this:
CREATE TABLE flights(id SERIAL PRIMARY KEY NOT NULL, origin VARCHAR NOT NULL, destination VARCHAR NOT NULL, duration INTEGER NOT NULL);
Then try running your application, it should work.
I am writing on a Dash app and using SQL alchemy for the database. I want to make the creation of the database automated so I have a few lines of code to create the database if it does not exist.
from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String, Float
from sqlalchemy_utils import create_database, database_exists
engine = create_engine(SQLALCHEMY_DATABASE_URI, echo = True)
create_database(SQLALCHEMY_DATABASE_URI)
meta = MetaData()
flagged_vendors_table = Table(
'flagged_vendors', meta,
Column(vendor_col, String, primary_key = True),
Column(flag_col, Integer)
)
meta.create_all(engine)
This seems to work well as per the echo:
2020-01-21 09:28:30,604 INFO sqlalchemy.engine.base.Engine
CREATE TABLE flagged_vendors (
"Vendor PRS_description" VARCHAR NOT NULL,
"Flag" INTEGER,
PRIMARY KEY ("Vendor PRS_description")
)
2020-01-21 09:28:30,608 INFO sqlalchemy.engine.base.Engine ()
2020-01-21 09:28:30,639 INFO sqlalchemy.engine.base.Engine COMMIT
However when I execute the app I get the following error:
Traceback (most recent call last):
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1246, in _execute_context
cursor, statement, parameters, context
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\engine\default.py", line 581, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: no such table: flagged_vendors
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\pandas\io\sql.py", line 518, in to_sql
method=method,
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\pandas\io\sql.py", line 1319, in to_sql
table.create()
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\pandas\io\sql.py", line 647, in create
self.pd_sql.drop_table(self.name, self.schema)
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\pandas\io\sql.py", line 1367, in drop_table
self.get_table(table_name, schema).drop()
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\sql\schema.py", line 884, in drop
bind._run_visitor(ddl.SchemaDropper, self, checkfirst=checkfirst)
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\engine\base.py", line 2049, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1618, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\sql\visitors.py", line 138, in traverse_single
return meth(obj, **kw)
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\sql\ddl.py", line 1002, in visit_table
self.connection.execute(DropTable(table))
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\engine\base.py", line 982, in execute
return meth(self, multiparams, params)
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\sql\ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1044, in _execute_ddl
compiled,
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1250, in _execute_context
e, statement, parameters, cursor, context
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1476, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\util\compat.py", line 398, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\util\compat.py", line 152, in reraise
raise value.with_traceback(tb)
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1246, in _execute_context
cursor, statement, parameters, context
File "C:\Users\LIAG8802\Documents\Procurement_analytics\venv\lib\site-packages\sqlalchemy\engine\default.py", line 581, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: flagged_vendors
[SQL:
DROP TABLE flagged_vendors]
(Background on this error at: http://sqlalche.me/e/e3q8)
The traceback is not leading to any line in my code, which is an issue that I am trying to investigate here, but I believe there is only one part in my code that could cause the error:
def to_storage(df, storage_info):
storage_type = storage_info["storage_type"]
if storage_type == "sql":
db_uri = storage_info["db_uri"]
engine = create_engine(db_uri, echo = False, pool_pre_ping=True)
table_name = storage_info["table_name"]
df.to_sql(table_name, engine, if_exists = "replace")
It is very strange because the code is supposed to drop the table only if it exists. One last element: this issue seems to appear only if there is no existing database at the specified URI when I launch the app.
I'm working with scrapy and have had a functioning pipeline which uses sql-alchemy to insert records into a sqllite db. This was working but then I added 2 new fields: name and address. Now I'm getting:
2016-06-14 10:12:23 [scrapy] ERROR: Error processing {'account': u'X6',
'address': u' the address',
'name': u'my name'}
Traceback (most recent call last):
File "C:\envs\virtalenvs\teat\lib\site-packages\twisted\internet\defer.py", line 588, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "C:\envs\r2\tutorial\tutorial\pipelines.py", line 54, in process_item
self.connection.execute(ins_query)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\base.py", line 914, in execute
return meth(self, multiparams, params)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\sql\elements.py", line 323, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\base.py", line 1010, in _execute_clauseelement
compiled_sql, distilled_params
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\base.py", line 1146, in _execute_context
context)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\base.py", line 1341, in _handle_dbapi_exception
exc_info
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\util\compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\base.py", line 1139, in _execute_context
context)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\default.py", line 450, in do_execute
cursor.execute(statement, parameters)
OperationalError: (sqlite3.OperationalError) table accounts has no column named name [SQL: u'INSERT INTO accounts (account, name, address) VALUES (?, ?, ?)'] [parameters: (u'X6', u' my name', u' the address')]
Here's the pipeline code:
class MYPipeline(object):
def __init__(self):
_engine = create_engine("sqlite:///data.db")
_connection = _engine.connect()
_metadata = MetaData()
_stack_items = Table("accounts", _metadata,
Column("id", Integer, primary_key=True),
Column("account", Text),
Column("name", Text),
Column("address", Text))
_metadata.create_all(_engine)
self.connection = _connection
self.stack_items = _stack_items
def process_item(self, item, spider):
is_valid = True
for data in item:
if not data:
is_valid = False
raise DropItem("Missing %s!" % data)
if is_valid:
ins_query = self.stack_items.insert().values(
account=item["account"],
name=item["name"],
address=item["address"])
self.connection.execute(ins_query)
return item
Whats weird is that if I run this using :
scrapy crawl myspider
using GIT-Bash (I'm using win7) this works normally. However with pycharm I get the errors above.
for pycharm the run config is in the screenshot, and man.py:
from scrapy import cmdline
cmdline.execute("scrapy crawl myspider".split())
What am I doing wrong?
I suspect you're running with two different working directories. Try configuring PyCharm with the working directory you're using on the command line. Alternatively, specify an absolute path for the SQLAlchemy URL.
I have followed the CKAN install guide from source (http://docs.ckan.org/en/latest/maintaining/installing/install-from-source.html) and managed to get all the way until point 6. Create database tables.
At that stage, running paster db init -c /etc/ckan/default/development.ini gives the following stack trace:
Traceback (most recent call last):
File "/usr/lib/ckan/default/bin/paster", line 9, in <module>
load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 104, in run
invoke(command, command_name, options, args[1:])
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 143, in invoke
exit_code = runner.run(args)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 238, in run
result = self.command()
File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 208, in command
self._load_config(cmd!='upgrade')
File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 164, in _load_config
self.site_user = logic.get_action('get_site_user')({'ignore_auth': True}, {})
File "/usr/lib/ckan/default/src/ckan/ckan/logic/__init__.py", line 424, in wrapped
result = _action(context, data_dict, **kw)
File "/usr/lib/ckan/default/src/ckan/ckan/logic/action/get.py", line 2209, in get_site_user
user = model.User.get(site_id)
File "/usr/lib/ckan/default/src/ckan/ckan/model/user.py", line 64, in get
return query.first()
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2334, in first
ret = list(self[0:1])
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2201, in __getitem__
return list(res)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2405, in __iter__
return self._execute_and_instances(context)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2420, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 727, in execute
return meth(self, multiparams, params)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 824, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 954, in _execute_context
context)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1116, in _handle_dbapi_exception
exc_info
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 189, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 947, in _execute_context
context)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 435, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (ProgrammingError) column user.password does not exist
LINE 1: SELECT "user".password AS user_password, "user".id AS user_i...
^
'SELECT "user".password AS user_password, "user".id AS user_id, "user".name AS user_name, "user".openid AS user_openid, "user".fullname AS user_fullname, "user".email AS user_email, "user".apikey AS user_apikey, "user".created AS user_created, "user".reset_key AS user_reset_key, "user".about AS user_about, "user".activity_streams_email_notifications AS user_activity_streams_email_notifications, "user".sysadmin AS user_sysadmin, "user".state AS user_state \nFROM "user" \nWHERE "user".name = %(name_1)s OR "user".openid = %(openid_1)s OR "user".id = %(id_1)s ORDER BY "user".name \n LIMIT %(param_1)s' {'param_1': 1, 'id_1': 'default', 'name_1': 'default', 'openid_1': 'default'}
I did not make any modifications to the installation or the schema. The installation is under a proxy, but that did not turn out to be a problem thus far, since there were many posts giving help in that regard.
Did anybody come across this error before and managed to resolve it? I don't suppose it is a proxy-related problem... though it seems to be related to the schema.
I ran into the same issue but dropping the DB and creating and initialising it again afterwards produced the same error. Also running paster db clear -c /etc/ckan/default/development.ini failed with said error.
It worked for me, when I deactivated all ckan.plugins in development.ini before running paster db init -c /etc/ckan/default/development.ini.
Managed to solve this one after all, so I'll answer my own question... :)
The problem was with the setup of the database. Once I purged the database
and completely reinstalled it, everything worked like a charm!
sqlalchemy.exc.ProgrammingError: (ProgrammingError) column user.password does not exist
LINE 1: SELECT "user".password AS user_password, "user".id AS user_i...
This error usually means that the CKAN database exists but is empty - the CKAN tables have not been created in it.
To solve it:
cd /usr/lib/ckan/default/src/ckan
paster db init -c /etc/ckan/default/development.ini
There's more about this in the docs:
http://docs.ckan.org/en/latest/maintaining/installing/install-from-source.html#create-database-tables