I'm trying to use LOAD LOCAL DATA INFILE with sqlalchemy to upload batch files to my server. I have edited my /etc/mysql/my.conf to contain local-infile == 1 under both [mysql] and [mysqld], yet I'm getting this traceback when I run the program.
The relevant line of code is:
Traceback (most recent call last):
File "main.py", line 48, in <module>
con.execute(sql)
File "/Users/eoddata/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 906, in execute
return self._execute_text(object, multiparams, params)
File "/Users/eoddata/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1054, in _execute_text
statement, parameters
File "/Users/eoddata/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
context)
File "/Users/eoddata/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1332, in _handle_dbapi_exception
exc_info
File "/Users/eoddata/venv/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/Users/eoddata/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
context)
File "/Users/eoddata/venv/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 442, in do_execute
cursor.execute(statement, parameters)
File "/Users/eoddata/venv/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/Users/eoddata/venv/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1148, 'The used command is not allowed with
this MySQL version') [SQL: "LOAD DATA LOCAL
INFILE 'stocks/AMEX_20150420.txt' INTO TABLE database.new_table\n
FIELDS TERMINATED BY ',' (Symbol,#Date,Open,High,Low,Close,Volume)\n
SET Date = STR_TO_DATE(#Date,'%%Y%%m%%d');"]
I've done some research from the docs and can't find a way to fix this error. With python MySQLdb (from pip install MySQL-Python) will allow me to pass in a local-infile paramter. I can't find an equivalent in SQLAlchemy. Any help is appreciated.
EDIT: I've worked on this all night and still haven't made any progress - a lot of dead ends. Any ideas?
Thanks, Jared
You should add local_infile=1 to the connection string.
SQLALCHEMY_DATABASE_URI = "mysql://user:pass#localhost/dbname?charset=utf8&local_infile=1"
Related
Ubuntu 20.04.3 LTS (WSL2)
fiona 1.8.21
geopandas 0.10.2
geopandas-base 0.10.2
geopy 2.2.0
numpy 1.22.3
pandas 1.4.1
psycopg2 2.9.3
pyproj 3.3.0
rtree 0.9.7
shapely 1.8.0
postgresql-14 14.2-1.pgdg20.04+1
postgresql-14-postgis-3 3.2.1+dfsg-1.pgdg20.04+1
trying to send a couple of polygons to postgresql database (postgis). Having no luck -.-' Always getting the same error message. Can someone help me?
Exception in thread Thread-2 (_create_searchwindows):
Traceback (most recent call last):
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1808, in _execute_context
self.dialect.do_execute(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.UndefinedObject: type "geometry" does not exist
LINE 4: geometry geometry(POLYGON,25832)
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/miniconda3/envs/geo/lib/python3.10/threading.py", line 1009, in _bootstrap_inner
self.run()
File "/home/user/miniconda3/envs/geo/lib/python3.10/threading.py", line 946, in run
self._target(*self._args, **self._kwargs)
File "/home/user/github/create_searchwindows/searchwindow_creator.py", line 143, in _create_searchwindows
grid.to_postgis(self.args.searchwindows_tablename,self.engine,if_exists="append",index=True,index_label="Index")
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/geopandas/geodataframe.py", line 1808, in to_postgis
geopandas.io.sql._write_postgis(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/geopandas/io/sql.py", line 431, in _write_postgis
gdf.to_sql(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/pandas/core/generic.py", line 2963, in to_sql
return sql.to_sql(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/pandas/io/sql.py", line 697, in to_sql
return pandas_sql.to_sql(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/pandas/io/sql.py", line 1729, in to_sql
table = self.prep_table(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/pandas/io/sql.py", line 1628, in prep_table
table.create()
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/pandas/io/sql.py", line 842, in create
self._execute_create()
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/pandas/io/sql.py", line 828, in _execute_create
self.table.create(bind=self.pd_sql.connectable)
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 950, in create
bind._run_ddl_visitor(ddl.SchemaGenerator, self, checkfirst=checkfirst)
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2119, in _run_ddl_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py", line 524, in traverse_single
return meth(obj, **kw)
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 893, in visit_table
self.connection.execute(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1295, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
return connection._execute_ddl(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1387, in _execute_ddl
ret = self._execute_context(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1851, in _execute_context
self._handle_dbapi_exception(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2032, in _handle_dbapi_exception
util.raise_(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1808, in _execute_context
self.dialect.do_execute(
File "/home/user/miniconda3/envs/geo/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedObject) type "geometry" does not exist
LINE 4: geometry geometry(POLYGON,25832)
^
[SQL:
CREATE TABLE public.sw32 (
"Index" BIGINT,
geometry geometry(POLYGON,25832)
)
]
(Background on this error at: https://sqlalche.me/e/14/f405)
I made sure, that the postgis extension is activated in the database. If I create the table with a geometry column in the database using psql its working perfectly fine. Something seems to be not correct with the python packages installed. Did someone came up with the same trouble?
code looks like this:
import geopandas as gpd
from sqlalchemy import create_engine
engine = create_engine(f'postgresql://{db_user}:{db_password}#{db_host}:{db_port}/')
grid = gpd.GeoDataFrame(polygons,geometry="geometry",crs="EPSG:25832")
grid.to_postgis("testtable",engine,if_exists="append",index=True,index_label="Index")
EDIT 1:
My current workaround is to store the polygons in a *.shp file and execute shp2psql to send the polygons to the database. The code looks as follows:
grid.to_file("tmp_name.shp")
cmd = f"shp2pgsql -s 25832 tmp_name.shp public.testtable |psql postgresql://{db_user}:{db_password}#{db_host}:{db_port}/{database}"
os.system(cmd)
Indeed my connection string was not correct. forgot to mention the database. with the adjustment of the connection string as follows:
engine = create_engine(f'postgresql://{db_user}:{db_password}#{db_host}:{db_port}/{database}')
everything works fine!
Probably it is related to your "geoalchemy2" version. Your geoalchemy2 version should be "0.9.2".
I am trying to connect, read and write from a mysql db using odbc(ODBC is a requirement and I'll have to use it) in python. I am using sqlalchemy and pyodbc. I am able to connect to the db and read. But when I am trying to write, pyodbc is giving me strange errors. The code is given below.
params = urllib3.parse.quote_plus("DRIVER={MySQL ODBC 8.0 ANSI Driver};"
f"SERVER={host}:{port};"
f"DATABASE={db};"
f"UID={username};"
f"PWD={password};"
f"charset=utf8")
db_engine = create_engine(f"mysql+pyodbc:///?odbc_connect={params}")
connection = db_engine.connect()
# main_df is a pd.DataFrame()
maindf.to_sql("mytable", connection, if_exists="append", index=False, chunksize=1000)
The Error is given below:
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\sqlalchemy\engine\base.py", line 1685, in _execute_context
self.dialect.do_executemany(
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\sqlalchemy\engine\default.py", line 713, in do_executemany
cursor.executemany(statement, parameters)
pyodbc.ProgrammingError: ('42000', "[42000] [MySQL][ODBC 8.0(a) Driver][mysqld-5.7.31-log]You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '4' at line 1 (1064) (SQLParamData)")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "E:/myusername/cx-index-score/nice_rpa/pipeline.py", line 17, in <module>
uploader.split_upload(os.path.abspath(Path('./datasets')))
File "E:\myusername\cx-index-score\nice_rpa\processandupload.py", line 163, in split_upload
self.writetosandbox()
File "E:\myusername\cx-index-score\nice_rpa\processandupload.py", line 216, in writetosandbox
self.maindf.to_sql("nice_daily_update", self.connection, if_exists="append",
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\pandas\core\generic.py", line 2779, in to_sql
sql.to_sql(
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\pandas\io\sql.py", line 590, in to_sql
pandas_sql.to_sql(
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\pandas\io\sql.py", line 1405, in to_sql
raise err
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\pandas\io\sql.py", line 1397, in to_sql
table.insert(chunksize, method=method)
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\pandas\io\sql.py", line 831, in insert
exec_insert(conn, keys, chunk_iter)
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\pandas\io\sql.py", line 748, in _execute_insert
conn.execute(self.table.insert(), data)
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\sqlalchemy\engine\base.py", line 1200, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\sqlalchemy\sql\elements.py", line 313, in _execute_on_connection
return connection._execute_clauseelement(
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\sqlalchemy\engine\base.py", line 1389, in _execute_clauseelement
ret = self._execute_context(
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\sqlalchemy\engine\base.py", line 1748, in _execute_context
self._handle_dbapi_exception(
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\sqlalchemy\engine\base.py", line 1929, in _handle_dbapi_exception
util.raise_(
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\sqlalchemy\util\compat.py", line 211, in raise_
raise exception
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\sqlalchemy\engine\base.py", line 1685, in _execute_context
self.dialect.do_executemany(
File "C:\ProgramData\Anaconda3\envs\nice_rpa\lib\site-packages\sqlalchemy\engine\default.py", line 713, in do_executemany
cursor.executemany(statement, parameters)
sqlalchemy.exc.ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000] [MySQL][ODBC 8.0(a) Driver][mysqld-5.7.31-log]You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '33\x03#333333\x03#333333\x03#' at line 1 (1064) (SQLParamData)")
If you see the syntax error part, it is actually supposed to be a simple string, which is utf-8. The Databse encoding is 'latin1'. Interestingly, even though each time the data and the error are same, but the "wrong syntax" part always changes. once, it was '4', then it was '33\x03#333333\x03#333333\x03#' and it changes on each run, the input data is always same though.
I have exhausted all the other stackoverflow questions related to this topic I can find. Please help.
Hi Everyone I Am working right now with Pandas and MSSQL. I have bene working ok but recently after an SQlalchemy update i am getting the following error when I am trying to upload information into the DB via df.to_sql
pyodbc.Error: ('HY000', 'The driver did not supply an error!')
My connection is defined as follows
engine = create_engine('mssql+pyodbc://Reporter:******#localhost:1433/SourcingDb?driver=SQL+Server')
df.to_sql('ProductivityDeck', con=engine, if_exists='replace', index= False)
and the full error I get is this one
Traceback (most recent call last):
File "C:\Python\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 1182, in _execute_context
context)
File "C:\Python\Python36\lib\site-packages\sqlalchemy\engine\default.py", line 470, in do_execute
cursor.execute(statement, parameters)
pyodbc.Error: ('HY000', 'The driver did not supply an error!')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:/Users/H212310/PycharmProjects/untitled2/UpdateCEDeckv2.py", line 23, in <module>
df.to_sql('ProductivityDeck', con=engine, if_exists='replace', index= False)
File "C:\Python\Python36\lib\site-packages\pandas\core\generic.py", line 2127, in to_sql
dtype=dtype)
File "C:\Python\Python36\lib\site-packages\pandas\io\sql.py", line 450, in to_sql
chunksize=chunksize, dtype=dtype)
File "C:\Python\Python36\lib\site-packages\pandas\io\sql.py", line 1149, in to_sql
table.insert(chunksize)
File "C:\Python\Python36\lib\site-packages\pandas\io\sql.py", line 663, in insert
self._execute_insert(conn, keys, chunk_iter)
File "C:\Python\Python36\lib\site-packages\pandas\io\sql.py", line 638, in _execute_insert
conn.execute(*self.insert_statement(data, conn))
File "C:\Python\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 945, in execute
return meth(self, multiparams, params)
File "C:\Python\Python36\lib\site-packages\sqlalchemy\sql\elements.py", line 263, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "C:\Python\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 1053, in _execute_clauseelement
compiled_sql, distilled_params
File "C:\Python\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 1189, in _execute_context
context)
File "C:\Python\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 1402, in _handle_dbapi_exception
exc_info
File "C:\Python\Python36\lib\site-packages\sqlalchemy\util\compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "C:\Python\Python36\lib\site-packages\sqlalchemy\util\compat.py", line 186, in reraise
raise value.with_traceback(tb)
File "C:\Python\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 1182, in _execute_context context)
File "C:\Python\Python36\lib\site-packages\sqlalchemy\engine\default.py", line 470, in do_execute
cursor.execute(statement, parameters)
I have tried to rollback to a lower version of SqlAlchemy, Update the ODBC driver in windows and upgrading the python installation and it keeps coming back.
Any ideas in how to fix this issue?
OK this was a "silent error for a known bug in pandas 23, downgrading to v22 makes the error go away and you can upload 1000+ rows.
this is being worked out on
https://github.com/pandas-dev/pandas/issues/21103
I had the same error ('HY000', 'The driver did not supply an error!'). The existing solutions on a few StackOverflow threads did not help.
In my case, the error came from pyodbc.connect().
I ended up reinstalling python and dependencies and the problem disappeared.
pyodbc.connect('Driver={ODBC Driver 17 for SQL Server};')
Just changing the driver worked for me!!
There are a couple of related questions regarding this, but in my case, all those solutions is not working out. Thats why I thought of asking again. I am getting this error while I am firing below query using sqlalchemy orm.
Traceback (most recent call last):
File "MyFile.py", line 1010, in <module>
handler.handle(line.split('\t'))
File "MyFile.py", line 849, in handle
self.getRecord(whatIfFlag, id)
File "MyFile.py", line 143, in getRecord
newRecord = self.recordSearcher.getRecordByParams(name, pId)
File "abc.py", line 67, in getRecord
File "/opt/product/python/prod/bin/2.7.6/lib/python2.7/site-packages/SQLAlchemy-0.9.4-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py", line 2361, in one
ret = list(self)
File "/opt/product/python/prod/bin/2.7.6/lib/python2.7/site-packages/SQLAlchemy-0.9.4-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py", line 2404, in __iter__
return self._execute_and_instances(context)
File "/opt/product/python/prod/bin/2.7.6/lib/python2.7/site-packages/SQLAlchemy-0.9.4-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py", line 2419, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/opt/product/python/prod/bin/2.7.6/lib/python2.7/site-packages/SQLAlchemy-0.9.4-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 720, in execute
return meth(self, multiparams, params)
File "/opt/product/python/prod/bin/2.7.6/lib/python2.7/site-packages/SQLAlchemy-0.9.4-py2.7-linux-x86_64.egg/sqlalchemy/sql/elements.py", line 317, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/product/python/prod/bin/2.7.6/lib/python2.7/site-packages/SQLAlchemy-0.9.4-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 817, in _execute_clauseelement
compiled_sql, distilled_params
File "/opt/product/python/prod/bin/2.7.6/lib/python2.7/site-packages/SQLAlchemy-0.9.4-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 947, in _execute_context
context)
File "/opt/product/python/prod/bin/2.7.6/lib/python2.7/site-packages/SQLAlchemy-0.9.4-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1108, in _handle_dbapi_exception
exc_info
File "/opt/product/python/prod/bin/2.7.6/lib/python2.7/site-packages/SQLAlchemy-0.9.4-py2.7-linux-x86_64.egg/sqlalchemy/util/compat.py", line 185, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/opt/product/python/prod/bin/2.7.6/lib/python2.7/site-packages/SQLAlchemy-0.9.4-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 940, in _execute_context
context)
File "/opt/product/python/prod/bin/2.7.6/lib/python2.7/site-packages/SQLAlchemy-0.9.4-py2.7-linux-x86_64.egg/sqlalchemy/engine/default.py", line 435, in do_execute
cursor.execute(statement, parameters)
File "build/bdist.linux-x86_64/egg/MySQLdb/cursors.py", line 205, in execute
File "build/bdist.linux-x86_64/egg/MySQLdb/connections.py", line 36, in defaulterrorhandler
sqlalchemy.exc.OperationalError: (OperationalError) (2013, 'Lost connection to MySQL server during query') ....
query = self.__session.query(MyTable).filter(and_(MyTable.NAME == name,MyTable.P_ID == p_id)
try:
record = query.one()
except NoResultFound:
new_record = MyTable(params)
self.__session.add(new_record)
self.__session.commit()
self.__session.close()
It is expected to return only one record. This is how I create my session.
sqlEngine = sqlalchemy.create_engine(self.getMySQLURI(), pool_recycle=10800, echo=False, echo_pool=False)
session = scoped_session(sessionmaker(autoflush=True,
autocommit=False,
bind=sqlEngine,
expire_on_commit=False))
These are my mysql configurations: interactive_timeout and wait_timeout is set to 28800 ~ 8 hours. net_write_timeout is set to 3600 ~ 60 mins and net_read_timeout is set 300 ~ 5 mins.
Any help is highly appreciated.
It is turned out to be problem with the tcp_connect_timeout between the application server and the database server. The tcp connect timeout was default of 1 hour and my pool recycle settings was 3 hrs. So anything between 1 and 3 were failing. Posting the answer to help others who might face this later.
I have followed the CKAN install guide from source (http://docs.ckan.org/en/latest/maintaining/installing/install-from-source.html) and managed to get all the way until point 6. Create database tables.
At that stage, running paster db init -c /etc/ckan/default/development.ini gives the following stack trace:
Traceback (most recent call last):
File "/usr/lib/ckan/default/bin/paster", line 9, in <module>
load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 104, in run
invoke(command, command_name, options, args[1:])
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 143, in invoke
exit_code = runner.run(args)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 238, in run
result = self.command()
File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 208, in command
self._load_config(cmd!='upgrade')
File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 164, in _load_config
self.site_user = logic.get_action('get_site_user')({'ignore_auth': True}, {})
File "/usr/lib/ckan/default/src/ckan/ckan/logic/__init__.py", line 424, in wrapped
result = _action(context, data_dict, **kw)
File "/usr/lib/ckan/default/src/ckan/ckan/logic/action/get.py", line 2209, in get_site_user
user = model.User.get(site_id)
File "/usr/lib/ckan/default/src/ckan/ckan/model/user.py", line 64, in get
return query.first()
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2334, in first
ret = list(self[0:1])
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2201, in __getitem__
return list(res)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2405, in __iter__
return self._execute_and_instances(context)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2420, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 727, in execute
return meth(self, multiparams, params)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 824, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 954, in _execute_context
context)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1116, in _handle_dbapi_exception
exc_info
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 189, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 947, in _execute_context
context)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 435, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (ProgrammingError) column user.password does not exist
LINE 1: SELECT "user".password AS user_password, "user".id AS user_i...
^
'SELECT "user".password AS user_password, "user".id AS user_id, "user".name AS user_name, "user".openid AS user_openid, "user".fullname AS user_fullname, "user".email AS user_email, "user".apikey AS user_apikey, "user".created AS user_created, "user".reset_key AS user_reset_key, "user".about AS user_about, "user".activity_streams_email_notifications AS user_activity_streams_email_notifications, "user".sysadmin AS user_sysadmin, "user".state AS user_state \nFROM "user" \nWHERE "user".name = %(name_1)s OR "user".openid = %(openid_1)s OR "user".id = %(id_1)s ORDER BY "user".name \n LIMIT %(param_1)s' {'param_1': 1, 'id_1': 'default', 'name_1': 'default', 'openid_1': 'default'}
I did not make any modifications to the installation or the schema. The installation is under a proxy, but that did not turn out to be a problem thus far, since there were many posts giving help in that regard.
Did anybody come across this error before and managed to resolve it? I don't suppose it is a proxy-related problem... though it seems to be related to the schema.
I ran into the same issue but dropping the DB and creating and initialising it again afterwards produced the same error. Also running paster db clear -c /etc/ckan/default/development.ini failed with said error.
It worked for me, when I deactivated all ckan.plugins in development.ini before running paster db init -c /etc/ckan/default/development.ini.
Managed to solve this one after all, so I'll answer my own question... :)
The problem was with the setup of the database. Once I purged the database
and completely reinstalled it, everything worked like a charm!
sqlalchemy.exc.ProgrammingError: (ProgrammingError) column user.password does not exist
LINE 1: SELECT "user".password AS user_password, "user".id AS user_i...
This error usually means that the CKAN database exists but is empty - the CKAN tables have not been created in it.
To solve it:
cd /usr/lib/ckan/default/src/ckan
paster db init -c /etc/ckan/default/development.ini
There's more about this in the docs:
http://docs.ckan.org/en/latest/maintaining/installing/install-from-source.html#create-database-tables