I used the following guide so far:
http://docs.wagtail.io/en/v1.9/advanced_topics/performance.html
I added:
WAGTAILSEARCH_BACKENDS = {
'default': {
'BACKEND': 'wagtail.wagtailsearch.backends.elasticsearch',
'INDEX': '{{ project_name }}',
},
}
to my base.py. When I run $ ./manage.py update_index, I get the following output:
Updating backend: default
default: Rebuilding index {{ project_name }}
Traceback (most recent call last):
File "./manage.py", line 12, in <module>
execute_from_command_line(sys.argv)
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
utility.execute()
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 359, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/django/core/management/base.py", line 294, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/django/core/management/base.py", line 345, in execute
output = self.handle(*args, **options)
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/wagtail/wagtailsearch/management/commands/update_index.py", line 120, in handle
self.update_backend(backend_name, schema_only=options.get('schema_only', False))
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/wagtail/wagtailsearch/management/commands/update_index.py", line 73, in update_backend
index = rebuilder.start()
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/wagtail/wagtailsearch/backends/elasticsearch.py", line 613, in start
self.reset_index()
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/wagtail/wagtailsearch/backends/elasticsearch.py", line 609, in reset_index
self.index.reset()
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/wagtail/wagtailsearch/backends/elasticsearch.py", line 598, in reset
self.delete()
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/wagtail/wagtailsearch/backends/elasticsearch.py", line 499, in delete
self.es.indices.delete(self.name)
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
return func(*args, params=params, **kwargs)
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/elasticsearch/client/indices.py", line 200, in delete
params=params)
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/home/emtr0/Env/emtr0dotcom/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 123, in perform_request
raise ConnectionError('N/A', str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f3b708fa210>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f3b708fa210>: Failed to establish a new connection: [Errno 111] Connection refused)
Thanks in advance.
EDIT UPDATE:
It looks like my Linode might not have enough RAM. I get the following error when trying to run ES with ./bin/elasticsearch after global install:
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085330000, 2060255232, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 2060255232 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /home/emtr0/elasticsearch-5.3.0/hs_err_pid24719.log
Thanks.
For people still having issues with this, make sure your Elasticsearch version and elasticsearch-py version are the same. Both elasticsearch and the py versions need to be installed. That fixed it for me.
sorry I don't have enough reputation to comment directly under your question, I hope this will help you to get things sorted out:
install elasticsearch locally, run it, and then check if it's running (should be on port 9200) using a simple curl http://localhost:9200. If you get a JSON response, it's running 🎉
OR
setup a bonsai (http://bonsai.io) instance in order to have a hosted elasticsearch server running, and just copy-paste the bonsai's server URL into your config, but it can need a bit more settings (SSL etc), and is more limited in term of elasticsearch version
Bonus: if you are a macOS user, installing/running Elastic Search is as easy as brew install elasticsearch (maybe set a specific version though), and then elasticsearch if you don't mind about having it running in the foreground.
Related
I had to install Jupyter in order to complete Udacity's projects. After following the instructions, and running Jupyter, none of the notebooks would open. This is the log output:
[W 06:54:04.215 NotebookApp] The signatures database cannot be opened; maybe it is corrupted or encrypted. You may need to rerun your notebooks to ensure that they are trusted to run Javascript. The old signatures database has been renamed to /home/aulon/.local/share/jupyter/nbsignatures.db.bak and a new one has been created.
[E 06:54:04.218 NotebookApp] Unhandled error in API request
Traceback (most recent call last):
File "/home/aulon/.local/lib/python2.7/site-packages/notebook/base/handlers.py", line 516, in wrapper
result = yield gen.maybe_future(method(self, *args, **kwargs))
File "/home/aulon/.local/lib/python2.7/site-packages/tornado/gen.py", line 1055, in run
value = future.result()
File "/home/aulon/.local/lib/python2.7/site-packages/tornado/concurrent.py", line 238, in result
raise_exc_info(self._exc_info)
File "/home/aulon/.local/lib/python2.7/site-packages/tornado/gen.py", line 307, in wrapper
yielded = next(result)
File "/home/aulon/.local/lib/python2.7/site-packages/notebook/services/contents/handlers.py", line 124, in get
path=path, type=type, format=format, content=content,
File "/home/aulon/.local/lib/python2.7/site-packages/notebook/services/contents/filemanager.py", line 384, in get
model = self._notebook_model(path, content=content)
File "/home/aulon/.local/lib/python2.7/site-packages/notebook/services/contents/filemanager.py", line 344, in _notebook_model
self.mark_trusted_cells(nb, path)
File "/home/aulon/.local/lib/python2.7/site-packages/notebook/services/contents/manager.py", line 456, in mark_trusted_cells
trusted = self.notary.check_signature(nb)
File "/home/aulon/.local/lib/python2.7/site-packages/traitlets/traitlets.py", line 556, in __get__
return self.get(obj, cls)
File "/home/aulon/.local/lib/python2.7/site-packages/traitlets/traitlets.py", line 535, in get
value = self._validate(obj, dynamic_default())
File "/home/aulon/.local/lib/python2.7/site-packages/notebook/services/contents/manager.py", line 58, in _notary_default
return sign.NotebookNotary(parent=self)
File "/home/aulon/.local/lib/python2.7/site-packages/nbformat/sign.py", line 390, in __init__
self.store = self.store_factory()
File "/home/aulon/.local/lib/python2.7/site-packages/nbformat/sign.py", line 338, in factory
return SQLiteSignatureStore(self.db_file)
File "/home/aulon/.local/lib/python2.7/site-packages/nbformat/sign.py", line 147, in __init__
self.db = self._connect_db(db_file)
File "/home/aulon/.local/lib/python2.7/site-packages/nbformat/sign.py", line 171, in _connect_db
os.rename(db_file, old_db_location)
OSError: [Errno 2] No such file or directory
...
I googled and found some similar problems here, here, and here, without success.
I installed Anaconda, still the same problem.
Any help is much appreciated.
If I cannot solve this, how can I fully uninstall everything related to jupyter and anaconda? I am using Ubuntu 16.04.
Otherwise, is there any other way I can open a python notebook without jupyter? Maybe Spyder?
SOLVED!
It was a permission error.
Follow the link here if you are having the same problem.
If you are working on Windows OS try this solution. The issue is because of permission by Windows Defender.
Go to "Windows Defender Security Center"
Scroll & Click on "Virus & threat protection settings"
Scroll down to "Controlled folder access"
Make sure is Disabled
I am using django-nonrel with mongodb-engine I am getting the following error when I run python manage.py syncdb:
Traceback (most recent call last):
File "manage.py", line 22, in <module> execute_from_command_line(sys.argv)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(*args, **options)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/django/core/management/base.py", line 415, in handle
return self.handle_noargs(**options)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 147, in handle_noargs
index_sql = connection.creation.sql_indexes_for_model(model, self.style)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/django_mongodb_engine/creation.py", line 49, in sql_indexes_for_model
self._handle_oldstyle_indexes(ensure_index, meta)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/django_mongodb_engine/creation.py", line 116, in _handle_oldstyle_indexes
sparse=field.name in sparse_indexes)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/django_mongodb_engine/creation.py", line 42, in ensure_index
return collection.ensure_index(*args, **kwargs)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/pymongo/collection.py", line 1420, in ensure_index
self.__create_index(keys, kwargs)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/pymongo/collection.py", line 1298, in __create_index
sock_info, cmd, read_preference=ReadPreference.PRIMARY)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/pymongo/collection.py", line 208, in _command
read_concern=read_concern)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/pymongo/pool.py", line 239, in command
read_concern)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/pymongo/network.py", line 102, in command
helpers._check_command_response(response_doc, None, allowable_errors)
File "/Users/<user>/site/qmcdb/lib/python2.7/site-packages/pymongo/helpers.py", line 205, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: The field 'sparse' is not valid for an _id index specification. Specification: { ns: "qmcdb_mongodb.django_admin_log", v: 2, sparse: false, unique: true, name: "_id_1", key: { _id: 1 } }
I have no idea what is wrong. I've been trying to get mongodb to work with django and I have been having a lot of issues. My settings.py:
DATABASES = {
'default': {
'ENGINE': 'django_mongodb_engine',
'NAME': 'qmcdb_mongodb',
'PORT': 27017,
'HOST': 'localhost'
}
}
I hope I have posted enough background information. I would really appreciate any help.
Try use Mongo database in version 3.2, There is some problem with mongoengine and Mongo Server 3.4
1.Tested syncdb in MongoDB version 3.2 3.4, it still raising error.
2.In the MongoDB 2.6 syncdb is running fine with a minor unrelated problem.I still got an error in version 2.6 and fixed with this(https://gist.github.com/ielshareef/2986459).
3.django_mongodb_engine is so buggy which was forked from django1.3 . Also, this repo is outdated that last commit was on Jul 13, 2015. I should not recommend this package.
4.I would recommend https://github.com/MongoEngine/django-mongoengine
My hacky solution to this was modifying lines 115-116 of Lib\site-packages\django_mongodb_engine\creation.py
from this:
ensure_index(column, unique=field.unique,
sparse=field.name in sparse_indexes)
to this:
ensure_index(column)
It seems to clear the error, but I'm sure it's gonna make something somewhere fail catastrophically because I have no idea what I'm doing.
I'll be waiting on a better answer too.
I recommend you use:
pip3 install git+https://github.com/MongoEngine/django-mongoengine
Django 1.11.2
Python 3.6
MongoDB 3.4
Always is good, use versions updated of packages, use isolation of environment.
https://virtualenvwrapper.readthedocs.io/en/latest/install.html
Now I use flask to develop web app.
But at first it works well,after operating web page for a while,the flask back-end shows error like these:
File "/usr/lib64/python2.6/BaseHTTPServer.py", line 329, in handle
self.handle_one_request()
File "/usr/lib/python2.6/site-packages/werkzeug/serving.py", line 251, in handle_one_request
return self.run_wsgi()
File "/usr/lib/python2.6/site-packages/werkzeug/serving.py", line 193, in run_wsgi
execute(self.server.app)
File "/usr/lib/python2.6/site-packages/werkzeug/serving.py", line 184, in execute
write(data)
File "/usr/lib/python2.6/site-packages/werkzeug/serving.py", line 152, in write
self.send_header(key, value)
File "/usr/lib64/python2.6/BaseHTTPServer.py", line 390, in send_header
self.wfile.write("%s: %s\r\n" % (keyword, value))
IOError: [Errno 32] Broken pipe
My app run on port 5000 app.run(debug=True,port=5000),
I use nginx as web server,and set proxy_pass http://127.0.0.1:5000 in nginx config file.
Now I really don't know where is the wrong,I use session['email'] = request.form['email'] and in other file I use email = session.get('email').
Is this usage right? How to set session active period?
or any other reason cause this error ?
then I set app.run(debug=False,port=5000),it shows new error
File "/usr/lib64/python2.6/SocketServer.py", line 671, in finish
self.wfile.flush()
File "/usr/lib64/python2.6/socket.py", line 303, in flush
self._sock.sendall(buffer(data, write_offset, buffer_size))
socket.error: [Errno 32] Broken pipe
why ?
Please help me,thks.
The built-in werkzeug server is not capable of handling the remote end closing the connection while the server is still churing its content out.
instead of app.run(debug=True,port=5000)
try
from gevent.wsgi import WSGIServer
http_server = WSGIServer(('', 5000), app)
http_server.serve_forever()
or if you are using nginx, use it with uwsgi as described here
It is rather a werkzeug issue I would argue
I have tried to use the below forum to item to fix the problem but it did not seam to work for me
https://stackoverflow.com/questions/21955234/ckan-install-paster-error
Amazingly I got the same issue when I tried to install CKAN on windows.
paster db init -c XXXX/development.ini not working for CKAN-command 'db' not know
This time I am trying to install CKAN on Ubuntu 12.04 (actually 12.04.5 as I couldn't get 12.0.4) as instructed in
http://docs.ckan.org/en/latest/maintaining/installing/install-from-source.html
I am having to install everything using a PROXY
I have added the password to the SQL Chemistry and the Development.ini does exist. This is my error (below)
Is this a proxy issue? I have used the chmod to change the access to the ini file as the other forum recommended. I also set the virtual path. The database base does exist as I check it.
:
(default)root#UbuntaDataServer:/usr/lib/ckan/default/src/ckan# paster db init -c /etc/ckan/default/development.ini
Traceback (most recent call last):
File "/usr/lib/ckan/default/bin/paster", line 9, in <module>
load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 104, in run
invoke(command, command_name, options, args[1:])
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 143, in invoke
exit_code = runner.run(args)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 238, in run
result = self.command()
File "/root/ckan/lib/default/src/ckan/ckan/lib/cli.py", line 156, in command
self._load_config()
File "/root/ckan/lib/default/src/ckan/ckan/lib/cli.py", line 98, in _load_config
load_environment(conf.global_conf, conf.local_conf)
File "/root/ckan/lib/default/src/ckan/ckan/config/environment.py", line 232, in load_environment
p.load_all(config)
File "/root/ckan/lib/default/src/ckan/ckan/plugins/core.py", line 124, in load_all
unload_all()
File "/root/ckan/lib/default/src/ckan/ckan/plugins/core.py", line 182, in unload_all
unload(*reversed(_PLUGINS))
File "/root/ckan/lib/default/src/ckan/ckan/plugins/core.py", line 210, in unload
plugins_update()
File "/root/ckan/lib/default/src/ckan/ckan/plugins/core.py", line 116, in plugins_update
environment.update_config()
File "/root/ckan/lib/default/src/ckan/ckan/config/environment.py", line 270, in update_config
search.check_solr_schema_version()
File "/root/ckan/lib/default/src/ckan/ckan/lib/search/__init__.py", line 291, in check_solr_schema_version
res = urllib2.urlopen(req)
File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 406, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 444, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 503: Service Unavailable
This part of the stacktrace:
File "/root/ckan/lib/default/src/ckan/ckan/lib/search/init.py", line 291, in check_solr_schema_version
res = urllib2.urlopen(req)
Suggests that there is a problem connecting to Solr. You should make sure solr is running, that you can connect to it, and that the setting in your .ini file for the location and port that solr is running on is correct.
This is not the complete answer. Maybe close.
This is what I see on http:||localhost|solr|
Solr Admin (ckan)
UbuntaDataServer:8983
cwd=/var/cache/jetty/tmp SolrHome=/usr/share/solr/
This is what is running on the URL. I assume this is close or correct?
Any more suggestions?
Using CKAN 2.2 I had the same problem with proxies that require authentication
If you are installing CKAN from sources, I suggest to move to 2.2.1 version (or newer).
In these versions I found no issues with auth proxy.
Anyway, if you're bound to a specific, older version of CKAN, you can manually add a proxy handler.
First of all, set your http_proxy env vars (both uppercase and lowercase)
Now you can edit the file ckan/ckan/lib/search/__init__.py and get your hands dirty.
We need to declare a handle_proxy() function:
import os
def handle_proxy():
proxy_settings = dict()
for k,v in os.environ.items():
if k.rfind('_proxy') > -1:
proxy_settings[k] = v
proxy_handler = urllib2.ProxyHandler(proxy_settings)
opener = urllib2.build_opener(proxy_handler)
urllib2.install_opener(opener)
Now we can call it in the check_solr_schema_version() function just before sending the request.
Replace
res = urllib2.urlopen(req)
with
handle_proxy()
res = urllib2.urlopen(req)
NOTE: this is a temporary workaround, just in case upgrading to newer versions (i currently use the 2.2.2 branch) does not fix the problem for you. I wouldn't suggest it for a production environment :)
I found another answer, if the above does not work, try:
Install this again:
sudo -E apt-get install python-pastescript
. /usr/lib/ckan/default/bin/activate
cd /usr/lib/ckan/default/src/ckan
paster make-config ckan /etc/ckan/default/development.ini
Change SOLR to your IP number and not localhost
paster db init -c /etc/ckan/default/development.ini
Hope that fixes your problem
I'm trying to use django-mssql to connect to MS SQL Server 2008 R2 with Django 1.4.2 These are my Database settings:
DATABASE_ENGINE = 'sqlserver_ado'
DATABASE_NAME = 'dbtest'
DATABASE_USER = 'App'
DATABASE_PASSWORD = '*********'
DATABASE_HOST = 'localhost'
DATABASE_OPTIONS = {
'provider': 'SQLNCLI10',
'extra_params': 'DataTypeCompatibility=80;MARS Connection=True;',
}
DATABASES = {
'default': {
'ENGINE': DATABASE_ENGINE,
'NAME': DATABASE_NAME,
'USER': DATABASE_USER,
'PASSWORD': DATABASE_PASSWORD,
'HOST': DATABASE_HOST,
'OPTIONS' : DATABASE_OPTIONS,
},
}
This is the error I get when I try to syncdb
Traceback (most recent call last):
File "C:\Python27\DataSatellite\manage.py", line 11, in <module>
execute_manager(settings)
File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line 459, in execute_manager
utility.execute()
File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line 382, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 196, in run_from_argv
self.execute(*args, **options.__dict__)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 232, in execute
output = self.handle(*args, **options)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 371, in handle
return self.handle_noargs(**options)
File "C:\Python27\lib\site-packages\django\core\management\commands\syncdb.py", line 57, in handle_noargs
cursor = connection.cursor()
File "C:\Python27\lib\site-packages\django\db\backends\__init__.py", line 306, in cursor
cursor = self.make_debug_cursor(self._cursor())
File "C:\Python27\lib\site-packages\sqlserver_ado\base.py", line 193, in _cursor
self.__connect()
File "C:\Python27\lib\site-packages\sqlserver_ado\base.py", line 168, in __connect
use_transactions=self.use_transactions,
File "C:\Python27\lib\site-packages\sqlserver_ado\dbapi.py", line 151, in connect
raise OperationalError(e, "Error opening connection: " + connection_string)
sqlserver_ado.dbapi.OperationalError: (AttributeError("'module' object has no attribute 'VARIANT'",), 'Error opening connection: DATA SOURCE=localhost;Initial Catalog=dbtest;UID=App;PWD=*********;PROVIDER=SQLNCLI10;MARS Connection=True;DataTypeCompatibility=80;MARS Connection=True;')
Finished "C:\Python27\DataSatellite\manage.py syncdb" execution.
I've looked everywhere and I cannot seem to understand and fix the problem. I hope someone can help!
Thanks!
Edit:
I've already created the database. I've also connected to the database using django-pyodbc, and I've successfully read and written from the database. But django-pyodbc causes problems when I use Apache, which was why I decided to try django-mssql. However, I do not understand the error it comes up with.
My Django (1.4.2) and Python (2.7) installs run on Windows, and I'm using an Apache webserver.
Update pywin32 to build 217 or greater. I found another question on stackoverflow with the same issue (on another topic):
Python COM server throws 'module' object has no attribute 'VARIANT'