I'm writing a site on Flask, I decided to make a search system on the site using Elasticsearch, I did it according to the guide https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xvi-full-text-search .
Here is my code
es = Elasticsearch(os.environ.get('ELASTICSEARCH_URL'))
app.elasticsearch = Elasticsearch([app.config['ELASTICSEARCH_URL']]) if app.config['ELASTICSEARCH_URL'] else None
But I get an error:
app.elasticsearch = Elasticsearch([app.config['ELASTICSEARCH_URL']]) if app.config['ELASTICSEARCH_URL'] else None
KeyError: 'ELASTICSEARCH_URL'
Please help to fix this
I think you have simply not degined the elasticsearch key inside your app config. According to the documentation you should first make
app.config['ELASTICSEARCH_URL'] = os.environ.get('ELASTICSEARCH_URL')
In the end the app config behaves as a dictionary, and you are trying to access a key that does not exist
Related
I am trying to export a tableau view as an image/csv (doesn't matter) using Python. I googled and found that REST API would help here, so I created a Personal Access Token and wrote the following command to connect: -
import tableauserverclient as TSC
from tableau_api_lib import TableauServerConnection
from tableau_api_lib.utils.querying import get_views_dataframe, get_view_data_dataframe
server_url = 'https://tableau.mariadb.com'
site = ''
mytoken_name = 'Marine'
mytoken_secret = '$32mcyTOkmjSFqKBeVKEZYpMUexseV197l2MuvRlwHghMacCOa'
server = TSC.Server(server_url, use_server_version=True)
tableau_auth = TSC.PersonalAccessTokenAuth(token_name=mytoken_name, personal_access_token=mytoken_secret, site_id=site)
with server.auth.sign_in_with_personal_access_token(tableau_auth):
print('[Logged in successfully to {}]'.format(server_url))
It entered successfully and gave the message: -
[Logged in successfully to https://tableau.mariadb.com]
However, Iam at a loss now on how to access the tableau workbooks using Python. I searched here:-
https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api_ref_workbooks_and_views.htm
but was unable to write these commands like GET or others in python.
Can anyone help?
I'm assuming you don't know the view_id of the view you're looking for
Adding this after the print in the with block will query all the views available on your site;
all_views, pagination_item = server.views.get()
print([view.name for view in all_views])
Then find the view you're looking for in the printed output and note the view_id for use like this;
view_item = server.view.get_by_id('d79634e1-6063-4ec9-95ff-50acbf609ff5')
From there, you can get the image like this;
server.views.populate_image(view_item)
with open('./view_image.png', 'wb') as f:
f.write(view_item.image)
The tableauserverclient-python docs should help you out a ton as well
https://tableau.github.io/server-client-python/docs/api-ref#views
The Jinja documentation states the following regarding line statements:
If line statements are enabled by the application, it’s possible to mark a line as a statement.
In this video, line statements are enabled/configured like this:
from flask import Flask
app = Flask(__name__)
app.jinja_env.line_statement_prefix = '%'
However, in Flask 1.0.4 my application object does not have this attribute.
How can I enable and configure line statements?
So according to the source, app.jinja_env is a locked_cached_property which is created the first time it is accessed. So we can't set options directly on app.jinja_env.
What we can do is set app.jinja_options when we are creating our app so that when jinja goes to load the environment it looks at the default app.jinja_options in Flask already which are
jinja_options = {"extensions": ["jinja2.ext.autoescape", "jinja2.ext.with_"]}
So with that, I believe the following should do what we need
from flask import Flask
Flask.jinja_options = {'extensions': ['jinja2.ext.autoescape', 'jinja2.ext.with_'], 'line_statement_prefix': '%'}
app = Flask(__name__)
Flask breaks up the options object, passes that to the Environment which is a subclass of Jinja Environment which then assigns the line_statement_prefix.
Using python pyramid and ElastiSearch. I looked at pythonelasticsearch-dsl which offers a nice ORM but I'm not sure how to integrate it with pyramid.
So far I made a "global connection" as per pythonelasticsearch-dsl and expose the connection via an attribute into pyramid's request.
Do you see anything wrong with this code ?!
from elasticsearch_dsl import connections
def _create_es_connection(config):
registry = config.registry
settings = registry.settings
es_servers = settings.get('elasticsearch.' + 'servers', ['localhost:9200'])
es_timeout = settings.get('elasticsearch.' + 'timeout', 20)
registry.es_connection = connections.create_connection(
hosts=es_servers,
timeout=es_timeout)
def get_es_connection(request):
return getattr(request.registry, 'es_connection',
connections.get_connection())
# main
def main(global_config, **settings):
...
config = Configurator(settings=settings)
config.add_request_method(
get_es_connection,
'es',
reify=True)
I use the connection as
#view
request.es ...
If there are any other ways I would appreciate any pointers - thank you.
A few things look weird, but I guess it comes from the copy/paste from your project (missing type cast in settings, connections undefined, etc.)
What you are trying to do is very similar to what you'd do with SQLAlchemy:
https://docs.pylonsproject.org/projects/pyramid_cookbook/en/latest/database/sqlalchemy.html
But according the docs of pythonelasticsearch-dsl you don't even have to bother with all that, since the lib allows you define a global default connection:
https://elasticsearch-dsl.readthedocs.io/en/latest/configuration.html#default-connection
I have following code written in python in order to communicate with ExistDB using eulexistdb module.
from eulexistdb import db
class TryExist:
def __init__(self):
self.db = db.ExistDB(server_url="http://localhost:8899/exist")
def get_data(self, query):
result = list()
qresult = self.db.executeQuery(query)
hits = self.db.getHits(qresult)
for i in range(hits):
result.append(str(self.db.retrieve(qresult, i)))
return result
query = '''
let $x:= doc("/db/sample/books.xml")
return $x/bookstore/book/author/text()
'''
a = TryExist()
response = a.get_data(query)
print response
I am amazed that this code runs fine in Aptana Studio 3 giving me the output I want, but when running from other IDE or using command "python.exe myfile.py" brings following error:
django.core.exceptions.ImproperlyConfigured: Requested setting EXISTDB_TIMEOUT, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
I used my own localsetting.py to solve the problem using following code:
import os
# must be set before importing anything from django
os.environ['DJANGO_SETTINGS_MODULE'] = 'localsettings'
... writing link for existdb here...
Then I get error as:
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
How do I configure the setting in Django to suit for ExistDB? Help me here please..
Never Mind. I found the answer with little research from this site. What I did was created a localsetting.py file with following configurations.
EXISTDB_SERVER_USER = 'user'
EXISTDB_SERVER_PASSWORD = 'admin'
EXISTDB_SERVER_URL = "http://localhost:8899/exist"
EXISTDB_ROOT_COLLECTION = "/db"
and in my main file myfile.py I used :
from localsettings import EXISTDB_SERVER_URL
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'localsettings.py'
and In the class TryExist I changed in __ init __() as:
def __init__(self):
self.db = db.ExistDB(server_url=EXISTDB_SERVER_URL)
PS: Using only os.environ['DJANGO_SETTINGS_MODULE'] = 'localsettings' brings the django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty..
The reason your code works in an IDE but not at the command line is probably that you have a difference in what Python environments are used to run your code.
I've done a couple of tests:
Virtualenv with eulexistdb installed but not Django. eulexistdb tries to load django.conf but fails and so does not try to get its configuration from a Django configuration. Ultimately, your code runs without error.
Virtualenv with 'eulexistdb*and* Django:eulexistdbtries to loaddjango.conf` and succeed. I then tries to get is configuration from the Django configuration but fails. I get the same error you describe in your question.
To prevent the error in the presence of a Django installation, the problem can be fixed by adding a Django configuration like you did in your accepted self-answer. But if the code you are writing does not otherwise use Django, that's a bit of a roundabout way to get your code to run. The most direct way to fix the problem is to simply add a timeout parameter to the code that creates the ExistDB instance:
self.db = db.ExistDB(
server_url="http://localhost:8080/exist", timeout=None)
If you do this, then there won't be any error. Setting the timeout to None leaves the default behavior in place but prevents eulexistdb from looking for a Django configuration.
I'm trying to learn Pyramid and having problems getting the message flash to work. I'm totally new but read the documentation and did the tutorials.
I did the tutorial on creating a wiki(tutorial here, Code here ). It worked great and was pretty easy so I decided to try to apply the flash message I saw in todo list tutorial I did(tutorial here, full code is in a single file at the bottom of the page). Basically when a todo list is created, the page is refreshed with a message saying 'New task was successfully added!'. I wanted to do that everytime someone updated a wiki article in the wiki tutorial.
So I re-read the session section in the documentaion and it says I really just need to do this:
from pyramid.session import UnencryptedCookieSessionFactoryConfig
my_session_factory = UnencryptedCookieSessionFactoryConfig('itsaseekreet')
from pyramid.config import Configurator
config = Configurator(session_factory = my_session_factory)
then in my code I need to add: request.session.flash('New wiki was successfully added!') but I get a error everytime: Pyramid error: AttributeError: No session factory registered
Here's my function(its the exact same from the tutorial except for the request.session.flash part):
#view_config(route_name='edit_page', renderer='templates/edit.pt', permission='edit')
def edit_page(request):
name = request.matchdict['pagename']
page = DBSession.query(Page).filter_by(name=name).one()
if 'form.submitted' in request.params:
page.data = request.params['body']
DBSession.add(page)
request.session.flash('page was successfully edited!')
return HTTPFound(location = request.route_url('view_page',
pagename=name))
return dict(
page=page,
save_url = request.route_url('edit_page', pagename=name),
logged_in=authenticated_userid(request),
)
(note: One thing that I think I could be doing wrong is in the todo example, all the data is in one file, but in the wiki example there are several files..I added my session imports in the view.py file because the flash message is being generated by the view itself).
What am I doing wrong? Any suggestions?
The code you provided is just an example, of course you need to apply it in a correct place. In Pyramid you should (in simple cases ;) have only 1 place in your code where you create just 1 Configurator instance, in the tutorial it is in the main function. A Configurator does not do anything by itself, except create a WSGI application with make_wsgi_app.
Thus, to add sessions there, modify wiki2/src/views/tutorial/__init__.py as follows:
from pyramid.config import Configurator
from sqlalchemy import engine_from_config
from pyramid.session import UnencryptedCookieSessionFactoryConfig
from .models import DBSession
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
engine = engine_from_config(settings, 'sqlalchemy.')
DBSession.configure(bind=engine)
my_session_factory = UnencryptedCookieSessionFactoryConfig('itsaseekreet')
config = Configurator(settings=settings, session_factory=my_session_factory)
...