I'm currently working through the Flask Mega-Tutorial (Part XVI) and have gotten stuck implementing elasticsearch. Specifically, I get this error when running the following from my flask shell command line:
from app.search import add_to_index, remove_from_index, query_index
>>> for post in Post.query.all():
... add_to_index('posts', post)
AttributeError: module 'flask.app' has no attribute 'elasticsearch'
I should mention that I did not implement the app restructuring from the previous lesson to use blueprints. Here's what my files look like:
__init__.py:
#
from elasticsearch import Elasticsearch
app.elasticsearch = Elasticsearch([app.config['ELASTICSEARCH_URL']]) \
if app.config['ELASTICSEARCH_URL'] else None
config.py:
class Config(object):
#
ELASTICSEARCH_URL = 'http://localhost:9200'
search.py:
from flask import app
def add_to_index(index, model):
if not app.elasticsearch:
return
payload = {}
for field in model.__searchable__:
payload[field] = getattr(model, field)
app.elasticsearch.index(index=index, id=model.id, body=payload)
def remove_from_index(index, model):
if not app.elasticsearch:
return
app.elasticsearch.delete(index=index, id=model.id)
def query_index(index, query, page, per_page):
if not app.elasticsearch:
return [], 0
search = app.elasticsearch.search(
index=index,
body={'query': {'multi_match': {'query': query, 'fields': ['*']}},
'from': (page - 1) * per_page, 'size': per_page})
ids = [int(hit['_id']) for hit in search['hits']['hits']]
return ids, search['hits']['total']['value']
I think I'm not importing elasticsearch correctly into search.py but I'm not sure how to represent it given that I didn't do the restructuring in the last lesson. Any ideas?
The correct way to write it in the search.py file should be from flask import current_app
Not sure if you got this working, but the way I implemented it was by still using app.elasticsearch but instead within search.py do:
from app import app
Related
I can't find anything on the internet about how to actually extract and utilize the OpenAPI specification generated by the apispec package to make a Swagger page.
I don't want to rely on a package that hasn't been being actively maintained like flask-apispec. I want to use flask-swagger-ui, apispec, and other standard/well-maintained packages only.
Here is my test app. I don't know if the APISpec is working right because the documentation doesn't tell you anything about what you do with the object, but the Flask app is functional.
from flask import Flask, request, abort
from marshmallow import Schema, fields
from apispec import APISpec
from apispec.ext.marshmallow import MarshmallowPlugin
from apispec_webframeworks.flask import FlaskPlugin
app = Flask(__name__)
spec = APISpec(
title="Doubler",
version="1.0.0",
openapi_version="3.0.2",
plugins=[FlaskPlugin(), MarshmallowPlugin()],
)
class InputSchema(Schema):
useless_1 = fields.String(required=True, description='A string')
useless_2 = fields.Int(missing=5, description='An integer')
class OutputSchema(Schema):
doublyuseless_1 = fields.String(required=True)
doublyuseless_2 = fields.Int(required=True)
inputschema = InputSchema()
outputschema = OutputSchema()
#app.route('/double', methods=['GET'])
def double():
"""A doubler.
---
get:
description: Double things
parameters:
schema:
InputSchema
responses:
200:
description: Double things
content:
application/json:
schema: OutputSchema"""
errors = inputschema.validate(request.args)
if errors:
abort(400, str(errors))
return_dict = {}
args = inputschema.load(request.args)
return_dict['doublyuseless_1'] = args['useless_1']*2
return_dict['doublyuseless_2'] = args['useless_2']*2
return outputschema.dump(return_dict)
with app.test_request_context():
spec.path(view=double)
UPDATE: with the following code, I now get a blank page at root with the Swagger title, but no content.
with app.test_request_context():
spec.path(view=double)
with open('swagger.json', 'w') as f:
dict_ = yaml.load(StringIO(spec.to_yaml()), Loader=yaml.SafeLoader)
print(dict_)
with open('swagger.json', 'w') as f:
json.dump(dict_, f)
SWAGGER_URL = '/'
API_URL = 'swagger.json'
swaggerui_blueprint = get_swaggerui_blueprint(
SWAGGER_URL, # Swagger UI static files will be mapped to '{SWAGGER_URL}/dist/'
API_URL,
config={ # Swagger UI config overrides
'app_name': "Doubler"
},
# oauth_config={ # OAuth config. See https://github.com/swagger-api/swagger-ui#oauth2-configuration .
# 'clientId': "your-client-id",
# 'clientSecret': "your-client-secret-if-required",
# 'realm': "your-realms",
# 'appName': "your-app-name",
# 'scopeSeparator': " ",
# 'additionalQueryStringParams': {'test': "hello"}
# }
)
The solution to my original problem was quite straightforward: ApiSpec has both a to_dict and a to_yaml method for exporting swagger.json. My second problem was more esoteric. I needed to use a SWAGGER_URL that was not /, because for some reason this caused the page to look for the core Swagger files at URLs like http://swagger-ui.js, which obviously didn't work. Once I changed my path to /doc, I still had a white screen, but that could be fixed by hosting the files myself at /doc (which I think flask_swagger_ui was supposed to do automatically, but hey, it worked).
I'm trying to use Celery on my Flask application.
I'm defining a task in a file insight_tasks.py.
In that file is defined a function:
#celery_app.task
def save_insights_task()
That function do some stuff and, here comes the error, I'm trying to save data into MongoDB and the console throws me:
MongoEngineConnectionError('You have not defined a default connection',)
So I think it's because MongoEngine has not been initialized, and here is my question:
How should I use MongoDB inside a Celery Task?, because when using MongoDB on my routes (flask app) It's working as expected.
Celery does not share the db instance?
Files:
__init__.py (Celery intialization)
celery_app = Celery('insights',
broker=config.CELERY_LOCATIONS_BROKER_URL,
backend=config.CELERY_LOCATIONS_RESULT_BACKEND,
include=['app.insight_tasks']
)
insight_tasks.py
from app.google import google_service
from app.models import LocationStats
from . import celery_app
from firebase_admin import db as firebase_db
import arrow
#celery_app.task
def save_insight_task(account_location, uid, gid, locations_obj, aggregation):
try:
insights, reviews = google_service.store_location_resources(
gid, uid,
start_datetime, end_datetime,
account_location, aggregation
)
except StandardError as err:
from pprint import pprint
import traceback
pprint(err)
pprint(traceback.print_exc())
path = 'saved_locations/{}/accounts/{}'.format(gid, account_location)
location = [loc for loc in locations_obj if loc['name'] == 'accounts/' + account_location]
if len(location) > 0:
firebase_db.reference(path).update(location[0])
Here google_service.store_location_resources() is the function that saves thew data into MongoDB. this function is used on another side, by the routes of my app, so it works as expected, except on the Celery task
---------
The Celery task is called into a POST request
accounts/routes.py
#account.route('/save/queue', methods=['POST'])
def save_all_locations():
data = request.data
dataDict = json.loads(data)
uid = request.headers.get('uid', None)
gid = request.headers.get('gid', None)
account_locations = dataDict['locations']
locations_obj = dataDict['locations_obj']
for path in account_locations:
save_insight_task.delay(account_location=path, uid=uid, gid=gid, locations_obj=locations_obj, aggregate='SOME_TEXT')
You are supposed to connect to the database inside the task. The reason is because child processes (created by Celery) must have their own instance of mongo client.
More details here : Using PyMongo with Multiprocessing
For example define a utils.py :
from pymodm import connect
def mongo_connect():
return connect("mongodb://{0}:{1}/{2}".format(MONGODB['host'],
MONGODB['port'],
MONGODB['db_name']),
alias=MONGODB['db_name'])
Then in insight_tasks.py
from utils import mongo_connect
#celery_app.task
def save_insight_task(account_location, uid, gid, locations_obj, aggregation):
# connect to mongodb
mongo_connect()
# do your db operations
try:
insights, reviews = google_service.store_location_resources(
gid, uid,
start_datetime, end_datetime,
account_location, aggregation
)
except StandardError as err:
from pprint import pprint
import traceback
pprint(err)
pprint(traceback.print_exc())
path = 'saved_locations/{}/accounts/{}'.format(gid, account_location)
location = [loc for loc in locations_obj if loc['name'] == 'accounts/' + account_location]
if len(location) > 0:
firebase_db.reference(path).update(location[0])
Note that I use pymodm package instead of mongoengine package as ODM for mongo.
i'm developing a little app for a University project and i need to json encode the result of a query to pass it to a js library, i've read elsewhere that i can use model_to_dict to accomplish that, but i'm getting this error
AttributeError: 'SelectQuery' object has no attribute '_meta'
and i don't know why or what to do, does anyone know how to solve that?
I'm using python 2.7 and the last version of peewee
#app.route('/ormt')
def orm():
doitch = Player.select().join(Nationality).where(Nationality.nation % 'Germany')
return model_to_dict(doitch)
This is because doitch is a SelectQuery instance it is not model, you have to call get()
from flask import jsonify
#app.route('/ormt')
def orm():
doitch = Player.select().join(Nationality).where(Nationality.nation % 'Germany')
return jsonify(model_to_dict(doitch.get()))
Also you could use dicts method to get data as dict. This omits creation a whole model stuff.
from flask import jsonify
#app.route('/ormt')
def orm():
doitch = Player.select().join(Nationality).where(Nationality.nation % 'Germany')
return jsonify(doitch.dicts().get())
edit
As #lord63 pointed out, you cannot simply return dict, it must be a Flask response so convert it to jsonify.
edit 2
#app.route('/ormt')
def orm():
doitch = Player.select().join(Nationality).where(Nationality.nation % 'Germany')
# another query
sth = Something.select()
return jsonify({
'doitch': doitch.dicts().get(),
'something': sth_query.dicts().get()
})
I'm building a bottle.py app that grabs some data from MongoDB and renders it into a web page using pygal.
The code produces a Error: 500 Internal Server Error in my browser.
On the server, I see: Exception: TypeError('serve_static() takes exactly 1 argument (0 given)',).
My question: how do I correct the code to render the .svg file?
The code:
import sys
import bottle
from bottle import get, post, request, route, run, static_file
import pymongo
import json
import pygal
connection = pymongo.MongoClient("mongodb://localhost", safe=True)
#get('/chart')
def serve_static(chart):
db = connection.control
chart = db.chart
cursor = chart.find({}, {"num":1, "x":1, "_id":0})
data = []
for doc in cursor:
data.append(doc)
list = [int(i.get('x')) for i in data]
line = pygal.Line()
line.title = 'widget quality'
line.x_labels = map(str, range(1, 20))
line.add('quality measure', list)
line.render_to_file('chart.svg')
try:
return static_file(chart.svg, root='/home/johnk/Desktop/chart/',mimetype='image/svg+xml')
except:
return "<p>Yikes! Somethin' wrong!</p>"
bottle.debug(True)
bottle.run(host='localhost', port=8080)
You didn't give a parameter to the route, so the function doesn't get any.
What you probably want to do, is either:
#get('/<chart>')
def serve_static(chart):
...
If you want /myfile.svg to work, or:
#get('/chart/<chart>')
def serve_static(chart):
...
If you want /chart/myfile.svg to work.
If you just want to show the same SVG file every time, you can just leave off the parameter:
#get('/chart')
def serve_static():
...
I'm trying to learn Pyramid and having problems getting the message flash to work. I'm totally new but read the documentation and did the tutorials.
I did the tutorial on creating a wiki(tutorial here, Code here ). It worked great and was pretty easy so I decided to try to apply the flash message I saw in todo list tutorial I did(tutorial here, full code is in a single file at the bottom of the page). Basically when a todo list is created, the page is refreshed with a message saying 'New task was successfully added!'. I wanted to do that everytime someone updated a wiki article in the wiki tutorial.
So I re-read the session section in the documentaion and it says I really just need to do this:
from pyramid.session import UnencryptedCookieSessionFactoryConfig
my_session_factory = UnencryptedCookieSessionFactoryConfig('itsaseekreet')
from pyramid.config import Configurator
config = Configurator(session_factory = my_session_factory)
then in my code I need to add: request.session.flash('New wiki was successfully added!') but I get a error everytime: Pyramid error: AttributeError: No session factory registered
Here's my function(its the exact same from the tutorial except for the request.session.flash part):
#view_config(route_name='edit_page', renderer='templates/edit.pt', permission='edit')
def edit_page(request):
name = request.matchdict['pagename']
page = DBSession.query(Page).filter_by(name=name).one()
if 'form.submitted' in request.params:
page.data = request.params['body']
DBSession.add(page)
request.session.flash('page was successfully edited!')
return HTTPFound(location = request.route_url('view_page',
pagename=name))
return dict(
page=page,
save_url = request.route_url('edit_page', pagename=name),
logged_in=authenticated_userid(request),
)
(note: One thing that I think I could be doing wrong is in the todo example, all the data is in one file, but in the wiki example there are several files..I added my session imports in the view.py file because the flash message is being generated by the view itself).
What am I doing wrong? Any suggestions?
The code you provided is just an example, of course you need to apply it in a correct place. In Pyramid you should (in simple cases ;) have only 1 place in your code where you create just 1 Configurator instance, in the tutorial it is in the main function. A Configurator does not do anything by itself, except create a WSGI application with make_wsgi_app.
Thus, to add sessions there, modify wiki2/src/views/tutorial/__init__.py as follows:
from pyramid.config import Configurator
from sqlalchemy import engine_from_config
from pyramid.session import UnencryptedCookieSessionFactoryConfig
from .models import DBSession
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
engine = engine_from_config(settings, 'sqlalchemy.')
DBSession.configure(bind=engine)
my_session_factory = UnencryptedCookieSessionFactoryConfig('itsaseekreet')
config = Configurator(settings=settings, session_factory=my_session_factory)
...