We Keep Coding

How to call celery task with class based method?

I have the class:
class Parser:
def __init__(self):
self.OM = Omni() # Creates an class object, which makes auth on the web site, for scrapping
#app.task
def foo(self, data):
self.OM.parse(data)
So how can I call task with foo method?
Because when I try to do like this, I takes error : Missing argument data. I think it is because calling the method get data as self parameter
prs = Parser()
prs.foo.delay(data)
How I can to resolve it?

Creating tasks from methods was possible in Celery 3.x, but it was removed in Celery 4.0 because it was too buggy.
I would create a little helper function:
class Parser:
def __init__(self):
self.OM = Omni() # Creates an class object, which makes auth on the web site, for scrapping
def foo(self, data):
self.OM.parse(data)
#app.task
def foo_task(data)
prs = Parser()
parser.foo(data)
foo_task.delay(data)

How to create a singleton object in Flask micro framework

I am creating a class for Producer which pushes messages to RabbitMQ. It makes use of pika module.
I would like to create a handler so that I have control over the number of connections that interact with Rabbit MQ.
Is there a way we can add this to app_context and later refer to that or is there way that we use init_app to define this handler.
Any code snippet would be of really good help.

In Python, using singleton pattern is not needed in most cases, because Python module is essentially singleton. But you can use it anyway.
class Singleton(object):
_instance = None
def __init__(self):
raise Error('call instance()')
#classmethod
def instance(cls):
if cls._instance is None:
cls._instance = cls.__new__(cls)
# more init operation here
return cls._instance
To use Flask (or any other web framework) app as singleton, simply try like this.
class AppContext(object):
_app = None
def __init__(self):
raise Error('call instance()')
#classmethod
def app(cls):
if cls._app is None:
cls._app = Flask(__name__)
# more init opration here
return cls._app
app = AppContext.app() # can be called as many times as you want
Or inherite Flask class and make itself as a singleton.

Django - create a class instance in AppConfig.ready() only once

I need to create a class instance (lets say backend requests session) on the app startup(runserver), and I don't want to rewrite this session after running other management command. How can I achieve this? I tried several approaches and I'm not sure why something like this doesn't work.
# app/apps.py
class MyConfig(AppConfig):
....
requests_session = None
....
def ready(self):
if MyConfig.requests_session is None:
MyConfig.requests_session = requests.Session()
Unfortunately, the condition is always met and the session is recreated. This approach is recommended in the documentation though.
Other solution for me would be to run MyConfig.ready() only after using selected subset of management commands, is that possible?
Is there completely different better way for me to store requests session?
TIA

I think it should work if you use an instance variable instead of a class variable:
# app/apps.py
class MyConfig(AppConfig):
def __init__(self, app_name, app_module):
super(MyConfig, self).__init__(app_name, app_module)
self.requests_session = None
def ready(self):
if self.requests_session is None:
self.requests_session = requests.Session()
The question now is how to access this instance variable elsewhere. You can do that like so:
from django.apps import apps
# Here myapp is the label of your app - change it as required
# This returns the instance of your app config that was initialised
# at startup.
my_app_config = apps.get_app_config('myapp')
# Use the stored request session
req = my_app_config.requests_session
Note that this instance variable only exists in the context of the current process. If you run a management command in a separate process (e.g., manage.py ...) then that will create a new instance of each app.

how to setup sqlalchemy session in celery tasks with no global variable

Summary: I want to use a sqlalchemy session in celery tasks without having a global variable containing that session.
I am using SQLAlchemy in a project with celery tasks, and I'm having
Currently, I have a global variable 'session' defined along with my celery app setup (celery.py), with a worker signal to set it up.
session = scoped_session(sessionmaker())
#celeryd_init.connect
def configure_workers(sender=None, conf=None, **kwargs):
# load the application configuration
# db_uri = conf['db_uri']
engine = create_engine(db_uri)
session.configure(bind=engine)
In the module defining the tasks, I simply import 'session' and use it. Tasks are defined with a custom class that closes the session after returning:
class DBTask(Task):
def after_return(self, *args, **kwargs):
session.remove()
That works well, however: when unit testing with CELERY_ALWAYS_EAGER=True, the session won't be configured. The only solution I've found so far is to mock that 'session' variable when running a task in a unit test:
with mock.patch('celerymodule.tasks.session', self.session):
do_something.delay(...)
While it works, I don't want to do that.
Is there any way to setup a session that will no be a global variable, that will work both for normal asynchronous behavior and without workers with CELERY_ALWAYS_EAGER=True?

The answer was right under my nose in the official documentation about custom task classes.
I modified the custom task class that I use for tasks accessing the database:
class DBTask(Task):
_session = None
def after_return(self, *args, **kwargs):
if self._session is not None:
self._session.remove()
#property
def session(self):
if self._session is None:
_, self._session = _get_engine_session(self.conf['db_uri'],
verbose=False)
return self._session
I define my tasks this way:
#app.task(base=DBTask, bind=True)
def do_stuff_with_db(self, conf, some_arg):
self.conf = conf
thing = self.session.query(Thing).filter_by(arg=some_arg).first()
That way, the SQLAlchemy session will only be created once for each celery worker process, and I don't need any global variable.
This solves the problem with my unit tests, since the SQLAlchemy session setup is now independant from the celery workers.

Flask: using app as an attribute and accessing decorators

using the python flask module, i would like to have the
app = flask.Flask(__name__)
as a attribute of a class:
class Handler(object):
def __init__(self):
self.datastores = {}
self.websocket_queue = gevent.queue.JoinableQueue()
self.app = flask.Flask(__name__)
the problem is how to access decorators then?
#self.app.route('/socket.io/<path:remaining>')
def socketio(self, remaining):
That generates the error NameError: name 'self' is not defined
Thanks

You could try to use Flask-Classy as it provides an easy way to use classes with Python-Flask.

It depends - if you are adding handlers inside of a method of the Handler class it should work without issue:
def add_routes(self):
#self.app.route("/some/route")
def some_route():
return "At some route"
If you are attempting to add routes outside of Handler you will need to use a reference to your Handler instance:
handler = Handler()
#handler.app.route("/some/route")
def some_route():
return "At some route"