I'm trying to set up a background task using celery and rabbitmq on django but I'm getting an error saying that my project has no attribute celery. I'm using PyCharm and installed celery through that.
I'm new to celery but I've read a lot of articles similar to this issue (AttributeError: module 'module' has no attribute 'celery' this one seems the closest but not sure it's quite the same)
Project structure:
project_name
├──project_name
| ├── settings.py
├──app_name1
| └──celery.py
├──app_name2
| └──tasks.py
I run the following command:
celery -A project_name worker -l info --pool=solo
But I get the following error:
Error: Invalid value for "-A" / "--app":
Unable to load celery application.
Module 'project_name' has no attribute 'celery'
celery.py file:
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project_name.settings')
app = Celery('project_name')
app.config_from_object('django.config:settings', namespace='CELERY')
app.autodiscover_tasks()
tasks.py file:
from __future__ import absolute_import, unicode_literals
from celery import shared_task
#shared_task
def add(x, y):
return x + y
Just for the record: Put celery.py file into the main project file, not inside the app.
Try to enter into the project via the terminal. Write:
cd project_name
It worked with me
Related
I am trying to execute a celery task that among other things, communicates with a specific Django view with a pipe.
I have been trying all day to import the celery tasks file (tasks.py) from the Django views (views.py) without any success.
I have also checked the file permissions but this was not the case.
I have added to sys.path the path of the file desired to import (tasks.py), and then import it, but I keep getting an ImportError. However, when trying to import another script on the same folder (script1), the import succeeds.
views.py
...
sys.path.append('/home/celery')
import script1 #SUCCEEDS
# Trying to import "/home/celery/tasks.py" here
import tasks #FAILS
...
tasks.py
...
#app.task
def start_operation(client,**kwargs):
# Pipe comes here
...
The celery directory structure is the following:
/home
├── celery
├── script1.py
tasks.py
And the Django proyect directory structure:
/myapp
├── myapp
├── views.py
Thanks in advance.
This question already has answers here:
How to access a standard-library module in Python when there is a local module with the same name? [duplicate]
(2 answers)
Importing installed package from script with the same name raises "AttributeError: module has no attribute" or an ImportError or NameError
(2 answers)
Closed 4 years ago.
This might have nothing to do with celery but here is my problem:
I have an app structured like this:
/app
/__init__.py
/api_1.0/foo.py
/proj
/__init__.py
/celery.py
/tasks.py
So in celery.py I create a celery app:
flask = create_app(os.getenv('FLASKCONFIG') or None)
celery = Celery(__name__,
broker=flask.config['CELERY_BROKER_URL'],
include=['proj.tasks'])
celery.conf.update(flask.config)
and in tasks.py there are lists of celery tasks. One of them is list_users
In foo.py I try to use the task:
from proj import tasks
but this is causing an importation problem when I do:
celery -A proj worker --logleve=info
error message:
from proj.celery import celery
ImportError: cannot import name celery
Strange enough, if I remove the creation of the flask app and simply create a celery app, the problem goes away.
it looks like a circular import problem. How to avoid this?
I am using celery for for an app that downloads data. The directory structure is:
proj
__init__.py
celeryApp.py
tasks.py
startup.py
fetcher.py
I run celery with celery worker --app=celeryApp:app -l info from inside the proj folder.
I run my app which queues tasks by running: python startup.py. Both startup.py and fetcher.py import tasks.py. In startup.py I create an object of a class Fetcher defined in fetcher.py and pass that as an argument to a task runFetcher in tasks.py.
This results in an error
DecodeError: No module named fetcher
What changes would I need to make, so that I can safely pass such an object to a task?
Update: I added import fetcher to tasks.py which was fine to do as these tasks were closely related to the fetcher. Can anyone suggest a solution which does not require importing fetcher in tasks?
I'm trying to setup Celery with Django. I have followed the guide:
project/project/celery.py:
from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
app = Celery('proj')
# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
then in...
project/project/__init__.py:
from __future__ import absolute_import
# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app
project/apps/test/tasks.py
from __future__ import absolute_import
from celery import shared_task
#shared_task
def add(x, y):
return x + y
Then run:
celery -A proj worker -l info
which gives the error:
File "/Users/user/Documents/workspace/test-api/env/lib/python2.7/site-packages/kombu/utils/__init__.py", line 92, in symbol_by_name
module = imp(module_name, package=package, **kwargs)
File "/Users/user/Documents/workspace/test-api/env/lib/python2.7/site-packages/celery/utils/imports.py", line 101, in import_from_cwd
return imp(module, package=package)
File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named proj
Your project is named project, not proj. You should use that name throughout.
I just want to say that if you have correctly named everything and you still have the module not found error (I spent hours trying to figure this out)... Then you must place the command "celery -A proj worker -l info" above the apps branch
"The celery program can be used to start the worker (you need to run the worker in the directory above proj):" as per the docs
I have a django application I've added celery. In django application I have a package named 'parser', 'api'. I configured the celery as I followed the following tutorial: First steps with Django. In parser package I have 'models.py'. Do you 'task.py' package 'api'. When I try to do 'from parser import models' in api package . I get the following error: No module named models
I looked and found that the following import file: lib/python2.7/lib-dynload/parser.x86_64-linux-gnu.so
webapp/
manage.py
api/
__init__.py
models.py
views.py
tasks.py
...
parser/
__init__.py
models.py
views.py
...
settings/
__init__.py
base.py
celery.py
dev.py
live.py
local.py
urls.py
wsgi.py
In case I need 'models.py' of parser package. Command you use to start the celery is following: celery -A settings worker --loglevel=info. When I run celery in manage.py then take the right file: python manage.py celery -A settings worker --loglevel=info
api/task.py
from __future__ import absolute_import, division, print_function, unicode_literals
import time
from celery import task
from parser.models import FileUploadProcess # Error import
#task()
def test_task(param1):
print("Test task called. Param: {}".format(param1))
return 42
#task()
def parse_file(file_candidate, candidate_id):
FileUploadProcess(candidate_id=candidate_id, is_process=True).save()
# parse file
time.sleep(15)
FileUploadProcess.objects.filter(candidate_id=candidate_id).update(is_process=False)
Can somehow tell me Imports right package?
'from parser import models'
You need to use is so:
from parser.models import ClassName
where ClassName is name of class you want to import
or just
import parser.models as models