We Keep Coding

Module 'project_name' has no attribute 'celery'

I'm trying to set up a background task using celery and rabbitmq on django but I'm getting an error saying that my project has no attribute celery. I'm using PyCharm and installed celery through that.
I'm new to celery but I've read a lot of articles similar to this issue (AttributeError: module 'module' has no attribute 'celery' this one seems the closest but not sure it's quite the same)
Project structure:
project_name
├──project_name
| ├── settings.py
├──app_name1
| └──celery.py
├──app_name2
| └──tasks.py
I run the following command:
celery -A project_name worker -l info --pool=solo
But I get the following error:
Error: Invalid value for "-A" / "--app":
Unable to load celery application.
Module 'project_name' has no attribute 'celery'
celery.py file:
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project_name.settings')
app = Celery('project_name')
app.config_from_object('django.config:settings', namespace='CELERY')
app.autodiscover_tasks()
tasks.py file:
from __future__ import absolute_import, unicode_literals
from celery import shared_task
#shared_task
def add(x, y):
return x + y

Just for the record: Put celery.py file into the main project file, not inside the app.

Try to enter into the project via the terminal. Write:
cd project_name
It worked with me

Why is my_task.delay() working from shell but not apache/wsgi

Celery worker and beat are working fine.
Site loads fine except when I call a url that passes a task to Celery.
manage.py
#!/usr/bin/env python
import os
import sys
def main():
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'ada.settings')
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
if __name__ == '__main__':
main()
This is my WSGI file.
import os
import sys
import dotenv
from django.core.wsgi import get_wsgi_application
dotenv.load_dotenv()
def addpath(path):
if path not in sys.path:
sys.path.insert(0, path)
addpath('/opt/ada')
os.environ['DJANGO_SETTINGS_MODULE'] = 'ada.settings'
os.environ['PYTHONPATH'] = '/opt/ada'
os.environ['CELERY_LOADER'] = "django"
application = get_wsgi_application()
celery_config.py
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'ada.settings')
app = Celery('ada')
app.config_from_object('django.conf:settings', namespace='')
app.autodiscover_tasks()
init.py
from __future__ import absolute_import, unicode_literals
from ada.celery_config import app as celery_app
__all__ = ('celery_app', )
I don't understand this error.
mod_wsgi (pid=2547): Exception occurred processing WSGI script '/opt/ada/ada/wsgi.py'.
Traceback (most recent call last):
File "/opt/ada/venv/lib/python3.8/site-packages/kombu/utils/objects.py", line 42, in __get__
return obj.__dict__[self.__name__]
KeyError: 'tasks'
Also, I can try and call the same task from the shell and it works.
The problem only manifests when calling a url that and so must be related to my apache2 site conf file, wsgi.py, or something related
What am I missing?

REMOVING the following line from wsgi.py fixes things.
os.environ['CELERY_LOADER'] = "django"
I do not know why it works and would love to understand.

Django - Celery Worker won't start; finds multiple errors in apps

So I've been working on Django project for some months now, and a while back I set up a test installation of Celery which worked fine. Since that time, I've done a few things, upgraded my third-party apps and Django itself to 1.9. Today I wanted to implement celery again and couldn't get it to start. It's throwing multiple errors in my apps that Django itself isn't complaining about. It's also ignoring apps that I've overriden.
My file structure
src/
apps/
core/
settings/
production.py
development.py
__init__.py
apps.py
celery.py
wsgi.py
...
app1/
app2/
...
manage.py
My celery.py
from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'core.settings.development')
from django.conf import settings
app = Celery('core')
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
Using this command:
celery --app=core worker --loglevel=INFO
I'm getting things like:
File "/Users/../src/apps/core/urls.py", line 17, in <module>
from users.views import *
File "/Users/../src/apps/users/views.py", line 21, in <module>
from allauth.account.views import SignupView as AllAuthSignupView
File "/Users/../site/lib/python2.7/site-packages/allauth/account/views.py", line 19, in <module>
from .forms import (
File "/Users/../site/lib/python2.7/site-packages/allauth/account/forms.py", line 206, in <module>
class BaseSignupForm(_base_signup_form_class()):
File "/Users/../site/lib/python2.7/site-packages/allauth/account/forms.py", line 188, in _base_signup_form_class
' "%s"' % (fc_module, e))
django.core.exceptions.ImproperlyConfigured: Error importing form class core.forms: "cannot import name generic"
and
File "/Users/../site/lib/python2.7/site-packages/review/models.py", line 3, in <module>
from django.contrib.contenttypes import generic
ImportError: cannot import name generic
This last error shouldn't be happening at all because I've overriden the app in my project. But it's still reading the universally installed option. Django, however, runs just fine.
Any ideas on what I'm doing wrong here?

How to import the right package (python-django)

I have a django application I've added celery. In django application I have a package named 'parser', 'api'. I configured the celery as I followed the following tutorial: First steps with Django. In parser package I have 'models.py'. Do you 'task.py' package 'api'. When I try to do 'from parser import models' in api package . I get the following error: No module named models
I looked and found that the following import file: lib/python2.7/lib-dynload/parser.x86_64-linux-gnu.so
webapp/
manage.py
api/
__init__.py
models.py
views.py
tasks.py
...
parser/
__init__.py
models.py
views.py
...
settings/
__init__.py
base.py
celery.py
dev.py
live.py
local.py
urls.py
wsgi.py
In case I need 'models.py' of parser package. Command you use to start the celery is following: celery -A settings worker --loglevel=info. When I run celery in manage.py then take the right file: python manage.py celery -A settings worker --loglevel=info
api/task.py
from __future__ import absolute_import, division, print_function, unicode_literals
import time
from celery import task
from parser.models import FileUploadProcess # Error import
#task()
def test_task(param1):
print("Test task called. Param: {}".format(param1))
return 42
#task()
def parse_file(file_candidate, candidate_id):
FileUploadProcess(candidate_id=candidate_id, is_process=True).save()
# parse file
time.sleep(15)
FileUploadProcess.objects.filter(candidate_id=candidate_id).update(is_process=False)
Can somehow tell me Imports right package?

'from parser import models'
You need to use is so:
from parser.models import ClassName
where ClassName is name of class you want to import
or just
import parser.models as models

How to force import a lib from python site-package?

I am using custom django struct as below:
all settings file in conf, all app in src
and need use below manage.py:
if __name__ == "__main__":
ROOT_PATH = os.path.dirname(os.path.abspath(__file__))
SRC_PATH = os.path.join(ROOT_PATH, 'src')
CONF_PATH = os.path.join(ROOT_PATH, 'conf')
sys.path.insert(0, SRC_PATH)
sys.path.insert(0, CONF_PATH)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
And I also need to use celery in django.
But django-celery need a celery.py file in same directory with settings.py.
When runserver ,it would raise ImportError: cannot import name Celery because below code:
sys.path.insert(0, SRC_PATH)
sys.path.insert(0, CONF_PATH)
It import itself! not from site-package, because CONF_PATH is before site-package.
But I can't change that to
sys.path.append(SRC_PATH)
sys.path.append(CONF_PATH)
This way would cause django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
For now, the only way I know is changing celery.py to celery_somename.py, and I have to type this name every time when I start a celery job.
How do I force import a lib from python site-package? Or is there another way to start celery?

You can add at the top of the celery.py module and other modules importing celery:
from __future__ import absolute_import
This will make imports absolute by default, now:
import celery
Will import the installed celery package and not this module.