Following the instructions on https://github.com/linuxlewis/djorm-ext-pgfulltext I added a search_index to one of my existing PostgreSQL models, as follows:
search_index = VectorField()
objects = SearchManager(
fields = ('materials','origin','style'),
config = 'pg_catalog.english',
search_field = 'search_index',
auto_update_search_field = True
)
Using my virtual-env and having pgfulltext installed, with this version djorm-ext-pgfulltext==0.9.3 according to pip freeze, I cd to the directory where manage.pyis located. And tried to run:
./manage.py update_search_field app_name
Wich resulted in the following error:
Unknown command: 'update_search_field' Type 'manage.py help' for
usage.
Am I missing something on how this command is used? Should I add the index directly to my db instance with psql? And if so, how could I achieve this?
Any help will be much appreciated.
"Unknown command"
...this usually means you forgot to add the app to your INSTALLED_APPS in settings.py
That's not surprising since it seems they forgot to mention that on the readme for the app, but you need something like:
INSTALLED_APPS = (
# ... (your existing apps)
'djorm_pgfulltext',
)
Django needs to be told about the app in this way so that it knows to look for the management/commands directory in the app.
Related
Terminal says that i didn't defined DJANGO_SETTINGS_MODULE but i tried as i thought every method to do it and so far nothing helper (coding on windows)
Can someone help me out please? I am stuck and dont know what to do
Maby this will help - i run my virtual env through Anaconda -
conda activate djangoenv
raise ImproperlyConfigured(django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
from faker import Faker
from app.models import AccessRecrod, Webpage, Topic
import random
import django
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
django.setup()
fakegen = Faker()
topics = ['Search', 'Social', 'Marketplace', 'News', 'Games']
def add_topic():
t = Topic.objects.get_or_create(top_name=random.choice(topics))[0]
t.save()
return t
def populate(N=5):
for entry in range(N):
# get topic for entry
top = add_topic()
# create fake data for entry
fake_url = fakegen.url()
fake_date = fakegen.date()
fake_name = fakegen.company()
# create new webpage entry
webpg = Webpage.objects.get_or_create(
topic=top, url=fake_url, name=fake_name)[0]
# create fake access record
acc_rec = AccessRecrod.objects.get_or_create(
name=webpg, date=fake_date)[0]
if __name__ == '__main__':
print('populating script!')
populate(20)
print('populating complete!')
The Problem is simply your import of models. You just need to setup django before you import anything related to django:
import django
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
django.setup()
from faker import Faker
from app.models import AccessRecrod, Webpage, Topic
import random
rest of your code
If you check the code in manage.py and django code on github you will essentially find the same sequence of code (besides a lot of additional stuff) when you run a management command by "python manage.py your_command"
To use the management command directly as described in the answer from #Code-Apprentice would be the other "normal" or "more django way" to integrate your script.
But you asked for the reason of the error message in your way of doing it.
If you study the complete error trace you will also find that it starts at the import command.
Instead of trying to shoe horn a regular python script into the Django environment like this, I suggest you create a command which you can run with ./manage.py. You can learn how to create your own custom commands here. When you do this correctly, you can run a command like ./manage.py create_topics or whatever name you give your command. This allows manage.py to load the Django environment for you.
Alternatively, you could look at creating fixtures with ./manage.py dumpdata and ./manage.py loaddata. These commands will allow you to create data and save them to a .json file which you can load into your database at any time.
We are using Django as backend for a website that provides various things, among others using a Neural Network using Tensorflow to answer to certain requests.
For that, we created an AppConfig and added loading of this app config to the INSTALLED_APPS in Django's settings.py. This AppConfig then loads the Neural Network as soon as it is initialized:
settings.py:
INSTALLED_APPS = [
...
'bert_app.apps.BertAppConfig',
]
.../bert_apps/app.py:
class BertAppConfig(AppConfig):
name = 'bert_app'
if 'bert_app.apps.BertAppConfig' in settings.INSTALLED_APPS:
predictor = BertPredictor() #loads the ANN.
Now while that works and does what it should, the ANN is now loaded for every single command run through manage.py. While we of course want it to be executed if you call manage.py runserver, we don't want it to be run for manage.py migrate, or manage.py help and all other commands.
I am generally not sure if this is the proper way how to load an ANN for a Django-Backend in general, so does anybody have any tips how to do this properly? I can imagine that loading the model on startup is not quite best practice, and I am very open to suggestions on how to do that properly instead.
However, there is also some other code besides the actual model-loading that also takes a few seconds and that is definitely supposed to be executed as soon as the server starts up (so on manage.py runserver), but also not on manage.py help (as it takes a few seconds as well), so is there some quick fix for how to tell Django to execute it only on runserver and not for its other commands?
I had a similar problem, solved it with checking argv.
class SomeAppConfig(AppConfig):
def ready(self, *args, **kwargs):
is_manage_py = any(arg.casefold().endswith("manage.py") for arg in sys.argv)
is_runserver = any(arg.casefold() == "runserver" for arg in sys.argv)
if (is_manage_py and is_runserver) or (not is_manage_py):
init_your_thing_here()
Now a bit closer to the if not is_manage_py part: in production you run your web server with uwsgi/uvicorn/..., which is still a web server, except it's not run with manage.py. Most likely, it's the only thing that you will ever run without manage.py
Use AppConfig.ready() - it's intended for it:
Subclasses can override this method to perform initialization tasks such as registering signals. It is called as soon as the registry is fully populated. - [django documentation]
To get your AppConfig back, use:
from django.apps import apps
apps.get_app_config(app_name)
# apps.get_app_configs() # all
This is another way, in your manage.py will have something probably look like this
def main():
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'slambook.settings')
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
# check if has runserver
if `runserver` in sys.argv:
#execute your custom function
if __name__ == '__main__':
main()
you can check sys.argv if it have runserver, if so then execute your script or function
Is it allowed to group custom Django commands to separate folders inside the same Django app?
I have a lot of them and wanted to group them logically by purpose. Created folders but Django can't find them.
Maybe I'm trying to run them wrong. Tried:
python manage.py process_A_related_data
the same plus imported all commands in __init__.py
python manage.py folderA process_A_related_data
python manage.py folderA.process_A_related_data
python manage.py folderA/process_A_related_data
Got following error:
Unknown command: 'folderA/process_A_related_data'
Type 'manage.py help' for usage.
I think you can create a basic custom command which will run other commands from relevent folders. Here is an approach you can take:
First make a folder structure like this:
management/
commands/
folder_a/
process_A_related_data.py
folder_b/
process_A_related_data.py
process_data.py
Then inside process_data.py, update the command like this:
from django.core import management
from django.core.management.base import BaseCommand
import importlib
class Command(BaseCommand):
help = 'Folder Process Commands'
def add_arguments(self, parser):
parser.add_argument('-u', '--use', type=str, nargs='?', default='folder_a.process_A_related_data')
def handle(self, *args, **options):
try:
folder_file_module = options['use'] if options['use'].startswith('.') else '.' + options['use']
command = importlib.import_module(folder_file_module, package='your_app.management.commands')
management.call_command(command.Command())
except ModuleNotFoundError:
self.stderr.write(f"No relevent folder found: {e.name}")
Here I am using call_command method to call other managment commands.
Then run commands like this:
python manage.py process_data --use folder_a.process_A_related_data
Finally, if you want to run commands like python manage.py folder_a.process_A_related_data, then probably you need to change in manage.py. Like this:
import re
...
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
if re.search('folder_[a-z].*', sys.argv[-1]):
new_arguments = sys.argv[:-1] + ['process_data','--use', sys.argv[-1]]
execute_from_command_line(new_arguments)
else:
execute_from_command_line(sys.argv)
You should be able to partition the code by using mixins (I have not tried this in this context, though)
A standard management command looks like
from django.core.management.base import BaseCommand
class Command(BaseCommand):
help = 'FIXME A helpful comment goes here'
def add_arguments(self, parser):
parser.add_argument( 'name', ...)
# more argument definitions
def handle(self, *args, **options):
# do stuff
Which can probably be replaced by a "stub" in app/management/commands:
from wherever.commands import FooCommandMixin
from django.core.management.base import BaseCommand
class Command(FooCommandMixin, BaseCommand):
# autogenerated -- do not put any code in here!
pass
and in wherever/commands
class FooCommandMixin( object):
help = 'FIXME A helpful comment goes here'
def add_arguments(self, parser):
parser.add_argument( 'name', ...)
# more argument definitions
def handle(self, *args, **options):
# do the work
It would not be hard to write a script to go through a list of file names or paths (using glob.glob) using re.findall to identify appropriate class declarations, and to (re)generate a matching stub for each in the app's management/commands folder.
Also/instead Python's argparse allows for the definition of sub-commands. So you should be able to define a command that works like
./manage.py foo bar --aa --bb something --cc and
./manage.py foo baz --bazzy a b c
where the syntax after foo is determined by the next word (bar or baz or ...). Again I have no experience of using subcommands in this context.
I found no mention of support for this feature in the release notes. It looks to be that this is still not supported as of version Django 3.0. I would suggest that you use meaningful names for your files that help you specify. You could always come up w/ a naming convention!
A workaround could be: create a specific Django "satellite" app for each group of management commands.
In recent version of Django, the requirements for a Python module to be an app are minimal: you won't need to provide any fake models.py or other specific files as happened in the old days.
While far from perfect from a stylistic point of view, you still gain a few advantages:
no need to hack the framework at all
python manage.py will list the commands grouped by app
you can control the grouping by providing suitable names to the apps
you can use these satellite apps as container for specific unit tests
I always try to avoid fighting against the framework, even when this means to compromise, and sometimes accept it's occasional design limitations.
I am having some issues getting gcloud to run in a Bazel genrule. Looks like python path related issues.
genrule(
name="foo",
outs=["bar"],
srcs=[":bar.enc"],
cmd="gcloud decrypt --location=global --keyring=foo --key=bar --plaintext-file $# --ciphertext-file $(location bar.enc)"
)
The exception is:
ImportError: No module named traceback
From:
try:
gcloud_main = _import_gcloud_main()
except Exception as err: # pylint: disable=broad-except
# We want to catch *everything* here to display a nice message to the user
# pylint:disable=g-import-not-at-top
import traceback
# We DON'T want to suggest `gcloud components reinstall` here (ex. as
# opposed to the similar message in gcloud_main.py), as we know that no
# commands will work.
sys.stderr.write(
('ERROR: gcloud failed to load: {0}\n{1}\n\n'
'This usually indicates corruption in your gcloud installation or '
'problems with your Python interpreter.\n\n'
'Please verify that the following is the path to a working Python 2.7 '
'executable:\n'
' {2}\n\n'
'If it is not, please set the CLOUDSDK_PYTHON environment variable to '
'point to a working Python 2.7 executable.\n\n'
'If you are still experiencing problems, please reinstall the Cloud '
'SDK using the instructions here:\n'
' https://cloud.google.com/sdk/\n').format(
err,
'\n'.join(traceback.format_exc().splitlines()[2::2]),
sys.executable))
sys.exit(1)
My questions are:
How do I best call gcloud from a genrule?
What are the parameters needed to specify the python path?
How is Bazel blocking this?
Update:
Able to get it to run by specifying the CLOUDSDK_PYTHON.
Indeed, bazel runs in a sandbox, hence gcloud cannot find its dependencies. Acutally, I'm surprised gcloud can be invoked at all.
To proceed, I would wrap gcloud in a bazel py_binary and refer it with tools attribute in the genrule. You also need to wrap it with location in the cmd. In the end, you will have
genrule(
name = "foo",
outs = ["bar"],
srcs = [":bar.enc"],
cmd = "$(location //third_party/google/gcloud) decrypt --location=global --keyring=foo --key=bar --plaintext-file $# --ciphertext-file $(location bar.enc)",
tools = ["//third_party/google/gcloud"],
)
And for that you define in third_party/google/gcloud/BUILD (or anywhere your want, I just used a path that makes sense to me)
py_binary(
name = "gcloud",
srcs = ["gcloud.py"],
main = "gcloud.py",
visibility = ["//visibility:public"],
deps = [
":gcloud_sdk",
],
)
py_library(
name = "gcloud_sdk",
srcs = glob(
["**/*.py"],
exclude = ["gcloud.py"],
# maybe exclude tests and unrelated code, too.
),
deps = [
# Whatever extra deps are needed by gcloud to compile
]
)
I had a similar issue, worked for me running this command:
export CLOUDSDK_PYTHON=/usr/bin/python
(this was answered above as an update but I felt to post the whole command for future people coming here)
I want to run a python scripts which should do:
Create a django project: django-admin startproject foobar
Create a app in the project: python manage.py barfoo
Add an entry of newly created app barfoo in the setting's INSTALLED_APP.
How can I achieve this?
There seems to be a pythonic way to do #1 and #2
https://docs.djangoproject.com/en/dev/ref/django-admin/#running-management-commands-from-your-code
from django.core import management
management.call_command('flush', verbosity=0, interactive=False)
management.call_command('loaddata', 'test_data', verbosity=0)
6 years later I stumbled upon this question trying to figure out how to write some tests for an app which only add a custom template tag that interact with other apps in the project. Hope this can help someone.
Building on #groovehunter answer: the official documentation now (Django 1.10) inculdes this feature outside dev.
Note that you need to change current directory to the created project before call startapp. See this answer for more details
from django.core import management
import os
management.call_command('startproject', 'foobar')
os.chdir('foobar')
management.call_command('startapp', 'barfoo')
or you can use the additional argumento to startproject to create the project in the current directory, if you're sure there won't be problems:
from django.core import management
management.call_command('startproject', 'foobar', '.')
management.call_command('startapp', 'barfoo')
Read a little abour subprocess and Popen method. This might be what you're looking for.
Popen(["django-admin", "startproject", "%s" % your_name ], stdout=PIPE).communicate()
Popen(["python", "manage.py", "%s" % your_app_name ], stdout=PIPE).communicate()
3.
I know that's not a perfect code, but I'm just giving an idea.
with open("settings.py", 'r') as file:
settings = file.readlines()
new_settings = []
for line in settings:
if "INSTALLED APPS" in line:
new_settings.append(line.replace("INSTALLED_APPS = (", "INSTALLED_APPS = (\n'%s'," % your_app_name))
else:
new_settings.append(line)
with open("settings.py", 'w') as file:
file.write("".join(new_settings))