We Keep Coding

How and where to run (heavy) functions in the server for a django web app / where to store them

I am quite beginner to Django and I am confronted with this question.
provided I have the following structure:
django_app/
django_app/
__init__.py
asgi.py
settings.py
urls.py
views.py
wsgi.py
loaddata_app/
>migrations
__init__.py
admin.py
apps.py
models.py
test.py
urls.py
views.py
>static
>templates
loaddata.html
My loaddata app is a super easy app that contains a text field to enter (paste) text.
The app should send the text to the server where a series of methods are applied to extract information out of the text and generate output data that will be presented to back to the user.
The first question is: If you have a bunch of methods in various files, where do you actually "store" those methods. And where are those methods supposed to be imported (if they need at all to be imported)?
What happens if those methods take (so to say) 10 minutes to run? is the connection still maintained with the browser of the user in the meanwhile?
So looks the simple app:
NOTE:
The question is partially answered here. But it is not entirely clear where those utils are imported and run in order to process the data.
Django: where to store functions common to all apps

Assuming that the view class for your particular URL which does this job of getting input text and then processing and showing output is in the loaddata_app, and you have a bunch of functions common that you require.
You can keep the function in a python package at the root level of your project. You can name the package as utils or utilities or even as common. You can then create a file common_functions.py to put all your functions in.
So your project structure would look something like this.
django_app/
django_app/
__init__.py
asgi.py
settings.py
urls.py
views.py
wsgi.py
loaddata_app/
>migrations
__init__.py
admin.py
apps.py
models.py
test.py
urls.py
views.py
>static
>templates
loaddata.html
>utils
__init__.py
common_functions.py
You can then import these functions (eg: func1, funct2) in your loaddata_app/views.py as from utils.common_functions import func1, func2
Hope I have answered your doubt.
You can also check how to create a python package in this link
Now, to answer your second question about the time, by default when you are running a django application using the development server, ie, python manage.py runserver and the request - response takes 10 mins, the connection will be live and it will not timeout.
But, say, if you deployed this application in Heroku or behind any application gateway (from AWS or Azure, etc) then, there will be a timeout and the connection will not persist (ie, it will be terminated). By default for heroku, the timeout is 30 seconds. In such cases, you should use something like Celery to handle large tasks as background processes and still send some response to the user's browser or client.

What is the best practice in structuring templates in your Django project?

I know there's a lot of questions regarding best practices of how to structure an entire Django project but i cannot find a concrete explanation on how to structure templates for a Django project.
Obviously, I've just started learning Python/Django and i'm quite confused why most of the tutorials/examples i found, the directory name under templates directory is always the same as the applications directory name.
For example:
gazette/
__init__.py
models.py
views.py
static/
...
templates/
gazette/
__base.html
__l_single_col.html
__l_right_sidebar.html
__l_left_sidebar.html
_article_full.html
_article_summary.html
_author_list.html
_author_name.html
_category_list.html
_navigation.html
_tag_list.html
article_detail.html
article_list.html
author_list.html
category_list.html
tag_list.html
this actually came from a blog: https://oncampus.oberlin.edu/webteam/2012/09/architecture-django-templates
In the application structure above, the application gazette/ have templates/ directory under it. In turn, under templates/, there's a single directory called gazette/, which is the same name as the application directory where it's contained. Under this folder are the actual html template files.
Question: Why can't we just directly put all the html template files under templates/ directory?
I would understand the logic of the structure if there are several directories under templates folder. However, i never found a single example from any tutorials in YouTube or blogs that shows a project structure where there are several directories under templates directory. Instead, they have to name their one and only directory (under templates) same as the the application directory.
If you could explain the principle behind it and direct me to any reference/link, i would greatly appreciate it.
Thanks in advance

One alternative for storing templates is to store each app's templates inside the templates directory of the app.
So the project structure would look something like this
Project
app1
templates
app1
template1.html
template2.html
app2
templates
app2
template1.html
This way you can render the template from the view by passing 'app1/templates1.html'

Your Django Project Template at the end of the day is a personal choice, yes there are known templates that optimize your workflow for most projects, but at the end of the day every project is different, for example this is the template i use when i work for someone who expect me to follow the guidelines
Project
app1
app2
..etc
static
app1
images
css
js
app2
...etc
templates
app1
file.html
...etc
app2
..etc
log_files
app1_info.log
app2_info.log
app1_erros.log
..etc
this is because the Django philosophy is based on separation, Django is built around encouraging modularity
when i'm working on personal projects, i like to mix everything because even if use different apps for different things, i'll still have the same style and same templates across most of them.

How to easy switch db in Django 1.9.x across multiple servers

I have one application in Django 1.9.x, and I want to use multidb.
Have any configuration on the settings that I can enter multiple databases, and when it is on a specific server it uses the correct database. For example:
When I'm programming localhost use default, When I put in test serve, automatically switch to testserverdb and when I put in production server use productiondb, I tried use multi-db documentation but it's not what I want 'cuze this case is to work with legacy database, not my case.
How I do it?

In your settings file:
try:
from [app_name].local_settings import *
except ImportError:
pass
Changes you make in this local_settings file will be overridden. So now you can have a different local_settings file for your localhost, development or production. You can specify a separate db in these files separately.

It sounds like you want to have environment specific databases, not necessarily a single app that connects to many databases. You can easily accomplish this with a custom settings module for each of these environments.
You might have a structure like the following :
myproject/
- settings/
- __init__.py
- common.py
You'll want to put all your common settings under common.py. This will serve as the basis for all your other environment settings. From here,there are a few setups that you can use to do what you want, but I'm going to suggest that you use common.py as a base settings module that can be overridden locall.y
To do this, you can set your DJANGO_SETTINGS_MODULE to by myproject.settings, and in your __init__.py,
from .common import *
try:
from .local import *
except ImportError:
pass
Then on each environment (production/development/etc), you'll want to include a file named local.py in myproject/settings. Any settings you put in that local.py file will override your common.py when your settings module gets loaded up.

Importance of apps orders in INSTALLED_APPS

Is the order of apps in INSTALLED_APPS important? I ask it because I have settings folder with two settings files: base.py and production.py and I put all my settings in base.py and then in production.py, I have:
from base import *
and then I override some settings.
Also in my base.py, I make INSTALLED_APPS a list, not a tuple. Because I want to remove some apps for production settings.
In production.py I want to write:
NOT_USED_APPS = ['debut_toolbar', 'other_odd_app',]
INSTALLED_APPS = list(set(INSTALLED_APPS) - set(NOT_USED_APPS))
In this case, the order of apps in INSTALLED_APPS is not like in base.py

Yes, the order is quite important.
From Django official docs on INSTALLED_APPS settings:
When several applications provide different versions of the same
resource (template, static file, management command, translation), the
application listed first in INSTALLED_APPS has precedence.
Example-1 Templates:
django.template.loaders.app_directories.Loader
If this template loader is enabled in your DjangoTemplates backend in the TEMPLATES setting or if you have passed it as a loaders argument to Engine, then it loads templates from Django apps on the filesystem.
For each app in INSTALLED_APPS, the loader looks for a templates subdirectory. If the directory exists, Django will look for templates in there.
Lets say in my project, i have defined INSTALLED_APPS as:
INSTALLED_APPS = ('myproject.app1', 'myproject.app2')
Now, i want to get the template some_template.html. Then get_template('some_template.html') will look for some_template.html in these directories, in this order:
/path/to/myproject/app1/templates/ # checks here first
/path/to/myproject/app2/templates/ # Then checks here
It will then use the one which it finds first.
Quoting from that section:
The order of INSTALLED_APPS is significant!
Example-2: Translations
Django applies the following algorithm for discovering translations:
The directories listed in LOCALE_PATHS have the highest precedence, with the ones appearing first having higher precedence than the ones appearing later.
Then, it looks for and uses if it exists a locale directory in each of the installed apps listed in INSTALLED_APPS. The ones appearing first have higher precedence than the ones appearing later.
Finally, the Django-provided base translation in django/conf/locale is used as a fallback.
We can see that order is important here also.
Example-3 Management Commands:
From Django 1.7 release notes on management commands and order of INSTALLED_APPS:
When several applications provide management commands with the same
name, Django loads the command from the application that comes first
in INSTALLED_APPS. Previous versions loaded the command from the
application that came last.
This brings discovery of management commands in line with other parts
of Django that rely on the order of INSTALLED_APPS, such as static
files, templates, and translations.

I experimented a bit and found two other things that I deemed useful to know:
The order in INSTALLED_APPS doesn't seem to effect when the models are created. Django figures out that certain models depend on others and run them in the correct order.
The apps ready method in the AppConfig object seems to run in the order they appear in INSTALLED_APPS.

Python and App Engine project structure

I am relatively new to python and app engine, and I just finished my first project.
It consists of several *.py files (usually py file for every page on the site) and respectively temple files for each py file.
In addition, I have one big PY file that has many functions that are common to a lot of pages, in I also declared the classes of db.Model (that is the datastore kinds).
My question is what is the convention (if there is one) of arranging these files.
If I create a model.py with the datastore classes, should it be in different package?
Where should I put my template files and all of the py files that handle every page (should they be in the same directory as the one big common PY file)?
I have tried to look for MVC and such implementations online but there are very few.
Thanks,
Joel

I usually organize my projects in this way:
project
main.py
README
models
bar.py
foo.py
views
foolist.hml
barlist.hml
controllers
controller1.py
controller2.py
api
controllerapi.py
helpers
utilities.py
lib
extfoo.py
db
foo.db
test
test.py
Look at this post; it's a really great article on how to structure a project (not in python but it does not matter).

Typically I organize like so:
project/
main.py
models.py
app.yaml
index.yaml
templates/
main.html
foo.html
...
styles/
project.css
js/
jquery.js
project.js
images/
icon.png
something.jpg
And I have all of my handlers in main.py, all of my models in models.py, etc.
If I have a lot of handlers, and I can easily split the functionality of some handlers from the others (like taskqueue handlers vs. request handlers vs. xmpp/email handlers) I'll add another foo_handlers.py to the mix, but usually I just cram them all in main.py
But then again, I tend not to write hugely complex Python App Engine apps...