We Keep Coding

How and where to run (heavy) functions in the server for a django web app / where to store them

I am quite beginner to Django and I am confronted with this question.
provided I have the following structure:
django_app/
django_app/
__init__.py
asgi.py
settings.py
urls.py
views.py
wsgi.py
loaddata_app/
>migrations
__init__.py
admin.py
apps.py
models.py
test.py
urls.py
views.py
>static
>templates
loaddata.html
My loaddata app is a super easy app that contains a text field to enter (paste) text.
The app should send the text to the server where a series of methods are applied to extract information out of the text and generate output data that will be presented to back to the user.
The first question is: If you have a bunch of methods in various files, where do you actually "store" those methods. And where are those methods supposed to be imported (if they need at all to be imported)?
What happens if those methods take (so to say) 10 minutes to run? is the connection still maintained with the browser of the user in the meanwhile?
So looks the simple app:
NOTE:
The question is partially answered here. But it is not entirely clear where those utils are imported and run in order to process the data.
Django: where to store functions common to all apps

Assuming that the view class for your particular URL which does this job of getting input text and then processing and showing output is in the loaddata_app, and you have a bunch of functions common that you require.
You can keep the function in a python package at the root level of your project. You can name the package as utils or utilities or even as common. You can then create a file common_functions.py to put all your functions in.
So your project structure would look something like this.
django_app/
django_app/
__init__.py
asgi.py
settings.py
urls.py
views.py
wsgi.py
loaddata_app/
>migrations
__init__.py
admin.py
apps.py
models.py
test.py
urls.py
views.py
>static
>templates
loaddata.html
>utils
__init__.py
common_functions.py
You can then import these functions (eg: func1, funct2) in your loaddata_app/views.py as from utils.common_functions import func1, func2
Hope I have answered your doubt.
You can also check how to create a python package in this link
Now, to answer your second question about the time, by default when you are running a django application using the development server, ie, python manage.py runserver and the request - response takes 10 mins, the connection will be live and it will not timeout.
But, say, if you deployed this application in Heroku or behind any application gateway (from AWS or Azure, etc) then, there will be a timeout and the connection will not persist (ie, it will be terminated). By default for heroku, the timeout is 30 seconds. In such cases, you should use something like Celery to handle large tasks as background processes and still send some response to the user's browser or client.

What is the best practice in structuring templates in your Django project?

I know there's a lot of questions regarding best practices of how to structure an entire Django project but i cannot find a concrete explanation on how to structure templates for a Django project.
Obviously, I've just started learning Python/Django and i'm quite confused why most of the tutorials/examples i found, the directory name under templates directory is always the same as the applications directory name.
For example:
gazette/
__init__.py
models.py
views.py
static/
...
templates/
gazette/
__base.html
__l_single_col.html
__l_right_sidebar.html
__l_left_sidebar.html
_article_full.html
_article_summary.html
_author_list.html
_author_name.html
_category_list.html
_navigation.html
_tag_list.html
article_detail.html
article_list.html
author_list.html
category_list.html
tag_list.html
this actually came from a blog: https://oncampus.oberlin.edu/webteam/2012/09/architecture-django-templates
In the application structure above, the application gazette/ have templates/ directory under it. In turn, under templates/, there's a single directory called gazette/, which is the same name as the application directory where it's contained. Under this folder are the actual html template files.
Question: Why can't we just directly put all the html template files under templates/ directory?
I would understand the logic of the structure if there are several directories under templates folder. However, i never found a single example from any tutorials in YouTube or blogs that shows a project structure where there are several directories under templates directory. Instead, they have to name their one and only directory (under templates) same as the the application directory.
If you could explain the principle behind it and direct me to any reference/link, i would greatly appreciate it.
Thanks in advance

One alternative for storing templates is to store each app's templates inside the templates directory of the app.
So the project structure would look something like this
Project
app1
templates
app1
template1.html
template2.html
app2
templates
app2
template1.html
This way you can render the template from the view by passing 'app1/templates1.html'

Your Django Project Template at the end of the day is a personal choice, yes there are known templates that optimize your workflow for most projects, but at the end of the day every project is different, for example this is the template i use when i work for someone who expect me to follow the guidelines
Project
app1
app2
..etc
static
app1
images
css
js
app2
...etc
templates
app1
file.html
...etc
app2
..etc
log_files
app1_info.log
app2_info.log
app1_erros.log
..etc
this is because the Django philosophy is based on separation, Django is built around encouraging modularity
when i'm working on personal projects, i like to mix everything because even if use different apps for different things, i'll still have the same style and same templates across most of them.

django deploying separate web & api endpoints on heroku

I have a web application with an associated API and database.
I'd like to use the same Django models in the API, but have it served separately by different processes so I can scale it independently.
I also don't need the API to serve static assets, or any of the other views.
The complication is that the routes I have defined have the API and the webapp sharing the root domain:
http://domain.com/normal/application/stuff
http://domain.com/api/different/stuff
and additionally my Django apps depend on each other's models and constants (so two different settings.py files with different INSTALLED_APPS doesn't quite solve it).
I guess one way is I could define different processes in my Procfile which just start the Django app, but that in one of the processes it might have different environment variables? I don't think I can change the environment per Proc with heroku:config, I think it would actually have to be a directive in the Procfile.
Anyone have any experience or insight with this? Thanks!

As Daniel said you could just use two settings files, with a shared base. If you want to serve a subset of the urls you should just also create separate url definitions in the ROOT_URLCONF setting.
So your project structure would be something like this:
project/
project/
settings/
__init__.py
base.py
normal.py
api.py
urls/
__init__.py
base.py
normal.py
api.py
wsgi/
__init__.py
normal.py
api.py
settings/normal.py (analog for api) would be somthing like this:
from .base import *
ROOT_URLCONF = 'project.urls.normal

I don't think you need different environment variables, just a separate WSGI file pointing to a different settings.py. Those settings file can import shared settings from a common file, then set their particular values for INSTALLED_APPS. Then the Procfile can refer to those wsgi files in separate processes.

Tastypie and django project structure

I am new to Django. I want to include rest api to my project using tastypie framework. I want to know what are best practices of including tastypie in project. I see two options:
Separate app where all api related modules are stored (this app will end up rather fat)
api folder in every app with api resources for this particular app (not sure that understand how to organize urls in this case)
So what is the best practice for this?
Thanks

Actually I prefer the first approach. I create an api folder and then i create subfolders for each separate application. Each folder contains an api.py file for the specific need.
This way it is really simple and clear to support versions in the api and make it more clear. Also in the api folder i keep a urls.py which contains all the apis from the subfolders and it is easier to maintain.
EXAMPLE
api/
app1/ api.py
app2/ api.py
urls.py
and the urls.py should look a lot like,
from tastypie.api import Api
from .app1.api import App1Resource
from .app2.api import App2Resource
api = Api(api_name='myPath')
api.register(App1Resource())
api.register(App2Resource())
urlpatterns = api.urls
This is of-course just my personal preference which have helped me over the years to maintain a variety of different apis.
I hope i helped a bit.
Regards,
Michael.

django package organization

I plan to build my project in Django framework. However, I noticed that all Django packages have models.py file. Now, let say I have a set of general purpose functions that I share between several apps in the project and I plan to put these functions definitions in a separate package (or app for that matter?). So, should I create an app "general" and copy-paste these functions into the models.py file? Or can I just create a general.py file in the "general" app directory and leave models.py empty? What is the "Django" way to do that?
Thanks.

models.py file is used to define the structure of database. So you should leave it for defining your database entries. You can make an app named generals and put general.py in that app, and from there you can use it by calling it in any app.

I usually create a utils.py file under my main app that is created from the django-admin.py when starting the project.

I plan to put these functions definitions in a separate package (or app for that matter?)
Before you decide to make this an app (and if you do decide to make it an app), I recommend you take a look at James Bennet keynote on Developing reusable apps and hist post on laying out an application. From one of his slides:
Should this be its own application?
Is it orthogonal to whatever else I’m doing?
Will I need similar functionality on other sites?
Yes? Then I should break it out into a separate application.
If you're cramming too much functionality in one single general purpose app, it might be better to split your general purpose app into multiple reusable apps.
Going back to your original question, Django is expecting a models.py file in every app. So you must have the file even if it's empty.
Inside your models.py, you should only have the application’s model classes. So, you wouldn't be following a best practice if you put inside models.py some miscellaneous code you want to reuse.
From the laying out an application post I mentioned before:
At the application level, I usually drop in a few more files depending on exactly what the application is going to be using:
If the application defines any custom manipulators, I put them in a file called forms.py instead of in the views file.
If there are multiple custom managers in the app, I put them in a file called managers.py instead of the models file.
If I’m defining any custom context processors, I put them in a file called context_processors.py.
If I’m setting up any custom dispatcher signals, they go in a file called signals.py.
If the application is setting up any syndication feeds, the feed classes go in a file called feeds.py. Similarly, sitemap classes go in sitemaps.py.
Middleware classes go in a file called middleware.py.
Any miscellaneous code which doesn’t clearly go anywhere else goes in a file or module called utils.
All of this does not answer directly your original question:
can I just create a general.py file in the "general" app directory and leave models.py empty?
But I hope this gives you additional information to make a decision that better fits your project and requirements.