We Keep Coding

Why Django template loader is case-sensitive in production but not in development

I'm using manage.py runserver on a MacOs Catalina OS as development. I have some templates that fit my built-in class based views. For example:
CuadroDeControl_detail.html LoteDeMedio_list.html TipoDeMedio_detail_tablaCuadros.html
CuadroDeControl_detail_resumen.html LoteDeMedio_list_tabla.html TipoDeMedio_list.html
CuadroDeControl_detail_tablaMetodos.html MetodoDeControl_detail.html TipoDeMedio_list_tabla.html
LoteDeMedio_confirm_delete.html MetodoDeControl_detail_resumen.html dropdown_CuadroDeControl.html
LoteDeMedio_create.html TipoDeMedio_confirm_delete.html dropwdown_CuadroDeControl.html
LoteDeMedio_detail.html TipoDeMedio_detail.html
LoteDeMedio_detail_resumen.html TipoDeMedio_detail_resumen.html
Here is an example of a working view:
class TipoDeMedioDetailView(AreaCalidadMixin, DashboardMixin, DetailView):
model = TipoDeMedio
Note that my views do not explicitly set template_name. In my production
environment, all my views load just fine. Django's template loader knows that the corresponding template to the view is TipoDeMedio_detail.html
However, in my production environment, which is set up with apache2 and mod_wsgi on a Ubuntu 20.04.2 LTS x86_64 Linode VM, the template loader fails to load the template of the same view, because it searches for it in all lowercase. Here is an example:
Request Method: GET
Request URL: http://45.79.4.38/calidad/TipoDeMedio/lista
Django Version: 3.1.6
Exception Type: TemplateDoesNotExist
Exception Value:
calidad/tipodemedio_list.html
Exception Location: /home/jonatan/django-app/venv/lib/python3.8/site-packages/django/template/loader.py, line 47, in select_template
Python Executable: /home/jonatan/django-app/venv/bin/python
Python Version: 3.8.5
Python Path:
['/home/jonatan/django-app/mysite',
'/usr/lib/python38.zip',
'/usr/lib/python3.8',
'/usr/lib/python3.8/lib-dynload',
'/home/jonatan/django-app/venv/lib/python3.8/site-packages']
Server time: Mon, 21 Jun 2021 18:24:19 -0500
Template-loader postmortem
Django tried loading these templates, in this order:
Using engine django:
django.template.loaders.filesystem.Loader: /home/jonatan/django-app/mysite/templates/calidad/tipodemedio_list.html (Source does not exist)
django.template.loaders.app_directories.Loader: /home/jonatan/django-app/mysite/login/templates/calidad/tipodemedio_list.html (Source does not exist)
django.template.loaders.app_directories.Loader: /home/jonatan/django-app/venv/lib/python3.8/site-packages/django/contrib/admin/templates/calidad/tipodemedio_list.html (Source does not exist)
django.template.loaders.app_directories.Loader: /home/jonatan/django-app/venv/lib/python3.8/site-packages/django/contrib/auth/templates/calidad/tipodemedio_list.html (Source does not exist)
An easy fix is to manually specify my template_name attribute on each of my CBV and point to the correct case-sensitive template name (for instance TipoDeMedio_detail.html. However, I would really like to avoid that.
I'm just trying to understand what is the root cause of the change in behavior between the environments. It just leads me to believe I will encounter similar problems in other aspects of Django's behavior.

The default Apple File System (APFS) is not case sensitive. This means that during development on your Mac, Django was able to find templates even if the filenames used incorrect case.
Now that you have moved to Ubuntu, it uses a case-sensitive file system by default. Django is not able to find the templates if the case is wrong.
Note that the Django docs for class-based generic views state:
... in the absence of an explicit template Django will infer one from the object’s name ... is the lowercased version of the model’s name.
The preferable solution is to rename your templates using lowercase.

MattRowbum pointed out to me APFS isn't case sensitive. I consider this sufficient explanation for the change in behavior.

Add local python module to the classpath in IntelliJ

I am trying to run/debug my python project from IntelliJ Ultimate 2018.1. I have defined a python SDK, a Django local server (as the project uses Django as a template language), the PYTHONPATH is properly defined, etc. If I execute
python manage.py runserver
from my MacOS terminal, the server starts normally.
When I am trying to run my IntelliJ configuration, it fails, with message:
/usr/bin/python2.7 -d /Users/my_user/dev/github/my_project/manage.py runserver 8000
Traceback (most recent call last):
File "/Users/my_user/dev/github/my_project/manage.py", line 19, in <module>
paths = importlib.import_module('settings.' + server + '.paths')
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/Users/my_user/dev/github/my_project/settings/__init__.py", line 11, in <module>
paths = importlib.import_module('my_project.settings.' + server + '.paths')
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named my_project.settings.local.paths
The settings.local folder does contains the __init__.py file, and also settings folder and the root folder of my project.
Doing a few hours of research over the internet I have realised that the main problem is to include local modules on the classpath, so they are available for server startup. For PyCharm, this is clearly explained here, and I was able to run the server from PyCharm. However, when trying to do the same thing in IntelliJ, this never works. Some people write about it here for example, but that didn't worked out for me.
I have tried using a system interpreter and also a virtual environment, created with PyCharm, with no luck. I am using python 2.7, Django 1.11 and MacOS High Sierra. Is this even possible with IntelliJ? I wonder why would people buy PyCharm if the same things can be accomplished (and so much more) with IntelliJ. Also, I have noticed that when I change a setting in IntelliJ run profile (the Django local server profile), this change is reflected back in PyCharm and viceversa. This seems to me like a bug from JetBrains, as two distinct apps somehow share the same config, probably due to the fact that the same project is loaded in both apps.

The problem was in fact trivial, as clarified with JetBrains support: I needed to set the parent folder of my_project as content root (/Users/my_user/dev/github/), instead of using /Users/my_user/dev/github/my_project/ directly. This was needed because the source code contained imports from module my_project which could not be resolved, as there was no subfolder my_project inside of my main project folder. I implemented this change in Project settings -> Modules view, Project settings -> Facets view and also in the run configuration.

"No module" error when using Python's importlib in Django -- is the path at fault?

I'm doing something wrong with Python's importlib in my Django project, and can't figure it out. I think there is a gotcha related to paths that I don't understand.
The simplest example illustrating the error occurs when I add these statements to my project's views.py file:
import test32 # a file I created. Is legal Python.
import importlib # Django is using Python 2.7
importlib.import_module # confirms that the method exists
#importlib.import_module( 'test32' )
This works fine until I uncomment the last line. Then I get an error:
ImportError at /
No module named test32
Request Method: GET
Request URL: http://localhost:4321/
Django Version: 1.8
Exception Type: ImportError
Exception Value:
No module named test32
Exception Location: /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/importlib/__init__.py in import_module, line 37
Python Executable: /Volumes/project663/bin/python
Python Version: 2.7.3
Python Path:
['/Volumes/project663/overall_project',
'/Volumes/project663/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg',
'/Volumes/project663/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg',
'/Volumes/project663/lib/python2.7/site-packages/Django-1.8-py2.7.egg',
'/Volumes/project663/lib/python27.zip',
'/Volumes/project663/lib/python2.7',
'/Volumes/project663/lib/python2.7/plat-darwin',
'/Volumes/project663/lib/python2.7/plat-mac',
'/Volumes/project663/lib/python2.7/plat-mac/lib-scriptpackages',
'/Volumes/project663/lib/python2.7/lib-tk',
'/Volumes/project663/lib/python2.7/lib-old',
'/Volumes/project663/lib/python2.7/lib-dynload',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/plat-darwin',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/lib-tk',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/plat-mac',
'/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/plat-mac/lib-scriptpackages',
'/Volumes/project663/lib/python2.7/site-packages']
I have tried numerous variants of the import_module statement, with dots preceding names, and using the package keyword, but none of them worked. This method can't be that hard to use, so I'm sure I'm missing something obvious.
What I really want to do is put test32.py in a subdirectory, but first I'd like to get it working in the same directory as views.py.
Thank you!

One possibility is that while, in Python 2, you can still have a relative import using the
import test32
syntax, this won't work with importlib. The importlib documentation for Python 2.7 says:
This module is a minor subset of what is available in the more
full-featured package of the same name from Python 3.1 that provides a
complete implementation of import. What is here has been provided to
help ease in transitioning from 2.7 to 3.1.
I read that as importlib functioning much like Python3's import, where relative imports should be explicitly specified using the dot-syntax.
So something like:
importlib.import_module('.test32', 'full.app.path')
may work. Note the dot in front of 'test32', and I don't know your full app path, but it'll be something like 'projectname.appname' (basically, the imports used elsewhere in your Django project.

Did I give a virus to my Django? manage.py disappearing

My Django installation is falling a part - could I have a virus?
I recently installed : https://github.com/chrisdev/django-pandas/
using pip and now very scary things are happening:
my manage.py in the root directory clears after every execution
even when I rename it something else, the starting script becomes a blank file
The admin section of the app wont run at all. I get errors
like:
[06/Feb/2015 06:27:48] "GET /admin/ HTTP/1.1" 200 4308
Exception happened during processing of request from ('127.0.0.1', 51159)
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 295, in _handle_request_noblock
self.process_request(request, client_address)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 601, in process_request
t = threading.Thread(target = self.process_request_thread,
AttributeError: 'NoneType' object has no attribute 'Thread'
What is really confusing is I had to comment out a settings line pertaining to “SessionAuthenticationMiddleware” or I get this
File "/Users/rpanos/.virtualenvs/JiraStat_Local_JB/lib/python2.7/site-packages/django/utils/module_loading.py", line 31, in import_by_path
error_prefix, module_path, class_name))
ImproperlyConfigured: Module "django.contrib.auth.middleware" does not define a "SessionAuthenticationMiddleware" attribute/class
Why would that suddenly be necessary?
If any one has any idea how these things are happening, please let me know.
I already did a pip uninstall on django-pandas but I imagine there could be some nefarious remnant doing this?
My sincere apologies if django-pandas has nothing to do with this but my django install has been working great for a year and now all is going to crap!
Just in case another package might be the issue, please see my virtual environement here:
(JiraStat_Local_JB)Rs-Mac-mini:JiraStats rXXXXs$ lssitepackages
Django-1.6.10-py2.7.egg-info lxml-3.4.0-py2.7.egg-info queuelib
OpenSSL model_utils queuelib-1.2.2-py2.7.egg-info
Scrapy-0.24.4-py2.7.egg-info numpy requests
Twisted-14.0.2-py2.7.egg-info numpy-1.9.0-py2.7.egg-info requests-2.4.3-py2.7.egg-info
_cffi_backend.so oauthlib requests_oauthlib
_markerlib oauthlib-0.6.3-py2.7.egg-info requests_oauthlib-0.4.1-py2.7.egg-info
cffi pandas scrapy
cffi-0.8.6-py2.7.egg-info pandas-0.14.1-py2.7.egg-info setuptools
cryptography pip setuptools-1.1.5-py2.7.egg-info
cryptography-0.6-py2.7.egg-info pip-1.4.1-py2.7.egg-info six-1.8.0-py2.7.egg-info
cssselect pkg_resources.py six.py
cssselect-0.9.1-py2.7.egg-info pkg_resources.pyc six.pyc
dateutil psycopg2 tlslite
django psycopg2-2.5.4-py2.7.egg-info tlslite-0.4.6-py2.7.egg-info
django_model_utils-2.2-py2.7.egg-info pyOpenSSL-0.14-py2.7.egg-info twisted
easy_install.py pycparser w3lib
easy_install.pyc pycparser-2.10-py2.7.egg-info w3lib-1.10.0-py2.7.egg-info
jira python_dateutil-2.2-py2.7.egg-info zope
jira-0.32-py2.7.egg-info pytz zope.interface-4.1.1-py2.7-nspkg.pth
lxml pytz-2014.7-py2.7.egg-info zope.interface-4.1.1-py2.7.egg-info
UPDATE
django-pandas did downgrade my Django and that solved a couple problems. Thank you Alasdair and Bernhard! However, there seems to be another issue.
I had two poorly formed packages - ones that I made by hand because I thought I knew what I was doing - when I remove them, all of the issues with the manage.py script go away. I usually trust PyCharm to do it, but these two I just copied the init files from another directory an started coding.
I have replaced and removed both of them to test this theory and its pretty solid.
Could any of you Django experts guess why a bad package would cause Django to crash AND delete its own manage.py?
It was still deleting "itself" after I re-upgraded Django.

Installing django-pandas most likely downgraded your Django installation as it explicitly requires Django < 1.7.

SessionAuthenticationMiddleware was added in Django 1.7. The traceback suggests that you may have Django 1.6 or earlier installed.

ImportError no module named accounts

Trying to create simple login functionality using Django, and I'm pretty new to using python and django. I've been searching for a while, but haven't found anything that fixes the problem. I'm running Django using MS VS2010, so I'm working under Windows instead of the typical linux environment. When run, I get this:
Request Method: GET
Request URL: http://localhost:1214/accounts/signup
Django Version: 1.4.3
Exception Type: ImportError
Exception Value:
No module named accounts
Exception Location: C:\Python27\lib\site-packages\django\utils\importlib.py in import_module, line 35
Python Executable: C:\Python27\python.exe
Python Version: 2.7.3
Python Path:
['C:\\Users\\brandon\\Desktop\\AdvancedLogin\\AdvancedLogin',
'C:\\Windows\\system32\\python27.zip',
'C:\\Python27\\DLLs',
'C:\\Python27\\lib',
'C:\\Python27\\lib\\plat-win',
'C:\\Python27\\lib\\lib-tk',
'C:\\Python27',
'C:\\Python27\\lib\\site-packages']
I've got the
__init__.py
file in bot the main project directory, and the app directory, though I haven't modified them at all.
I'm not really sure where to go from here. Any help is appreciated!

Did you add the accounts to your settings.py?

For other people be sure to use the correct AUTHENTICATION_BACKENDS. I've set up one for test but forgot it and i've got the same error.
Before i was :
settings.py
AUTHENTICATION_BACKENDS = (
'accounts.backends.MyAuthBackend',
)
After : settings.py
#Default one
AUTHENTICATION_BACKENDS = (
'django.contrib.auth.backends.ModelBackend',
)

This same error I was getting when I included 'accounts' in my setting.py file in "INSTALLED_APPS". But I was using sublime editor so I forgot to save that file and got this error.
Simply you have to add this "accounts" this to INSTALLED_APPS. this worked for me.