We Keep Coding

Testing with configuration files

my Google-fu has failed me by giving me results I don't understand, so I'm asking here.
I'm working on a Python project and I currently have a configuration file, which is also .py, that holds various python objects to load when everything starts. I'm trying to get some practice unit testing with pytest and I don't know exactly how to go about this issue. My guess is that I am probably going to make a dedicated testing config file that doesn't change, but I have no clue how to tell my code when to use the actual config file and when to use the testing config file. My current setup in my code is just using import config and setting values from it. I would greatly appreciate some help here!

An approach I've used to solve this problem is manipulating the path when running tests so that an additional package (/tests/resources) can shadow the normal resources package (/main/resources) that I set up for containing assets such as configuration files and making them available for loading at runtime.
Note: This structure is inspired by a pattern that I've brought from Java/Maven, so I won't claim it's Pythonic, and I don't like the path manipulation where the tests are concerned (I know a lot of others won't, either, so beware!). In fact, I found this question while looking for a better way to do this.
To accomplish this you first have to set up a folder to serve as your 'resources' package (See the docs). Once that's done, you create a similar version for your tests, in the tests folder. The directory structure will look something like this:
project
|-main
| |-resources
| | |-__init__.py
| | \-application.yml
| \-[...other source modules/packages...]
|-tests
| |-resources
| | |-__init__.py
| | \-application.yml
| \-[...other modules/packages with tests...]
\-run_tests.py
Note that main and tests are NOT packages. This is what contradicts a lot of conventional guidance in Python project structure. Instead, they're both added to the path, with tests being inserted into the path in front of main, by the run_tests.py script using code like this (some bits borrowed from this post:
import os.path
import sys
import pytest
ROOT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(ROOT_DIR, 'main'))
sys.path.insert(0, os.path.join(ROOT_DIR, 'tests'))
# Add any required args for testing.
args = []
pytest.main(args)
I'll typcially have additional options I want to feed pytest, and that's what args is for, but it's not relevant to this answer, so I left it empty here.
I know this is a late answer, but I hope this helps someone. Also, I know it's a controversial approach, but it's the most satisfying approach I've found so far. That may be because I'm experienced in Java as well as Python. To others, the approach of trying the import of tests.resources, and importing main.resources if it fails may be preferable.
I hope anyone with other approaches to this will share in comments, or post additional answers.

Figured out something based on this.
I ended up creating a testing config file in my folder named "tests" and having this code at the top of every file that used it:
import sys
if "pytest" in sys.modules:
import tests.testing_config as config
else:
import config
I don't think it's quite optimal since I feel something like this should be in the testing code, plus it kind of makes the new config file a dependency that should never be changed lest you break everything, but I guess it works for now if you have no clue how these cracked testing libraries work like me.

Django: Python path syntax to my custom validators.py?

I want to implement the solution detailed in this thread but I can't for the life of me figure out the Python path syntax in settings.py needed to link back to my custom validators.py I'm not even sure where to put it. I feel like a simple explanation of how the Python path syntax works in Django ought to be in the docs but after almost 20 minutes of looking through them I don't see anything. Any help would be greatly appreciated.

I'm assuming your custom validators.py is within your project, so like any of your other Python project's module it is accessed using dot notation.
The only thing "different" here is that you append the actual class name to this module string. For example:
AUTH_PASSWORD_VALIDATORS = [
'yourproject.validators.YourValidatorClass',
]
(although the format is not specified in this specific setting in the docs, it is a constant pattern used throughout Django settings)

How does python resolve imports (especially in context of django)?

I am doing a tutorial to get to know Django.
The last chapter I did was this one http://www.tangowithdjango.com/book17/chapters/models.html
I am using a virtual-environment managed via virtualenvironmentwrapper.
What I was wondering about is, how the imports really work. For example
we have this statement:
from rango.models import Category, Page
where we work on the file admin.py which is located in the same folder .../rango/ as models.py so I would expect
from models import Category, Page
to be the import statement (which seems to work, too as far as i can test it).
Ok, .../rango/ contains an __init__.py and therefore is considered as package as far as I know but why would I use the longer import-statement over the shorter one?
But most important to me would be:
When I use the shell to see sys.path I only get the directories created by my virtualenvwrapper where i would have expected it to contain (besides others) something like
[...]/rango/..
(the path to the parent-directory of rango).
How does the interpreter know where to search for those modules/packages in the first place?

How can you define a persistent variable in Django

I have a Django project, and what I would like to do is be able to create a reference to a variable defined in urls.py that is kept for the duration of the process.
I've tried an approach using global vars - but for whatever reason, the value of the variable in urls.py is the initial value - even if I change it somewhere else. I've followed the concepts as explained here, but just doesn't work.
However, when I try such a scenario outside of Django, it works as expected.
I think I'm missing a trick (or two) with Django - it's great, I'm new to it, and I think I'm going down the wrong path. Should I be using the caching stuff included with Django to store the variable, or is that even more off track?
Many thanks for any pointers.

I agree with S.Lott's comment, your question is a little vague. I think you're trying to reference some variable defined in urls? You know you can import urls.py into any other python script as long as it's on your PYTHONPATH. Assuming you're talking about a Django view when you say a "process", you could import a URL directly into a view function:
def foo_view(request):
from bar_app.urls import BAR_URL_PATTERN
Please give us more information on your specific use case, and I'll amend my answer to give you a better response.
Good Luck! I hope you're enjoying your first foray into the wonderful world of Django!

Correct place to put extra startup code in django?

I would like to run some environment checks when my django process starts and die noisily in the case of an error. I'm thinking things like the database has an incorrect encoding or the machine has a python version we don't support.
I'd rather our team be faced with a fatal error that they have to fix, rather than be able to ignore it.
I'm Ok with writing these checks but I'm curious about where the best place to put them is. How do I get them to execute as part of django's startup process? I thought there might be a signal I could listen too, but I can't find a relevant one in the docs.

If you don't want to use settings module, then try project's __init__.py.

If you want to check that the system is correctly installed, I think that you should write your own admin command and run it as post-installation check.
I think that it doesn't worth to check if the python version is correctly installed too often especially if you are installing the django app on shared-host. My app is hosted at alwaysdata and they restart the FastCgi process every hour. These checks can have an impact on the application response time.

We use the top-level urls.py for this.

I would put them in settings.py. In the past, I have put system checks like this:
try:
from local_settings import *
except ImportError:
print "Missing %s" % os.path.join(PROJECT_ROOT, "local_settings.py")
if DEBUG:
for p in [PROJECT_ROOT, MEDIA_ROOT, THEME_DIR, ADMIN_MEDIA_ROOT] + list(TEMPLATE_DIRS):
p = os.path.normpath(p)
if not os.path.exists(p):
print "Missing path: %s" % p

I have tested all three __init__.py Settings.py and urls.py methods and this is what I have found.
When code is run from either __init__.py or Settings.py, the start up functions are run twice upon web server start up; when the start up functions are run from urls.py the code is run once, however it is run only upon the first request made to the we server leading to a potentially long wait for the first user to visit your site.
It is standard practise to call a 'warming' page when bringing large web applications back online so I don't see that calling a start up function from a CLEARLY identified location in the urls.py should be a problem.

Most of the answers here are extremely old, so I assume that's why Django's checks framework isn't mentioned.
It's been a part of django since at least v2 (django==3.1 at the time of writing).
That's probably the right place for most of the checks you're requiring.
I'd still consider using settings, as mentioned by other answers, for checks where settings contents are required but which also need to be run prior to readying the apps.

You can put it in settings.py as mentioned by others, but having code in the settings is not ideal. There is also the option of adding a handler for django.db.models.signals.class_prepared that does the desired start up checks after a specific model class is prepared.