We Keep Coding

How can run django tests in Pycharm Pro, getting ImproperlyConfigured and AppRegistryNotReady errors

I am having issues running django tests in PyCharm Pro.
I am able to run python manage.py runserver just fine.
I am able to migrate, makemigrations, manage.py test just fine. Actually, everything with manage.py runs as expected.
Whenever I run tests in PyCharm (by clicking the green run button), I get this error:
django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
I have done several things to fix this:
I have, in my test file, put this:
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings' (with mysite being the correct dir I promise)
I have a settings file and a wsgi file, and my wsgi file does set the DJANGO_SETTINGS_MODULE:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")
application = get_wsgi_application()
The only thing that has taken me out of this error was by setting up the configuration for that test class by going to Edit Configuations -> Environment -> Environment Variables and adding the DJANGO_SETTINGS_MODULE there. This only gets me a different error:
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
Plus, I have to do this for every test class and I can't run individual tests in the class, but must rather run the entire test suite in that class. Not a good solution...
I have manually (in the virtualenv) run export DJANGO_SETTINGS_MODULE=mysite.settings. This has not changed anything.
So, I am out of ideas and have followed all the advice I could find on SO. Is there something else I have to do in Pycharm? Again, the cmdline works fine, but I'd really rather not run tests this way - plus, I can't debug.
Edit: It may be worth noting that I am running the unit tests while my server is running (this should not make a difference), and I am also have the python interpreter pointed to the one in my virtualenv.

Try to open the Run/Debug Configuration window (you should find it in the top toolbar, clicking the dropdown menu near the play / debug button and Edit Configuration)
From here, you should see a list with the existing saved configurations where the last element should be Templates. Search Django tests under Templates. In the environment part, add DJANGO_SETTINGS_MODULE=mysite.settings to the environment variables. The next time you'll run a test that is not existing in your run/debug configurations, the template with the setting module set will be used.
If I were you, I will delete all the existing test configurations. Also, I suspect from your point 3, that you have probably a python test configured instead of a Django test (this is why you had the error django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet because Django was not loaded). If you have it, delete it too. Apply the configurations and retry to run the test.
When you run the test, ensure that a Django test is run, not a python one (you should see the Django logo near the test name in the run section and in the dropdown in the toolbar)

PythonAnywhere: manage.py returns "Access denied for user 'user'#'%' to database 'user'"

I am having very frustrating issues with PythonAnywhere. I've managed to get my site up and running by doing some hacky work arounds whenever I need to run manage.py but now that I need to set an automated task, this is no longer an option.
The issue lies in the fact that in PythonAnywhere databases have to be named in the convention "user$database". Because of the dollar sign, whenever I run something like python manage.py inspectdb I get the error mentioned in the title as anything attached to the dollar signed is assumed to be an environment variable by the shell.
Currently, I am storing my production environment variables in a .env file and setting them in to the settings.py file. This works internally with Django but does not work when running manage.py commands in the shell. I've attempted to fix the problem by adding the following code in my manage.py file:
# update DATABASE_NAME to use \$ instead of $ else bash commands wont work
os.putenv(
"DATABASE_NAME",
str(os.getenv("DATABASE_NAME")).replace('$', '\\$')
)
however, this does not resolve the issue.
I've also attempted to resolve the by creating a file called production_settings.py containing my DATABASES settings and wrapping the database name in single quotes alongside my settings.py file then inserting the following code in my settings.py file to set my DATABASES settings:
try:
from production_settings import DATABASES
except ImportError:
pass
Again, this did not work. I've tried finding support with PythonAnywhere but there doesn't seem to be any so I've come here in hopes someone here knows how to resolve the issue. I am days behind now... What should I do???

That sounds like you need to put single quotes around the values in your env file.

How to configure Django settings for different environments in a modular way?

I have already searched on the web on this doubt, but they don't really seem to apply to my case.
I have 3 different config files - Dev, Staging, Prod (of course)
I want to modularize settings properly without repetition. So, I have made base_settings.py and I am importing it to dev_settings.py, stg_settings.py etc.
Problem - How to invoke the scripts on each env properly with minimal changes?
Right now, I'm doing this (taking dev env as an example)-
python manage.py runserver --settings=core.dev_settings
This works so far, but I am not convinced on how good workaround is this.
Because wsgi.py and a couple of other services have -
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'core.settings')
I am looking to do something without changing the config files of other services. Thank you everyone in advance.
PS - I've tried to be as clear as possible, but please excuse me if anything is unclear.

Just set DJANGO_SETTINGS_MODULE in environment variables to your desired config file.
That won't make you to change any of other services config files, and you don't even need to change django settings files.

Have a look at the Django Configurations package.

In pycharm can I run every file for django?

I'm new to Django. My localhost site is running fine. Since I am using pycharm it is easy to run any file. I decided to run each file in my django project, and came across several errors, such as this one in my views.py:
django.core.exceptions.ImproperlyConfigured: Requested setting DEFAULT_INDEX_TABLESPACE, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Even though the site is running, what seems like properly, I'm seeing this message. What is causing this, and is it typical?

You cannot run each file present in your django project individual.
No matter those are file with .py extension. They depend on the django framework to get the project running.
The reason you might be seeing that error is because you might be using the attributes present in the settings.py file which in turn requires django to set the application running like starting the WSGI server, getting all the dependencies and the installed apps ready before you actually use anything.
Understand that Django is a Framework and it relies on many underlying components to actually work. Even thought you can technically run any file in any manner, you cannot start the application itself.
There are other ways to do it if you like to test the application like using django shell by python manage.py shell to check and test the application, which would be a better way of doing individual testing rather than running each file standalone.

You can run any individual python file in a Django Project with Django, but keep in mind that the settings for Django must be supplied. This is not a good practise to run individual file with Django but for debugging purposes you may use it (for example. to test a parser that you wrote with database access),
You have to configure the Django settings before you can do anything with Django on a single file.
from django.conf import settings
settings.configure()
def do_something_with_a_model():
# does something with a model
print "I am done!!"
Note that relative imports may break when running on single files.
(for example. imports like from .another_module import AnotherClass may break when working with single file.)

How to setup django application for production and open source, with one repository

I have a python/django project that I've set up for development and production using git revision control. I have three settings.py files:
-settings.py (which has dummy variables for potential open source project),
-settings_production.py (for production variables), and
-settings_local.py (to override settings just for my local environment). This file is not tracked by git.
I use this method, which works great:
try:
from settings_production import *
except ImportError, e:
print 'Unable to load settings_production.py:', e
try:
from settings_local import *
except ImportError, e:
print 'Unable to load settings_local.py:', e
HOWEVER, I want this to be an open source project. I've set up two git remotes, one called 'heroku-production' and one called 'github-opensource'. How can I set it up so that the 'heroku-remote' includes settings_production.py while 'github-opensource' doesn't, so that I can keep those settings private?
Help! I've look at most of the resources over the internets, but they don't seem to address this use case. Is this the right way? Is there a better approach?
The dream would be to be able to push my local environment to either heroku-production or github-opensource without haveing to mess with the settings files.
Note: I've looked at the setup where you use environment variables or don't track the production settings, but that feels overly complicated. I like to see everything in front of me in my local setup. See this method.
I've also looked through all these methods, and they don't quite seem to fit the bill.

There's a very similar question here. One of the answers suggests git submodules
which I would say are the easiest way to go about this. This is a problem for your VCS, not your Python code.

I think using environment variables, as described on the Two Scoops of Django book, is the best way to do this.
I'm following this approach and I have an application running out of a private GitHub repository in production (with an average of half a million page views per month), staging and two development environments and I use a directory structure like this:
MyProject
-settings
--__init__.py
--base.py
--production.py
--staging.py
--development_1.py
--development_2.py
I keep everything that's common to all the environments in base.py and then make the appropiate changes on production.py, staging.py, development_1.py or development_2.py.
My deployment process for production includes virtualenv, Fabric, upstart, a bash script (used by upstart), gunicorn and Nginx. I have a slightly modified version of the bash script I use with upstart to run the test server; it is something like this:
#!/bin/bash -e
# starts the development server using environment variables and django-admin.py
PROJECTDIR=/home/user/project
PROJECTENV=/home/user/.virtualenvs/project_virtualenv
source $PROJECTENV/bin/activate
cd $PROJECTDIR
export LC_ALL="en_US.UTF-8"
export HOME="/home/user"
export DATABASES_DEFAULT_NAME_DEVELOPMENT="xxxx"
export DATABASES_DEFAULT_USER_DEVELOPMENT="xxxxx"
export DATABASES_DEFAULT_PASSWORD_DEVELOPMENT="xxx"
export DATABASES_DEFAULT_HOST_DEVELOPMENT="127.0.0.1"
export DATABASES_DEFAULT_PORT_DEVELOPMENT="5432"
export REDIS_HOST_DEVELOPMENT="127.0.0.1:6379"
django-admin.py runserver --pythonpath=`pwd` --settings=MyProject.settings.development_1 0.0.0.0:8006
Notice this is not the complete story and I'm simplifying to make my point. I have some extra Python code in base.py that takes the values from these environment variables too.
Play with this and make sure to check the relevant chapter in Two Scoops of Django, I was also using the import approach you mentioned but having settings out of the repository wasn't easy to manage and I made the switch a few months ago; it's helped me a lot.