I have moved my SECRET_KEY value out of my settings file, and it gets set when I load my virtualenv. I can confirm the value is present from python manage.py shell.
When I run the Django Console, SECRET_KEY is missing, as it should. So in preferences, I go to Console>Django Console and load SECRET_KEY and the appropriate value. I go back into the Django Console, and SECRET_KEY is there.
As expected, I cannot yet run a manage.py Task because it has yet to find the SECRET_KEY. So I go into Run>Edit Configurations to add SECRET_KEY into Django server and Django Tests, and into the project server. Restart Pycharm, confirm keys.
When I run a manage.py Task, such as runserver, I still get KeyError: 'SECRET_KEY'.
Where do I put this key?
Because Pycharm is not launching from a terminal, your environment will not be loaded. In short, any GUI program will not inherit the SHELL variables. See this for reasons (assuming a Mac).
However, there are several basic solutions to this problem. As #user3228589 posted, you can set this up as a variable within PyCharm. This has several pros and cons. I personally don't like this approach because it's not a single source. To fix this, I use a small function at the top of my settings.py file which looks up the variable inside a local .env file. I put all of my "private" stuff in there. I also can reference this in my virtualenv.
Here is what it looks like.
-- settings.py
def get_env_variable(var_name, default=False):
"""
Get the environment variable or return exception
:param var_name: Environment Variable to lookup
"""
try:
return os.environ[var_name]
except KeyError:
import StringIO
import ConfigParser
env_file = os.environ.get('PROJECT_ENV_FILE', SITE_ROOT + "/.env")
try:
config = StringIO.StringIO()
config.write("[DATA]\n")
config.write(open(env_file).read())
config.seek(0, os.SEEK_SET)
cp = ConfigParser.ConfigParser()
cp.readfp(config)
value = dict(cp.items('DATA'))[var_name.lower()]
if value.startswith('"') and value.endswith('"'):
value = value[1:-1]
elif value.startswith("'") and value.endswith("'"):
value = value[1:-1]
os.environ.setdefault(var_name, value)
return value
except (KeyError, IOError):
if default is not False:
return default
from django.core.exceptions import ImproperlyConfigured
error_msg = "Either set the env variable '{var}' or place it in your " \
"{env_file} file as '{var} = VALUE'"
raise ImproperlyConfigured(error_msg.format(var=var_name, env_file=env_file))
# Make this unique, and don't share it with anybody.
SECRET_KEY = get_env_variable('SECRET_KEY')
Then the env file looks like this:
#!/bin/sh
#
# This should normally be placed in the ${SITE_ROOT}/.env
#
# DEPLOYMENT DO NOT MODIFY THESE..
SECRET_KEY='XXXSECRETKEY'
And finally your virtualenv/bin/postactivate can source this file. You could go further and export the variables as described here if you'd like, but since settings file directly calls the .env, there isn't really a need.
To set your environment variables in PyCharm do the following:
Open the 'File' menu
Click 'Settings'
Click the '+' sign next to 'Console'
Click Python Console
Click the '...' button next to environment variables
Click the '+' to add environment variables
You can set the manage.py task environment variables via:
Preferences| Languages&Frameworks| Django| Manage.py tasks
Setting the env vars via the run/debug/console configuration won't affect the built in pycharm's manage.py task.
Another option that's worked for me:
Open a terminal
Activate the virtualenv of the project which will cause the hooks to run and set the environment variables
Launch PyCharm from this command line.
Pycharm will then have access to the environment variables. Likely because of something having to do with the PyCharm process being a child of the shell.
Same here, for some reason PyCharm cant see exported env vars.
For now i set SECRET_KEY in PyCharm Run/Debug Configurations -> "Environment variables"
In Pycharm manage.py tasks run in an isolated process that do not have access to your environment variables (not the same as Run/Debug).
The simplest solution is to make your python code aware of environment variables by reading them from your .env file directly.
Take a look at: https://github.com/joke2k/django-environ
import environ
env = environ.Env.read_env() # reading .env file
SECRET_KEY = env('SECRET_KEY')
That way you have a single source of configurations, and less settings in the IDE.
Hope that helps,
When using PyCharm along with Django, there are multiple sections where you can set environment variables (EVs):
File > Settings > Languages and Frameworks > Django
There's no purpose to set EVs here
File > Settings > Build, Execution, Deployment > Console > Python Console
There's no purpose to set EVs here
File > Settings > Build, Execution, Deployment > Console > Django Console
Set EVs here and they'll be accesible when using the PyCharm Python Console (which in a Django project opens a Django shell)
File > Settings > Tools > Terminal
Set EVs here and they'll be accesible when using the PyCharm Terminal (i.e when running python manage.py commands in the terminal)
Run > Edit configurations > [Your Django run configuration]
Set EVs here and they'll be accesible when using the PyCharm Run button
Tested on PyCharm 2020.2 using a virtual environment.
To paraphrase #antero-ortiz's answer, instead of using the default Mac double-click or spotlight search to launch PyCharm, you can launch it directly from the terminal. This will inherit all of your environment variables from your terminal session to the PyCharm app.
Here's how I solve this issue.
From your terminal program, run the snippet below.
Validate that things worked by following the instructions to Edit your Run/Debug Configurations and,
Clicking the ... next to Environment variables
Then clicking Show at the bottom right of the screen that pops up. This will show all of the environment variables inherited by the Debug process.
Find your SECRET_KEY in this list. If it's not there, post a comment on this answer and we can try to figure it out!
You'll likely need to launch PyCharm this way every time you start it.
Snippet:
# This file would presumably export SECRET_KEY
source /path/to/env/file
# This is just a shortcut to finding out where you installed PyCharm
# If you used brew cask, it's probably in /opt/homebrew-cask/Caskroom/pycharm
open $(locate PyCharm.app | egrep 'PyCharm.app$')
Side note
Including environment variables in a file that is then source'd is a common pattern for Django development, and the discussion of that best practice is best left out of answers.
based on #rh0dium amazing answer i've made this class:
here's my settings.py:
import os
class EnvironmentSettings():
"""Access to environment variables via system os or .env file for development
"""
def __init__(self, root_folder_path):
self._root_folder_path = root_folder_path
def __getitem__(self, key):
return self._get_env_variable(key)
def __setitem__(self, key, value):
raise InvalidOperationException('Environment Settings are read-only')
def __delitem__(self, key):
raise InvalidOperationException('Environment Settings are read-only')
def _get_env_variable(self, var_name, default=False):
"""
Get the environment variable or return exception
:param var_name: Environment Variable to lookup
"""
try:
return os.environ[var_name]
except KeyError:
from io import StringIO
from configparser import ConfigParser
env_file = os.environ.get('PROJECT_ENV_FILE', self._root_folder_path + "/.env")
try:
config = StringIO()
config.write("[DATA]\n")
config.write(open(env_file).read())
config.seek(0, os.SEEK_SET)
cp = ConfigParser()
cp.readfp(config)
value = dict(cp.items('DATA'))[var_name.lower()]
if value.startswith('"') and value.endswith('"'):
value = value[1:-1]
elif value.startswith("'") and value.endswith("'"):
value = value[1:-1]
os.environ.setdefault(var_name, value)
return value
except (KeyError, IOError):
if default is not False:
return default
error_msg = "Either set the env variable '{var}' or place it in your " \
"{env_file} file as '{var} = VALUE'"
raise ConfigurationError(error_msg.format(var=var_name, env_file=env_file))
class ConfigurationError(Exception):
pass
class InvalidOperationException(Exception):
pass
and inside my runserver.py i have this calling code:
from settings import EnvironmentSettings
root_folder_path = os.path.dirname(os.path.abspath(__file__))
env_settings = EnvironmentSettings(root_folder_path)
config_name = env_settings['APP_SETTINGS']
I'm trying this setup currently. I have an env.local file in my project root. My manage.py looks like this:
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.local")
# Load environment variables from env.local, if option --load-env specified.
# Good for injecting environment into PyCharm run configurations for example and no need to
# manually load the env values for manage.py commands
if "--load-env" in sys.argv:
with open("env.local") as envfile:
for line in envfile:
if line.strip():
setting = line.strip().split("=", maxsplit=1)
os.environ.setdefault(setting[0], setting[1])
sys.argv.remove("--load-env")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
Then I just pass --load-env in the PyCharm run dialog and all the environment variables will be loaded in for the time of the run.
In PyCharm 2018 you can use django manage.py environment variables option in settings
File
Settings
Language and Frameworks
Django
Environment variables
You can see in image
I consider that a good approach is to use django-environ which allow you to read environment variables from a .env file.
Usage
First, install django-environ
pip install django-environ
Second, create a .env file, in the same directory of your settings.py file.
$ cd project-settings-directory/
$ touch .env
Third, add the environment variables you want to use to .env:
# .env file
DEBUG=on
SECRET_KEY=your-secret-key
DATABASE_URL=psql://urser:un-githubbedpassword#127.0.0.1:8458/database
SQLITE_URL=sqlite:///my-local-sqlite.db
Fourth, in your settings file, add the following just after your import statements:
# other import statements
import environ
# the followin line expects .env file to exist. Otherwise, an IOError exception will be raised.
env = environ.Env.read_env()
Fifth, load the environment variables:
DEBUG = env('DEBUG')
SECRET_KEY = env('SECRET_KEY')
That's it. If you want further information about please review the docs.
Note: this configuration will also help you to set environment variables when running a server from PyCharm.
The answer by #(nu everest) did not work for me. Whatever you described in the question worked for me. You can set all environment variables in the Run/Debug Configurations dialog. This is w.r.t PyCharm 2016.1 and also as per https://www.jetbrains.com/help/pycharm/2016.1/run-debug-configuration-python.html?origin=old_help
Please note that the answer mentioned by "nu everest" works but you will not see Console tab available unless you create a project in pycharm. As individual files can be run without creating a project in pycharm some people might be confused. Although you have also to note that settings are lost to run-configurations that do not belong to a project when pycharm is closed.
Here is my manage.py file in case it helps. It reads all the variables from .env file located in root directory of my project.
In PyCharm 2019+, the debug environment settings are described on their site here. I had similar issues, even with environment variables defined in the other parts of PyCharm (Console, Terminal, etc.)
Summary:
Top-right where your Django project name is (next to the "run" icon), click the drop-down arrow.
i. Note: If the project hasn't been configured for Django, you might not see it. There should be templates available to use in the next window, though.
"Edit Configurations..."
You'll see an "Environment variables" field apart from the console settings. That's the one you actually want to use if you're running the Django server via PyCharm's debugger.
Related
I'm making a simple blog application on Django, using PyCharm as IDE.
If I try to use Python Console, I get this error every time.
You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
If I open console command line in Terminal, I have to specify my settings using this command every time:
set DJANGO_SETTINGS_MODULE=mysite.settings
It looks like manage.py file already has a settings reference but for some reason it seems to be ignored:
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "Example.settings")
try:
from django.core.management import execute_from_command_line
except ImportError:
...
In your PyCharm virtual environment. You have a option Edit Configurations, By click this a popup window is open. Then find Environment variable, and set DJANGO_SETTINGS_MODULE=Example.settings
From my Windows 10 command line:
heroku config
yields the output:
=== test-bot Config Vars
GROUPME_BOT_ID: [111111111111111111]
My Python 3 code:
data = {
'bot_id' : os.getenv('GROUPME_BOT_ID'),
'text' : "hi",
}
print(data)
yields:
{'bot_id': None, 'text': 'hi'}
The bot_id is None instead of 111111111111111111. Why?
For testing, I execute my code from the command line of my Windows computer via:
python app.py
heroku:config only sets environment variables on Heroku. To run your code locally you will have to set your environment variables locally. (They can be different in development and production. That's the whole idea—they're environment-specific.)
There are several ways to do this. Here are two:
On your local machine you can use a .env file containing data in the format
VAR1=value
VAR2=value2
and then run your application using Foreman, which will automatically read the .env file and set appropriate environment variables:
foreman start
You can set environment variables using something like direnv, which will read an .envrc file for your project whenever you cd into its directory and set environment variables accordingly. In this case you don't need to use Foreman; python app.py should work fine.
There are editor and IDE plugins for direnv that can automate this in your editor as well.
I need to set some access token environment variables for my python project that I am running in a pipenv. I will want to set these environment variables every time I start the pipenv.
How do I do this?
If you want to load automatically some environment variables each time you start the project, you can set a .env file at the root folder of the project, next to the Pipfile. See Automatic Loading of .env.
You can run the following command from the right folder to create this .env file :
echo MY_TOKEN=SuperToKen >.env # create the file and write into
echo MY_VAR=SuperVar >>.env # append to the file
or just create it manually to obtain:
MY_TOKEN=SuperToKen
MY_VAR=SuperVar
This file will be loaded automatically with pipenv shell or pipenv run your_command and the environment variables will be available.
You can access/check them in your code with :
print(os.getenv('MY_TOKEN', 'Token Not found'))
Not sure about other IDE, but within Pycharm you need the plugin Env File to load it automatically (access Env File tab from the Run/Debug configurations).
You can add comments in this file with a leading #
# My test token
MY_TOKEN=SuperToKen
Note : Of course you must exclude this file from your version control (like git).
In the following code snippet (meant to work in an init.d environment) I would like to execute test.ClassPath. However, I'm having trouble setting and passing the CLASSPATH environment variable as defined in the user's .bashrc.
Here is the source of my frustration:
When the below script is run in use mode, it prints out the CLASSPATH OK (from $HOME/.bashrc)
when I run it as root, it also displays CLASSPATH fine (I've set up /etc/bash.bashrc with CLASSPATH)
BUT when I do "sudo script.py" (to simulate what happens at init.d startup time), the CLASSPATH is missing !!
The CLASSPATH is quite large, so I'd like to read it from a file .. say $HOME/.classpath
#!/usr/bin/python
import subprocess
import os.path as osp
import os
user = "USERNAME"
logDir = "/home/USERNAME/temp/"
print os.environ["HOME"]
if "CLASSPATH" in os.environ:
print os.environ["CLASSPATH"]
else:
print "Missing CLASSPATH"
procLog = open(osp.join(logDir, 'test.log'), 'w')
cmdStr = 'sudo -u %s -i java test.ClassPath'%(user, ) # run in user
proc = subprocess.Popen(cmdStr, shell=True, bufsize=0, stderr=procLog, stdout=procLog)
procLog.close()
sudo will not pass environment variables by default. From the man page:
By default, the env_reset option is enabled. This causes
commands to be executed with a minimal environment containing
TERM, PATH, HOME, MAIL, SHELL, LOGNAME, USER and USERNAME in
addition to variables from the invoking process permitted by
the env_check and env_keep options. This is effectively a
whitelist for environment variables.
There are a few ways of addressing this.
You can edit /etc/sudoers to explicitly pass the CLASSPATH
variable using the env_keep configuration directive. That might
look something like:
Defaults env_keep += "CLASSPATH"
You can run your command using the env command, which lets you set the environment explicitly. A typical command line invocation might look like this:
sudo env CLASSPATH=/path1:/path2 java test.ClassPath
The obvious advantage to option (2) is that it doesn't require mucking about with the sudoers configuration.
You could put source ~/.bashrc before starting your python script to get the environment variables set.
I want to use an embedded IPython shell with a user_ns dictionary and a my profile configuration (ipython_config.py and the startup files). The purpose is to run a Django shell with models imported on startup. django-extensions implements a command called shell_plus that does this:
https://github.com/django-extensions/django-extensions/blob/master/django_extensions/management/commands/shell_plus.py
from IPython import embed
embed(user_ns=imported_objects)
The problem is that this does not load my startup files. embed() calls load_default_config() which I figure loads ipython_config.py.
How do I make the embedded IPython instance run my profile startup files?
I used the following workaround to run my own IPython startup script but still take advantage of shell_plus:
Create a file called shell_plus_startup.py in the same directory as manage.py. For example:
# File: shell_plus_startup.py
# Extra python code to run after shell_plus starts an embedded IPython shell.
# Run this file from IPython using '%run shell_plus_startup.py'
# Common imports
from datetime import date
# Common variables
tod = date.today()
Launch shell plus (which launches an embedded IPython shell).
python manage.py shell_plus
Manually run the startup script.
In [1]: %run shell_plus_startup.py
Then you can use variables you've defined, modules you've imported, etc.
In [2]: tod
Out[2]: datetime.date(2012, 7, 14)
Also see this answer: scripting ipython through django's shell_plus
I found a way that works if you are using django-extensions-shell_plus. It is a bit hacky, but with this way your startup file is loaded fully automatically and you don't have to type any run-command at the beginning of your ipython-session.
Therefore I edited the file shells.py from the django_extensions dir, which is in my case located in /usr/local/lib/python2.7/dist-packages/django_extensions/management/shells.py. I added these lines inside the function import_objects(options, style):, so it imports the content of the file startup.py defined by the environment param PYTHONSTARTUP.
def import_objects(options, style):
# (...)
import os, sys, pkgutil
if 'PYTHONSTARTUP' in os.environ:
try:
sys.path.append(os.environ['PYTHONSTARTUP'])
import startup
content = [element for element in dir(startup) if not element.startswith('__')]
for element in content:
imported_objects[element] = getattr(startup, element)
except Exception, ex:
sys.exit("Could not import startup module content, Error:\n%s" % ex)
Now when I launch the shell_plus-shell, I give it the environment variable to my startup python script. My bash script to launch the shell with everything in place looks like this:
#!/bin/bash
export PYTHONSTARTUP=/home/ifischer/src/myproject/startup.py # tells shell_plus to load this file
python /home/ifischer/src/myproject/manage.py shell_plus --ipython
Now I have access to all methods and variables defined in startup.py from the beginning of the ipython session.
So you can reuse that and have custom startup files for every project, pre-loading different aspects.
Maybe there is a cleaner way where to include the lines I added to the shells.py? But this approach works fine for me for now.
It does automatically load your ipython configuration starting from django-extensions==1.5.6. You can also pass additional arguments to ipython via IPYTHON_ARGUMENTS. Docs:
http://django-extensions.readthedocs.org/en/latest/shell_plus.html#configuration
Another way is using a class that derives from InteractiveShellEmbed and InteractiveShellApp. Sample, incomplete code:
from IPython.terminal.embed import InteractiveShellEmbed
from IPython.terminal.ipapp import InteractiveShellApp
class ISE(InteractiveShellEmbed, InteractiveShellApp):
def init_shell(self):
self.shell = self.instance()
self.extra_args = []
ise = ISE()
ise.init_path()
ise.init_shell()
ise.init_code()
ise.shell()