I've got a situation where I need to import * from a list of modules which may or may not exist. So far, I've got a working solution that goes a little something like this:
for module in modules:
try:
imported = __import__(module, globals(), locals(), [], -1)
for sub in module.split(".")[1:]:
imported = getattr(imported, sub)
for name in [n for n in dir(imported) if not n.startswith("_")]:
globals()[name] = getattr(imported, name)
except (ImportError, AttributeError):
pass
This works, but is a total mess and confuses linters no end when they're looking for the variables that get imported in this way. I'm certain I'm missing something in the way __import__ works, since surely from foo import * doesn't generate such a horrible call as I've got above. Can anyone enlighten me?
Update:
The actual use case here is refactoring a django project settings file, to move app-specific settings into a settings file within that app. Currently, all settings are in the main settings file, which has become unmaintainably large. I need to retain the ability to override any of the settings defined in that way from another settings file (client_settings.py), which gets imported as from foo_project.client_settings import * at the bottom of the settings file.
Doing from app.settings import * for each app would work, but I want to drive this from the installed apps setting to maintain DRY.
Related
I have a Django app that is up and running, mostly in an easy-to-understand way. But I have one quirky problem.
My code sits one level below where urls.py and views.py sit. I have a number of Python files, some of which are used as imports across the actual code in the other impoPython files.
I have found that, as part of a Django app, I need to precede import filenames with a '.' for them to import. So, if I have a file of code called foo.py, and an import file called vars.py, this won't import right:
# foo.py
from vars import *
but this works:
# foo.py
from .vars import *
The problem comes if I try to be more Pythonic (and safe) and name the import, so if I import everything it gets a prefix. But neither of the following work at all:
# foo.py
import vars as v
import .vars as v
my presumption is Django (using virtualenv or through other means) defines certain paths to look up import files, and the '.' becomes necessary to point the Python interpreter to the same directory in which the importing file sits to locate the import files it needs to find. But, somehow, this 'breaks' the ability to give the imported file contents its own prefix (or namespace, I guess).
Any thoughts?
I have a module that wraps another module to insert some shim logic in some functions. The wrapped module uses a settings module mod.settings which I want to expose, but I don't want the users to import it from there, in case I would like to shim something there as well in the future. I want them to import wrapmod.settings.
Importing the module and exporting it works, but is a bit verbose on the client side. It results in having to write settings.thing instead of just thing.
I want the users to be able to do from wrapmod.settings import * and get the same results as if they did from mod.settings import * but right now, only from wrapmod import settings is available. How to I work around this?
If I understand the situation correctly, you're writing a module wrapmod that is intended to transform parts of an existing package mod. The specific part you're transforming is the submodule mod.settings. You've imported the settings module and made your changes to it, but even though it is available as wrapmod.settings, you can't use that module name in an from ... import ... statement.
I think the best way to fix that is to insert the modified module into sys.modules under the new dotted name. This makes Python accept that name as valid even though wrapmod isn't really a package.
So wrapmod would look something like:
import sys
from mod import settings
# modify settings here
sys.modules['wrapmod.settings'] = settings # add this line!
I ended up making a code-generator for a thin wrapper module instead, since the sys.module hacking broke all IDE integration.
from ... import mod
# this is just a pass-through wrapper around mod.settings
__all__ = mod.__all__
# generate pass-through wrapper around mod.settings; doesn't break IDE integration, unlike manual sys.modules editing.
if __name__ == "__main__":
for thing in settings.__all__:
print(thing + " = mod." + thing)
which when run as a script, outputs code that can then be appended to the end of this file.
I apologize if this is a basic question but I can't seem to find the answer here or on Google. Basically I'm trying to create a single config module that would be available to all other modules imported in a python application. Of course it works if I have import config in each file but I would like to make my config dynamic based on the environment the application is running in and I'd prefer not to have to copy the logic into every file.
Here's an example:
app.py:
import config
import submodule
# do other stuff
submodule.py:
print config.some_config_variable
But python of course complains that config isn't defined.
I did find some stuff about global variables but that didn't seem to work either. Here's what I tried:
Edit I changed this to show that I'd like the actual config being imported to be dynamic. However I do currently have a static config modle for my tests just to figure out how to import globally and then worry about that logic
app.py
# logic here that defines some_dynamic_config
global config
config = __import__(some_dynamic_config)
import submodule
submodule.py
print config.some_config_variable
But config still isn't defined.
I'm aware that I could create a single config.py and place logic to set the variables but I dislike that. I prefer the config file to just configuration and not contain a bunch of logic.
You've got to put your logic somewhere. Your config.py could be a module that determines which config files to load, something like this:
#config.py
import sys
from common_config import *
if sys.platform == 'darwin':
from mac_config import *
elif sys.platform == 'win32':
from win32_config import *
else:
from linux_config import *
With this approach, you can put common settings in common_settings.py, and platform-specific settings in their respective files. Since the platform-specific settings are imported after common_settings, you can also override anything in common_settings by defining it again in the platform-specific files.
This is a common pattern used in Django settings files, and works quite well there.
You could also wrap each import call with try... except ImportError: blocks if need be.
Modules are shared, so each module can import config without issue.
config.py itself can have logic to set it's global variables howver you like. As an example:
config.py:
import sys
if sys.platform == "win32":
temp = "c:\\temp"
else:
temp = "/tmp"
Now import config in any module to use it:
import config
print "Using tmp dir: %s" % config.temp
If you have a module that you know will be initialized before anything else, you can create an empty config.py, and then set it externally:
import config
config.temp = "c:\\temp"
But you'll need to run this code before anything else that uses it. The empty module can be used as a singleton.
I am relatively new to Python. I am looking to create a "settings" module where various application-specific constants will be stored.
Here is how I am wanting to set up my code:
settings.py
CONSTANT = 'value'
script.py
import settings
def func():
var = CONSTANT
# do some more coding
return var
I am getting a Python error stating:
global name 'CONSTANT' is not defined.
I have noticed on Django's source code their settings.py file has constants named just like I do. I am confused on how they can be imported to a script and referenced through the application.
EDIT
Thank you for all your answers! I tried the following:
import settings
print settings.CONSTANT
I get the same error
ImportError: cannot import name CONSTANT
The easiest way to do this is to just have settings be a module.
(settings.py)
CONSTANT1 = "value1"
CONSTANT2 = "value2"
(consumer.py)
import settings
print settings.CONSTANT1
print settings.CONSTANT2
When you import a python module, you have to prefix the the variables that you pull from it with the module name. If you know exactly what values you want to use from it in a given file and you are not worried about them changing during execution, then you can do
from settings import CONSTANT1, CONSTANT2
print CONSTANT1
print CONSTANT2
but I wouldn't get carried away with that last one. It makes it difficult for people reading your code to tell where values are coming from. and precludes those values being updated if another client module changes them. One final way to do it is
import settings as s
print s.CONSTANT1
print s.CONSTANT2
This saves you typing, will propagate updates and only requires readers to remember that anything after s is from the settings module.
step 1: create a new file settings.py on the same directory for easier access.
#database configuration settings
database = dict(
DATABASE = "mysql",
USER = "Lark",
PASS = ""
)
#application predefined constants
app = dict(
VERSION = 1.0,
GITHUB = "{url}"
)
step 2: importing settings module into your application file.
import settings as s # s is aliasing settings & settings is the actual file you do not have to add .py
print(s.database['DATABASE']) # should output mysql
print(s.app['VERSION']) # should output 1.0
if you do not like to use alias like s you can use a different syntax
from settings import database, app
print(database['DATABASE']) # should output mysql
print(app['VERSION']) # should output 1.0
notice on the second import method you can use the dict names directly
A small tip you can import all the code on the settings file by using * in case you have a large file and you will be using most of the settings on it on your application
from settings import * # * represent all the code on the file, it will work like step 2
print(database['USER']) # should output lark
print(app['VERSION']) # should output 1.0
i hope that helps.
When you import settings, a module object called settings is placed in the global namespace - and this object carries has that was in settings.py as attributes. I.e. outside of settings.py, you refer to CONSTANT as settings.CONSTANT.
Leave your settings.py exactly as it is, then you can use it just as Django does:
import settings
def func():
var = settings.CONSTANT
...Or, if you really want all the constants from settings.py to be imported into the global namespace, you can run
from settings import *
...but otherwise using settings.CONSTANT, as everyone else has mentioned here, is quite right.
See the answer I posted to Can I prevent modifying an object in Python? which does what you want (as well as force the use of UPPERCASE identifiers). It might actually be a better answer for this question than it was for the the other.
This way is more efficient since it loads/evaluates your settings variables only once. It works well for all my Python projects.
pip install python-settings
Docs here: https://github.com/charlsagente/python-settings
You need a settings.py file with all your defined constants like:
# settings.py
DATABASE_HOST = '10.0.0.1'
Then you need to either set an env variable (export SETTINGS_MODULE=settings) or manually calling the configure method:
# something_else.py
from python_settings import settings
from . import settings as my_local_settings
settings.configure(my_local_settings) # configure() receives a python module
The utility also supports Lazy initialization for heavy to load objects, so when you run your python project it loads faster since it only evaluates the settings variable when its needed
# settings.py
from python_settings import LazySetting
from my_awesome_library import HeavyInitializationClass # Heavy to initialize object
LAZY_INITIALIZATION = LazySetting(HeavyInitializationClass, "127.0.0.1:4222")
# LazySetting(Class, *args, **kwargs)
Just configure once and now call your variables where is needed:
# my_awesome_file.py
from python_settings import settings
print(settings.DATABASE_HOST) # Will print '10.0.0.1'
I'm new in python but if we define an constant like a function
on setings.py
def CONST1():
return "some value"
main.py
import setings
print setings.CONST1() ##take an constant value
here I see only one, value cant be changed but its work like a function..
Try this:
In settings.py:
CONSTANT = 5
In your main file:
from settings import CONSTANT
class A:
b = CONSTANT
def printb(self):
print self.b
I think your above error is coming from the settings file being imported too late. Make sure it's at the top of the file.
Also worth checking out is the simple-settings project which allows you to feed the settings into your script at runtim, which allows for environment-specific settings (think dev, test, prod,...)
Update: This is, as I was told, no principle Python related problem, but seems to be more specific. See below for more explanations to my problem.
I have a custom exception (let's call it CustomException), that lives in a file named exceptions.py. Now imagine, that I can import this file via two paths:
import application.exceptions
or
import some.application.exceptions
with the same result. Furthermore I have no control over which way the module is imported in other modules.
Now to show my problem: Assume that the function do_something comes from another module that imports exceptions.py in a way I don't know. If I do this:
import application.exceptions
try:
do_something ()
except application.exceptions.CustomException:
catch_me ()
it might work or not, depending on how the sub-module imported exceptions.py (which I do not know).
Question: Is there a way to circumvent this problem, i.e., a name for the exception that will always be understood regardless of inclusion path? If not, what would be best practices to avoid these name clashes?
Cheers,
Update
It is a Django app. some would be the name of the Django 'project', application the name of one Django app. My code with the try..except clause sits in another app, frontend, and lives there as a view in a file some/frontend/views.py.
The PYTHONPATH is clean, that is, from my project only /path/to/project is in the path. In the frontend/views.py I import the exceptions.py via import application.exceptions, which seems to work. (Now, in retrospective, I don't know exactly, why it works...)
The exception is raised in the exceptions.py file itself.
Update 2
It might be interesting for some readers, that I finally found the place, where imports went wrong.
The sys.path didn't show any suspect irregularities. My Django project lay in /var/www/django/project. I had installed the apps app1 and app2, but noted them in the settings.py as
INSTALLED_APPS = [
'project.app1',
'project.app2',
]
The additional project. was the culprit for messing up sys.modules. Rewriting the settings to
INSTALLED_APPS = [
'app1',
'app2',
]
solved the problem.
Why that would be a problem? exception would me matched based on class type and it would be same however it is imported e.g.
import exceptions
l=[]
try:
l[1]
except exceptions.IndexError,e:
print e
try:
l[1]
except IndexError,e:
print e
both catch the same exception
you can even assign it to a new name, though not recommended usually
import os
os.myerror = exceptions.IndexError
try:
l[1]
except os.myerror,e:
print e
"If not, what would be best practices to avoid these name clashes?"
That depends entirely on why they happen. In a normal installation, you can not import from both application.exceptions and somepath.application.exceptions, unless the first case is a relative path from within the module somepath. And in that case Python will understand that the modules are the same, and you won't have a problem.
You are unclear on if you really have a problem or if it's theory. If you do have a problem, I'd guess that there is something fishy with your PYTHONPATH. Maybe both a directory and it's subdirectory is in the PATH?
Even if the same module is imported several times and in different ways, the CustomException class is still the same object, so it doesn't matter how you refer to it.
I don't know if there is a way to handle this inclusion path issue.
My suggestion would be to use the 'as' keyword in your import
Something like:
import some.application.exceptions as my_exceptions
or
import application.exceptions as my_exceptions