I want to make a map containing various attributes of a single item and share it across several modules. The most obvious answer is to use a string, but since the map will be used across several modules, I don't want maintainers to have to know about all existing attributes. For example, if two modules want something associated with "color", they shouldn't clobber each other by accidentally picking the same name.
A few ideas I've thought of which don't work:
1) Strings, integer keys: As said above, this isn't easily extensible.
2) Create of a particular class, use id(): I thought it would work to make a Key() class, for example, but this doesn't work in the case of modules imported from two different places. For example, you might want to make the "color" attribute described above as follows:
color = Key()
This doesn't work if you do
# module named foo
bar = Key()
import foo
id(foo.bar) == id(bar)
3) Look at the stack trace to see where the Key was created and use that as part of the identifier. Definitely wouldn't want to use line number, as this would be too brittle. Using module and name of item fails in the case of the main module, though. For example:
in foo.py:
key = Key() # will be named foo.py-key
elsewhere, in bar.py:
import foo
foo.key # will be named /Users/person/foo.py-key
Thanks! Any advice is appreciated.
Use a string, but prefix it with the current module name. Then you won't get any collisions.
key = __name__ + '.color'
Related
I have a set of named tuple instances. E.g.:
CostFN = namedtuple('CostFN', ['name', 'func', 'args'])
tuple_lin_1 = CostFN(name="lin_1", args=args_for_1, func1=func_1)
tuple_lin_2 = CostFN(name="lin_2", args=args_for_1, func1=func_2)
tuple_lin_3 = CostFN(name="lin_3", args=args_for_2, func1=func_1)
tuple_lin_4 = CostFN(name="lin_4", args=args_for_2, func1=func_2)
I am saving the instance name, and I want to access the relevant instance from its name.
Something like:
my_tuple = CostFN.get("lin4")
How do I achieve that?
You can't in any reasonable way. Objects don't have names in the way you think they do, names are bound to objects. So tuple_lin_4 is a name that contains a binding to the associated instance of CostFN, but the instance has no idea it's bound to that name. Similarly, the instance knows it is of type CostFN, but CostFN doesn't know about its own instances. So you've got two layers where the linkages don't work the way you intend.
If looking up instances by name is an issue, I'd suggest making a dict mapping names to instances, rather than using individual named variables. If you really wanted to, you could do terrible things with a custom __new__ to cache instances by name on a dict tied to the class, but frankly, even that's going too far; you likely have an XY problem if you think you need a behavior like this at all.
Suppose I have a module PyFoo.py that has a function bar. I want bar to print all of the local variables associated with the namespace that called it.
For example:
#! /usr/bin/env python
import PyFoo as pf
var1 = 'hi'
print locals()
pf.bar()
The two last lines would give the same output. So far I've tried defining bar as such:
def bar(x=locals):
print x()
def bar(x=locals()):
print x
But neither works. The first ends up being what's local to bar's namespace (which I guess is because that's when it's evaluated), and the second is as if I passed in globals (which I assume is because it's evaluated during import).
Is there a way I can have the default value of argument x of bar be all variables in the namespace which called bar?
EDIT 2018-07-29:
As has been pointed out, what was given was an XY Problem; as such, I'll give the specifics.
The module I'm putting together will allow the user to create various objects that represent different aspects of a numerical problem (e.x. various topology definitions, boundary conditions, constitutive models, ect.) and define how any given object interacts with any other object(s). The idea is for the user to import the module, define the various model entities that they need, and then call a function which will take all objects passed to it, make needed adjustments to ensure capability between them, and then write out a file that represents the entire numerical problem as a text file.
The module has a function generate that accepts each of the various types of aspects of the numerical problem. The default value for all arguments is an empty list. If a non-empty list is passed, then generate will use those instances for generating the completed numerical problem. If an argument is an empty list, then I'd like it to take in all instances in the namespace that called generate (which I will then parse out the appropriate instances for the argument).
EDIT 2018-07-29:
Sorry for any lack of understanding on my part (I'm not that strong of a programmer), but I think I might understand what you're saying with respect to an instance being declared or registered.
From my limited understanding, could this be done by creating some sort of registry dataset (like a list or dict) in the module that will be created when the module is imported, and that all module classes take this registry object in by default. During class initialization self can be appended to said dataset, and then the genereate function will take the registry as a default value for one of the arguments?
There's no way you can do what you want directly.
locals just returns the local variables in whatever namespace it's called in. As you've seen, you have access to the namespace the function is defined in at the time of definition, and you have access to the namespace of the function itself from within the function, but you don't have access to any other namespaces.
You can do what you want indirectly… but it's almost certainly a bad idea. At least this smells like an XY problem, and whatever it is you're actually trying to do, there's probably a better way to do it.
But occasionally it is necessary, so in case you have one of those cases:
The main good reason to want to know the locals of your caller is for some kind of debugging or other introspection function. And the way to do introspection is almost always through the inspect library.
In this case, what you want to inspect is the interpreter call stack. The calling function will be the first frame on the call stack behind your function's own frame.
You can get the raw stack frame:
inspect.currentframe().f_back
… or you can get a FrameInfo representing it:
inspect.stack()[1]
As explained at the top of the inspect docs, a frame object's local namespace is available as:
frame.f_locals
Note that this has all the same caveats that apply to getting your own locals with locals: what you get isn't the live namespace, but a mapping that, even if it is mutable, can't be used to modify the namespace (or, worse in 2.x, one that may or may not modify the namespace, unpredictably), and that has all cell and free variables flattened into their values rather than their cell references.
Also, see the big warning in the docs about not keeping frame objects alive unnecessarily (or calling their clear method if you need to keep a snapshot but not all of the references, but I think that only exists in 3.x).
The Python docs (Python2 and Python3) state that identifiers must not start with a digit. From my understanding this is solely a compiler constraint (see also this question). So is there anything wrong about starting dynamically created identifiers with a digit? For example:
type('3Tuple', (object,), {})
setattr(some_object, '123', 123)
Edit
Admittedly the second example (using setattr) from above might be less relevant, as one could introspect the object via dir, discovers the attribute '123' but cannot retrieve it via some_object.123.
So I'll elaborate a bit more on the first example (which appears more relevant to me).
The user should be provided with fixed length tuples and because the tuple length is arbitrary and not known in advance a proxy function for retrieving such tuples is used (could also be a class implementing __call__ or __getattr__):
def NTuple(number_of_elements):
# Add methods here.
return type('{0}Tuple'.format(number_of_elements),
(object,),
{'number_of_elements': number_of_elements})
The typical use case involves referencing instances of those dynamically create classes, not the classes themselves, as for example:
limits = NTuple(3)(1, 2, 3)
But still the class name provides some useful information (as opposed to just using 'Tuple'):
>>> limits.__class__.__name__
'3Tuple'
Also those class names will not be relevant for any code at compile time, hence it doesn't introduce any obstacles for the programmer/user.
Is there a way to get the dictionary containing the global variables in a specific module namespace? It looks that you can achieve it if the module contains at least one function with something like:
import mymodule
d = mymodule.func.__globals__
Is this right?
Is there a more general way? (For instance, how could this be done if the module wouldn't contain any function but only variables?)
I am not expecting a "What are you trying to do with that? There is probably another way of doing it." answer; my question is rather a theoretical one.
I am more interested here by a Python3 answer if it matters.
Just grab use vars which grabs its __dict__ which corresponds to the global scope.
vars(mymodule)
func.__globals__ simply returns this dictionary for you see how:
vars(mymodule) is mymodule.func.__globals__
returns True.
The question
Are there any valid use cases for the ol' module switcheroo, where you replace the module with a class instance? By a valid use case, I mean a case where it would be generally agreed that using this trick would be the best way of solving a problem. For example, the module:
VERSION = (1, 2, 8)
VERSION_NAME = '1.2.8'
Could be converted to this:
import sys
class ConstantsModule(object):
def __init__(self):
self.VERSION = (1, 2, 8)
#property
def VERSION_NAME(self):
return u'{}.{}.{}'.format(*self.VERSION)
sys.modules[__name__] = ConstantsModule()
And now VERSION_NAME is a property with logic behind it.
I have googled around for this without finding anything relevant. I learned of this trick in a SO answer I read some time ago, and I know this is something referred to as "black magic" and to be avoided, but I'm curious about the valid use cases.
My specific use case
I have a small problem with one of my modules that could easily be solved if the module was a class instance. I have a "constant" called VERSION_NAME, which is a string version of VERSION, which in turn is a tuple with my application's version information. The VERSION_NAME is used throughout my project and in several other projects based on this one. Now I would like VERSION_NAME to include some logic - I would like it to be based on VERSION so that I don't have to edit it manually all the time, and I would like it to be formatted slightly differently depending on a couple of environmental circumstances. The way I see it I have two choices:
Hunt down every use-case of VERSION_NAME in my project and all its sub-projects and change it to a function call like get_version_name.
Invoke black magic like shown above.
This question is not about my use case though, this is just an example of what I figure it could be used for.
Since everything in Python is an object, there is no black magic about it; this is simply duck typing; if you create an object that walks and talks like a module, then the rest of Python is none the wiser.
However, for your specific use-case, you don't need to resort to this level of deception. Simply calculate version name at import time:
VERSION_NAME = u'{}.{}.{}'.format(*VERSION)
Nowhere is it stated that module globals can only be literal values; just use a Python expression for them instead.
After all, your VERSION_NAME variable is not going to change during the lifetime of your program, you only need to generate it once. Use a property only when you need an attribute that needs to be re-calculated every time you access it.