Related
What is a good design pattern to implement templated object generation (not sure that's the name) in python?
By that, I mean having a function such as:
from typing import TypeVar
T = TypeVar('T')
def mk_templated_obj_factory(template: T) -> Callable[..., T]:
"""Returns a f(**kwargs) function that returns an object of type T created by a template of the same type."""
Python has templated strings. Something like `"this {is} a {template}".format' would be how one could achieve the above. If we want to get a "proper" function that has a signature (useful for a user so they know what arguments they need to provide!), we could do this:
from inspect import signature, Signature, Parameter
from operator import itemgetter
from typing import Callable
f = "hello {name} how are you {verb}?".format
def templated_string_func(template: str) -> Callable:
"""A function making templated strings. Like template.format, but with a signature"""
f = partial(str.format, template)
names = filter(None, map(itemgetter(1), string.Formatter().parse(template)))
params = [Parameter(name=name, kind=Parameter.KEYWORD_ONLY) for name in names]
f.__signature__ = Signature(params)
return f
f = templated_string_func("hello {name} how are you {verb}?")
assert f(name='Christian', verb='doing') == 'hello Christian how are you doing?'
assert str(signature(f)) == '(*, name, verb)'
But would if we want to make dict factories? Something having this behavior:
g = templated_dict_func(template={'hello': '$name', 'how are you': ['$verb', 2]})
assert g(name='Christian', verb='doing') == {'hello': '$name', 'how are you': ['doing', 2]}
What about other types of objects?
It seems like something that would have a solid design pattern...
I would recommend using decorators to register your template function generating functions in a dictionary that maps from types to the functions that handle them. The dictionary is needed in order to be able to template objects of any type in an extensible way, without writing all the templating logic in a single big function, but instead adding handling logic for new types as needed.
The core code is in the Templater class, just grouped here for organisation:
class Templater:
templater_registry: dict[type, Callable[[Any], TemplateFunc]] = {}
#classmethod
def register(cls, handles_type: type):
def decorator(f):
cls.templater_registry[handles_type] = f
return f
return decorator
...
Where TemplateFunc is defined as Generator[str, None, Callable[..., T]], a generator that yields strs and returns a function that returns some type T. This is chosen so that the template handlers can yield the names of their keyword arguments and then return their template function. The Templater.template_func method uses a something of type TemplateFunc to generate a function with the correct signature.
The register decorator presented above is written such that:
#Templater.register(dict)
def templated_dict_func(template: dict[K, V]):
pass
is equivalent to:
def templated_dict_func(template: dict[K, V]):
pass
Templater.templater_registry[dict] = templated_dict_func
The code for templating any type is fairly self-explainatory:
class Templater:
...
#classmethod
def template_func_generator(cls, template: T) -> TemplateFunc[T]:
# if it is a type that can be a template
if type(template) in cls.templater_registry:
# then return the template handler
template_factory = cls.templater_registry[type(template)]
return template_factory(template)
else:
# else: an empty generator that returns a function that returns the template unchanged,
# since we don't know how to handle it
def just_return():
return lambda: template
yield # this yield is needed to tell python that this is a generator
return just_return()
The code for templating strings is fairly unchanged, except that the argument names are yielded instead of put in the function signature:
#Templater.register(str)
def templated_string_func(template: str) -> TemplateFunc[str]:
"""A function making templated strings. Like template.format, but with a signature"""
f = partial(str.format, template)
yield from filter(None, map(itemgetter(1), string.Formatter().parse(template)))
return f
The list template function could look like this:
#Templater.register(list)
def templated_list_func(template: list[T]) -> TemplateFunc[list[T]]:
entries = []
for item in template:
item_template_func = yield from Templater.template_func_generator(item)
entries.append(item_template_func)
def template_func(**kwargs):
return [
item_template_func(**kwargs)
for item_template_func in entries
]
return template_func
Although, if you cannot guarantee that every template function can handle extra arguments, you need to track which arguments belong to which elements and only pass the necessary ones. I use the get_generator_return utility function (defined later on) to capture both the yielded values and the return value of the recursive calls.
#Templater.register(list)
def templated_list_func(template: list[T]) -> TemplateFunc[list[T]]:
entries = []
for item in template:
params, item_template_func = get_generator_return(Templater.template_func_generator(item))
params = tuple(params)
yield from params
entries.append((item_template_func, params))
def template_func(**kwargs):
return [
item_template_func(**{arg: kwargs[arg] for arg in args})
for item_template_func, args in entries
]
return template_func
The dict handler is implemented similarly. This system could be extended to support all kinds of different objects, including arbitrary dataclasses and more, but I leave that as an exercise for the reader!
Here is the entire working example:
import string
from functools import partial
from inspect import Signature, Parameter
from operator import itemgetter
from typing import Callable, Any, TypeVar, Generator, Tuple, Dict, List
from collections import namedtuple
T = TypeVar('T')
U = TypeVar('U')
def get_generator_return(gen: Generator[T, Any, U]) -> Tuple[Generator[T, Any, U], U]:
return_value = None
def inner():
nonlocal return_value
return_value = yield from gen
gen_items = list(inner())
def new_gen():
yield from gen_items
return return_value
return new_gen(), return_value
# TemplateFunc: TypeAlias = Generator[str, None, Callable[..., T]]
TemplateFunc = Generator[str, None, Callable[..., T]]
class Templater:
templater_registry: Dict[type, Callable[[Any], TemplateFunc]] = {}
#classmethod
def register(cls, handles_type: type):
def decorator(f):
cls.templater_registry[handles_type] = f
return f
return decorator
#classmethod
def template_func_generator(cls, template: T) -> TemplateFunc[T]:
if type(template) in cls.templater_registry:
template_factory = cls.templater_registry[type(template)]
return template_factory(template)
else:
# an empty generator that returns a function that returns the template unchanged,
# since we don't know how to handle it
def just_return():
return lambda: template
yield # this yield is needed to tell python that this is a generator
return just_return()
#classmethod
def template_func(cls, template: T) -> Callable[..., T]:
gen = cls.template_func_generator(template)
params, f = get_generator_return(gen)
f.__signature__ = Signature(Parameter(name=param, kind=Parameter.KEYWORD_ONLY) for param in params)
return f
#Templater.register(str)
def templated_string_func(template: str) -> TemplateFunc[str]:
"""A function making templated strings. Like template.format, but with a signature"""
f = partial(str.format, template)
yield from filter(None, map(itemgetter(1), string.Formatter().parse(template)))
return f
K = TypeVar('K')
V = TypeVar('V')
#Templater.register(dict)
def templated_dict_func(template: Dict[K, V]) -> TemplateFunc[Dict[K, V]]:
DictEntryInfo = namedtuple('DictEntryInfo', ['key_func', 'value_func', 'key_args', 'value_args'])
entries: list[DictEntryInfo] = []
for key, value in template.items():
key_params, key_template_func = get_generator_return(Templater.template_func_generator(key))
value_params, value_template_func = get_generator_return(Templater.template_func_generator(value))
key_params = tuple(key_params)
value_params = tuple(value_params)
yield from key_params
yield from value_params
entries.append(DictEntryInfo(key_template_func, value_template_func, key_params, value_params))
def template_func(**kwargs):
return {
entry_info.key_func(**{arg: kwargs[arg] for arg in entry_info.key_args}):
entry_info.value_func(**{arg: kwargs[arg] for arg in entry_info.value_args})
for entry_info in entries
}
return template_func
#Templater.register(list)
def templated_list_func(template: List[T]) -> TemplateFunc[List[T]]:
entries = []
for item in template:
params, item_template_func = get_generator_return(Templater.template_func_generator(item))
params = tuple(params)
yield from params
entries.append((item_template_func, params))
def template_func(**kwargs):
return [
item_template_func(**{arg: kwargs[arg] for arg in args})
for item_template_func, args in entries
]
return template_func
g = Templater.template_func(template={'hello': '{name}', 'how are you': ['{verb}', 2]})
assert g(name='Christian', verb='doing') == {'hello': 'Christian', 'how are you': ['doing', 2]}
print(g.__signature__)
Let's say I have a python function whose single argument is a non-trivial type:
from typing import List, Dict
ArgType = List[Dict[str, int]] # this could be any non-trivial type
def myfun(a: ArgType) -> None:
...
... and then I have a data structure that I have unpacked from a JSON source:
import json
data = json.loads(...)
My question is: How can I check at runtime that data has the correct type to be used as an argument to myfun() before using it as an argument for myfun()?
if not isCorrectType(data, ArgType):
raise TypeError("data is not correct type")
else:
myfun(data)
Validating a type annotation is a non-trivial task. Python does not do it automatically, and writing your own validator is difficult because the typing module doesn't offer much of a useful interface. (In fact the internals of the typing module have changed so much since its introduction in python 3.5 that it's honestly a nightmare to work with.)
Here's a type validator function taken from one of my personal projects (wall of code warning):
import inspect
import typing
__all__ = ['is_instance', 'is_subtype', 'python_type', 'is_generic', 'is_base_generic', 'is_qualified_generic']
if hasattr(typing, '_GenericAlias'):
# python 3.7
def _is_generic(cls):
if isinstance(cls, typing._GenericAlias):
return True
if isinstance(cls, typing._SpecialForm):
return cls not in {typing.Any}
return False
def _is_base_generic(cls):
if isinstance(cls, typing._GenericAlias):
if cls.__origin__ in {typing.Generic, typing._Protocol}:
return False
if isinstance(cls, typing._VariadicGenericAlias):
return True
return len(cls.__parameters__) > 0
if isinstance(cls, typing._SpecialForm):
return cls._name in {'ClassVar', 'Union', 'Optional'}
return False
def _get_base_generic(cls):
# subclasses of Generic will have their _name set to None, but
# their __origin__ will point to the base generic
if cls._name is None:
return cls.__origin__
else:
return getattr(typing, cls._name)
def _get_python_type(cls):
"""
Like `python_type`, but only works with `typing` classes.
"""
return cls.__origin__
def _get_name(cls):
return cls._name
else:
# python <3.7
if hasattr(typing, '_Union'):
# python 3.6
def _is_generic(cls):
if isinstance(cls, (typing.GenericMeta, typing._Union, typing._Optional, typing._ClassVar)):
return True
return False
def _is_base_generic(cls):
if isinstance(cls, (typing.GenericMeta, typing._Union)):
return cls.__args__ in {None, ()}
if isinstance(cls, typing._Optional):
return True
return False
else:
# python 3.5
def _is_generic(cls):
if isinstance(cls, (typing.GenericMeta, typing.UnionMeta, typing.OptionalMeta, typing.CallableMeta, typing.TupleMeta)):
return True
return False
def _is_base_generic(cls):
if isinstance(cls, typing.GenericMeta):
return all(isinstance(arg, typing.TypeVar) for arg in cls.__parameters__)
if isinstance(cls, typing.UnionMeta):
return cls.__union_params__ is None
if isinstance(cls, typing.TupleMeta):
return cls.__tuple_params__ is None
if isinstance(cls, typing.CallableMeta):
return cls.__args__ is None
if isinstance(cls, typing.OptionalMeta):
return True
return False
def _get_base_generic(cls):
try:
return cls.__origin__
except AttributeError:
pass
name = type(cls).__name__
if not name.endswith('Meta'):
raise NotImplementedError("Cannot determine base of {}".format(cls))
name = name[:-4]
return getattr(typing, name)
def _get_python_type(cls):
"""
Like `python_type`, but only works with `typing` classes.
"""
# Many classes actually reference their corresponding abstract base class from the abc module
# instead of their builtin variant (i.e. typing.List references MutableSequence instead of list).
# We're interested in the builtin class (if any), so we'll traverse the MRO and look for it there.
for typ in cls.mro():
if typ.__module__ == 'builtins' and typ is not object:
return typ
try:
return cls.__extra__
except AttributeError:
pass
if is_qualified_generic(cls):
cls = get_base_generic(cls)
if cls is typing.Tuple:
return tuple
raise NotImplementedError("Cannot determine python type of {}".format(cls))
def _get_name(cls):
try:
return cls.__name__
except AttributeError:
return type(cls).__name__[1:]
if hasattr(typing.List, '__args__'):
# python 3.6+
def _get_subtypes(cls):
subtypes = cls.__args__
if get_base_generic(cls) is typing.Callable:
if len(subtypes) != 2 or subtypes[0] is not ...:
subtypes = (subtypes[:-1], subtypes[-1])
return subtypes
else:
# python 3.5
def _get_subtypes(cls):
if isinstance(cls, typing.CallableMeta):
if cls.__args__ is None:
return ()
return cls.__args__, cls.__result__
for name in ['__parameters__', '__union_params__', '__tuple_params__']:
try:
subtypes = getattr(cls, name)
break
except AttributeError:
pass
else:
raise NotImplementedError("Cannot extract subtypes from {}".format(cls))
subtypes = [typ for typ in subtypes if not isinstance(typ, typing.TypeVar)]
return subtypes
def is_generic(cls):
"""
Detects any kind of generic, for example `List` or `List[int]`. This includes "special" types like
Union and Tuple - anything that's subscriptable, basically.
"""
return _is_generic(cls)
def is_base_generic(cls):
"""
Detects generic base classes, for example `List` (but not `List[int]`)
"""
return _is_base_generic(cls)
def is_qualified_generic(cls):
"""
Detects generics with arguments, for example `List[int]` (but not `List`)
"""
return is_generic(cls) and not is_base_generic(cls)
def get_base_generic(cls):
if not is_qualified_generic(cls):
raise TypeError('{} is not a qualified Generic and thus has no base'.format(cls))
return _get_base_generic(cls)
def get_subtypes(cls):
return _get_subtypes(cls)
def _instancecheck_iterable(iterable, type_args):
if len(type_args) != 1:
raise TypeError("Generic iterables must have exactly 1 type argument; found {}".format(type_args))
type_ = type_args[0]
return all(is_instance(val, type_) for val in iterable)
def _instancecheck_mapping(mapping, type_args):
return _instancecheck_itemsview(mapping.items(), type_args)
def _instancecheck_itemsview(itemsview, type_args):
if len(type_args) != 2:
raise TypeError("Generic mappings must have exactly 2 type arguments; found {}".format(type_args))
key_type, value_type = type_args
return all(is_instance(key, key_type) and is_instance(val, value_type) for key, val in itemsview)
def _instancecheck_tuple(tup, type_args):
if len(tup) != len(type_args):
return False
return all(is_instance(val, type_) for val, type_ in zip(tup, type_args))
_ORIGIN_TYPE_CHECKERS = {}
for class_path, check_func in {
# iterables
'typing.Container': _instancecheck_iterable,
'typing.Collection': _instancecheck_iterable,
'typing.AbstractSet': _instancecheck_iterable,
'typing.MutableSet': _instancecheck_iterable,
'typing.Sequence': _instancecheck_iterable,
'typing.MutableSequence': _instancecheck_iterable,
'typing.ByteString': _instancecheck_iterable,
'typing.Deque': _instancecheck_iterable,
'typing.List': _instancecheck_iterable,
'typing.Set': _instancecheck_iterable,
'typing.FrozenSet': _instancecheck_iterable,
'typing.KeysView': _instancecheck_iterable,
'typing.ValuesView': _instancecheck_iterable,
'typing.AsyncIterable': _instancecheck_iterable,
# mappings
'typing.Mapping': _instancecheck_mapping,
'typing.MutableMapping': _instancecheck_mapping,
'typing.MappingView': _instancecheck_mapping,
'typing.ItemsView': _instancecheck_itemsview,
'typing.Dict': _instancecheck_mapping,
'typing.DefaultDict': _instancecheck_mapping,
'typing.Counter': _instancecheck_mapping,
'typing.ChainMap': _instancecheck_mapping,
# other
'typing.Tuple': _instancecheck_tuple,
}.items():
try:
cls = eval(class_path)
except AttributeError:
continue
_ORIGIN_TYPE_CHECKERS[cls] = check_func
def _instancecheck_callable(value, type_):
if not callable(value):
return False
if is_base_generic(type_):
return True
param_types, ret_type = get_subtypes(type_)
sig = inspect.signature(value)
missing_annotations = []
if param_types is not ...:
if len(param_types) != len(sig.parameters):
return False
# FIXME: add support for TypeVars
# if any of the existing annotations don't match the type, we'll return False.
# Then, if any annotations are missing, we'll throw an exception.
for param, expected_type in zip(sig.parameters.values(), param_types):
param_type = param.annotation
if param_type is inspect.Parameter.empty:
missing_annotations.append(param)
continue
if not is_subtype(param_type, expected_type):
return False
if sig.return_annotation is inspect.Signature.empty:
missing_annotations.append('return')
else:
if not is_subtype(sig.return_annotation, ret_type):
return False
if missing_annotations:
raise ValueError("Missing annotations: {}".format(missing_annotations))
return True
def _instancecheck_union(value, type_):
types = get_subtypes(type_)
return any(is_instance(value, typ) for typ in types)
def _instancecheck_type(value, type_):
# if it's not a class, return False
if not isinstance(value, type):
return False
if is_base_generic(type_):
return True
type_args = get_subtypes(type_)
if len(type_args) != 1:
raise TypeError("Type must have exactly 1 type argument; found {}".format(type_args))
return is_subtype(value, type_args[0])
_SPECIAL_INSTANCE_CHECKERS = {
'Union': _instancecheck_union,
'Callable': _instancecheck_callable,
'Type': _instancecheck_type,
'Any': lambda v, t: True,
}
def is_instance(obj, type_):
if type_.__module__ == 'typing':
if is_qualified_generic(type_):
base_generic = get_base_generic(type_)
else:
base_generic = type_
name = _get_name(base_generic)
try:
validator = _SPECIAL_INSTANCE_CHECKERS[name]
except KeyError:
pass
else:
return validator(obj, type_)
if is_base_generic(type_):
python_type = _get_python_type(type_)
return isinstance(obj, python_type)
if is_qualified_generic(type_):
python_type = _get_python_type(type_)
if not isinstance(obj, python_type):
return False
base = get_base_generic(type_)
try:
validator = _ORIGIN_TYPE_CHECKERS[base]
except KeyError:
raise NotImplementedError("Cannot perform isinstance check for type {}".format(type_))
type_args = get_subtypes(type_)
return validator(obj, type_args)
return isinstance(obj, type_)
def is_subtype(sub_type, super_type):
if not is_generic(sub_type):
python_super = python_type(super_type)
return issubclass(sub_type, python_super)
# at this point we know `sub_type` is a generic
python_sub = python_type(sub_type)
python_super = python_type(super_type)
if not issubclass(python_sub, python_super):
return False
# at this point we know that `sub_type`'s base type is a subtype of `super_type`'s base type.
# If `super_type` isn't qualified, then there's nothing more to do.
if not is_generic(super_type) or is_base_generic(super_type):
return True
# at this point we know that `super_type` is a qualified generic... so if `sub_type` isn't
# qualified, it can't be a subtype.
if is_base_generic(sub_type):
return False
# at this point we know that both types are qualified generics, so we just have to
# compare their sub-types.
sub_args = get_subtypes(sub_type)
super_args = get_subtypes(super_type)
return all(is_subtype(sub_arg, super_arg) for sub_arg, super_arg in zip(sub_args, super_args))
def python_type(annotation):
"""
Given a type annotation or a class as input, returns the corresponding python class.
Examples:
::
>>> python_type(typing.Dict)
<class 'dict'>
>>> python_type(typing.List[int])
<class 'list'>
>>> python_type(int)
<class 'int'>
"""
try:
mro = annotation.mro()
except AttributeError:
# if it doesn't have an mro method, it must be a weird typing object
return _get_python_type(annotation)
if Type in mro:
return annotation.python_type
elif annotation.__module__ == 'typing':
return _get_python_type(annotation)
else:
return annotation
Demonstration:
>>> is_instance([{'x': 3}], List[Dict[str, int]])
True
>>> is_instance([{'x': 3}, {'y': 7.5}], List[Dict[str, int]])
False
(As far as I'm aware, this supports all python versions, even the ones <3.5 using the typing module backport.)
It's awkward that there's no built-in function for this but typeguard comes with a convenient check_type() function:
>>> from typeguard import check_type
>>> from typing import List
>>> check_type("foo", [1,2,"3"], List[int])
Traceback (most recent call last):
...
TypeError: type of foo[2] must be int; got str instead
type of foo[2] must be int; got str instead
For more see: https://typeguard.readthedocs.io/en/latest/api.html#typeguard.check_type
First of all, even though I think you are aware but rather for the sake of completeness, the typing library contains types for type hints. These type hints are used by IDE's to check if your code is somewhat sane, and also serves as documentation what types a developer expects.
To check whether a variable is a type of something, we have to use the isinstance function. Amazingly, we can use direct types of the typing library function, eg.
from typing import List
value = []
isinstance(value, List)
However, for nested structures such as List[Dict[str, int]] we cannot use this directly, because you funny enough get a TypeError. What you have to do is:
Check if the initial value is a list
Check if each item of the list is of type dict
Check if each key of each dict is in fact a string and if each value is in fact an int
Unfortunately, for strict checking python is a bit cumbersome. However, do be aware that python makes use of duck typing: if it is like a duck and behaves like a duck, then it definitely is a duck.
The common way to handle this is by making use of the fact that if whatever object you pass to myfun doesn't have the required functionality a corresponding exception will be raised (usually TypeError or AttributeError). So you would do the following:
try:
myfun(data)
except (TypeError, AttributeError) as err:
# Fallback for invalid types here.
You indicate in your question that you would raise a TypeError if the passed object does not have the appropriate structure but Python does this already for you. The critical question is how you would handle this case. You could also move the try / except block into myfun, if appropriate. When it comes to typing in Python you usually rely on duck typing: if the object has the required functionality then you don't care much about what type it is, as long as it serves the purpose.
Consider the following example. We just pass the data into the function and then get the AttributeError for free (which we can then except); no need for manual type checking:
>>> def myfun(data):
... for x in data:
... print(x.items())
...
>>> data = json.loads('[[["a", 1], ["b", 2]], [["c", 3], ["d", 4]]]')
>>> myfun(data)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in myfun
AttributeError: 'list' object has no attribute 'items'
In case you are concerned about the usefulness of the resulting error, you could still except and then re-raise a custom exception (or even change the exception's message):
try:
myfun(data)
except (TypeError, AttributeError) as err:
raise TypeError('Data has incorrect structure') from err
try:
myfun(data)
except (TypeError, AttributeError) as err:
err.args = ('Data has incorrect structure',)
raise
When using third-party code one should always check the documentation for exceptions that will be raised. For example numpy.inner reports that it will raise a ValueError under certain circumstances. When using that function we don't need to perform any checks ourselves but rely on the fact that it will raise the error if needed. When using third-party code for which it is not clear how it will behave in some corner-cases, i.m.o. it is easier and clearer to just hardcode a corresponding type checker (see below) instead of using a generic solution that works for any type. These cases should be rare anyway and leaving a corresponding comment makes your fellow developers aware of the situation.
The typing library is for type-hinting and as such it won't be checking the types at runtime. Sure you could do this manually but it is rather cumbersome:
def type_checker(data):
return (
isinstance(data, list)
and all(isinstance(x, dict) for x in list)
and all(isinstance(k, str) and isinstance(v, int) for x in list for k, v in x.items())
)
This together with an appropriate comment is still an acceptable solution and it is reusable where a similar data structure is expected. The intent is clear and the code is easily verifiable.
You would have to check your nested type structure manually - the type hint's are not enforced.
Checking like this ist best done using ABC (Abstract Meta Classes) - so users can provide their derived classes that support the same accessing as default dict/lists:
import collections.abc
def isCorrectType(data):
if isinstance(data, collections.abc.Collection):
for d in data:
if isinstance(d,collections.abc.MutableMapping):
for key in d:
if isinstance(key,str) and isinstance(d[key],int):
pass
else:
return False
else:
return False
else:
return False
return True
Output:
print ( isCorrectType( [ {"a":2} ] )) # True
print ( isCorrectType( [ {2:2} ] )) # False
print ( isCorrectType( [ {"a":"a"} ] )) # False
print ( isCorrectType( [ {"a":2},1 ] )) # False
Doku:
ABC - abstract meta classes
Related:
What is duck typing?
The other way round would be to follow the "Ask forgiveness not permission" - explain paradigm and simyply use your data in the form you want and try:/except: around if if it does not conform to what you wanted. This fits better with What is duck typing? - and allows (similar to ABC-checking) the consumer to provide you with derived classes from list/dict while it still will work...
If all you want to do is json-parsing, you should just use pydantic.
But, I encountered the same problem where I wanted to check the type of python objects, so I created a simpler solution than in other answers that handles at least complex types with nested lists and dictionaries.
I created a gist with this method at https://gist.github.com/ramraj07/f537bf9f80b4133c65dd76c958d4c461
Some example uses of this method include:
from typing import List, Dict, Union, Type, Optional
check_type('a', str)
check_type({'a': 1}, Dict[str, int])
check_type([{'a': [1.0]}, 'ten'], List[Union[Dict[str, List[float]], str]])
check_type(None, Optional[str])
check_type('abc', Optional[str])
Here's the code below for reference:
import typing
def check_type(obj: typing.Any, type_to_check: typing.Any, _external=True) -> None:
try:
if not hasattr(type_to_check, "_name"):
# base-case
if not isinstance(obj, type_to_check):
raise TypeError
return
# type_to_check is from typing library
type_name = type_to_check._name
if type_to_check is typing.Any:
pass
elif type_name in ("List", "Tuple"):
if (type_name == "List" and not isinstance(obj, list)) or (
type_name == "Tuple" and not isinstance(obj, tuple)
):
raise TypeError
element_type = type_to_check.__args__[0]
for element in obj:
check_type(element, element_type, _external=False)
elif type_name == "Dict":
if not isinstance(obj, dict):
raise TypeError
if len(type_to_check.__args__) != 2:
raise NotImplementedError(
"check_type can only accept Dict typing with separate annotations for key and values"
)
key_type, value_type = type_to_check.__args__
for key, value in obj.items():
check_type(key, key_type, _external=False)
check_type(value, value_type, _external=False)
elif type_name is None and type_to_check.__origin__ is typing.Union:
type_options = type_to_check.__args__
no_option_matched = True
for type_option in type_options:
try:
check_type(obj, type_option, _external=False)
no_option_matched = False
break
except TypeError:
pass
if no_option_matched:
raise TypeError
else:
raise NotImplementedError(
f"check_type method currently does not support checking typing of form '{type_name}'"
)
except TypeError:
if _external:
raise TypeError(
f"Object {repr(obj)} is of type {_construct_type_description(obj)} "
f"when {type_to_check} was expected"
)
raise TypeError()
def _construct_type_description(obj) -> str:
def get_types_in_iterable(iterable) -> str:
types = {_construct_type_description(element) for element in iterable}
return types.pop() if len(types) == 1 else f"Union[{','.join(types)}]"
if isinstance(obj, list):
return f"List[{get_types_in_iterable(obj)}]"
elif isinstance(obj, dict):
key_types = get_types_in_iterable(obj.keys())
val_types = get_types_in_iterable(obj.values())
return f"Dict[{key_types}, {val_types}]"
else:
return type(obj).__name__
I am subclassing Template from string to give it some extra defaulting capabilities. The idea is for its look-up to extend beyond the passed dict to the locals() first, then to the globals() and finally default (e.g., to '-'). So this is what I wrote:
class MyTemplate(Template):
def substitute_default(*args, **kws):
if not args:
raise TypeError("descriptor 'substitute' of 'Template' object needs an argument")
self, *args = args # allow the "self" keyword be passed
if len(args) > 1:
raise TypeError('Too many positional arguments')
if not args:
mapping = kws
elif kws:
mapping = ChainMap(kws, args[0])
else:
mapping = args[0]
def convert(mo):
named = mo.group('named') or mo.group('braced')
if named is not None:
val = mapping.get(named, locals().get(named, globals().get(named, '-')))
return '%s' % (val,)
if mo.group('escaped') is not None:
return self.delimiter
if mo.group('invalid') is not None:
self._invalid(mo)
raise ValueError('Unrecognized named group in pattern', self.pattern)
return self.pattern.sub(convert, self.template)
The line with the juice is this:
val = mapping.get(named, locals().get(named, globals().get(named, '-')))
I am testing it like so:
a = 'global_foo'
def f():
b = 'local_foo'
t = MyTemplate('''a=$a, b=$b, c=$c, d=$d''')
text = t.substitute_default({'c': 'foo', 'd': 'bar'})
print(text)
f() # -> a=global_foo, b=-, c=foo, d=bar
As you can see, the globals() look-up works but the locals() one does not..
Does anyone have an idea as to why this might be the case?
Is there a better way to do it?
The problem is that locals() is local to your convert function when you'd want it to refer to f locals.
You have to pass the locals() dictionary somehow, either in constructor or somewhere else for it to work.
I'm using a namedtuple to hold sets of strings and their corresponding values.
I'm not using a dictionary, because I want the strings accessible as attributes.
Here's my code:
from collections import namedtuple
# Shortened for readability :-)
strings = namedtuple("strings", ['a0', 'a1', 'a2', ..., 'a400'])
my_strings = strings(value0, value1, value2, ..., value400)
Ideally, once my_strings is initialized, I should be able to do this:
print(my_strings.a1)
and get value1 printed back.
However, I get the following error instead:
strings(value0, value1, value2, ...value400)
^SyntaxError: more than 255 arguments
It seems python functions (including namedtuple's init()), do not accept more than 255 arguments when called.
Is there any way to bypass this issue and have named tuples with more than 255 items? Why is there a 255 arguments limit anyway?
This is a limit to CPython function definitions; in versions before Python 3.7, you cannot specify more than 255 explicit arguments to a callable. This applies to any function definition, not just named tuples.
Note that this limit has been lifted in Python 3.7 and newer, where the new limit is sys.maxint. See What is a maximum number of arguments in a Python function?
It is the generated code for the class that is hitting this limit. You cannot define a function with more than 255 arguments; the __new__ class method of the resulting class is thus not achievable in the CPython implementation.
You'll have to ask yourself, however, if you really should be using a different structure instead. It looks like you have a list-like piece of data to me; 400 numbered names is a sure sign of your data bleeding into your names.
You can work around this by creating your own subclass, manually:
from operator import itemgetter
from collections import OrderedDict
class strings(tuple):
__slots__ = ()
_fields = tuple('a{}'.format(i) for i in range(400))
def __new__(cls, *args, **kwargs):
req = len(cls._fields)
if len(args) + len(kwargs) > req:
raise TypeError(
'__new__() takes {} positional arguments but {} were given'.format(
req, len(args) + len(kwargs)))
if kwargs.keys() > set(cls._fields):
raise TypeError(
'__new__() got an unexpected keyword argument {!r}'.format(
(kwargs.keys() - set(cls._fields)).pop()))
missing = req - len(args)
if kwargs.keys() & set(cls._fields[:-missing]):
raise TypeError(
'__new__() got multiple values for argument {!r}'.format(
(kwargs.keys() & set(cls._fields[:-missing])).pop()))
try:
for field in cls._fields[-missing:]:
args += (kwargs[field],)
missing -= 1
except KeyError:
pass
if len(args) < req:
raise TypeError('__new__() missing {} positional argument{}: {}'.format(
missing, 's' if missing > 1 else '',
' and '.join(filter(None, [', '.join(map(repr, cls._fields[-missing:-1])), repr(cls._fields[-1])]))))
return tuple.__new__(cls, args)
#classmethod
def _make(cls, iterable, new=tuple.__new__, len=len):
'Make a new strings object from a sequence or iterable'
result = new(cls, iterable)
if len(result) != len(cls._fields):
raise TypeError('Expected %d arguments, got %d' % (len(cls._fields), len(result)))
return result
def __repr__(self):
'Return a nicely formatted representation string'
format = '{}({})'.format(self.__class__.__name__, ', '.join('{}=%r'.format(n) for n in self._fields))
return format % self
def _asdict(self):
'Return a new OrderedDict which maps field names to their values'
return OrderedDict(zip(self._fields, self))
__dict__ = property(_asdict)
def _replace(self, **kwds):
'Return a new strings object replacing specified fields with new values'
result = self._make(map(kwds.pop, self._fields, self))
if kwds:
raise ValueError('Got unexpected field names: %r' % list(kwds))
return result
def __getnewargs__(self):
'Return self as a plain tuple. Used by copy and pickle.'
return tuple(self)
def __getstate__(self):
'Exclude the OrderedDict from pickling'
return None
for i, name in enumerate(strings._fields):
setattr(strings, name,
property(itemgetter(i), doc='Alias for field number {}'.format(i)))
This version of the named tuple avoids the long argument lists altogether, but otherwise behaves exactly like the original. The somewhat verbose __new__ method is not strictly needed but does closely emulate the original behaviour when arguments are incomplete. Note the construction of the _fields attribute; replace this with your own to name your tuple fields.
Pass in a generator expression to set your arguments:
s = strings(i for i in range(400))
or if you have a list of values:
s = strings(iter(list_of_values))
Either technique bypasses the limits on function signatures and function call argument counts.
Demo:
>>> s = strings(i for i in range(400))
>>> s
strings(a0=0, a1=1, a2=2, a3=3, a4=4, a5=5, a6=6, a7=7, a8=8, a9=9, a10=10, a11=11, a12=12, a13=13, a14=14, a15=15, a16=16, a17=17, a18=18, a19=19, a20=20, a21=21, a22=22, a23=23, a24=24, a25=25, a26=26, a27=27, a28=28, a29=29, a30=30, a31=31, a32=32, a33=33, a34=34, a35=35, a36=36, a37=37, a38=38, a39=39, a40=40, a41=41, a42=42, a43=43, a44=44, a45=45, a46=46, a47=47, a48=48, a49=49, a50=50, a51=51, a52=52, a53=53, a54=54, a55=55, a56=56, a57=57, a58=58, a59=59, a60=60, a61=61, a62=62, a63=63, a64=64, a65=65, a66=66, a67=67, a68=68, a69=69, a70=70, a71=71, a72=72, a73=73, a74=74, a75=75, a76=76, a77=77, a78=78, a79=79, a80=80, a81=81, a82=82, a83=83, a84=84, a85=85, a86=86, a87=87, a88=88, a89=89, a90=90, a91=91, a92=92, a93=93, a94=94, a95=95, a96=96, a97=97, a98=98, a99=99, a100=100, a101=101, a102=102, a103=103, a104=104, a105=105, a106=106, a107=107, a108=108, a109=109, a110=110, a111=111, a112=112, a113=113, a114=114, a115=115, a116=116, a117=117, a118=118, a119=119, a120=120, a121=121, a122=122, a123=123, a124=124, a125=125, a126=126, a127=127, a128=128, a129=129, a130=130, a131=131, a132=132, a133=133, a134=134, a135=135, a136=136, a137=137, a138=138, a139=139, a140=140, a141=141, a142=142, a143=143, a144=144, a145=145, a146=146, a147=147, a148=148, a149=149, a150=150, a151=151, a152=152, a153=153, a154=154, a155=155, a156=156, a157=157, a158=158, a159=159, a160=160, a161=161, a162=162, a163=163, a164=164, a165=165, a166=166, a167=167, a168=168, a169=169, a170=170, a171=171, a172=172, a173=173, a174=174, a175=175, a176=176, a177=177, a178=178, a179=179, a180=180, a181=181, a182=182, a183=183, a184=184, a185=185, a186=186, a187=187, a188=188, a189=189, a190=190, a191=191, a192=192, a193=193, a194=194, a195=195, a196=196, a197=197, a198=198, a199=199, a200=200, a201=201, a202=202, a203=203, a204=204, a205=205, a206=206, a207=207, a208=208, a209=209, a210=210, a211=211, a212=212, a213=213, a214=214, a215=215, a216=216, a217=217, a218=218, a219=219, a220=220, a221=221, a222=222, a223=223, a224=224, a225=225, a226=226, a227=227, a228=228, a229=229, a230=230, a231=231, a232=232, a233=233, a234=234, a235=235, a236=236, a237=237, a238=238, a239=239, a240=240, a241=241, a242=242, a243=243, a244=244, a245=245, a246=246, a247=247, a248=248, a249=249, a250=250, a251=251, a252=252, a253=253, a254=254, a255=255, a256=256, a257=257, a258=258, a259=259, a260=260, a261=261, a262=262, a263=263, a264=264, a265=265, a266=266, a267=267, a268=268, a269=269, a270=270, a271=271, a272=272, a273=273, a274=274, a275=275, a276=276, a277=277, a278=278, a279=279, a280=280, a281=281, a282=282, a283=283, a284=284, a285=285, a286=286, a287=287, a288=288, a289=289, a290=290, a291=291, a292=292, a293=293, a294=294, a295=295, a296=296, a297=297, a298=298, a299=299, a300=300, a301=301, a302=302, a303=303, a304=304, a305=305, a306=306, a307=307, a308=308, a309=309, a310=310, a311=311, a312=312, a313=313, a314=314, a315=315, a316=316, a317=317, a318=318, a319=319, a320=320, a321=321, a322=322, a323=323, a324=324, a325=325, a326=326, a327=327, a328=328, a329=329, a330=330, a331=331, a332=332, a333=333, a334=334, a335=335, a336=336, a337=337, a338=338, a339=339, a340=340, a341=341, a342=342, a343=343, a344=344, a345=345, a346=346, a347=347, a348=348, a349=349, a350=350, a351=351, a352=352, a353=353, a354=354, a355=355, a356=356, a357=357, a358=358, a359=359, a360=360, a361=361, a362=362, a363=363, a364=364, a365=365, a366=366, a367=367, a368=368, a369=369, a370=370, a371=371, a372=372, a373=373, a374=374, a375=375, a376=376, a377=377, a378=378, a379=379, a380=380, a381=381, a382=382, a383=383, a384=384, a385=385, a386=386, a387=387, a388=388, a389=389, a390=390, a391=391, a392=392, a393=393, a394=394, a395=395, a396=396, a397=397, a398=398, a399=399)
>>> s.a391
391
namedtuple out of the box doesn't support what you are trying to do.
So the following might achieve the goal, which might change from 400 to 450 arguments, or lesser and saner.
def customtuple(*keys):
class string:
_keys = keys
_dict = {}
def __init__(self, *args):
args = list(args)
if len(args) != len(self._keys):
raise Exception("No go forward")
for key in range(len(args)):
self._dict[self._keys[key]] = args[key]
def __setattr__(self, *args):
raise BaseException("Not allowed")
def __getattr__(self, arg):
try:
return self._dict[arg]
except:
raise BaseException("Name not defined")
def __repr__(self):
return ("string(%s)"
%(", ".join(["%s=%r"
%(self._keys[key],
self._dict[self._keys[key]])
for key in range(len(self._dict))])))
return string
>>> strings = customtuple(*['a'+str(x) for x in range(1, 401)])
>>> s = strings(*['a'+str(x) for x in range(2, 402)])
>>> s.a1
'a2'
>>> s.a1 = 1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/hus787/p.py", line 15, in __setattr__
def __setattr__(self, *args):
BaseException: Not allowed
For more light on the subject.
Here is my version of a replacement for namedtuple that supports more than 255 arguments. The idea was not to be functionally equivalent but rather to improve on some aspects (in my opinion). This is for Python 3.4+ only:
class SequenceAttrReader(object):
""" Class to function similar to collections.namedtuple but allowing more than 255 keys.
Initialize with attribute string (space separated), then load in data via a sequence, then access the list keys as properties
i.e.
csv_line = SequenceAttrReader('a b c')
csv_line = csv_line.load([1, 2, 3])
print(csv_line.b)
>> 2
"""
_attr_string = None
_attr_list = []
_data_list = []
def __init__(self, attr_string):
if not attr_string:
raise AttributeError('SequenceAttrReader not properly initialized, please use a non-empty string')
self._attr_string = attr_string
self._attr_list = attr_string.split(' ')
def __getattr__(self, name):
if not self._attr_string or not self._attr_list or not self._data_list:
raise AttributeError('SequenceAttrReader not properly initialized or loaded')
try:
index = self._attr_list.index(name)
except ValueError:
raise AttributeError("'{name}'' not in attribute string".format(name=name)) from None
try:
value = self._data_list[index]
except IndexError:
raise AttributeError("No attribute named '{name}'' in".format(name=name)) from None
return value
def __str__(self):
return str(self._data_list)
def __repr__(self):
return 'SequenceAttrReader("{attr_string}")'.format(attr_string=self._attr_string)
def load_data(self, data_list):
if not self._attr_list:
raise AttributeError('SequenceAttrReader not properly initialized')
if not data_list:
raise AttributeError('SequenceAttrReader needs to load a non-empty sequence')
self._data_list = data_list
This is probably not the most efficient way if you are doing a lot of individual lookups, converting it internally to a dict may be better. I'll work on an optimized version once I have more time or at least see what the performance difference is.
Suppose I have a function like f(a, b, c=None). The aim is to call the function like f(*args, **kwargs), and then construct a new set of args and kwargs such that:
If the function had default values, I should be able to acquire their values. For example, if I call it like f(1, 2), I should be able to get the tuple (1, 2, None) and/or the dictionary {'c': None}.
If the value of any of the arguments was modified inside the function, get the new value. For example, if I call it like f(1, 100000, 3) and the function does if b > 500: b = 5 modifying the local variable, I should be able to get the the tuple (1, 5, 3).
The aim here is to create a a decorator that finishes the job of a function. The original function acts as a preamble setting up the data for the actual execution, and the decorator finishes the job.
Edit: I'm adding an example of what I'm trying to do. It's a module for making proxies for other classes.
class Spam(object):
"""A fictional class that we'll make a proxy for"""
def eggs(self, start, stop, step):
"""A fictional method"""
return range(start, stop, step)
class ProxyForSpam(clsproxy.Proxy):
proxy_for = Spam
#clsproxy.signature_preamble
def eggs(self, start, stop, step=1):
start = max(0, start)
stop = min(100, stop)
And then, we'll have that:
ProxyForSpam().eggs(-10, 200) -> Spam().eggs(0, 100, 1)
ProxyForSpam().eggs(3, 4) -> Spam().eggs(3, 4, 1)
There are two recipes available here, one which requires an external library and another that uses only the standard library. They don't quite do what you want, in that they actually modify the function being executed to obtain its locals() rather than obtain the locals() after function execution, which is impossible, since the local stack no longer exists after the function finishes execution.
Another option is to see what debuggers, such as WinPDB or even the pdb module do. I suspect they use the inspect module (possibly along with others), to get the frame inside which a function is executing and retrieve locals() that way.
EDIT: After reading some code in the standard library, the file you want to look at is probably bdb.py, which should be wherever the rest of your Python standard library is. Specifically, look at set_trace() and related functions. This will give you an idea of how the Python debugger breaks into the class. You might even be able to use it directly. To get the frame to pass to set_trace() look at the inspect module.
I've stumbled upon this very need today and wanted to share my solution.
import sys
def call_function_get_frame(func, *args, **kwargs):
"""
Calls the function *func* with the specified arguments and keyword
arguments and snatches its local frame before it actually executes.
"""
frame = None
trace = sys.gettrace()
def snatch_locals(_frame, name, arg):
nonlocal frame
if frame is None and name == 'call':
frame = _frame
sys.settrace(trace)
return trace
sys.settrace(snatch_locals)
try:
result = func(*args, **kwargs)
finally:
sys.settrace(trace)
return frame, result
The idea is to use sys.trace() to catch the frame of the next 'call'. Tested on CPython 3.6.
Example usage
import types
def namespace_decorator(func):
frame, result = call_function_get_frame(func)
try:
module = types.ModuleType(func.__name__)
module.__dict__.update(frame.f_locals)
return module
finally:
del frame
#namespace_decorator
def mynamespace():
eggs = 'spam'
class Bar:
def hello(self):
print("Hello, World!")
assert mynamespace.eggs == 'spam'
mynamespace.Bar().hello()
I don't see how you could do this non-intrusively -- after the function is done executing, it doesn't exist any more -- there's no way you can reach inside something that doesn't exist.
If you can control the functions that are being used, you can do an intrusive approach like
def fn(x, y, z, vars):
'''
vars is an empty dict that we use to pass things back to the caller
'''
x += 1
y -= 1
z *= 2
vars.update(locals())
>>> updated = {}
>>> fn(1, 2, 3, updated)
>>> print updated
{'y': 1, 'x': 2, 'z': 6, 'vars': {...}}
>>>
...or you can just require that those functions return locals() -- as #Thomas K asks above, what are you really trying to do here?
Witchcraft below read on your OWN danger(!)
I have no clue what you want to do with this, it's possible but it's an awful hack...
Anyways, I HAVE WARNED YOU(!), be lucky if such things don't work in your favorite language...
from inspect import getargspec, ismethod
import inspect
def main():
#get_modified_values
def foo(a, f, b):
print a, f, b
a = 10
if a == 2:
return a
f = 'Hello World'
b = 1223
e = 1
c = 2
foo(e, 1000, b = c)
# intercept a function and retrieve the modifed values
def get_modified_values(target):
def wrapper(*args, **kwargs):
# get the applied args
kargs = getcallargs(target, *args, **kwargs)
# get the source code
src = inspect.getsource(target)
lines = src.split('\n')
# oh noes string patching of the function
unindent = len(lines[0]) - len(lines[0].lstrip())
indent = lines[0][:len(lines[0]) - len(lines[0].lstrip())]
lines[0] = ''
lines[1] = indent + 'def _temp(_args, ' + lines[1].split('(')[1]
setter = []
for k in kargs.keys():
setter.append('_args["%s"] = %s' % (k, k))
i = 0
while i < len(lines):
indent = lines[i][:len(lines[i]) - len(lines[i].lstrip())]
if lines[i].find('return ') != -1 or lines[i].find('return\n') != -1:
for e in setter:
lines.insert(i, indent + e)
i += len(setter)
elif i == len(lines) - 2:
for e in setter:
lines.insert(i + 1, indent + e)
break
i += 1
for i in range(0, len(lines)):
lines[i] = lines[i][unindent:]
data = '\n'.join(lines) + "\n"
# setup variables
frame = inspect.currentframe()
loc = inspect.getouterframes(frame)[1][0].f_locals
glob = inspect.getouterframes(frame)[1][0].f_globals
loc['_temp'] = None
# compile patched function and call it
func = compile(data, '<witchstuff>', 'exec')
eval(func, glob, loc)
loc['_temp'](kargs, *args, **kwargs)
# there you go....
print kargs
# >> {'a': 10, 'b': 1223, 'f': 'Hello World'}
return wrapper
# from python 2.7 inspect module
def getcallargs(func, *positional, **named):
"""Get the mapping of arguments to values.
A dict is returned, with keys the function argument names (including the
names of the * and ** arguments, if any), and values the respective bound
values from 'positional' and 'named'."""
args, varargs, varkw, defaults = getargspec(func)
f_name = func.__name__
arg2value = {}
# The following closures are basically because of tuple parameter unpacking.
assigned_tuple_params = []
def assign(arg, value):
if isinstance(arg, str):
arg2value[arg] = value
else:
assigned_tuple_params.append(arg)
value = iter(value)
for i, subarg in enumerate(arg):
try:
subvalue = next(value)
except StopIteration:
raise ValueError('need more than %d %s to unpack' %
(i, 'values' if i > 1 else 'value'))
assign(subarg,subvalue)
try:
next(value)
except StopIteration:
pass
else:
raise ValueError('too many values to unpack')
def is_assigned(arg):
if isinstance(arg,str):
return arg in arg2value
return arg in assigned_tuple_params
if ismethod(func) and func.im_self is not None:
# implicit 'self' (or 'cls' for classmethods) argument
positional = (func.im_self,) + positional
num_pos = len(positional)
num_total = num_pos + len(named)
num_args = len(args)
num_defaults = len(defaults) if defaults else 0
for arg, value in zip(args, positional):
assign(arg, value)
if varargs:
if num_pos > num_args:
assign(varargs, positional[-(num_pos-num_args):])
else:
assign(varargs, ())
elif 0 < num_args < num_pos:
raise TypeError('%s() takes %s %d %s (%d given)' % (
f_name, 'at most' if defaults else 'exactly', num_args,
'arguments' if num_args > 1 else 'argument', num_total))
elif num_args == 0 and num_total:
raise TypeError('%s() takes no arguments (%d given)' %
(f_name, num_total))
for arg in args:
if isinstance(arg, str) and arg in named:
if is_assigned(arg):
raise TypeError("%s() got multiple values for keyword "
"argument '%s'" % (f_name, arg))
else:
assign(arg, named.pop(arg))
if defaults: # fill in any missing values with the defaults
for arg, value in zip(args[-num_defaults:], defaults):
if not is_assigned(arg):
assign(arg, value)
if varkw:
assign(varkw, named)
elif named:
unexpected = next(iter(named))
if isinstance(unexpected, unicode):
unexpected = unexpected.encode(sys.getdefaultencoding(), 'replace')
raise TypeError("%s() got an unexpected keyword argument '%s'" %
(f_name, unexpected))
unassigned = num_args - len([arg for arg in args if is_assigned(arg)])
if unassigned:
num_required = num_args - num_defaults
raise TypeError('%s() takes %s %d %s (%d given)' % (
f_name, 'at least' if defaults else 'exactly', num_required,
'arguments' if num_required > 1 else 'argument', num_total))
return arg2value
main()
Output:
1 1000 2
{'a': 10, 'b': 1223, 'f': 'Hello World'}
There you go... I'm not responsible for any small children that get eaten by demons or something the like (or if it breaks on complicated functions).
PS: The inspect module is the pure EVIL.
Since you are trying to manipulate variables in one function, and do some job based on those variables on another function, the cleanest way to do it is having these variables to be an object's attributes.
It could be a dictionary - that could be defined inside the decorator - therefore access to it inside the decorated function would be as a "nonlocal" variable. That cleans up the default parameter tuple of this dictionary, that #bgporter proposed.:
def eggs(self, a, b, c=None):
# nonlocal parms ## uncomment in Python 3
parms["a"] = a
...
To be even more clean, you probably should have all these parameters as attributes of the instance (self) - so that no "magical" variable has to be used inside the decorated function.
As for doing it "magically" without having the parameters set as attributes of certain object explicitly, nor having the decorated function to return the parameters themselves (which is also an option) - that is, to have it to work transparently with any decorated function - I can't think of a way that does not involve manipulating the bytecode of the function itself.
If you can think of a way to make the wrapped function raise an exception at return time, you could trap the exception and check the execution trace.
If it is so important to do it automatically that you consider altering the function bytecode an option, feel free to ask me further.