Alternate Values for MongoDB fields setup via Mongoengine - python
I have a somewhat different need/requirement in how I use mongoengine and wanted to share the below to get any ideas on how to deal with it.
My requirement is that for some fields in a document, I want to be able to keep two values. One is a system-default value and the other a user-specified value. When accessing this field, it should return back a value in the following manner:
if a user-specified value exists, return that OR
if a user-specified value doesn’t exist, return the default
I can not use the default attribute that mongoengine already provides because:
It is applied to the mongoengine class model rather than the document
The default value would need to be updated based on system changes in newer versions of the software
There would not be a memory of what the default is at the document level so it would be difficult to determine if the default needs to be changed for certain documents only
I thought of creating something called an AlternateValueField, which is based off a MapField. Below is the code for this new field type:
import mongoengine
class AlternateValueField(mongoengine.MapField):
def __init__(self, field=None, *args, **kwargs):
self.allowed_keys = (
'_active', # Active Value - changeable by user
'_default', # Default Value - provided by system
)
# Remove the field's default since it now gets handled
# as _default
self.kwargs_default = kwargs.pop('default', None)
super().__init__(field=field, *args, **kwargs)
def __get__(self, instance, owner):
result = super().__get__(instance, owner)
if isinstance(result, dict):
if '_active' in result.keys():
return result['_active']
if '_default' in result.keys():
return result['_default']
return result
def __set__(self, instance, value):
instance_dict = instance._data.get(self.name)
if isinstance(value, dict):
if isinstance(instance_dict, dict):
# To keep the order (_active followed by _default)
new_value = {}
for key in self.allowed_keys:
if (key in instance_dict.keys()
and key not in value.keys()):
new_value[key] = instance_dict[key]
elif key in value.keys():
new_value[key] = value[key]
value = new_value
elif value is not None:
new_value = {
'_active': value,
}
if isinstance(instance_dict, dict):
if '_default' in instance_dict.keys():
new_value['_default'] = instance_dict['_default']
value = new_value
else:
if self.required and self.default is not None:
new_value = {
'_default': self.kwargs_default,
}
value = new_value
self._check_for_allowed_keys(value)
super().__set__(instance, value)
def _check_for_allowed_keys(self, value):
if value is None:
return
for key in value.keys():
if key in self.allowed_keys:
continue
err_msg = (
f"Key '{key}' is not allowed."
f" Allowed keys: {self.allowed_keys}"
)
raise KeyError(err_msg)
def validate(self, value, **kwargs):
super().validate(value, **kwargs)
self._check_for_allowed_keys(value)
A document example that uses it:
class MySchedule(mongoengine.Document):
my_name = AlternateValueField(mongoengine.StringField())
sched1 = mongoengine.StringField()
Using this model to create a document:
>>> s2 = MySchedule(my_name={'_active': 'one', '_default': 'two'}, sched1='yes')
>>> s2.validate()
>>> s2.my_name # returns just the value (of active or default) instead of dict
'one'
>>> s2.to_json()
'{"my_name": {"_active": "one", "_default": "two"}, "sched1": "yes"}'
>>> s2.to_mongo()
SON([('my_name', {'_active': 'one', '_default': 'two'}), ('sched1', 'yes')])
>>> s2._fields
{'my_name': <__main__.AlternateValueField object at 0x7fd37fbaf1c0>, 'sched1': <mongoengine.fields.StringField object at 0x7fd37f968250>, 'id': <mongoengine.base.fields.ObjectIdField object at 0x7fd378225640>}
>>> s2._data
{'id': None, 'my_name': {'_active': 'one', '_default': 'two'}, 'sched1': 'yes'}
>>> s2.my_name = 'anotherone' # works due to the __set__() implementation above
>>> s2.my_name
'anotherone'
>>> s2._data
{'id': None, 'my_name': {'_active': 'anotherone', '_default': 'two'}, 'sched1': 'yes'}
>>> s2.my_name['_active'] = 'blah' # won't work now
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'str' object does not support item assignment
>>> s2._data['my_name']['_active'] = 'blah' # will work
>>> s2.my_name
'blah'
>>> s2._data
{'id': None, 'my_name': {'_active': 'blah', '_default': 'two'}, 'sched1': 'yes'}
My issue is, I would like to be able to use these cases:
s2.my_name # works - returns the _active value or _default value, not dict of them
s2.my_name = 'something' # works - via the __set__() implementation
s2.my_name['_active'] = 'blah' # causes error as shown above
The 3rd case above is what I'd like to be able to do without causing an error.
I tried the following in an attempt to get around it:
overriding __get__(), which doesn't work because it's job is done prior to recognizing there is a getitem call
thought about overriding BaseDict.__getitem__() to recognize this, but that won't work for when there is no getitem call
I would need something after the AlternateField's __get__() call and before BaseDict's __getitem__() call to recognize whether a getitem is following it or not.
Any thoughts on best way to get around this? Obviously, the above may just all be a bad idea. Any alternatives (pardon the pun :)) on creating fields with alternative values that can be changed on a per-document basis and remembered across restarts of the software? Fields can have all the types mongoengine supports, but mostly will be string, int, and potentially, list.
Related
How checking lookup depth into nested dictionary as class attribute?
I created a nested dictionary based on AttrDict found there : Object-like attribute access for nested dictionary I modified it to contain str commands in "leaves" that gets executed when the value is requested/written to : commands = {'root': {'com': {'read': 'READ_CMD', 'write': 'WRITE_CMD'} } } class AttrTest() def __init__: self.__dict__['attr'] = AttrDict(commands) test = AttrTest() data = test.attr.root.com.read # data = value read with the command test.attr.root.com.write = data # data = value written on the com port While it works beautifully, I'd like to : Avoid people getting access to attr/root/com as these returns a sub-level dictonary People accessing attr.root.com directly (through __getattribute__/__setattr__) Currently, I'm facing the following problems : As said, when accessing the 'trunk' of the nested dict, I get a partial dict of the 'leaves' When accessing attr.root.com it returns {'read': 'READ_CMD', 'write': 'WRITE_CMD'} If detecting a read I do a forward lookup and return the value, but then attr.root.com.read fails Is it possible to know what is the final level Python will request in the "path" ? To block access to attr/root To read/write the value accessing attr.root.com directly (using forward lookup) To return the needed partial dict only if attr.root.com.read or attr.root.com.write are requested Currently I've found nothing that allows me to control how deep the lookup is expected to go. Thanks for your consideration.
For a given attribute lookup you cannot determine how many others will follow; this is how Python works. In order to resolve x.y.z, first the object x.y needs to be retrieved before the subsequent attribute lookup (x.y).z can be performed. What you can do however, is return a proxy object that represents the (partial) path instead of the actual underlying object which is stored in the dict. So for example if you did test.attr.com then this would return a proxy object which represents the path attr.com to-be-looked up on the test object. Only when you encounter a read or write leaf in the path, you would resolve the path and read/write the data. The following is a sample implementation which uses an AttrDict based on __getattr__ to provide the Proxy objects (so you don't have to intercept __getattribute__): from functools import reduce class AttrDict(dict): def __getattr__(self, name): return Proxy(self, (name,)) def _resolve(self, path): return reduce(lambda d, k: d[k], path, self) class Proxy: def __init__(self, obj, path): object.__setattr__(self, '_obj', obj) object.__setattr__(self, '_path', path) def __str__(self): return f"Path<{'.'.join(self._path)}>" def __getattr__(self, name): if name == 'read': return self._obj._resolve(self._path)[name] else: return type(self)(self._obj, (*self._path, name)) def __setattr__(self, name, value): if name != 'write' or name not in (_dict := self._obj._resolve(self._path)): raise AttributeError(f'Cannot set attribute {name!r} for {self}') _dict[name] = value commands = {'root': {'com': {'read': 'READ_CMD', 'write': 'WRITE_CMD'} } } test = AttrDict({'attr': commands}) print(f'{test.attr = !s}') # Path<attr> print(f'{test.attr.root = !s}') # Path<attr.root> print(f'{test.attr.root.com = !s}') # Path<attr.root.com> print(f'{test.attr.root.com.read = !s}') # READ_CMD test.attr.root.com.write = 'test' test.attr.root.write = 'illegal' # raises AttributeError
Nested dictionary that acts as defaultdict when setting items but not when getting items
I want to implement a dict-like data structure that has the following properties: from collections import UserDict class TestDict(UserDict): pass test_dict = TestDict() # Create empty dictionaries at 'level_1' and 'level_2' and insert 'Hello' at the 'level_3' key. test_dict['level_1']['level_2']['level_3'] = 'Hello' >>> test_dict { 'level_1': { 'level_2': { 'level_3': 'Hello' } } } # However, this should not return an empty dictionary but raise a KeyError. >>> test_dict['unknown_key'] KeyError: 'unknown_key' The problem, to my knowledge, is that python does not know whether __getitem__ is being called in the context of setting an item, i.e. the first example, or in the context of getting and item, the second example. I have already seen Python `defaultdict`: Use default when setting, but not when getting, but I do not think that this question is a duplicate, or that it answers my question. Please let me know if you have any ideas. Thanks in advance. EDIT: It is possible to achieve something similar using: def set_nested_item(dict_in: Union[dict, TestDict], value, keys): for i, key in enumerate(keys): is_last = i == (len(keys) - 1) if is_last: dict_in[key] = value else: if key not in dict_in: dict_in[key] = {} else: if not isinstance(dict_in[key], (dict, TestDict)): dict_in[key] = {} dict_in[key] = set_nested_item(dict_in[key], value, keys[(i + 1):]) return dict_in class TestDict(UserDict): def __init__(self): super().__init__() def __setitem__(self, key, value): if isinstance(key, list): self.update(set_nested_item(self, value, key)) else: super().__setitem__(key, value) test_dict[['level_1', 'level_2', 'level_3']] = 'Hello' >>> test_dict { 'level_1': { 'level_2': { 'level_3': 'Hello' } } }
It's impossible. test_dict['level_1']['level_2']['level_3'] = 'Hello' is semantically equivalent to: temp1 = test_dict['level_1'] # Should this line fail? temp1['level_2']['level_3'] = 'Hello' But... if determined to implement it anyway, you could inspect the Python stack to grab/parse the calling line of code, and then vary the behaviour depending on whether the calling line of code contains an assignment! Unfortunately, sometimes the calling code isn't available in the stack trace (e.g. when called interactively), in which case you need to work with Python bytecode. import dis import inspect from collections import UserDict def get_opcodes(code_object, lineno): """Utility function to extract Python VM opcodes for line of code""" line_ops = [] instructions = dis.get_instructions(code_object).__iter__() for instruction in instructions: if instruction.starts_line == lineno: # found start of our line line_ops.append(instruction.opcode) break for instruction in instructions: if not instruction.starts_line: line_ops.append(instruction.opcode) else: # start of next line break return line_ops class TestDict(UserDict): def __getitem__(self, key): try: return super().__getitem__(key) except KeyError: # inspect the stack to get calling line of code frame = inspect.stack()[1].frame opcodes = get_opcodes(frame.f_code, frame.f_lineno) # STORE_SUBSCR is Python opcode for TOS1[TOS] = TOS2 if dis.opmap['STORE_SUBSCR'] in opcodes: # calling line of code contains a dict/array assignment default = TestDict() super().__setitem__(key, default) return default else: raise test_dict = TestDict() test_dict['level_1']['level_2']['level_3'] = 'Hello' print(test_dict) # {'level_1': {'level_2': {'level_3': 'Hello'}}} test_dict['unknown_key'] # KeyError: 'unknown_key' The above is just a partial solution. It can still be fooled if there are other dictionary/array assignments on the same line, e.g. other['key'] = test_dict['unknown_key']. A more complete solution would need to actually parse the line of code to figure out where the variable occurs in the assignment.
Dynamically subclass an Enum base class
I've set up a metaclass and base class pair for creating the line specifications of several different file types I have to parse. I have decided to go with using enumerations because many of the individual parts of the different lines in the same file often have the same name. Enums make it easy to tell them apart. Additionally, the specification is rigid and there will be no need to add more members, or extend the line specifications later. The specification classes work as expected. However, I am having some trouble dynamically creating them: >>> C1 = LineMakerMeta('C1', (LineMakerBase,), dict(a = 0)) AttributeError: 'dict' object has no attribute '_member_names' Is there a way around this? The example below works just fine: class A1(LineMakerBase): Mode = 0, dict(fill=' ', align='>', type='s') Level = 8, dict(fill=' ', align='>', type='d') Method = 10, dict(fill=' ', align='>', type='d') _dummy = 20 # so that Method has a known length A1.format(**dict(Mode='DESIGN', Level=3, Method=1)) # produces ' DESIGN 3 1' The metaclass is based on enum.EnumMeta, and looks like this: import enum class LineMakerMeta(enum.EnumMeta): "Metaclass to produce formattable LineMaker child classes." def _iter_format(cls): "Iteratively generate formatters for the class members." for member in cls: yield member.formatter def __str__(cls): "Returns string line with all default values." return cls.format() def format(cls, **kwargs): "Create formatted version of the line populated by the kwargs members." # build resulting string by iterating through members result = '' for member in cls: # determine value to be injected into member try: try: value = kwargs[member] except KeyError: value = kwargs[member.name] except KeyError: value = member.default value_str = member.populate(value) result = result + value_str return result And the base class is as follows: class LineMakerBase(enum.Enum, metaclass=LineMakerMeta): """A base class for creating Enum subclasses used for populating lines of a file. Usage: class LineMaker(LineMakerBase): a = 0, dict(align='>', fill=' ', type='f'), 3.14 b = 10, dict(align='>', fill=' ', type='d'), 1 b = 15, dict(align='>', fill=' ', type='s'), 'foo' # ^-start ^---spec dictionary ^--default """ def __init__(member, start, spec={}, default=None): member.start = start member.spec = spec if default is not None: member.default = default else: # assume value is numerical for all provided types other than 's' (string) default_or_set_type = member.spec.get('type','s') default = {'s': ''}.get(default_or_set_type, 0) member.default = default #property def formatter(member): """Produces a formatter in form of '{0:<format>}' based on the member.spec dictionary. The member.spec dictionary makes use of these keys ONLY (see the string.format docs): fill align sign width grouping_option precision type""" try: # get cached value return '{{0:{}}}'.format(member._formatter) except AttributeError: # add width to format spec if not there member.spec.setdefault('width', member.length if member.length != 0 else '') # build formatter using the available parts in the member.spec dictionary # any missing parts will simply not be present in the formatter formatter = '' for part in 'fill align sign width grouping_option precision type'.split(): try: spec_value = member.spec[part] except KeyError: # missing part continue else: # add part sub_formatter = '{!s}'.format(spec_value) formatter = formatter + sub_formatter member._formatter = formatter return '{{0:{}}}'.format(formatter) def populate(member, value=None): "Injects the value into the member's formatter and returns the formatted string." formatter = member.formatter if value is not None: value_str = formatter.format(value) else: value_str = formatter.format(member.default) if len(value_str) > len(member) and len(member) != 0: raise ValueError( 'Length of object string {} ({}) exceeds available' ' field length for {} ({}).' .format(value_str, len(value_str), member.name, len(member))) return value_str #property def length(member): return len(member) def __len__(member): """Returns the length of the member field. The last member has no length. Length are based on simple subtraction of starting positions.""" # get cached value try: return member._length # calculate member length except AttributeError: # compare by member values because member could be an alias members = list(type(member)) try: next_index = next( i+1 for i,m in enumerate(type(member)) if m.value == member.value ) except StopIteration: raise TypeError( 'The member value {} was not located in the {}.' .format(member.value, type(member).__name__) ) try: next_member = members[next_index] except IndexError: # last member defaults to no length length = 0 else: length = next_member.start - member.start member._length = length return length
This line: C1 = enum.EnumMeta('C1', (), dict(a = 0)) fails with exactly the same error message. The __new__ method of EnumMeta expects an instance of enum._EnumDict as its last argument. _EnumDict is a subclass of dict and provides an instance variable named _member_names, which of course a regular dict doesn't have. When you go through the standard mechanism of enum creation, this all happens correctly behind the scenes. That's why your other example works just fine. This line: C1 = enum.EnumMeta('C1', (), enum._EnumDict()) runs with no error. Unfortunately, the constructor of _EnumDict is defined as taking no arguments, so you can't initialize it with keywords as you apparently want to do. In the implementation of enum that's backported to Python3.3, the following block of code appears in the constructor of EnumMeta. You could do something similar in your LineMakerMeta class: def __new__(metacls, cls, bases, classdict): if type(classdict) is dict: original_dict = classdict classdict = _EnumDict() for k, v in original_dict.items(): classdict[k] = v In the official implementation, in Python3.5, the if statement and the subsequent block of code is gone for some reason. Therefore classdict must be an honest-to-god _EnumDict, and I don't see why this was done. In any case the implementation of Enum is extremely complicated and handles a lot of corner cases. I realize this is not a cut-and-dried answer to your question but I hope it will point you to a solution.
Create your LineMakerBase class, and then use it like so: C1 = LineMakerBase('C1', dict(a=0)) The metaclass was not meant to be used the way you are trying to use it. Check out this answer for advice on when metaclass subclasses are needed. Some suggestions for your code: the double try/except in format seems clearer as: for member in cls: if member in kwargs: value = kwargs[member] elif member.name in kwargs: value = kwargs[member.name] else: value = member.default this code: # compare by member values because member could be an alias members = list(type(member)) would be clearer with list(member.__class__) has a false comment: listing an Enum class will never include the aliases (unless you have overridden that part of EnumMeta) instead of the complicated __len__ code you have now, and as long as you are subclassing EnumMeta you should extend __new__ to automatically calculate the lengths once: # untested def __new__(metacls, cls, bases, clsdict): # let the main EnumMeta code do the heavy lifting enum_cls = super(LineMakerMeta, metacls).__new__(cls, bases, clsdict) # go through the members and calculate the lengths canonical_members = [ member for name, member in enum_cls.__members__.items() if name == member.name ] last_member = None for next_member in canonical_members: next_member.length = 0 if last_member is not None: last_member.length = next_member.start - last_member.start
The simplest way to create Enum subclasses on the fly is using Enum itself: >>> from enum import Enum >>> MyEnum = Enum('MyEnum', {'a': 0}) >>> MyEnum <enum 'MyEnum'> >>> MyEnum.a <MyEnum.a: 0> >>> type(MyEnum) <class 'enum.EnumMeta'> As for your custom methods, it might be simpler if you used regular functions, precisely because Enum implementation is so special.
How do I use dictionary key,value pair to set class instance attributes "pythonic"ly?
I have created some Python classes to use as multivariate data structures, which are then used for various tasks. In some instances, I like to populate the classes with various value sets. The default parameter filename "ho2.defaults" would look something like this: name = 'ho2' mass_option = 'h1o16' permutation = 'odd' parity = 'odd' j_total = 10 lr = 40 br = 60 jmax = 60 mass_lr = 14578.471659 mass_br = 1781.041591 length_lr = ( 1.0, 11.0, 2.65 ) length_br = ( 0.0, 11.0, 2.46 ) use_spline = True energy_units = 'au' pes_zpe = -7.407998138300982E-2 pes_cutoff = 0.293994 Currently, I create a dictionary from reading the desired key,value pairs from file, and now I'd like a "pythonic" way of making those dictionary keys be class instance variable names, i.e. # Instantiate Molecule Class molecule = Molecule() # Create Dictionary of default values default_dict = read_dict_from_file(filename) # Set populate class instance variables with dictionary values for key,value in default_dict: molecule.key = value So the Class's instance variable "molecule.name" could be set with the dictionary key,value pair. I could do this by hand, but I'ms sure there is a better way to loop through it. In actuality, the dictionary could be large, and I'd rather allow the user to choose which values they want to populate, so the dictionary could change. What am I missing here?
You would use setattr: setattr(molecule, key, value)
The simple way is: vars(molecule).update(default_dict) This will clobber any pre-existing attributes though. For a more delicate approach try: for name, value in default_dict.items(): if not hasattr(molecule, name): setattr(molecule, name value)
I'd invert the logic so that the object dynamically answers questions: class Settings(object): ATTRS = {'foo', 'bar'} def __init__(self, defaults): self.__dict__['data'] = defaults.copy() def __getattr__(self, key): if key not in self.ATTRS or key not in self.data: raise AttributeError("'{}' object has no attribute '{}'".format( self.__class__.__name__, key)) return self.data[key] def __setattr__(self, key, value): self.data[key] = value s = Settings({'a': 'b', 'foo': 'foo!', 'spam': 'eggs'}) print s.foo try: print s.spam except AttributeError: pass else: raise AssertionError("That should have failed because 'spam' isn't in Settings.ATTRS") try: print s.bar except AttributeError: pass else: raise AssertionError("That should have failed because 'bar' wasn't passed in") class Molecule(settings): ATTRS = {'name', 'mass_option', ...} molecule = Molecule(default_dict)
Any way to bypass namedtuple 255 arguments limitation?
I'm using a namedtuple to hold sets of strings and their corresponding values. I'm not using a dictionary, because I want the strings accessible as attributes. Here's my code: from collections import namedtuple # Shortened for readability :-) strings = namedtuple("strings", ['a0', 'a1', 'a2', ..., 'a400']) my_strings = strings(value0, value1, value2, ..., value400) Ideally, once my_strings is initialized, I should be able to do this: print(my_strings.a1) and get value1 printed back. However, I get the following error instead: strings(value0, value1, value2, ...value400) ^SyntaxError: more than 255 arguments It seems python functions (including namedtuple's init()), do not accept more than 255 arguments when called. Is there any way to bypass this issue and have named tuples with more than 255 items? Why is there a 255 arguments limit anyway?
This is a limit to CPython function definitions; in versions before Python 3.7, you cannot specify more than 255 explicit arguments to a callable. This applies to any function definition, not just named tuples. Note that this limit has been lifted in Python 3.7 and newer, where the new limit is sys.maxint. See What is a maximum number of arguments in a Python function? It is the generated code for the class that is hitting this limit. You cannot define a function with more than 255 arguments; the __new__ class method of the resulting class is thus not achievable in the CPython implementation. You'll have to ask yourself, however, if you really should be using a different structure instead. It looks like you have a list-like piece of data to me; 400 numbered names is a sure sign of your data bleeding into your names. You can work around this by creating your own subclass, manually: from operator import itemgetter from collections import OrderedDict class strings(tuple): __slots__ = () _fields = tuple('a{}'.format(i) for i in range(400)) def __new__(cls, *args, **kwargs): req = len(cls._fields) if len(args) + len(kwargs) > req: raise TypeError( '__new__() takes {} positional arguments but {} were given'.format( req, len(args) + len(kwargs))) if kwargs.keys() > set(cls._fields): raise TypeError( '__new__() got an unexpected keyword argument {!r}'.format( (kwargs.keys() - set(cls._fields)).pop())) missing = req - len(args) if kwargs.keys() & set(cls._fields[:-missing]): raise TypeError( '__new__() got multiple values for argument {!r}'.format( (kwargs.keys() & set(cls._fields[:-missing])).pop())) try: for field in cls._fields[-missing:]: args += (kwargs[field],) missing -= 1 except KeyError: pass if len(args) < req: raise TypeError('__new__() missing {} positional argument{}: {}'.format( missing, 's' if missing > 1 else '', ' and '.join(filter(None, [', '.join(map(repr, cls._fields[-missing:-1])), repr(cls._fields[-1])])))) return tuple.__new__(cls, args) #classmethod def _make(cls, iterable, new=tuple.__new__, len=len): 'Make a new strings object from a sequence or iterable' result = new(cls, iterable) if len(result) != len(cls._fields): raise TypeError('Expected %d arguments, got %d' % (len(cls._fields), len(result))) return result def __repr__(self): 'Return a nicely formatted representation string' format = '{}({})'.format(self.__class__.__name__, ', '.join('{}=%r'.format(n) for n in self._fields)) return format % self def _asdict(self): 'Return a new OrderedDict which maps field names to their values' return OrderedDict(zip(self._fields, self)) __dict__ = property(_asdict) def _replace(self, **kwds): 'Return a new strings object replacing specified fields with new values' result = self._make(map(kwds.pop, self._fields, self)) if kwds: raise ValueError('Got unexpected field names: %r' % list(kwds)) return result def __getnewargs__(self): 'Return self as a plain tuple. Used by copy and pickle.' return tuple(self) def __getstate__(self): 'Exclude the OrderedDict from pickling' return None for i, name in enumerate(strings._fields): setattr(strings, name, property(itemgetter(i), doc='Alias for field number {}'.format(i))) This version of the named tuple avoids the long argument lists altogether, but otherwise behaves exactly like the original. The somewhat verbose __new__ method is not strictly needed but does closely emulate the original behaviour when arguments are incomplete. Note the construction of the _fields attribute; replace this with your own to name your tuple fields. Pass in a generator expression to set your arguments: s = strings(i for i in range(400)) or if you have a list of values: s = strings(iter(list_of_values)) Either technique bypasses the limits on function signatures and function call argument counts. Demo: >>> s = strings(i for i in range(400)) >>> s strings(a0=0, a1=1, a2=2, a3=3, a4=4, a5=5, a6=6, a7=7, a8=8, a9=9, a10=10, a11=11, a12=12, a13=13, a14=14, a15=15, a16=16, a17=17, a18=18, a19=19, a20=20, a21=21, a22=22, a23=23, a24=24, a25=25, a26=26, a27=27, a28=28, a29=29, a30=30, a31=31, a32=32, a33=33, a34=34, a35=35, a36=36, a37=37, a38=38, a39=39, a40=40, a41=41, a42=42, a43=43, a44=44, a45=45, a46=46, a47=47, a48=48, a49=49, a50=50, a51=51, a52=52, a53=53, a54=54, a55=55, a56=56, a57=57, a58=58, a59=59, a60=60, a61=61, a62=62, a63=63, a64=64, a65=65, a66=66, a67=67, a68=68, a69=69, a70=70, a71=71, a72=72, a73=73, a74=74, a75=75, a76=76, a77=77, a78=78, a79=79, a80=80, a81=81, a82=82, a83=83, a84=84, a85=85, a86=86, a87=87, a88=88, a89=89, a90=90, a91=91, a92=92, a93=93, a94=94, a95=95, a96=96, a97=97, a98=98, a99=99, a100=100, a101=101, a102=102, a103=103, a104=104, a105=105, a106=106, a107=107, a108=108, a109=109, a110=110, a111=111, a112=112, a113=113, a114=114, a115=115, a116=116, a117=117, a118=118, a119=119, a120=120, a121=121, a122=122, a123=123, a124=124, a125=125, a126=126, a127=127, a128=128, a129=129, a130=130, a131=131, a132=132, a133=133, a134=134, a135=135, a136=136, a137=137, a138=138, a139=139, a140=140, a141=141, a142=142, a143=143, a144=144, a145=145, a146=146, a147=147, a148=148, a149=149, a150=150, a151=151, a152=152, a153=153, a154=154, a155=155, a156=156, a157=157, a158=158, a159=159, a160=160, a161=161, a162=162, a163=163, a164=164, a165=165, a166=166, a167=167, a168=168, a169=169, a170=170, a171=171, a172=172, a173=173, a174=174, a175=175, a176=176, a177=177, a178=178, a179=179, a180=180, a181=181, a182=182, a183=183, a184=184, a185=185, a186=186, a187=187, a188=188, a189=189, a190=190, a191=191, a192=192, a193=193, a194=194, a195=195, a196=196, a197=197, a198=198, a199=199, a200=200, a201=201, a202=202, a203=203, a204=204, a205=205, a206=206, a207=207, a208=208, a209=209, a210=210, a211=211, a212=212, a213=213, a214=214, a215=215, a216=216, a217=217, a218=218, a219=219, a220=220, a221=221, a222=222, a223=223, a224=224, a225=225, a226=226, a227=227, a228=228, a229=229, a230=230, a231=231, a232=232, a233=233, a234=234, a235=235, a236=236, a237=237, a238=238, a239=239, a240=240, a241=241, a242=242, a243=243, a244=244, a245=245, a246=246, a247=247, a248=248, a249=249, a250=250, a251=251, a252=252, a253=253, a254=254, a255=255, a256=256, a257=257, a258=258, a259=259, a260=260, a261=261, a262=262, a263=263, a264=264, a265=265, a266=266, a267=267, a268=268, a269=269, a270=270, a271=271, a272=272, a273=273, a274=274, a275=275, a276=276, a277=277, a278=278, a279=279, a280=280, a281=281, a282=282, a283=283, a284=284, a285=285, a286=286, a287=287, a288=288, a289=289, a290=290, a291=291, a292=292, a293=293, a294=294, a295=295, a296=296, a297=297, a298=298, a299=299, a300=300, a301=301, a302=302, a303=303, a304=304, a305=305, a306=306, a307=307, a308=308, a309=309, a310=310, a311=311, a312=312, a313=313, a314=314, a315=315, a316=316, a317=317, a318=318, a319=319, a320=320, a321=321, a322=322, a323=323, a324=324, a325=325, a326=326, a327=327, a328=328, a329=329, a330=330, a331=331, a332=332, a333=333, a334=334, a335=335, a336=336, a337=337, a338=338, a339=339, a340=340, a341=341, a342=342, a343=343, a344=344, a345=345, a346=346, a347=347, a348=348, a349=349, a350=350, a351=351, a352=352, a353=353, a354=354, a355=355, a356=356, a357=357, a358=358, a359=359, a360=360, a361=361, a362=362, a363=363, a364=364, a365=365, a366=366, a367=367, a368=368, a369=369, a370=370, a371=371, a372=372, a373=373, a374=374, a375=375, a376=376, a377=377, a378=378, a379=379, a380=380, a381=381, a382=382, a383=383, a384=384, a385=385, a386=386, a387=387, a388=388, a389=389, a390=390, a391=391, a392=392, a393=393, a394=394, a395=395, a396=396, a397=397, a398=398, a399=399) >>> s.a391 391
namedtuple out of the box doesn't support what you are trying to do. So the following might achieve the goal, which might change from 400 to 450 arguments, or lesser and saner. def customtuple(*keys): class string: _keys = keys _dict = {} def __init__(self, *args): args = list(args) if len(args) != len(self._keys): raise Exception("No go forward") for key in range(len(args)): self._dict[self._keys[key]] = args[key] def __setattr__(self, *args): raise BaseException("Not allowed") def __getattr__(self, arg): try: return self._dict[arg] except: raise BaseException("Name not defined") def __repr__(self): return ("string(%s)" %(", ".join(["%s=%r" %(self._keys[key], self._dict[self._keys[key]]) for key in range(len(self._dict))]))) return string >>> strings = customtuple(*['a'+str(x) for x in range(1, 401)]) >>> s = strings(*['a'+str(x) for x in range(2, 402)]) >>> s.a1 'a2' >>> s.a1 = 1 Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/hus787/p.py", line 15, in __setattr__ def __setattr__(self, *args): BaseException: Not allowed For more light on the subject.
Here is my version of a replacement for namedtuple that supports more than 255 arguments. The idea was not to be functionally equivalent but rather to improve on some aspects (in my opinion). This is for Python 3.4+ only: class SequenceAttrReader(object): """ Class to function similar to collections.namedtuple but allowing more than 255 keys. Initialize with attribute string (space separated), then load in data via a sequence, then access the list keys as properties i.e. csv_line = SequenceAttrReader('a b c') csv_line = csv_line.load([1, 2, 3]) print(csv_line.b) >> 2 """ _attr_string = None _attr_list = [] _data_list = [] def __init__(self, attr_string): if not attr_string: raise AttributeError('SequenceAttrReader not properly initialized, please use a non-empty string') self._attr_string = attr_string self._attr_list = attr_string.split(' ') def __getattr__(self, name): if not self._attr_string or not self._attr_list or not self._data_list: raise AttributeError('SequenceAttrReader not properly initialized or loaded') try: index = self._attr_list.index(name) except ValueError: raise AttributeError("'{name}'' not in attribute string".format(name=name)) from None try: value = self._data_list[index] except IndexError: raise AttributeError("No attribute named '{name}'' in".format(name=name)) from None return value def __str__(self): return str(self._data_list) def __repr__(self): return 'SequenceAttrReader("{attr_string}")'.format(attr_string=self._attr_string) def load_data(self, data_list): if not self._attr_list: raise AttributeError('SequenceAttrReader not properly initialized') if not data_list: raise AttributeError('SequenceAttrReader needs to load a non-empty sequence') self._data_list = data_list This is probably not the most efficient way if you are doing a lot of individual lookups, converting it internally to a dict may be better. I'll work on an optimized version once I have more time or at least see what the performance difference is.