Related
class NiceClass():
some_value = SomeObject(...)
some_other_value = SomeOtherObject(...)
#classmethod
def get_all_vars(cls):
...
I want get_all_vars() to return [SomeObject(...), SomeOtherObject(...)], or more specifically, the values of the variables in cls.
Solutions tried that didn't work out for me:
return [cls.some_value, cls.some_other_value, ...] (requires listing the variable manually)
subclassing Enum then using list(cls) (requires using some_value.value to access the value elsewhere in the program, also type hinting would be a mess)
namedtuples (nope not touching that subject, heard it was much more complicated than Enum)
[value for key, value in vars(cls).items() if not callable(value) and not key.startswith("__")] (too hacky due to using vars(cls), also for some reason it also includes get_all_vars due to it being a classmethod)
There are two ways. This is a straight answer to your question:
class Foo:
pass
class Bar:
x: int = 1
y: str = 'hello'
z: Foo = Foo()
#classmethod
def get_all(cls):
xs = []
for name, value in vars(cls).items():
if not (name.startswith('__') or isinstance(value, classmethod)):
xs.append(value)
return xs
This is what I suggest:
from dataclasses import dataclass, fields
class Foo:
pass
#dataclass
class Bar:
x: int = 1
y: str = 'hello'
z: Foo = Foo()
#classmethod
def get_defaults(cls):
return [f.default for f in fields(cls)]
#classmethod
def get_all(cls):
return [getattr(cls, f.name) for f in fields(cls)]
results:
Bar.get_defaults() == Bar.get_all()
# True -> [1, 'hello', __main__.Foo]
Bar.x = 10
Bar.get_defaults() == Bar.get_all()
# False -> [1, 'hello', __main__.Foo] != [10, 'hello', __main__.Foo]
You can create a list of values and define individual attributes at the same time with minimal boilerplate.
class NiceClass():
(some_value,
some_other_value,
) = _all_values = [
SomeObject(...),
SomeOtherObject(...)
]
#classmethod
def get_all_vars(cls):
return cls._all_values
The most obvious drawback to this is that it's easy to get the order of names and values out of sync.
Ideally, you might like to do something like
class NiceClass:
_attributes = {
'some_value': SomeObject(...),
'some_other_value': SomeOtherObject(...)
}
#classmethod
def get_all_vars(cls):
return cls._attributes.values()
and have some way to "inject" the contents of _attributes into the class namespace directly. The simplest way to do this is with a mix-in class that defines __init_subclass__:
class AddAttributes:
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
cls.__dict__.update(cls._attributes)
class NiceClass(AddAttributes):
# As above
...
This might sound like a https://xyproblem.info/ but my solution might work in the other case as well. You can get the fields of an object by using __dict__ or vars (which is considered more pythonic given: Python dictionary from an object's fields)
You could do something like:
class ClassTest:
def __init__(self):
self.one = 1
self.two = 2
list(vars(ClassTest()).values())
But you will see that it has some limitations. It doesn't recognize the not in self.variable_name defined variables like you have used above. It might help you nonetheless, because you can simply define them in init.
Is there a way to conveniently define a C-like structure in Python? I'm tired of writing stuff like:
class MyStruct():
def __init__(self, field1, field2, field3):
self.field1 = field1
self.field2 = field2
self.field3 = field3
Update: Data Classes
With the introduction of Data Classes in Python 3.7 we get very close.
The following example is similar to the NamedTuple example below, but the resulting object is mutable and it allows for default values.
from dataclasses import dataclass
#dataclass
class Point:
x: float
y: float
z: float = 0.0
p = Point(1.5, 2.5)
print(p) # Point(x=1.5, y=2.5, z=0.0)
This plays nicely with the new typing module in case you want to use more specific type annotations.
I've been waiting desperately for this! If you ask me, Data Classes and the new NamedTuple declaration, combined with the typing module are a godsend!
Improved NamedTuple declaration
Since Python 3.6 it became quite simple and beautiful (IMHO), as long as you can live with immutability.
A new way of declaring NamedTuples was introduced, which allows for type annotations as well:
from typing import NamedTuple
class User(NamedTuple):
name: str
class MyStruct(NamedTuple):
foo: str
bar: int
baz: list
qux: User
my_item = MyStruct('foo', 0, ['baz'], User('peter'))
print(my_item) # MyStruct(foo='foo', bar=0, baz=['baz'], qux=User(name='peter'))
Use a named tuple, which was added to the collections module in the standard library in Python 2.6. It's also possible to use Raymond Hettinger's named tuple recipe if you need to support Python 2.4.
It's nice for your basic example, but also covers a bunch of edge cases you might run into later as well. Your fragment above would be written as:
from collections import namedtuple
MyStruct = namedtuple("MyStruct", "field1 field2 field3")
The newly created type can be used like this:
m = MyStruct("foo", "bar", "baz")
You can also use named arguments:
m = MyStruct(field1="foo", field2="bar", field3="baz")
You can use a tuple for a lot of things where you would use a struct in C (something like x,y coordinates or RGB colors for example).
For everything else you can use dictionary, or a utility class like this one:
>>> class Bunch:
... def __init__(self, **kwds):
... self.__dict__.update(kwds)
...
>>> mystruct = Bunch(field1=value1, field2=value2)
I think the "definitive" discussion is here, in the published version of the Python Cookbook.
Perhaps you are looking for Structs without constructors:
class Sample:
name = ''
average = 0.0
values = None # list cannot be initialized here!
s1 = Sample()
s1.name = "sample 1"
s1.values = []
s1.values.append(1)
s1.values.append(2)
s1.values.append(3)
s2 = Sample()
s2.name = "sample 2"
s2.values = []
s2.values.append(4)
for v in s1.values: # prints 1,2,3 --> OK.
print v
print "***"
for v in s2.values: # prints 4 --> OK.
print v
How about a dictionary?
Something like this:
myStruct = {'field1': 'some val', 'field2': 'some val'}
Then you can use this to manipulate values:
print myStruct['field1']
myStruct['field2'] = 'some other values'
And the values don't have to be strings. They can be pretty much any other object.
dF: that's pretty cool... I didn't
know that I could access the fields in
a class using dict.
Mark: the situations that I wish I had
this are precisely when I want a tuple
but nothing as "heavy" as a
dictionary.
You can access the fields of a class using a dictionary because the fields of a class, its methods and all its properties are stored internally using dicts (at least in CPython).
...Which leads us to your second comment. Believing that Python dicts are "heavy" is an extremely non-pythonistic concept. And reading such comments kills my Python Zen. That's not good.
You see, when you declare a class you are actually creating a pretty complex wrapper around a dictionary - so, if anything, you are adding more overhead than by using a simple dictionary. An overhead which, by the way, is meaningless in any case. If you are working on performance critical applications, use C or something.
I would also like to add a solution that uses slots:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
Definitely check the documentation for slots but a quick explanation of slots is that it is python's way of saying: "If you can lock these attributes and only these attributes into the class such that you commit that you will not add any new attributes once the class is instantiated (yes you can add new attributes to a class instance, see example below) then I will do away with the large memory allocation that allows for adding new attributes to a class instance and use just what I need for these slotted attributes".
Example of adding attributes to class instance (thus not using slots):
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
print(p1.z)
Output: 8
Example of trying to add attributes to class instance where slots was used:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
Output: AttributeError: 'Point' object has no attribute 'z'
This can effectively works as a struct and uses less memory than a class (like a struct would, although I have not researched exactly how much). It is recommended to use slots if you will be creating a large amount of instances of the object and do not need to add attributes. A point object is a good example of this as it is likely that one may instantiate many points to describe a dataset.
You can subclass the C structure that is available in the standard library. The ctypes module provides a Structure class. The example from the docs:
>>> from ctypes import *
>>> class POINT(Structure):
... _fields_ = [("x", c_int),
... ("y", c_int)]
...
>>> point = POINT(10, 20)
>>> print point.x, point.y
10 20
>>> point = POINT(y=5)
>>> print point.x, point.y
0 5
>>> POINT(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: too many initializers
>>>
>>> class RECT(Structure):
... _fields_ = [("upperleft", POINT),
... ("lowerright", POINT)]
...
>>> rc = RECT(point)
>>> print rc.upperleft.x, rc.upperleft.y
0 5
>>> print rc.lowerright.x, rc.lowerright.y
0 0
>>>
You can also pass the init parameters to the instance variables by position
# Abstract struct class
class Struct:
def __init__ (self, *argv, **argd):
if len(argd):
# Update by dictionary
self.__dict__.update (argd)
else:
# Update by position
attrs = filter (lambda x: x[0:2] != "__", dir(self))
for n in range(len(argv)):
setattr(self, attrs[n], argv[n])
# Specific class
class Point3dStruct (Struct):
x = 0
y = 0
z = 0
pt1 = Point3dStruct()
pt1.x = 10
print pt1.x
print "-"*10
pt2 = Point3dStruct(5, 6)
print pt2.x, pt2.y
print "-"*10
pt3 = Point3dStruct (x=1, y=2, z=3)
print pt3.x, pt3.y, pt3.z
print "-"*10
Whenever I need an "instant data object that also behaves like a dictionary" (I don't think of C structs!), I think of this cute hack:
class Map(dict):
def __init__(self, **kwargs):
super(Map, self).__init__(**kwargs)
self.__dict__ = self
Now you can just say:
struct = Map(field1='foo', field2='bar', field3=42)
self.assertEquals('bar', struct.field2)
self.assertEquals(42, struct['field3'])
Perfectly handy for those times when you need a "data bag that's NOT a class", and for when namedtuples are incomprehensible...
Some the answers here are massively elaborate. The simplest option I've found is (from: http://norvig.com/python-iaq.html):
class Struct:
"A structure that can have any fields defined."
def __init__(self, **entries): self.__dict__.update(entries)
Initialising:
>>> options = Struct(answer=42, linelen=80, font='courier')
>>> options.answer
42
adding more:
>>> options.cat = "dog"
>>> options.cat
dog
edit: Sorry didn't see this example already further down.
You access C-Style struct in python in following way.
class cstruct:
var_i = 0
var_f = 0.0
var_str = ""
if you just want use object of cstruct
obj = cstruct()
obj.var_i = 50
obj.var_f = 50.00
obj.var_str = "fifty"
print "cstruct: obj i=%d f=%f s=%s" %(obj.var_i, obj.var_f, obj.var_str)
if you want to create an array of objects of cstruct
obj_array = [cstruct() for i in range(10)]
obj_array[0].var_i = 10
obj_array[0].var_f = 10.00
obj_array[0].var_str = "ten"
#go ahead and fill rest of array instaces of struct
#print all the value
for i in range(10):
print "cstruct: obj_array i=%d f=%f s=%s" %(obj_array[i].var_i, obj_array[i].var_f, obj_array[i].var_str)
Note:
instead of 'cstruct' name, please use your struct name
instead of var_i, var_f, var_str, please define your structure's member variable.
This might be a bit late but I made a solution using Python Meta-Classes (decorator version below too).
When __init__ is called during run time, it grabs each of the arguments and their value and assigns them as instance variables to your class. This way you can make a struct-like class without having to assign every value manually.
My example has no error checking so it is easier to follow.
class MyStruct(type):
def __call__(cls, *args, **kwargs):
names = cls.__init__.func_code.co_varnames[1:]
self = type.__call__(cls, *args, **kwargs)
for name, value in zip(names, args):
setattr(self , name, value)
for name, value in kwargs.iteritems():
setattr(self , name, value)
return self
Here it is in action.
>>> class MyClass(object):
__metaclass__ = MyStruct
def __init__(self, a, b, c):
pass
>>> my_instance = MyClass(1, 2, 3)
>>> my_instance.a
1
>>>
I posted it on reddit and /u/matchu posted a decorator version which is cleaner. I'd encourage you to use it unless you want to expand the metaclass version.
>>> def init_all_args(fn):
#wraps(fn)
def wrapped_init(self, *args, **kwargs):
names = fn.func_code.co_varnames[1:]
for name, value in zip(names, args):
setattr(self, name, value)
for name, value in kwargs.iteritems():
setattr(self, name, value)
return wrapped_init
>>> class Test(object):
#init_all_args
def __init__(self, a, b):
pass
>>> a = Test(1, 2)
>>> a.a
1
>>>
I wrote a decorator which you can use on any method to make it so that all of the arguments passed in, or any defaults, are assigned to the instance.
def argumentsToAttributes(method):
argumentNames = method.func_code.co_varnames[1:]
# Generate a dictionary of default values:
defaultsDict = {}
defaults = method.func_defaults if method.func_defaults else ()
for i, default in enumerate(defaults, start = len(argumentNames) - len(defaults)):
defaultsDict[argumentNames[i]] = default
def newMethod(self, *args, **kwargs):
# Use the positional arguments.
for name, value in zip(argumentNames, args):
setattr(self, name, value)
# Add the key word arguments. If anything is missing, use the default.
for name in argumentNames[len(args):]:
setattr(self, name, kwargs.get(name, defaultsDict[name]))
# Run whatever else the method needs to do.
method(self, *args, **kwargs)
return newMethod
A quick demonstration. Note that I use a positional argument a, use the default value for b, and a named argument c. I then print all 3 referencing self, to show that they've been properly assigned before the method is entered.
class A(object):
#argumentsToAttributes
def __init__(self, a, b = 'Invisible', c = 'Hello'):
print(self.a)
print(self.b)
print(self.c)
A('Why', c = 'Nothing')
Note that my decorator should work with any method, not just __init__.
I don't see this answer here, so I figure I'll add it since I'm leaning Python right now and just discovered it. The Python tutorial (Python 2 in this case) gives the following simple and effective example:
class Employee:
pass
john = Employee() # Create an empty employee record
# Fill the fields of the record
john.name = 'John Doe'
john.dept = 'computer lab'
john.salary = 1000
That is, an empty class object is created, then instantiated, and the fields are added dynamically.
The up-side to this is its really simple. The downside is it isn't particularly self-documenting (the intended members aren't listed anywhere in the class "definition"), and unset fields can cause problems when accessed. Those two problems can be solved by:
class Employee:
def __init__ (self):
self.name = None # or whatever
self.dept = None
self.salary = None
Now at a glance you can at least see what fields the program will be expecting.
Both are prone to typos, john.slarly = 1000 will succeed. Still, it works.
Here is a solution which uses a class (never instantiated) to hold data. I like that this way involves very little typing and does not require any additional packages etc.
class myStruct:
field1 = "one"
field2 = "2"
You can add more fields later, as needed:
myStruct.field3 = 3
To get the values, the fields are accessed as usual:
>>> myStruct.field1
'one'
Personally, I like this variant too. It extends #dF's answer.
class struct:
def __init__(self, *sequential, **named):
fields = dict(zip(sequential, [None]*len(sequential)), **named)
self.__dict__.update(fields)
def __repr__(self):
return str(self.__dict__)
It supports two modes of initialization (that can be blended):
# Struct with field1, field2, field3 that are initialized to None.
mystruct1 = struct("field1", "field2", "field3")
# Struct with field1, field2, field3 that are initialized according to arguments.
mystruct2 = struct(field1=1, field2=2, field3=3)
Also, it prints nicer:
print(mystruct2)
# Prints: {'field3': 3, 'field1': 1, 'field2': 2}
There is a python package exactly for this purpose. see cstruct2py
cstruct2py is a pure python library for generate python classes from C code and use them to pack and unpack data. The library can parse C headres (structs, unions, enums, and arrays declarations) and emulate them in python. The generated pythonic classes can parse and pack the data.
For example:
typedef struct {
int x;
int y;
} Point;
after generating pythonic class...
p = Point(x=0x1234, y=0x5678)
p.packed == "\x34\x12\x00\x00\x78\x56\x00\x00"
How to use
First we need to generate the pythonic structs:
import cstruct2py
parser = cstruct2py.c2py.Parser()
parser.parse_file('examples/example.h')
Now we can import all names from the C code:
parser.update_globals(globals())
We can also do that directly:
A = parser.parse_string('struct A { int x; int y;};')
Using types and defines from the C code
a = A()
a.x = 45
print a
buf = a.packed
b = A(buf)
print b
c = A('aaaa11112222', 2)
print c
print repr(c)
The output will be:
{'x':0x2d, 'y':0x0}
{'x':0x2d, 'y':0x0}
{'x':0x31316161, 'y':0x32323131}
A('aa111122', x=0x31316161, y=0x32323131)
Clone
For clone cstruct2py run:
git clone https://github.com/st0ky/cstruct2py.git --recursive
Here is a quick and dirty trick:
>>> ms = Warning()
>>> ms.foo = 123
>>> ms.bar = 'akafrit'
How does it works? It just re-use the builtin class Warning (derived from Exception) and use it as it was you own defined class.
The good points are that you do not need to import or define anything first, that "Warning" is a short name, and that it also makes clear you are doing something dirty which should not be used elsewhere than a small script of yours.
By the way, I tried to find something even simpler like ms = object() but could not (this last exemple is not working). If you have one, I am interested.
NamedTuple is comfortable. but there no one shares the performance and storage.
from typing import NamedTuple
import guppy # pip install guppy
import timeit
class User:
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserSlot:
__slots__ = ('name', 'uid')
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserTuple(NamedTuple):
# __slots__ = () # AttributeError: Cannot overwrite NamedTuple attribute __slots__
name: str
uid: int
def get_fn(obj, attr_name: str):
def get():
getattr(obj, attr_name)
return get
if 'memory test':
obj = [User('Carson', 1) for _ in range(1000000)] # Cumulative: 189138883
obj_slot = [UserSlot('Carson', 1) for _ in range(1000000)] # 77718299 <-- winner
obj_namedtuple = [UserTuple('Carson', 1) for _ in range(1000000)] # 85718297
print(guppy.hpy().heap()) # Run this function individually.
"""
Index Count % Size % Cumulative % Kind (class / dict of class)
0 1000000 24 112000000 34 112000000 34 dict of __main__.User
1 1000000 24 64000000 19 176000000 53 __main__.UserTuple
2 1000000 24 56000000 17 232000000 70 __main__.User
3 1000000 24 56000000 17 288000000 87 __main__.UserSlot
...
"""
if 'performance test':
obj = User('Carson', 1)
obj_slot = UserSlot('Carson', 1)
obj_tuple = UserTuple('Carson', 1)
time_normal = min(timeit.repeat(get_fn(obj, 'name'), repeat=20))
print(time_normal) # 0.12550550000000005
time_slot = min(timeit.repeat(get_fn(obj_slot, 'name'), repeat=20))
print(time_slot) # 0.1368690000000008
time_tuple = min(timeit.repeat(get_fn(obj_tuple, 'name'), repeat=20))
print(time_tuple) # 0.16006120000000124
print(time_tuple/time_slot) # 1.1694481584580898 # The slot is almost 17% faster than NamedTuple on Windows. (Python 3.7.7)
If your __dict__ is not using, please choose between __slots__ (higher performance and storage) and NamedTuple (clear for reading and use)
You can review this link(Usage of slots
) to get more __slots__ information.
https://stackoverflow.com/a/32448434/159695 does not work in Python3.
https://stackoverflow.com/a/35993/159695 works in Python3.
And I extends it to add default values.
class myStruct:
def __init__(self, **kwds):
self.x=0
self.__dict__.update(kwds) # Must be last to accept assigned member variable.
def __repr__(self):
args = ['%s=%s' % (k, repr(v)) for (k,v) in vars(self).items()]
return '%s(%s)' % ( self.__class__.__qualname__, ', '.join(args) )
a=myStruct()
b=myStruct(x=3,y='test')
c=myStruct(x='str')
>>> a
myStruct(x=0)
>>> b
myStruct(x=3, y='test')
>>> c
myStruct(x='str')
The following solution to a struct is inspired by the namedtuple implementation and some of the previous answers. However, unlike the namedtuple it is mutable, in it's values, but like the c-style struct immutable in the names/attributes, which a normal class or dict isn't.
_class_template = """\
class {typename}:
def __init__(self, *args, **kwargs):
fields = {field_names!r}
for x in fields:
setattr(self, x, None)
for name, value in zip(fields, args):
setattr(self, name, value)
for name, value in kwargs.items():
setattr(self, name, value)
def __repr__(self):
return str(vars(self))
def __setattr__(self, name, value):
if name not in {field_names!r}:
raise KeyError("invalid name: %s" % name)
object.__setattr__(self, name, value)
"""
def struct(typename, field_names):
class_definition = _class_template.format(
typename = typename,
field_names = field_names)
namespace = dict(__name__='struct_%s' % typename)
exec(class_definition, namespace)
result = namespace[typename]
result._source = class_definition
return result
Usage:
Person = struct('Person', ['firstname','lastname'])
generic = Person()
michael = Person('Michael')
jones = Person(lastname = 'Jones')
In [168]: michael.middlename = 'ben'
Traceback (most recent call last):
File "<ipython-input-168-b31c393c0d67>", line 1, in <module>
michael.middlename = 'ben'
File "<string>", line 19, in __setattr__
KeyError: 'invalid name: middlename'
If you don't have a 3.7 for #dataclass and need mutability, the following code might work for you. It's quite self-documenting and IDE-friendly (auto-complete), prevents writing things twice, is easily extendable and it is very simple to test that all instance variables are completely initialized:
class Params():
def __init__(self):
self.var1 : int = None
self.var2 : str = None
def are_all_defined(self):
for key, value in self.__dict__.items():
assert (value is not None), "instance variable {} is still None".format(key)
return True
params = Params()
params.var1 = 2
params.var2 = 'hello'
assert(params.are_all_defined)
The best way I found to do this was to use a custom dictionary class as explained in this post: https://stackoverflow.com/a/14620633/8484485
If iPython autocompletion support is needed, simply define the dir() function like this:
class AttrDict(dict):
def __init__(self, *args, **kwargs):
super(AttrDict, self).__init__(*args, **kwargs)
self.__dict__ = self
def __dir__(self):
return self.keys()
You then define your pseudo struct like so: (this one is nested)
my_struct=AttrDict ({
'com1':AttrDict ({
'inst':[0x05],
'numbytes':2,
'canpayload':False,
'payload':None
})
})
You can then access the values inside my_struct like this:
print(my_struct.com1.inst)
=>[5]
The cleanest way I can think of is to use a class decorator that lets you declare a static class and rewrite it to act as a struct with normal, named properties:
from as_struct import struct
#struct
class Product():
name = 'unknown product'
quantity = -1
sku = '-'
# create instance
p = Product('plush toy', sku='12-345-6789')
# check content:
p.name # plush toy
p.quantity # -1
p.sku # 12-345-6789
Using the following decorator code:
def struct(struct_class):
# create a new init
def struct_init(self, *args, **kwargs):
i = 0 # we really don't need enumerate() here...
for value in args:
name = member_names[i]
default_value = member_values[i]
setattr(self, name, value if value is not None else default_value)
i += 1 # ...we just need to inc an int
for key,value in kwargs.items():
i = member_names.index(key)
default_value = member_values[i]
setattr(self, key, value if value is not None else default_value)
# extract the struct members
member_names = []
member_values = []
for attr_name in dir(struct_class):
if not attr_name.startswith('_'):
value = getattr(struct_class, attr_name)
if not callable(value):
member_names.append(attr_name)
member_values.append(value)
# rebind and return
struct_class.init = struct_init
return struct_class
Which works by taking the class, extracting the field names and their default values, then rewriting the class's __init__ function to set self attributes based on knowing which argument index maps to which property name.
I think Python structure dictionary is suitable for this requirement.
d = dict{}
d[field1] = field1
d[field2] = field2
d[field2] = field3
Extending #gz.'s (generally superior to this one) answer, for a quick and dirty namedtuple structure we can do:
import collections
x = collections.namedtuple('foobar', 'foo bar')(foo=1,bar=2)
y = collections.namedtuple('foobar', 'foo bar')(foo=3,bar=4)
print(x,y)
>foobar(foo=1, bar=2) foobar(foo=3, bar=4)
Instead of writing code like this every time I define a class:
class Foo(object):
def __init__(self, a, b, c, d, e, f, g):
self.a = a
self.b = b
self.c = c
self.d = d
self.e = e
self.f = f
self.g = g
I could use this recipe for automatic attribute assignment.
class Foo(object):
#autoassign
def __init__(self, a, b, c, d, e, f, g):
pass
Two questions:
Are there drawbacks or pitfalls associated with this shortcut?
Is there a better way to achieve similar convenience?
There are some things about the autoassign code that bug me (mostly stylistic, but one more serious problem):
autoassign does not assign an
'args' attribute:
class Foo(object):
#autoassign
def __init__(self,a,b,c=False,*args):
pass
a=Foo('IBM','/tmp',True, 100, 101)
print(a.args)
# AttributeError: 'Foo' object has no attribute 'args'
autoassign acts like a decorator.
But autoassign(*argnames) calls a
function which returns a decorator.
To achieve this magic, autoassign
needs to test the type of its first
argument. If given a choice, I
prefer functions not test
the type of its arguments.
There seems to be a considerable
amount of code devoted to setting up
sieve, lambdas within lambdas,
ifilters, and lots of conditions.
if kwargs:
exclude, f = set(kwargs['exclude']), None
sieve = lambda l:itertools.ifilter(lambda nv: nv[0] not in exclude, l)
elif len(names) == 1 and inspect.isfunction(names[0]):
f = names[0]
sieve = lambda l:l
else:
names, f = set(names), None
sieve = lambda l: itertools.ifilter(lambda nv: nv[0] in names, l)
I think there might be a simpler way. (See
below).
for _ in
itertools.starmap(assigned.setdefault,
defaults): pass. I don't think
map or starmap was meant to call
functions, whose only purpose is their
side effects. It could have been
written more clearly with the mundane:
for key,value in defaults.iteritems():
assigned.setdefault(key,value)
Here is an alternative simpler implementation which has the same functionality as autoassign (e.g. can do includes and excludes), and which addresses the above points:
import inspect
import functools
def autoargs(*include, **kwargs):
def _autoargs(func):
attrs, varargs, varkw, defaults = inspect.getargspec(func)
def sieve(attr):
if kwargs and attr in kwargs['exclude']:
return False
if not include or attr in include:
return True
else:
return False
#functools.wraps(func)
def wrapper(self, *args, **kwargs):
# handle default values
if defaults:
for attr, val in zip(reversed(attrs), reversed(defaults)):
if sieve(attr):
setattr(self, attr, val)
# handle positional arguments
positional_attrs = attrs[1:]
for attr, val in zip(positional_attrs, args):
if sieve(attr):
setattr(self, attr, val)
# handle varargs
if varargs:
remaining_args = args[len(positional_attrs):]
if sieve(varargs):
setattr(self, varargs, remaining_args)
# handle varkw
if kwargs:
for attr, val in kwargs.items():
if sieve(attr):
setattr(self, attr, val)
return func(self, *args, **kwargs)
return wrapper
return _autoargs
And here is the unit test I used to check its behavior:
import sys
import unittest
import utils_method as um
class Test(unittest.TestCase):
def test_autoargs(self):
class A(object):
#um.autoargs()
def __init__(self,foo,path,debug=False):
pass
a=A('rhubarb','pie',debug=True)
self.assertTrue(a.foo=='rhubarb')
self.assertTrue(a.path=='pie')
self.assertTrue(a.debug==True)
class B(object):
#um.autoargs()
def __init__(self,foo,path,debug=False,*args):
pass
a=B('rhubarb','pie',True, 100, 101)
self.assertTrue(a.foo=='rhubarb')
self.assertTrue(a.path=='pie')
self.assertTrue(a.debug==True)
self.assertTrue(a.args==(100,101))
class C(object):
#um.autoargs()
def __init__(self,foo,path,debug=False,*args,**kw):
pass
a=C('rhubarb','pie',True, 100, 101,verbose=True)
self.assertTrue(a.foo=='rhubarb')
self.assertTrue(a.path=='pie')
self.assertTrue(a.debug==True)
self.assertTrue(a.verbose==True)
self.assertTrue(a.args==(100,101))
def test_autoargs_names(self):
class C(object):
#um.autoargs('bar','baz','verbose')
def __init__(self,foo,bar,baz,verbose=False):
pass
a=C('rhubarb','pie',1)
self.assertTrue(a.bar=='pie')
self.assertTrue(a.baz==1)
self.assertTrue(a.verbose==False)
self.assertRaises(AttributeError,getattr,a,'foo')
def test_autoargs_exclude(self):
class C(object):
#um.autoargs(exclude=('bar','baz','verbose'))
def __init__(self,foo,bar,baz,verbose=False):
pass
a=C('rhubarb','pie',1)
self.assertTrue(a.foo=='rhubarb')
self.assertRaises(AttributeError,getattr,a,'bar')
def test_defaults_none(self):
class A(object):
#um.autoargs()
def __init__(self,foo,path,debug):
pass
a=A('rhubarb','pie',debug=True)
self.assertTrue(a.foo=='rhubarb')
self.assertTrue(a.path=='pie')
self.assertTrue(a.debug==True)
if __name__ == '__main__':
unittest.main(argv = sys.argv + ['--verbose'])
PS. Using autoassign or autoargs is compatible with IPython code completion.
From Python 3.7+ you can use a Data Class, which achieves what you want and more.
It allows you to define fields for your class, which are attributes automatically assigned.
It would look something like that:
#dataclass
class Foo:
a: str
b: int
c: str
...
The __init__ method will be automatically created in your class, and it will assign the arguments of instance creation to those attributes (and validate the arguments).
Note that here type hinting is required, that is why I have used int and str in the example. If you don't know the type of your field, you can use Any from the typing module.
Is there a better way to achieve similar convenience?
I don't know if it is necessarily better, but you could do this:
class Foo(object):
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
>>> foo = Foo(a = 1, b = 'bar', c = [1, 2])
>>> foo.a
1
>>> foo.b
'bar'
>>> foo.c
[1, 2]
>>>
Courtesy Peter Norvig's Python: Infrequently Answered Questions.
One drawback: many IDEs parse __init__.py to discover an object's attributes. If you want automatic code completion in your IDE to be more functional, then you may be better off spelling it out the old-fashioned way.
If you have a lot of variables, you could pass one single configuration dict or object.
Similar to the above, though not the same... the following is very short, deals with args and kwargs:
def autoassign(lcls):
for key in lcls.keys():
if key!="self":
lcls["self"].__dict__[key]=lcls[key]
Use like this:
class Test(object):
def __init__(self, a, b, *args, **kwargs):
autoassign(locals())
This a simple implementation by judy2k:
from inspect import signature
def auto_args(f):
sig = signature(f) # Get a signature object for the target:
def replacement(self, *args, **kwargs):
# Parse the provided arguments using the target's signature:
bound_args = sig.bind(self, *args, **kwargs)
# Save away the arguments on `self`:
for k, v in bound_args.arguments.items():
if k != 'self':
setattr(self, k, v)
# Call the actual constructor for anything else:
f(self, *args, **kwargs)
return replacement
class MyClass:
#auto_args
def __init__(self, a, b, c=None):
pass
m = MyClass('A', 'B', 'C')
print(m.__dict__)
# {'a': 'A', 'b': 'B', 'c': 'C'}
In this package you can now find
#autoargs inspired by answer-3653049
#autoprops to transform the fields generated by #autoargs into #property, for use in combination with a validation library such as enforce or pycontracts.
Note that this has been validated for python 3.5+
class MyClass(object):
def __init__(self, **kwargs):
for key, value in kwargs.iteritems():
setattr(self, key, value)
You just can't use *args, but you can store in some instance list (like self.args, don't know)
Recently I've gone through an existing code base containing many classes where instance attributes reflect values stored in a database. I've refactored a lot of these attributes to have their database lookups be deferred, ie. not be initialised in the constructor but only upon first read. These attributes do not change over the lifetime of the instance, but they're a real bottleneck to calculate that first time and only really accessed for special cases. Hence they can also be cached after they've been retrieved from the database (this therefore fits the definition of memoisation where the input is simply "no input").
I find myself typing the following snippet of code over and over again for various attributes across various classes:
class testA(object):
def __init__(self):
self._a = None
self._b = None
#property
def a(self):
if self._a is None:
# Calculate the attribute now
self._a = 7
return self._a
#property
def b(self):
#etc
Is there an existing decorator to do this already in Python that I'm simply unaware of? Or, is there a reasonably simple way to define a decorator that does this?
I'm working under Python 2.5, but 2.6 answers might still be interesting if they are significantly different.
Note
This question was asked before Python included a lot of ready-made decorators for this. I have updated it only to correct terminology.
Here is an example implementation of a lazy property decorator:
import functools
def lazyprop(fn):
attr_name = '_lazy_' + fn.__name__
#property
#functools.wraps(fn)
def _lazyprop(self):
if not hasattr(self, attr_name):
setattr(self, attr_name, fn(self))
return getattr(self, attr_name)
return _lazyprop
class Test(object):
#lazyprop
def a(self):
print 'generating "a"'
return range(5)
Interactive session:
>>> t = Test()
>>> t.__dict__
{}
>>> t.a
generating "a"
[0, 1, 2, 3, 4]
>>> t.__dict__
{'_lazy_a': [0, 1, 2, 3, 4]}
>>> t.a
[0, 1, 2, 3, 4]
I wrote this one for myself... To be used for true one-time calculated lazy properties. I like it because it avoids sticking extra attributes on objects, and once activated does not waste time checking for attribute presence, etc.:
import functools
class lazy_property(object):
'''
meant to be used for lazy evaluation of an object attribute.
property should represent non-mutable data, as it replaces itself.
'''
def __init__(self, fget):
self.fget = fget
# copy the getter function's docstring and other attributes
functools.update_wrapper(self, fget)
def __get__(self, obj, cls):
if obj is None:
return self
value = self.fget(obj)
setattr(obj, self.fget.__name__, value)
return value
class Test(object):
#lazy_property
def results(self):
calcs = 1 # Do a lot of calculation here
return calcs
Note: The lazy_property class is a non-data descriptor, which means it is read-only. Adding a __set__ method would prevent it from working correctly.
For all sorts of great utilities I'm using boltons.
As part of that library you have cachedproperty:
from boltons.cacheutils import cachedproperty
class Foo(object):
def __init__(self):
self.value = 4
#cachedproperty
def cached_prop(self):
self.value += 1
return self.value
f = Foo()
print(f.value) # initial value
print(f.cached_prop) # cached property is calculated
f.value = 1
print(f.cached_prop) # same value for the cached property - it isn't calculated again
print(f.value) # the backing value is different (it's essentially unrelated value)
property is a class. A descriptor to be exact. Simply derive from it and implement the desired behavior.
class lazyproperty(property):
....
class testA(object):
....
a = lazyproperty('_a')
b = lazyproperty('_b')
Here's a callable that takes an optional timeout argument, in the __call__ you could also copy over the __name__, __doc__, __module__ from func's namespace:
import time
class Lazyproperty(object):
def __init__(self, timeout=None):
self.timeout = timeout
self._cache = {}
def __call__(self, func):
self.func = func
return self
def __get__(self, obj, objcls):
if obj not in self._cache or \
(self.timeout and time.time() - self._cache[key][1] > self.timeout):
self._cache[obj] = (self.func(obj), time.time())
return self._cache[obj]
ex:
class Foo(object):
#Lazyproperty(10)
def bar(self):
print('calculating')
return 'bar'
>>> x = Foo()
>>> print(x.bar)
calculating
bar
>>> print(x.bar)
bar
...(waiting 10 seconds)...
>>> print(x.bar)
calculating
bar
What you really want is the reify (source linked!) decorator from Pyramid:
Use as a class method decorator. It operates almost exactly like the Python #property decorator, but it puts the result of the method it decorates into the instance dict after the first call, effectively replacing the function it decorates with an instance variable. It is, in Python parlance, a non-data descriptor. The following is an example and its usage:
>>> from pyramid.decorator import reify
>>> class Foo(object):
... #reify
... def jammy(self):
... print('jammy called')
... return 1
>>> f = Foo()
>>> v = f.jammy
jammy called
>>> print(v)
1
>>> f.jammy
1
>>> # jammy func not called the second time; it replaced itself with 1
>>> # Note: reassignment is possible
>>> f.jammy = 2
>>> f.jammy
2
They added exactly what you're looking for in python 3.8
Transform a method of a class into a property whose value is computed once and then cached as a normal attribute for the life of the instance.
Similar to property(), with the addition of caching.
Use it just like #property :
#cached_property
def a(self):
self._a = 7
return self._a
There is a mix up of terms and/or confusion of concepts both in question and in answers so far.
Lazy evaluation just means that something is evaluated at runtime at the last possible moment when a value is needed. The standard #property decorator does just that.(*) The decorated function is evaluated only and every time you need the value of that property. (see wikipedia article about lazy evaluation)
(*)Actually a true lazy evaluation (compare e.g. haskell) is very hard to achieve in python (and results in code which is far from idiomatic).
Memoization is the correct term for what the asker seems to be looking for. Pure functions that do not depend on side effects for return value evaluation can be safely memoized and there is actually a decorator in functools #functools.lru_cache so no need for writing own decorators unless you need specialized behavior.
You can do this nice and easily by building a class from Python native property:
class cached_property(property):
def __init__(self, func, name=None, doc=None):
self.__name__ = name or func.__name__
self.__module__ = func.__module__
self.__doc__ = doc or func.__doc__
self.func = func
def __set__(self, obj, value):
obj.__dict__[self.__name__] = value
def __get__(self, obj, type=None):
if obj is None:
return self
value = obj.__dict__.get(self.__name__, None)
if value is None:
value = self.func(obj)
obj.__dict__[self.__name__] = value
return value
We can use this property class like regular class property ( It's also support item assignment as you can see)
class SampleClass():
#cached_property
def cached_property(self):
print('I am calculating value')
return 'My calculated value'
c = SampleClass()
print(c.cached_property)
print(c.cached_property)
c.cached_property = 2
print(c.cached_property)
print(c.cached_property)
Value only calculated first time and after that we used our saved value
Output:
I am calculating value
My calculated value
My calculated value
2
2
I agree with #jason
When I think about lazy evaluation, Asyncio immediately comes to mind.
The possibility of delaying the expensive calculation till the last minute is the sole benefit of lazy evaluation.
Caching / memozition on the other hand could be beneficial but on the expense that the calculation is static and won't change with time / inputs.
A practice I often do for expensive calculations of these sorts is to calculate then cache with TTL.
Is there a way to conveniently define a C-like structure in Python? I'm tired of writing stuff like:
class MyStruct():
def __init__(self, field1, field2, field3):
self.field1 = field1
self.field2 = field2
self.field3 = field3
Update: Data Classes
With the introduction of Data Classes in Python 3.7 we get very close.
The following example is similar to the NamedTuple example below, but the resulting object is mutable and it allows for default values.
from dataclasses import dataclass
#dataclass
class Point:
x: float
y: float
z: float = 0.0
p = Point(1.5, 2.5)
print(p) # Point(x=1.5, y=2.5, z=0.0)
This plays nicely with the new typing module in case you want to use more specific type annotations.
I've been waiting desperately for this! If you ask me, Data Classes and the new NamedTuple declaration, combined with the typing module are a godsend!
Improved NamedTuple declaration
Since Python 3.6 it became quite simple and beautiful (IMHO), as long as you can live with immutability.
A new way of declaring NamedTuples was introduced, which allows for type annotations as well:
from typing import NamedTuple
class User(NamedTuple):
name: str
class MyStruct(NamedTuple):
foo: str
bar: int
baz: list
qux: User
my_item = MyStruct('foo', 0, ['baz'], User('peter'))
print(my_item) # MyStruct(foo='foo', bar=0, baz=['baz'], qux=User(name='peter'))
Use a named tuple, which was added to the collections module in the standard library in Python 2.6. It's also possible to use Raymond Hettinger's named tuple recipe if you need to support Python 2.4.
It's nice for your basic example, but also covers a bunch of edge cases you might run into later as well. Your fragment above would be written as:
from collections import namedtuple
MyStruct = namedtuple("MyStruct", "field1 field2 field3")
The newly created type can be used like this:
m = MyStruct("foo", "bar", "baz")
You can also use named arguments:
m = MyStruct(field1="foo", field2="bar", field3="baz")
You can use a tuple for a lot of things where you would use a struct in C (something like x,y coordinates or RGB colors for example).
For everything else you can use dictionary, or a utility class like this one:
>>> class Bunch:
... def __init__(self, **kwds):
... self.__dict__.update(kwds)
...
>>> mystruct = Bunch(field1=value1, field2=value2)
I think the "definitive" discussion is here, in the published version of the Python Cookbook.
Perhaps you are looking for Structs without constructors:
class Sample:
name = ''
average = 0.0
values = None # list cannot be initialized here!
s1 = Sample()
s1.name = "sample 1"
s1.values = []
s1.values.append(1)
s1.values.append(2)
s1.values.append(3)
s2 = Sample()
s2.name = "sample 2"
s2.values = []
s2.values.append(4)
for v in s1.values: # prints 1,2,3 --> OK.
print v
print "***"
for v in s2.values: # prints 4 --> OK.
print v
How about a dictionary?
Something like this:
myStruct = {'field1': 'some val', 'field2': 'some val'}
Then you can use this to manipulate values:
print myStruct['field1']
myStruct['field2'] = 'some other values'
And the values don't have to be strings. They can be pretty much any other object.
dF: that's pretty cool... I didn't
know that I could access the fields in
a class using dict.
Mark: the situations that I wish I had
this are precisely when I want a tuple
but nothing as "heavy" as a
dictionary.
You can access the fields of a class using a dictionary because the fields of a class, its methods and all its properties are stored internally using dicts (at least in CPython).
...Which leads us to your second comment. Believing that Python dicts are "heavy" is an extremely non-pythonistic concept. And reading such comments kills my Python Zen. That's not good.
You see, when you declare a class you are actually creating a pretty complex wrapper around a dictionary - so, if anything, you are adding more overhead than by using a simple dictionary. An overhead which, by the way, is meaningless in any case. If you are working on performance critical applications, use C or something.
I would also like to add a solution that uses slots:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
Definitely check the documentation for slots but a quick explanation of slots is that it is python's way of saying: "If you can lock these attributes and only these attributes into the class such that you commit that you will not add any new attributes once the class is instantiated (yes you can add new attributes to a class instance, see example below) then I will do away with the large memory allocation that allows for adding new attributes to a class instance and use just what I need for these slotted attributes".
Example of adding attributes to class instance (thus not using slots):
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
print(p1.z)
Output: 8
Example of trying to add attributes to class instance where slots was used:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
Output: AttributeError: 'Point' object has no attribute 'z'
This can effectively works as a struct and uses less memory than a class (like a struct would, although I have not researched exactly how much). It is recommended to use slots if you will be creating a large amount of instances of the object and do not need to add attributes. A point object is a good example of this as it is likely that one may instantiate many points to describe a dataset.
You can subclass the C structure that is available in the standard library. The ctypes module provides a Structure class. The example from the docs:
>>> from ctypes import *
>>> class POINT(Structure):
... _fields_ = [("x", c_int),
... ("y", c_int)]
...
>>> point = POINT(10, 20)
>>> print point.x, point.y
10 20
>>> point = POINT(y=5)
>>> print point.x, point.y
0 5
>>> POINT(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: too many initializers
>>>
>>> class RECT(Structure):
... _fields_ = [("upperleft", POINT),
... ("lowerright", POINT)]
...
>>> rc = RECT(point)
>>> print rc.upperleft.x, rc.upperleft.y
0 5
>>> print rc.lowerright.x, rc.lowerright.y
0 0
>>>
You can also pass the init parameters to the instance variables by position
# Abstract struct class
class Struct:
def __init__ (self, *argv, **argd):
if len(argd):
# Update by dictionary
self.__dict__.update (argd)
else:
# Update by position
attrs = filter (lambda x: x[0:2] != "__", dir(self))
for n in range(len(argv)):
setattr(self, attrs[n], argv[n])
# Specific class
class Point3dStruct (Struct):
x = 0
y = 0
z = 0
pt1 = Point3dStruct()
pt1.x = 10
print pt1.x
print "-"*10
pt2 = Point3dStruct(5, 6)
print pt2.x, pt2.y
print "-"*10
pt3 = Point3dStruct (x=1, y=2, z=3)
print pt3.x, pt3.y, pt3.z
print "-"*10
Whenever I need an "instant data object that also behaves like a dictionary" (I don't think of C structs!), I think of this cute hack:
class Map(dict):
def __init__(self, **kwargs):
super(Map, self).__init__(**kwargs)
self.__dict__ = self
Now you can just say:
struct = Map(field1='foo', field2='bar', field3=42)
self.assertEquals('bar', struct.field2)
self.assertEquals(42, struct['field3'])
Perfectly handy for those times when you need a "data bag that's NOT a class", and for when namedtuples are incomprehensible...
Some the answers here are massively elaborate. The simplest option I've found is (from: http://norvig.com/python-iaq.html):
class Struct:
"A structure that can have any fields defined."
def __init__(self, **entries): self.__dict__.update(entries)
Initialising:
>>> options = Struct(answer=42, linelen=80, font='courier')
>>> options.answer
42
adding more:
>>> options.cat = "dog"
>>> options.cat
dog
edit: Sorry didn't see this example already further down.
You access C-Style struct in python in following way.
class cstruct:
var_i = 0
var_f = 0.0
var_str = ""
if you just want use object of cstruct
obj = cstruct()
obj.var_i = 50
obj.var_f = 50.00
obj.var_str = "fifty"
print "cstruct: obj i=%d f=%f s=%s" %(obj.var_i, obj.var_f, obj.var_str)
if you want to create an array of objects of cstruct
obj_array = [cstruct() for i in range(10)]
obj_array[0].var_i = 10
obj_array[0].var_f = 10.00
obj_array[0].var_str = "ten"
#go ahead and fill rest of array instaces of struct
#print all the value
for i in range(10):
print "cstruct: obj_array i=%d f=%f s=%s" %(obj_array[i].var_i, obj_array[i].var_f, obj_array[i].var_str)
Note:
instead of 'cstruct' name, please use your struct name
instead of var_i, var_f, var_str, please define your structure's member variable.
This might be a bit late but I made a solution using Python Meta-Classes (decorator version below too).
When __init__ is called during run time, it grabs each of the arguments and their value and assigns them as instance variables to your class. This way you can make a struct-like class without having to assign every value manually.
My example has no error checking so it is easier to follow.
class MyStruct(type):
def __call__(cls, *args, **kwargs):
names = cls.__init__.func_code.co_varnames[1:]
self = type.__call__(cls, *args, **kwargs)
for name, value in zip(names, args):
setattr(self , name, value)
for name, value in kwargs.iteritems():
setattr(self , name, value)
return self
Here it is in action.
>>> class MyClass(object):
__metaclass__ = MyStruct
def __init__(self, a, b, c):
pass
>>> my_instance = MyClass(1, 2, 3)
>>> my_instance.a
1
>>>
I posted it on reddit and /u/matchu posted a decorator version which is cleaner. I'd encourage you to use it unless you want to expand the metaclass version.
>>> def init_all_args(fn):
#wraps(fn)
def wrapped_init(self, *args, **kwargs):
names = fn.func_code.co_varnames[1:]
for name, value in zip(names, args):
setattr(self, name, value)
for name, value in kwargs.iteritems():
setattr(self, name, value)
return wrapped_init
>>> class Test(object):
#init_all_args
def __init__(self, a, b):
pass
>>> a = Test(1, 2)
>>> a.a
1
>>>
I wrote a decorator which you can use on any method to make it so that all of the arguments passed in, or any defaults, are assigned to the instance.
def argumentsToAttributes(method):
argumentNames = method.func_code.co_varnames[1:]
# Generate a dictionary of default values:
defaultsDict = {}
defaults = method.func_defaults if method.func_defaults else ()
for i, default in enumerate(defaults, start = len(argumentNames) - len(defaults)):
defaultsDict[argumentNames[i]] = default
def newMethod(self, *args, **kwargs):
# Use the positional arguments.
for name, value in zip(argumentNames, args):
setattr(self, name, value)
# Add the key word arguments. If anything is missing, use the default.
for name in argumentNames[len(args):]:
setattr(self, name, kwargs.get(name, defaultsDict[name]))
# Run whatever else the method needs to do.
method(self, *args, **kwargs)
return newMethod
A quick demonstration. Note that I use a positional argument a, use the default value for b, and a named argument c. I then print all 3 referencing self, to show that they've been properly assigned before the method is entered.
class A(object):
#argumentsToAttributes
def __init__(self, a, b = 'Invisible', c = 'Hello'):
print(self.a)
print(self.b)
print(self.c)
A('Why', c = 'Nothing')
Note that my decorator should work with any method, not just __init__.
I don't see this answer here, so I figure I'll add it since I'm leaning Python right now and just discovered it. The Python tutorial (Python 2 in this case) gives the following simple and effective example:
class Employee:
pass
john = Employee() # Create an empty employee record
# Fill the fields of the record
john.name = 'John Doe'
john.dept = 'computer lab'
john.salary = 1000
That is, an empty class object is created, then instantiated, and the fields are added dynamically.
The up-side to this is its really simple. The downside is it isn't particularly self-documenting (the intended members aren't listed anywhere in the class "definition"), and unset fields can cause problems when accessed. Those two problems can be solved by:
class Employee:
def __init__ (self):
self.name = None # or whatever
self.dept = None
self.salary = None
Now at a glance you can at least see what fields the program will be expecting.
Both are prone to typos, john.slarly = 1000 will succeed. Still, it works.
Here is a solution which uses a class (never instantiated) to hold data. I like that this way involves very little typing and does not require any additional packages etc.
class myStruct:
field1 = "one"
field2 = "2"
You can add more fields later, as needed:
myStruct.field3 = 3
To get the values, the fields are accessed as usual:
>>> myStruct.field1
'one'
Personally, I like this variant too. It extends #dF's answer.
class struct:
def __init__(self, *sequential, **named):
fields = dict(zip(sequential, [None]*len(sequential)), **named)
self.__dict__.update(fields)
def __repr__(self):
return str(self.__dict__)
It supports two modes of initialization (that can be blended):
# Struct with field1, field2, field3 that are initialized to None.
mystruct1 = struct("field1", "field2", "field3")
# Struct with field1, field2, field3 that are initialized according to arguments.
mystruct2 = struct(field1=1, field2=2, field3=3)
Also, it prints nicer:
print(mystruct2)
# Prints: {'field3': 3, 'field1': 1, 'field2': 2}
There is a python package exactly for this purpose. see cstruct2py
cstruct2py is a pure python library for generate python classes from C code and use them to pack and unpack data. The library can parse C headres (structs, unions, enums, and arrays declarations) and emulate them in python. The generated pythonic classes can parse and pack the data.
For example:
typedef struct {
int x;
int y;
} Point;
after generating pythonic class...
p = Point(x=0x1234, y=0x5678)
p.packed == "\x34\x12\x00\x00\x78\x56\x00\x00"
How to use
First we need to generate the pythonic structs:
import cstruct2py
parser = cstruct2py.c2py.Parser()
parser.parse_file('examples/example.h')
Now we can import all names from the C code:
parser.update_globals(globals())
We can also do that directly:
A = parser.parse_string('struct A { int x; int y;};')
Using types and defines from the C code
a = A()
a.x = 45
print a
buf = a.packed
b = A(buf)
print b
c = A('aaaa11112222', 2)
print c
print repr(c)
The output will be:
{'x':0x2d, 'y':0x0}
{'x':0x2d, 'y':0x0}
{'x':0x31316161, 'y':0x32323131}
A('aa111122', x=0x31316161, y=0x32323131)
Clone
For clone cstruct2py run:
git clone https://github.com/st0ky/cstruct2py.git --recursive
Here is a quick and dirty trick:
>>> ms = Warning()
>>> ms.foo = 123
>>> ms.bar = 'akafrit'
How does it works? It just re-use the builtin class Warning (derived from Exception) and use it as it was you own defined class.
The good points are that you do not need to import or define anything first, that "Warning" is a short name, and that it also makes clear you are doing something dirty which should not be used elsewhere than a small script of yours.
By the way, I tried to find something even simpler like ms = object() but could not (this last exemple is not working). If you have one, I am interested.
NamedTuple is comfortable. but there no one shares the performance and storage.
from typing import NamedTuple
import guppy # pip install guppy
import timeit
class User:
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserSlot:
__slots__ = ('name', 'uid')
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserTuple(NamedTuple):
# __slots__ = () # AttributeError: Cannot overwrite NamedTuple attribute __slots__
name: str
uid: int
def get_fn(obj, attr_name: str):
def get():
getattr(obj, attr_name)
return get
if 'memory test':
obj = [User('Carson', 1) for _ in range(1000000)] # Cumulative: 189138883
obj_slot = [UserSlot('Carson', 1) for _ in range(1000000)] # 77718299 <-- winner
obj_namedtuple = [UserTuple('Carson', 1) for _ in range(1000000)] # 85718297
print(guppy.hpy().heap()) # Run this function individually.
"""
Index Count % Size % Cumulative % Kind (class / dict of class)
0 1000000 24 112000000 34 112000000 34 dict of __main__.User
1 1000000 24 64000000 19 176000000 53 __main__.UserTuple
2 1000000 24 56000000 17 232000000 70 __main__.User
3 1000000 24 56000000 17 288000000 87 __main__.UserSlot
...
"""
if 'performance test':
obj = User('Carson', 1)
obj_slot = UserSlot('Carson', 1)
obj_tuple = UserTuple('Carson', 1)
time_normal = min(timeit.repeat(get_fn(obj, 'name'), repeat=20))
print(time_normal) # 0.12550550000000005
time_slot = min(timeit.repeat(get_fn(obj_slot, 'name'), repeat=20))
print(time_slot) # 0.1368690000000008
time_tuple = min(timeit.repeat(get_fn(obj_tuple, 'name'), repeat=20))
print(time_tuple) # 0.16006120000000124
print(time_tuple/time_slot) # 1.1694481584580898 # The slot is almost 17% faster than NamedTuple on Windows. (Python 3.7.7)
If your __dict__ is not using, please choose between __slots__ (higher performance and storage) and NamedTuple (clear for reading and use)
You can review this link(Usage of slots
) to get more __slots__ information.
https://stackoverflow.com/a/32448434/159695 does not work in Python3.
https://stackoverflow.com/a/35993/159695 works in Python3.
And I extends it to add default values.
class myStruct:
def __init__(self, **kwds):
self.x=0
self.__dict__.update(kwds) # Must be last to accept assigned member variable.
def __repr__(self):
args = ['%s=%s' % (k, repr(v)) for (k,v) in vars(self).items()]
return '%s(%s)' % ( self.__class__.__qualname__, ', '.join(args) )
a=myStruct()
b=myStruct(x=3,y='test')
c=myStruct(x='str')
>>> a
myStruct(x=0)
>>> b
myStruct(x=3, y='test')
>>> c
myStruct(x='str')
The following solution to a struct is inspired by the namedtuple implementation and some of the previous answers. However, unlike the namedtuple it is mutable, in it's values, but like the c-style struct immutable in the names/attributes, which a normal class or dict isn't.
_class_template = """\
class {typename}:
def __init__(self, *args, **kwargs):
fields = {field_names!r}
for x in fields:
setattr(self, x, None)
for name, value in zip(fields, args):
setattr(self, name, value)
for name, value in kwargs.items():
setattr(self, name, value)
def __repr__(self):
return str(vars(self))
def __setattr__(self, name, value):
if name not in {field_names!r}:
raise KeyError("invalid name: %s" % name)
object.__setattr__(self, name, value)
"""
def struct(typename, field_names):
class_definition = _class_template.format(
typename = typename,
field_names = field_names)
namespace = dict(__name__='struct_%s' % typename)
exec(class_definition, namespace)
result = namespace[typename]
result._source = class_definition
return result
Usage:
Person = struct('Person', ['firstname','lastname'])
generic = Person()
michael = Person('Michael')
jones = Person(lastname = 'Jones')
In [168]: michael.middlename = 'ben'
Traceback (most recent call last):
File "<ipython-input-168-b31c393c0d67>", line 1, in <module>
michael.middlename = 'ben'
File "<string>", line 19, in __setattr__
KeyError: 'invalid name: middlename'
If you don't have a 3.7 for #dataclass and need mutability, the following code might work for you. It's quite self-documenting and IDE-friendly (auto-complete), prevents writing things twice, is easily extendable and it is very simple to test that all instance variables are completely initialized:
class Params():
def __init__(self):
self.var1 : int = None
self.var2 : str = None
def are_all_defined(self):
for key, value in self.__dict__.items():
assert (value is not None), "instance variable {} is still None".format(key)
return True
params = Params()
params.var1 = 2
params.var2 = 'hello'
assert(params.are_all_defined)
The best way I found to do this was to use a custom dictionary class as explained in this post: https://stackoverflow.com/a/14620633/8484485
If iPython autocompletion support is needed, simply define the dir() function like this:
class AttrDict(dict):
def __init__(self, *args, **kwargs):
super(AttrDict, self).__init__(*args, **kwargs)
self.__dict__ = self
def __dir__(self):
return self.keys()
You then define your pseudo struct like so: (this one is nested)
my_struct=AttrDict ({
'com1':AttrDict ({
'inst':[0x05],
'numbytes':2,
'canpayload':False,
'payload':None
})
})
You can then access the values inside my_struct like this:
print(my_struct.com1.inst)
=>[5]
The cleanest way I can think of is to use a class decorator that lets you declare a static class and rewrite it to act as a struct with normal, named properties:
from as_struct import struct
#struct
class Product():
name = 'unknown product'
quantity = -1
sku = '-'
# create instance
p = Product('plush toy', sku='12-345-6789')
# check content:
p.name # plush toy
p.quantity # -1
p.sku # 12-345-6789
Using the following decorator code:
def struct(struct_class):
# create a new init
def struct_init(self, *args, **kwargs):
i = 0 # we really don't need enumerate() here...
for value in args:
name = member_names[i]
default_value = member_values[i]
setattr(self, name, value if value is not None else default_value)
i += 1 # ...we just need to inc an int
for key,value in kwargs.items():
i = member_names.index(key)
default_value = member_values[i]
setattr(self, key, value if value is not None else default_value)
# extract the struct members
member_names = []
member_values = []
for attr_name in dir(struct_class):
if not attr_name.startswith('_'):
value = getattr(struct_class, attr_name)
if not callable(value):
member_names.append(attr_name)
member_values.append(value)
# rebind and return
struct_class.init = struct_init
return struct_class
Which works by taking the class, extracting the field names and their default values, then rewriting the class's __init__ function to set self attributes based on knowing which argument index maps to which property name.
I think Python structure dictionary is suitable for this requirement.
d = dict{}
d[field1] = field1
d[field2] = field2
d[field2] = field3
Extending #gz.'s (generally superior to this one) answer, for a quick and dirty namedtuple structure we can do:
import collections
x = collections.namedtuple('foobar', 'foo bar')(foo=1,bar=2)
y = collections.namedtuple('foobar', 'foo bar')(foo=3,bar=4)
print(x,y)
>foobar(foo=1, bar=2) foobar(foo=3, bar=4)