Related
I have an abstract base class GameNodeState that contains a Type enum:
import abc
import enum
class GameNodeState(metaclass=abc.ABCMeta):
class Type(enum.Enum):
INIT = enum.auto()
INTERMEDIATE = enum.auto()
END = enum.auto()
The names in the enum are generic because they must make sense for any subclass of GameNodeState. But when I subclass GameNodeState, as GameState and RoundState, I would like to be able to add concrete aliases to the members of GameNodeState.Type if the enum is accessed through the subclass. For example, if the GameState subclass aliases INTERMEDIATE as ROUND and RoundState aliases INTERMEDIATE as TURN, I would like the following behaviour:
>>> GameNodeState.Type.INTERMEDIATE
<Type.INTERMEDIATE: 2>
>>> RoundState.Type.TURN
<Type.INTERMEDIATE: 2>
>>> RoundState.Type.INTERMEDIATE
<Type.INTERMEDIATE: 2>
>>> GameNodeState.Type.TURN
AttributeError: TURN
My first thought was this:
class GameState(GameNodeState):
class Type(GameNodeState.Type):
ROUND = GameNodeState.Type.INTERMEDIATE.value
class RoundState(GameNodeState):
class Type(GameNodeState.Type):
TURN = GameNodeState.Type.INTERMEDIATE.value
But enums can't be subclassed.
Note: there are obviously more attributes and methods in the GameNodeState hierarchy, I stripped it down to the bare minimum here to focus on this particular thing.
Refinement
(Original solution below.)
I've extracted an intermediate concept from the code above, namely the concept of enum union. This can be used to obtain the behaviour above, and is also useful in other contexts too. The code can be foud here, and I've asked a Code Review question.
I'll add the code here as well for reference:
import enum
import itertools as itt
from functools import reduce
import operator
from typing import Literal, Union
import more_itertools as mitt
AUTO = object()
class UnionEnumMeta(enum.EnumMeta):
"""
The metaclass for enums which are the union of several sub-enums.
Union enums have the _subenums_ attribute which is a tuple of the enums forming the
union.
"""
#classmethod
def make_union(
mcs, *subenums: enum.EnumMeta, name: Union[str, Literal[AUTO], None] = AUTO
) -> enum.EnumMeta:
"""
Create an enum whose set of members is the union of members of several enums.
Order matters: where two members in the union have the same value, they will
be considered as aliases of each other, and the one appearing in the first
enum in the sequence will be used as the canonical members (the aliases will
be associated to this enum member).
:param subenums: Sequence of sub-enums to make a union of.
:param name: Name to use for the enum class. AUTO will result in a combination
of the names of all subenums, None will result in "UnionEnum".
:return: An enum class which is the union of the given subenums.
"""
subenums = mcs._normalize_subenums(subenums)
class UnionEnum(enum.Enum, metaclass=mcs):
pass
union_enum = UnionEnum
union_enum._subenums_ = subenums
if duplicate_names := reduce(
set.intersection, (set(subenum.__members__) for subenum in subenums)
):
raise ValueError(
f"Found duplicate member names in enum union: {duplicate_names}"
)
# If aliases are defined, the canonical member will be the one that appears
# first in the sequence of subenums.
# dict union keeps last key so we have to do it in reverse:
union_enum._value2member_map_ = value2member_map = reduce(
operator.or_, (subenum._value2member_map_ for subenum in reversed(subenums))
)
# union of the _member_map_'s but using the canonical member always:
union_enum._member_map_ = member_map = {
name: value2member_map[member.value]
for name, member in itt.chain.from_iterable(
subenum._member_map_.items() for subenum in subenums
)
}
# only include canonical aliases in _member_names_
union_enum._member_names_ = list(
mitt.unique_everseen(
itt.chain.from_iterable(subenum._member_names_ for subenum in subenums),
key=member_map.__getitem__,
)
)
if name is AUTO:
name = (
"".join(subenum.__name__.removesuffix("Enum") for subenum in subenums)
+ "UnionEnum"
)
UnionEnum.__name__ = name
elif name is not None:
UnionEnum.__name__ = name
return union_enum
def __repr__(cls):
return f"<union of {', '.join(map(str, cls._subenums_))}>"
def __instancecheck__(cls, instance):
return any(isinstance(instance, subenum) for subenum in cls._subenums_)
#classmethod
def _normalize_subenums(mcs, subenums):
"""Remove duplicate subenums and flatten nested unions"""
# we will need to collapse at most one level of nesting, with the inductive
# hypothesis that any previous unions are already flat
subenums = mitt.collapse(
(e._subenums_ if isinstance(e, mcs) else e for e in subenums),
base_type=enum.EnumMeta,
)
subenums = mitt.unique_everseen(subenums)
return tuple(subenums)
def enum_union(*enums, **kwargs):
return UnionEnumMeta.make_union(*enums, **kwargs)
Once we have that, we can just define the extend_enum decorator to compute the union of the base enum and the enum "extension", which will result in the desired behaviour:
def extend_enum(base_enum):
def decorator(extension_enum):
return enum_union(base_enum, extension_enum)
return decorator
Usage:
class GameNodeState(metaclass=abc.ABCMeta):
class Type(enum.Enum):
INIT = enum.auto()
INTERMEDIATE = enum.auto()
END = enum.auto()
class RoundState(GameNodeState):
#extend_enum(GameNodeState.Type)
class Type(enum.Enum):
TURN = GameNodeState.Type.INTERMEDIATE.value
class GameState(GameNodeState):
#extend_enum(GameNodeState.Type)
class Type(enum.Enum):
ROUND = GameNodeState.Type.INTERMEDIATE.value
Now all of the examples above produce the same output (plus the added instance check, i.e. isinstance(RoundState.Type.TURN, RoundState.Type) returns True).
I think this is a cleaner solution because it doesn't involve mucking around with descriptors; it doesn't need to know anything about the owner class (this works just as well with top-level classes).
Attribute lookup through subclasses and instances of GameNodeState should automatically link to the correct "extension" (i.e., union), as long as the extension enum is added with the same name as for the GameNodeState superclass so that it hides the original definition.
Original
Not sure how bad of an idea this is, but here is a solution using a descriptor wrapped around the enum that gets the set of aliases based on the class from which it is being accessed.
class ExtensibleClassEnum:
class ExtensionWrapperMeta(enum.EnumMeta):
#classmethod
def __prepare__(mcs, name, bases):
# noinspection PyTypeChecker
classdict: enum._EnumDict = super().__prepare__(name, bases)
classdict["_ignore_"] = ["base_descriptor", "extension_enum"]
return classdict
# noinspection PyProtectedMember
def __new__(mcs, cls, bases, classdict):
base_descriptor = classdict.pop("base_descriptor")
extension_enum = classdict.pop("extension_enum")
wrapper_enum = super().__new__(mcs, cls, bases, classdict)
wrapper_enum.base_descriptor = base_descriptor
wrapper_enum.extension_enum = extension_enum
base, extension = base_descriptor.base_enum, extension_enum
if set(base._member_map_.keys()) & set(extension._member_map_.keys()):
raise ValueError("Found duplicate names in extension")
# dict union keeps last key so we have to do it in reverse:
wrapper_enum._value2member_map_ = (
extension._value2member_map_ | base._value2member_map_
)
# union of both _member_map_'s but using the canonical member always:
wrapper_enum._member_map_ = {
name: wrapper_enum._value2member_map_[member.value]
for name, member in itertools.chain(
base._member_map_.items(), extension._member_map_.items()
)
}
# aliases shouldn't appear in _member_names_
wrapper_enum._member_names_ = list(
m.name for m in wrapper_enum._value2member_map_.values()
)
return wrapper_enum
def __repr__(self):
# have to use vars() to avoid triggering the descriptor
base_descriptor = vars(self)["base_descriptor"]
return (
f"<extension wrapper enum for {base_descriptor.base_enum}"
f" in {base_descriptor._extension2owner[self]}>"
)
def __init__(self, base_enum):
if not issubclass(base_enum, enum.Enum):
raise TypeError(base_enum)
self.base_enum = base_enum
# The user won't be able to retrieve the descriptor object itself, just
# the enum, so we have to forward calls to register_extension:
self.base_enum.register_extension = staticmethod(self.register_extension)
# mapping of owner class -> extension for subclasses that define an extension
self._extensions: Dict[Type, ExtensibleClassEnum.ExtensionWrapperMeta] = {}
# reverse mapping
self._extension2owner: Dict[ExtensibleClassEnum.ExtensionWrapperMeta, Type] = {}
# add the base enum as the base extension via __set_name__:
self._pending_extension = base_enum
#property
def base_owner(self):
# will be initialised after __set_name__ is called with base owner
return self._extension2owner[self.base_enum]
def __set_name__(self, owner, name):
# step 2 of register_extension: determine the class that defined it
self._extensions[owner] = self._pending_extension
self._extension2owner[self._pending_extension] = owner
del self._pending_extension
def __get__(self, instance, owner):
# Only compute extensions once:
if owner in self._extensions:
return self._extensions[owner]
# traverse in MRO until we find the closest supertype defining an extension
for supertype in owner.__mro__:
if supertype in self._extensions:
extension = self._extensions[supertype]
break
else:
raise TypeError(f"{owner} is not a subclass of {self.base_owner}")
# Cache the result
self._extensions[owner] = extension
return extension
def make_extension(self, extension: enum.EnumMeta):
class ExtensionWrapperEnum(
enum.Enum, metaclass=ExtensibleClassEnum.ExtensionWrapperMeta
):
base_descriptor = self
extension_enum = extension
return ExtensionWrapperEnum
def register_extension(self, extension_enum):
"""Decorator for enum extensions"""
# need a way to determine owner class
# add a temporary attribute that we will use when __set_name__ is called:
if hasattr(self, "_pending_extension"):
# __set_name__ not called after the previous call to register_extension
raise RuntimeError(
"An extension was created outside of a class definition",
self._pending_extension,
)
self._pending_extension = self.make_extension(extension_enum)
return self
Usage would be as follows:
class GameNodeState(metaclass=abc.ABCMeta):
#ExtensibleClassEnum
class Type(enum.Enum):
INIT = enum.auto()
INTERMEDIATE = enum.auto()
END = enum.auto()
class RoundState(GameNodeState):
#GameNodeState.Type.register_extension
class Type(enum.Enum):
TURN = GameNodeState.Type.INTERMEDIATE.value
class GameState(GameNodeState):
#GameNodeState.Type.register_extension
class Type(enum.Enum):
ROUND = GameNodeState.Type.INTERMEDIATE.value
Then:
>>> (RoundState.Type.TURN
... == RoundState.Type.INTERMEDIATE
... == GameNodeState.Type.INTERMEDIATE
... == GameState.Type.INTERMEDIATE
... == GameState.Type.ROUND)
...
True
>>> RoundState.Type.__members__
mappingproxy({'INIT': <Type.INIT: 1>,
'INTERMEDIATE': <Type.INTERMEDIATE: 2>,
'END': <Type.END: 3>,
'TURN': <Type.INTERMEDIATE: 2>})
>>> list(RoundState.Type)
[<Type.INTERMEDIATE: 2>, <Type.INIT: 1>, <Type.END: 3>]
>>> GameNodeState.Type.TURN
Traceback (most recent call last):
...
File "C:\Program Files\Python39\lib\enum.py", line 352, in __getattr__
raise AttributeError(name) from None
AttributeError: TURN
Is there a way to conveniently define a C-like structure in Python? I'm tired of writing stuff like:
class MyStruct():
def __init__(self, field1, field2, field3):
self.field1 = field1
self.field2 = field2
self.field3 = field3
Update: Data Classes
With the introduction of Data Classes in Python 3.7 we get very close.
The following example is similar to the NamedTuple example below, but the resulting object is mutable and it allows for default values.
from dataclasses import dataclass
#dataclass
class Point:
x: float
y: float
z: float = 0.0
p = Point(1.5, 2.5)
print(p) # Point(x=1.5, y=2.5, z=0.0)
This plays nicely with the new typing module in case you want to use more specific type annotations.
I've been waiting desperately for this! If you ask me, Data Classes and the new NamedTuple declaration, combined with the typing module are a godsend!
Improved NamedTuple declaration
Since Python 3.6 it became quite simple and beautiful (IMHO), as long as you can live with immutability.
A new way of declaring NamedTuples was introduced, which allows for type annotations as well:
from typing import NamedTuple
class User(NamedTuple):
name: str
class MyStruct(NamedTuple):
foo: str
bar: int
baz: list
qux: User
my_item = MyStruct('foo', 0, ['baz'], User('peter'))
print(my_item) # MyStruct(foo='foo', bar=0, baz=['baz'], qux=User(name='peter'))
Use a named tuple, which was added to the collections module in the standard library in Python 2.6. It's also possible to use Raymond Hettinger's named tuple recipe if you need to support Python 2.4.
It's nice for your basic example, but also covers a bunch of edge cases you might run into later as well. Your fragment above would be written as:
from collections import namedtuple
MyStruct = namedtuple("MyStruct", "field1 field2 field3")
The newly created type can be used like this:
m = MyStruct("foo", "bar", "baz")
You can also use named arguments:
m = MyStruct(field1="foo", field2="bar", field3="baz")
You can use a tuple for a lot of things where you would use a struct in C (something like x,y coordinates or RGB colors for example).
For everything else you can use dictionary, or a utility class like this one:
>>> class Bunch:
... def __init__(self, **kwds):
... self.__dict__.update(kwds)
...
>>> mystruct = Bunch(field1=value1, field2=value2)
I think the "definitive" discussion is here, in the published version of the Python Cookbook.
Perhaps you are looking for Structs without constructors:
class Sample:
name = ''
average = 0.0
values = None # list cannot be initialized here!
s1 = Sample()
s1.name = "sample 1"
s1.values = []
s1.values.append(1)
s1.values.append(2)
s1.values.append(3)
s2 = Sample()
s2.name = "sample 2"
s2.values = []
s2.values.append(4)
for v in s1.values: # prints 1,2,3 --> OK.
print v
print "***"
for v in s2.values: # prints 4 --> OK.
print v
How about a dictionary?
Something like this:
myStruct = {'field1': 'some val', 'field2': 'some val'}
Then you can use this to manipulate values:
print myStruct['field1']
myStruct['field2'] = 'some other values'
And the values don't have to be strings. They can be pretty much any other object.
dF: that's pretty cool... I didn't
know that I could access the fields in
a class using dict.
Mark: the situations that I wish I had
this are precisely when I want a tuple
but nothing as "heavy" as a
dictionary.
You can access the fields of a class using a dictionary because the fields of a class, its methods and all its properties are stored internally using dicts (at least in CPython).
...Which leads us to your second comment. Believing that Python dicts are "heavy" is an extremely non-pythonistic concept. And reading such comments kills my Python Zen. That's not good.
You see, when you declare a class you are actually creating a pretty complex wrapper around a dictionary - so, if anything, you are adding more overhead than by using a simple dictionary. An overhead which, by the way, is meaningless in any case. If you are working on performance critical applications, use C or something.
I would also like to add a solution that uses slots:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
Definitely check the documentation for slots but a quick explanation of slots is that it is python's way of saying: "If you can lock these attributes and only these attributes into the class such that you commit that you will not add any new attributes once the class is instantiated (yes you can add new attributes to a class instance, see example below) then I will do away with the large memory allocation that allows for adding new attributes to a class instance and use just what I need for these slotted attributes".
Example of adding attributes to class instance (thus not using slots):
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
print(p1.z)
Output: 8
Example of trying to add attributes to class instance where slots was used:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
Output: AttributeError: 'Point' object has no attribute 'z'
This can effectively works as a struct and uses less memory than a class (like a struct would, although I have not researched exactly how much). It is recommended to use slots if you will be creating a large amount of instances of the object and do not need to add attributes. A point object is a good example of this as it is likely that one may instantiate many points to describe a dataset.
You can subclass the C structure that is available in the standard library. The ctypes module provides a Structure class. The example from the docs:
>>> from ctypes import *
>>> class POINT(Structure):
... _fields_ = [("x", c_int),
... ("y", c_int)]
...
>>> point = POINT(10, 20)
>>> print point.x, point.y
10 20
>>> point = POINT(y=5)
>>> print point.x, point.y
0 5
>>> POINT(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: too many initializers
>>>
>>> class RECT(Structure):
... _fields_ = [("upperleft", POINT),
... ("lowerright", POINT)]
...
>>> rc = RECT(point)
>>> print rc.upperleft.x, rc.upperleft.y
0 5
>>> print rc.lowerright.x, rc.lowerright.y
0 0
>>>
You can also pass the init parameters to the instance variables by position
# Abstract struct class
class Struct:
def __init__ (self, *argv, **argd):
if len(argd):
# Update by dictionary
self.__dict__.update (argd)
else:
# Update by position
attrs = filter (lambda x: x[0:2] != "__", dir(self))
for n in range(len(argv)):
setattr(self, attrs[n], argv[n])
# Specific class
class Point3dStruct (Struct):
x = 0
y = 0
z = 0
pt1 = Point3dStruct()
pt1.x = 10
print pt1.x
print "-"*10
pt2 = Point3dStruct(5, 6)
print pt2.x, pt2.y
print "-"*10
pt3 = Point3dStruct (x=1, y=2, z=3)
print pt3.x, pt3.y, pt3.z
print "-"*10
Whenever I need an "instant data object that also behaves like a dictionary" (I don't think of C structs!), I think of this cute hack:
class Map(dict):
def __init__(self, **kwargs):
super(Map, self).__init__(**kwargs)
self.__dict__ = self
Now you can just say:
struct = Map(field1='foo', field2='bar', field3=42)
self.assertEquals('bar', struct.field2)
self.assertEquals(42, struct['field3'])
Perfectly handy for those times when you need a "data bag that's NOT a class", and for when namedtuples are incomprehensible...
Some the answers here are massively elaborate. The simplest option I've found is (from: http://norvig.com/python-iaq.html):
class Struct:
"A structure that can have any fields defined."
def __init__(self, **entries): self.__dict__.update(entries)
Initialising:
>>> options = Struct(answer=42, linelen=80, font='courier')
>>> options.answer
42
adding more:
>>> options.cat = "dog"
>>> options.cat
dog
edit: Sorry didn't see this example already further down.
You access C-Style struct in python in following way.
class cstruct:
var_i = 0
var_f = 0.0
var_str = ""
if you just want use object of cstruct
obj = cstruct()
obj.var_i = 50
obj.var_f = 50.00
obj.var_str = "fifty"
print "cstruct: obj i=%d f=%f s=%s" %(obj.var_i, obj.var_f, obj.var_str)
if you want to create an array of objects of cstruct
obj_array = [cstruct() for i in range(10)]
obj_array[0].var_i = 10
obj_array[0].var_f = 10.00
obj_array[0].var_str = "ten"
#go ahead and fill rest of array instaces of struct
#print all the value
for i in range(10):
print "cstruct: obj_array i=%d f=%f s=%s" %(obj_array[i].var_i, obj_array[i].var_f, obj_array[i].var_str)
Note:
instead of 'cstruct' name, please use your struct name
instead of var_i, var_f, var_str, please define your structure's member variable.
This might be a bit late but I made a solution using Python Meta-Classes (decorator version below too).
When __init__ is called during run time, it grabs each of the arguments and their value and assigns them as instance variables to your class. This way you can make a struct-like class without having to assign every value manually.
My example has no error checking so it is easier to follow.
class MyStruct(type):
def __call__(cls, *args, **kwargs):
names = cls.__init__.func_code.co_varnames[1:]
self = type.__call__(cls, *args, **kwargs)
for name, value in zip(names, args):
setattr(self , name, value)
for name, value in kwargs.iteritems():
setattr(self , name, value)
return self
Here it is in action.
>>> class MyClass(object):
__metaclass__ = MyStruct
def __init__(self, a, b, c):
pass
>>> my_instance = MyClass(1, 2, 3)
>>> my_instance.a
1
>>>
I posted it on reddit and /u/matchu posted a decorator version which is cleaner. I'd encourage you to use it unless you want to expand the metaclass version.
>>> def init_all_args(fn):
#wraps(fn)
def wrapped_init(self, *args, **kwargs):
names = fn.func_code.co_varnames[1:]
for name, value in zip(names, args):
setattr(self, name, value)
for name, value in kwargs.iteritems():
setattr(self, name, value)
return wrapped_init
>>> class Test(object):
#init_all_args
def __init__(self, a, b):
pass
>>> a = Test(1, 2)
>>> a.a
1
>>>
I wrote a decorator which you can use on any method to make it so that all of the arguments passed in, or any defaults, are assigned to the instance.
def argumentsToAttributes(method):
argumentNames = method.func_code.co_varnames[1:]
# Generate a dictionary of default values:
defaultsDict = {}
defaults = method.func_defaults if method.func_defaults else ()
for i, default in enumerate(defaults, start = len(argumentNames) - len(defaults)):
defaultsDict[argumentNames[i]] = default
def newMethod(self, *args, **kwargs):
# Use the positional arguments.
for name, value in zip(argumentNames, args):
setattr(self, name, value)
# Add the key word arguments. If anything is missing, use the default.
for name in argumentNames[len(args):]:
setattr(self, name, kwargs.get(name, defaultsDict[name]))
# Run whatever else the method needs to do.
method(self, *args, **kwargs)
return newMethod
A quick demonstration. Note that I use a positional argument a, use the default value for b, and a named argument c. I then print all 3 referencing self, to show that they've been properly assigned before the method is entered.
class A(object):
#argumentsToAttributes
def __init__(self, a, b = 'Invisible', c = 'Hello'):
print(self.a)
print(self.b)
print(self.c)
A('Why', c = 'Nothing')
Note that my decorator should work with any method, not just __init__.
I don't see this answer here, so I figure I'll add it since I'm leaning Python right now and just discovered it. The Python tutorial (Python 2 in this case) gives the following simple and effective example:
class Employee:
pass
john = Employee() # Create an empty employee record
# Fill the fields of the record
john.name = 'John Doe'
john.dept = 'computer lab'
john.salary = 1000
That is, an empty class object is created, then instantiated, and the fields are added dynamically.
The up-side to this is its really simple. The downside is it isn't particularly self-documenting (the intended members aren't listed anywhere in the class "definition"), and unset fields can cause problems when accessed. Those two problems can be solved by:
class Employee:
def __init__ (self):
self.name = None # or whatever
self.dept = None
self.salary = None
Now at a glance you can at least see what fields the program will be expecting.
Both are prone to typos, john.slarly = 1000 will succeed. Still, it works.
Here is a solution which uses a class (never instantiated) to hold data. I like that this way involves very little typing and does not require any additional packages etc.
class myStruct:
field1 = "one"
field2 = "2"
You can add more fields later, as needed:
myStruct.field3 = 3
To get the values, the fields are accessed as usual:
>>> myStruct.field1
'one'
Personally, I like this variant too. It extends #dF's answer.
class struct:
def __init__(self, *sequential, **named):
fields = dict(zip(sequential, [None]*len(sequential)), **named)
self.__dict__.update(fields)
def __repr__(self):
return str(self.__dict__)
It supports two modes of initialization (that can be blended):
# Struct with field1, field2, field3 that are initialized to None.
mystruct1 = struct("field1", "field2", "field3")
# Struct with field1, field2, field3 that are initialized according to arguments.
mystruct2 = struct(field1=1, field2=2, field3=3)
Also, it prints nicer:
print(mystruct2)
# Prints: {'field3': 3, 'field1': 1, 'field2': 2}
There is a python package exactly for this purpose. see cstruct2py
cstruct2py is a pure python library for generate python classes from C code and use them to pack and unpack data. The library can parse C headres (structs, unions, enums, and arrays declarations) and emulate them in python. The generated pythonic classes can parse and pack the data.
For example:
typedef struct {
int x;
int y;
} Point;
after generating pythonic class...
p = Point(x=0x1234, y=0x5678)
p.packed == "\x34\x12\x00\x00\x78\x56\x00\x00"
How to use
First we need to generate the pythonic structs:
import cstruct2py
parser = cstruct2py.c2py.Parser()
parser.parse_file('examples/example.h')
Now we can import all names from the C code:
parser.update_globals(globals())
We can also do that directly:
A = parser.parse_string('struct A { int x; int y;};')
Using types and defines from the C code
a = A()
a.x = 45
print a
buf = a.packed
b = A(buf)
print b
c = A('aaaa11112222', 2)
print c
print repr(c)
The output will be:
{'x':0x2d, 'y':0x0}
{'x':0x2d, 'y':0x0}
{'x':0x31316161, 'y':0x32323131}
A('aa111122', x=0x31316161, y=0x32323131)
Clone
For clone cstruct2py run:
git clone https://github.com/st0ky/cstruct2py.git --recursive
Here is a quick and dirty trick:
>>> ms = Warning()
>>> ms.foo = 123
>>> ms.bar = 'akafrit'
How does it works? It just re-use the builtin class Warning (derived from Exception) and use it as it was you own defined class.
The good points are that you do not need to import or define anything first, that "Warning" is a short name, and that it also makes clear you are doing something dirty which should not be used elsewhere than a small script of yours.
By the way, I tried to find something even simpler like ms = object() but could not (this last exemple is not working). If you have one, I am interested.
NamedTuple is comfortable. but there no one shares the performance and storage.
from typing import NamedTuple
import guppy # pip install guppy
import timeit
class User:
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserSlot:
__slots__ = ('name', 'uid')
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserTuple(NamedTuple):
# __slots__ = () # AttributeError: Cannot overwrite NamedTuple attribute __slots__
name: str
uid: int
def get_fn(obj, attr_name: str):
def get():
getattr(obj, attr_name)
return get
if 'memory test':
obj = [User('Carson', 1) for _ in range(1000000)] # Cumulative: 189138883
obj_slot = [UserSlot('Carson', 1) for _ in range(1000000)] # 77718299 <-- winner
obj_namedtuple = [UserTuple('Carson', 1) for _ in range(1000000)] # 85718297
print(guppy.hpy().heap()) # Run this function individually.
"""
Index Count % Size % Cumulative % Kind (class / dict of class)
0 1000000 24 112000000 34 112000000 34 dict of __main__.User
1 1000000 24 64000000 19 176000000 53 __main__.UserTuple
2 1000000 24 56000000 17 232000000 70 __main__.User
3 1000000 24 56000000 17 288000000 87 __main__.UserSlot
...
"""
if 'performance test':
obj = User('Carson', 1)
obj_slot = UserSlot('Carson', 1)
obj_tuple = UserTuple('Carson', 1)
time_normal = min(timeit.repeat(get_fn(obj, 'name'), repeat=20))
print(time_normal) # 0.12550550000000005
time_slot = min(timeit.repeat(get_fn(obj_slot, 'name'), repeat=20))
print(time_slot) # 0.1368690000000008
time_tuple = min(timeit.repeat(get_fn(obj_tuple, 'name'), repeat=20))
print(time_tuple) # 0.16006120000000124
print(time_tuple/time_slot) # 1.1694481584580898 # The slot is almost 17% faster than NamedTuple on Windows. (Python 3.7.7)
If your __dict__ is not using, please choose between __slots__ (higher performance and storage) and NamedTuple (clear for reading and use)
You can review this link(Usage of slots
) to get more __slots__ information.
https://stackoverflow.com/a/32448434/159695 does not work in Python3.
https://stackoverflow.com/a/35993/159695 works in Python3.
And I extends it to add default values.
class myStruct:
def __init__(self, **kwds):
self.x=0
self.__dict__.update(kwds) # Must be last to accept assigned member variable.
def __repr__(self):
args = ['%s=%s' % (k, repr(v)) for (k,v) in vars(self).items()]
return '%s(%s)' % ( self.__class__.__qualname__, ', '.join(args) )
a=myStruct()
b=myStruct(x=3,y='test')
c=myStruct(x='str')
>>> a
myStruct(x=0)
>>> b
myStruct(x=3, y='test')
>>> c
myStruct(x='str')
The following solution to a struct is inspired by the namedtuple implementation and some of the previous answers. However, unlike the namedtuple it is mutable, in it's values, but like the c-style struct immutable in the names/attributes, which a normal class or dict isn't.
_class_template = """\
class {typename}:
def __init__(self, *args, **kwargs):
fields = {field_names!r}
for x in fields:
setattr(self, x, None)
for name, value in zip(fields, args):
setattr(self, name, value)
for name, value in kwargs.items():
setattr(self, name, value)
def __repr__(self):
return str(vars(self))
def __setattr__(self, name, value):
if name not in {field_names!r}:
raise KeyError("invalid name: %s" % name)
object.__setattr__(self, name, value)
"""
def struct(typename, field_names):
class_definition = _class_template.format(
typename = typename,
field_names = field_names)
namespace = dict(__name__='struct_%s' % typename)
exec(class_definition, namespace)
result = namespace[typename]
result._source = class_definition
return result
Usage:
Person = struct('Person', ['firstname','lastname'])
generic = Person()
michael = Person('Michael')
jones = Person(lastname = 'Jones')
In [168]: michael.middlename = 'ben'
Traceback (most recent call last):
File "<ipython-input-168-b31c393c0d67>", line 1, in <module>
michael.middlename = 'ben'
File "<string>", line 19, in __setattr__
KeyError: 'invalid name: middlename'
If you don't have a 3.7 for #dataclass and need mutability, the following code might work for you. It's quite self-documenting and IDE-friendly (auto-complete), prevents writing things twice, is easily extendable and it is very simple to test that all instance variables are completely initialized:
class Params():
def __init__(self):
self.var1 : int = None
self.var2 : str = None
def are_all_defined(self):
for key, value in self.__dict__.items():
assert (value is not None), "instance variable {} is still None".format(key)
return True
params = Params()
params.var1 = 2
params.var2 = 'hello'
assert(params.are_all_defined)
The best way I found to do this was to use a custom dictionary class as explained in this post: https://stackoverflow.com/a/14620633/8484485
If iPython autocompletion support is needed, simply define the dir() function like this:
class AttrDict(dict):
def __init__(self, *args, **kwargs):
super(AttrDict, self).__init__(*args, **kwargs)
self.__dict__ = self
def __dir__(self):
return self.keys()
You then define your pseudo struct like so: (this one is nested)
my_struct=AttrDict ({
'com1':AttrDict ({
'inst':[0x05],
'numbytes':2,
'canpayload':False,
'payload':None
})
})
You can then access the values inside my_struct like this:
print(my_struct.com1.inst)
=>[5]
The cleanest way I can think of is to use a class decorator that lets you declare a static class and rewrite it to act as a struct with normal, named properties:
from as_struct import struct
#struct
class Product():
name = 'unknown product'
quantity = -1
sku = '-'
# create instance
p = Product('plush toy', sku='12-345-6789')
# check content:
p.name # plush toy
p.quantity # -1
p.sku # 12-345-6789
Using the following decorator code:
def struct(struct_class):
# create a new init
def struct_init(self, *args, **kwargs):
i = 0 # we really don't need enumerate() here...
for value in args:
name = member_names[i]
default_value = member_values[i]
setattr(self, name, value if value is not None else default_value)
i += 1 # ...we just need to inc an int
for key,value in kwargs.items():
i = member_names.index(key)
default_value = member_values[i]
setattr(self, key, value if value is not None else default_value)
# extract the struct members
member_names = []
member_values = []
for attr_name in dir(struct_class):
if not attr_name.startswith('_'):
value = getattr(struct_class, attr_name)
if not callable(value):
member_names.append(attr_name)
member_values.append(value)
# rebind and return
struct_class.init = struct_init
return struct_class
Which works by taking the class, extracting the field names and their default values, then rewriting the class's __init__ function to set self attributes based on knowing which argument index maps to which property name.
I think Python structure dictionary is suitable for this requirement.
d = dict{}
d[field1] = field1
d[field2] = field2
d[field2] = field3
Extending #gz.'s (generally superior to this one) answer, for a quick and dirty namedtuple structure we can do:
import collections
x = collections.namedtuple('foobar', 'foo bar')(foo=1,bar=2)
y = collections.namedtuple('foobar', 'foo bar')(foo=3,bar=4)
print(x,y)
>foobar(foo=1, bar=2) foobar(foo=3, bar=4)
I want to replace string literals in my code, as I want to minimize risk of typos, especially in dict key sets:
a['typoh'] = 'this is bad'
I don't want to type things in twice (risk of a missed typo on the value)
I want it to be "trackable" by various IDEs (i.e. click thru to see where it is defined and escape completion).
Enums are out: 'E.a.name' to get 'a' is dumb.
I have been told this can be done with slots, but I can't figure out how without a little trickery. I can think of a few ways below:
This is an unacceptable answer:
class TwiceIsNotNice(object):
this_is_a_string = 'this_is_a_string'
... (five thousand string constants in)
this_has_a_hard_to_spot_typographical_error =
'this_has_a_had_to_spot_typographical_error'
... (five thousand more string constants)
A clear but annoying way is with a "Stringspace" class/object where the attributes are set via a string list passed in. This solves the minimized typo risk, is VERY easy to read, but has neither IDE trackability nor autocompletion. It's okay, but makes people complain (please don't complain here, I am simply showing how it could be done):
string_consts = Stringspace('a', 'b',...,'asdfasdfasdf')
print(string_consts.a)
... where:
class Stringspace(object):
def __init__(self, *strlist):
for s in strlist:
setattr(self, s, s)
Another way is to define a class using a sentinel object, setting the value in a post phase. This is okay, is trackable, presents itself as an actual class, allows for aliases, etc. But it requires an annoying extra call at the end of the class:
same = object()
class StrList(object):
this_is_a_strval = same
this_is_another_strval = same
this_gets_aliased = "to something else"
# This would of course could become a function
for attr in dir(StrList):
if getattr(StrList, attr) is same:
setattr(StrList, attr, attr)
print(StrList.a)
If this is what the slot magic is supposedly about, then I am disappointed, as one would have to actually instantiate an object:
class SlotEnum(object):
__slots__ = []
def __init__(self):
for k in self.__slots__:
setattr(self, k, k)
class Foo(SlotEnum):
__slots__ = ['a', 'b']
foo_enum_OBJECT = Foo()
print(foo_enum_OBJECT.a)
Enums are out: E.a.name to get a is dumb.
from enum import Enum, auto
class StrEnum(str, Enum):
"base class for Enum members to be strings matching the member's name"
def __repr__(self):
return '<%s.%s>' % (self.__class__.__name__, self.name)
def __str__(self):
return self.name
class E(StrEnum):
a = auto()
this_is_a_string = auto()
no_typo_here = auto()
>>> print(repr(E.a))
<E.a>
>>> print(E.a)
a
>>> print('the answer is: %s!' % E.a)
the answer is: a!
I found one solution at this external link using a custom meta class, for your class containing the string member variables:
Step 1 of 2: The custom meta class can be defined like this:
class MetaForMyStrConstants(type):
def __new__(metacls, cls, bases, classdict):
object_attrs = set(dir(type(cls, (object,), {})))
simple_enum_cls = super().__new__(metacls, cls, bases, classdict)
simple_enum_cls._member_names_ = set(classdict.keys()) - object_attrs
non_members = set()
for attr in simple_enum_cls._member_names_:
if attr.startswith('_') and attr.endswith('_'):
non_members.add(attr)
else:
setattr(simple_enum_cls, attr, attr)
simple_enum_cls._member_names_.difference_update(non_members)
return simple_enum_cls
Step 2 of 2: The class defining your strings can be defined like this (with dummy values, eg, empty tuples):
class MyStrConstants(metaclass=MetaForMyStrConstants):
ONE_LONG_STR = ()
ANOTHER_LONG_STR = ()
THE_REAL_LONGEST_STR = ()
Testing it out:
print (MyStrConstants.ONE_LONG_STR)
print (MyStrConstants.ANOTHER_LONG_STR)
print (MyStrConstants.THE_REAL_LONGEST_STR)
Output:
ONE_LONG_STR
ANOTHER_LONG_STR
THE_REAL_LONGEST_STR
Having a simple Python class like this:
class Spam(object):
__init__(self, description, value):
self.description = description
self.value = value
I would like to check the following constraints:
"description cannot be empty"
"value must be greater than zero"
Should I:
1. validate data before creating spam object ?
2. check data on __init__ method ?
3. create an is_valid method on Spam class and call it with spam.isValid() ?
4. create an is_valid static method on Spam class and call it with Spam.isValid(description, value) ?
5. check data on setters declaration ?
6. etc.
Could you recommend a well designed/Pythonic/not verbose (on class with many attributes)/elegant approach?
You can use Python properties to cleanly apply rules to each field separately, and enforce them even when client code tries to change the field:
class Spam(object):
def __init__(self, description, value):
self.description = description
self.value = value
#property
def description(self):
return self._description
#description.setter
def description(self, d):
if not d: raise Exception("description cannot be empty")
self._description = d
#property
def value(self):
return self._value
#value.setter
def value(self, v):
if not (v > 0): raise Exception("value must be greater than zero")
self._value = v
An exception will be thrown on any attempt to violate the rules, even in the __init__ function, in which case object construction will fail.
UPDATE: Sometime between 2010 and now, I learned about operator.attrgetter:
import operator
class Spam(object):
def __init__(self, description, value):
self.description = description
self.value = value
description = property(operator.attrgetter('_description'))
#description.setter
def description(self, d):
if not d: raise Exception("description cannot be empty")
self._description = d
value = property(operator.attrgetter('_value'))
#value.setter
def value(self, v):
if not (v > 0): raise Exception("value must be greater than zero")
self._value = v
If you only want to validate the values when the object is created AND passing in invalid values is considered a programming error then I would use assertions:
class Spam(object):
def __init__(self, description:str, value:int):
assert description != ""
assert value > 0
self.description = description
self.value = value
This is about as concise as you are going to get, and clearly documents that these are preconditions for creating the object.
Unless you're hellbent on rolling your own, you can simply use formencode. It really shines with many attributes and schemas (just subclass schemas) and has a lot of useful validators builtin. As you can see this is the "validate data before creating spam object" approach.
from formencode import Schema, validators
class SpamSchema(Schema):
description = validators.String(not_empty=True)
value = validators.Int(min=0)
class Spam(object):
def __init__(self, description, value):
self.description = description
self.value = value
## how you actually validate depends on your application
def validate_input( cls, schema, **input):
data = schema.to_python(input) # validate `input` dict with the schema
return cls(**data) # it validated here, else there was an exception
# returns a Spam object
validate_input( Spam, SpamSchema, description='this works', value=5)
# raises an exception with all the invalid fields
validate_input( Spam, SpamSchema, description='', value=-1)
You could do the checks during __init__ too (and make them completely transparent with descriptors|decorators|metaclass), but I'm not a big fan of that. I like a clean barrier between user input and internal objects.
if you want to only validate those values passed to the constructor, you could do:
class Spam(object):
def __init__(self, description, value):
if not description or value <=0:
raise ValueError
self.description = description
self.value = value
This will of course will not prevent anyone from doing something like this:
>>> s = Spam('s', 5)
>>> s.value = 0
>>> s.value
0
So, correct approach depends on what you're trying to accomplish.
You can try pyfields:
from pyfields import field
class Spam(object):
description = field(validators={"description can not be empty": lambda s: len(s) > 0})
value = field(validators={"value must be greater than zero": lambda x: x > 0})
s = Spam()
s.description = "hello"
s.description = "" # <-- raises error, see below
It yields
ValidationError[ValueError]: Error validating [<...>.Spam.description=''].
InvalidValue: description can not be empty.
Function [<lambda>] returned [False] for value ''.
It is compliant with python 2 and 3.5 (as opposed to pydantic), and validation happens everytime the value is changed (not only the first time, as opposed to attrs). It can create the constructor for you, but does not do it by default as shown above.
Note that you may wish to optionally use mini-lambda instead of plain old lambda functions if you wish the error messages to be even more straightforward (they will display the failing expression).
See pyfields documentation for details (I'm the author by the way ;) )
I'm working on yet another validation library - convtools models (docs / github).
The vision of this library is:
validation first
no implicit type casting
no implicit data losses during type casting - e.g. casting 10.0 to int is fine, 10.1 is not
if there’s a model instance, it is valid.
from collections import namedtuple
from typing import Union
from convtools.contrib.models import ObjectModel, build, validate, validators
# input data to test
SpamTest = namedtuple("SpamTest", ["description", "value"])
class Spam(ObjectModel):
description: str = validate(validators.Length(min_length=1))
value: Union[int, float] = validate(validators.Gt(0))
spam, errors = build(Spam, SpamTest("", 0))
"""
>>> In [34]: errors
>>> Out[34]:
>>> {'description': {'__ERRORS': {'min_length': 'length is 0, but should be >= 1'}},
>>> 'value': {'__ERRORS': {'gt': 'should be > 0'}}
"""
spam, errors = build(Spam, SpamTest("foo", 1))
"""
>>> In [42]: spam
>>> Out[42]: Spam(description='foo', value=1)
>>> In [43]: spam.to_dict()
>>> Out[43]: {'description': 'foo', 'value': 1}
>>> In [44]: spam.description
>>> Out[44]: 'foo'
"""
Is there a way to conveniently define a C-like structure in Python? I'm tired of writing stuff like:
class MyStruct():
def __init__(self, field1, field2, field3):
self.field1 = field1
self.field2 = field2
self.field3 = field3
Update: Data Classes
With the introduction of Data Classes in Python 3.7 we get very close.
The following example is similar to the NamedTuple example below, but the resulting object is mutable and it allows for default values.
from dataclasses import dataclass
#dataclass
class Point:
x: float
y: float
z: float = 0.0
p = Point(1.5, 2.5)
print(p) # Point(x=1.5, y=2.5, z=0.0)
This plays nicely with the new typing module in case you want to use more specific type annotations.
I've been waiting desperately for this! If you ask me, Data Classes and the new NamedTuple declaration, combined with the typing module are a godsend!
Improved NamedTuple declaration
Since Python 3.6 it became quite simple and beautiful (IMHO), as long as you can live with immutability.
A new way of declaring NamedTuples was introduced, which allows for type annotations as well:
from typing import NamedTuple
class User(NamedTuple):
name: str
class MyStruct(NamedTuple):
foo: str
bar: int
baz: list
qux: User
my_item = MyStruct('foo', 0, ['baz'], User('peter'))
print(my_item) # MyStruct(foo='foo', bar=0, baz=['baz'], qux=User(name='peter'))
Use a named tuple, which was added to the collections module in the standard library in Python 2.6. It's also possible to use Raymond Hettinger's named tuple recipe if you need to support Python 2.4.
It's nice for your basic example, but also covers a bunch of edge cases you might run into later as well. Your fragment above would be written as:
from collections import namedtuple
MyStruct = namedtuple("MyStruct", "field1 field2 field3")
The newly created type can be used like this:
m = MyStruct("foo", "bar", "baz")
You can also use named arguments:
m = MyStruct(field1="foo", field2="bar", field3="baz")
You can use a tuple for a lot of things where you would use a struct in C (something like x,y coordinates or RGB colors for example).
For everything else you can use dictionary, or a utility class like this one:
>>> class Bunch:
... def __init__(self, **kwds):
... self.__dict__.update(kwds)
...
>>> mystruct = Bunch(field1=value1, field2=value2)
I think the "definitive" discussion is here, in the published version of the Python Cookbook.
Perhaps you are looking for Structs without constructors:
class Sample:
name = ''
average = 0.0
values = None # list cannot be initialized here!
s1 = Sample()
s1.name = "sample 1"
s1.values = []
s1.values.append(1)
s1.values.append(2)
s1.values.append(3)
s2 = Sample()
s2.name = "sample 2"
s2.values = []
s2.values.append(4)
for v in s1.values: # prints 1,2,3 --> OK.
print v
print "***"
for v in s2.values: # prints 4 --> OK.
print v
How about a dictionary?
Something like this:
myStruct = {'field1': 'some val', 'field2': 'some val'}
Then you can use this to manipulate values:
print myStruct['field1']
myStruct['field2'] = 'some other values'
And the values don't have to be strings. They can be pretty much any other object.
dF: that's pretty cool... I didn't
know that I could access the fields in
a class using dict.
Mark: the situations that I wish I had
this are precisely when I want a tuple
but nothing as "heavy" as a
dictionary.
You can access the fields of a class using a dictionary because the fields of a class, its methods and all its properties are stored internally using dicts (at least in CPython).
...Which leads us to your second comment. Believing that Python dicts are "heavy" is an extremely non-pythonistic concept. And reading such comments kills my Python Zen. That's not good.
You see, when you declare a class you are actually creating a pretty complex wrapper around a dictionary - so, if anything, you are adding more overhead than by using a simple dictionary. An overhead which, by the way, is meaningless in any case. If you are working on performance critical applications, use C or something.
I would also like to add a solution that uses slots:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
Definitely check the documentation for slots but a quick explanation of slots is that it is python's way of saying: "If you can lock these attributes and only these attributes into the class such that you commit that you will not add any new attributes once the class is instantiated (yes you can add new attributes to a class instance, see example below) then I will do away with the large memory allocation that allows for adding new attributes to a class instance and use just what I need for these slotted attributes".
Example of adding attributes to class instance (thus not using slots):
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
print(p1.z)
Output: 8
Example of trying to add attributes to class instance where slots was used:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
Output: AttributeError: 'Point' object has no attribute 'z'
This can effectively works as a struct and uses less memory than a class (like a struct would, although I have not researched exactly how much). It is recommended to use slots if you will be creating a large amount of instances of the object and do not need to add attributes. A point object is a good example of this as it is likely that one may instantiate many points to describe a dataset.
You can subclass the C structure that is available in the standard library. The ctypes module provides a Structure class. The example from the docs:
>>> from ctypes import *
>>> class POINT(Structure):
... _fields_ = [("x", c_int),
... ("y", c_int)]
...
>>> point = POINT(10, 20)
>>> print point.x, point.y
10 20
>>> point = POINT(y=5)
>>> print point.x, point.y
0 5
>>> POINT(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: too many initializers
>>>
>>> class RECT(Structure):
... _fields_ = [("upperleft", POINT),
... ("lowerright", POINT)]
...
>>> rc = RECT(point)
>>> print rc.upperleft.x, rc.upperleft.y
0 5
>>> print rc.lowerright.x, rc.lowerright.y
0 0
>>>
You can also pass the init parameters to the instance variables by position
# Abstract struct class
class Struct:
def __init__ (self, *argv, **argd):
if len(argd):
# Update by dictionary
self.__dict__.update (argd)
else:
# Update by position
attrs = filter (lambda x: x[0:2] != "__", dir(self))
for n in range(len(argv)):
setattr(self, attrs[n], argv[n])
# Specific class
class Point3dStruct (Struct):
x = 0
y = 0
z = 0
pt1 = Point3dStruct()
pt1.x = 10
print pt1.x
print "-"*10
pt2 = Point3dStruct(5, 6)
print pt2.x, pt2.y
print "-"*10
pt3 = Point3dStruct (x=1, y=2, z=3)
print pt3.x, pt3.y, pt3.z
print "-"*10
Whenever I need an "instant data object that also behaves like a dictionary" (I don't think of C structs!), I think of this cute hack:
class Map(dict):
def __init__(self, **kwargs):
super(Map, self).__init__(**kwargs)
self.__dict__ = self
Now you can just say:
struct = Map(field1='foo', field2='bar', field3=42)
self.assertEquals('bar', struct.field2)
self.assertEquals(42, struct['field3'])
Perfectly handy for those times when you need a "data bag that's NOT a class", and for when namedtuples are incomprehensible...
Some the answers here are massively elaborate. The simplest option I've found is (from: http://norvig.com/python-iaq.html):
class Struct:
"A structure that can have any fields defined."
def __init__(self, **entries): self.__dict__.update(entries)
Initialising:
>>> options = Struct(answer=42, linelen=80, font='courier')
>>> options.answer
42
adding more:
>>> options.cat = "dog"
>>> options.cat
dog
edit: Sorry didn't see this example already further down.
You access C-Style struct in python in following way.
class cstruct:
var_i = 0
var_f = 0.0
var_str = ""
if you just want use object of cstruct
obj = cstruct()
obj.var_i = 50
obj.var_f = 50.00
obj.var_str = "fifty"
print "cstruct: obj i=%d f=%f s=%s" %(obj.var_i, obj.var_f, obj.var_str)
if you want to create an array of objects of cstruct
obj_array = [cstruct() for i in range(10)]
obj_array[0].var_i = 10
obj_array[0].var_f = 10.00
obj_array[0].var_str = "ten"
#go ahead and fill rest of array instaces of struct
#print all the value
for i in range(10):
print "cstruct: obj_array i=%d f=%f s=%s" %(obj_array[i].var_i, obj_array[i].var_f, obj_array[i].var_str)
Note:
instead of 'cstruct' name, please use your struct name
instead of var_i, var_f, var_str, please define your structure's member variable.
This might be a bit late but I made a solution using Python Meta-Classes (decorator version below too).
When __init__ is called during run time, it grabs each of the arguments and their value and assigns them as instance variables to your class. This way you can make a struct-like class without having to assign every value manually.
My example has no error checking so it is easier to follow.
class MyStruct(type):
def __call__(cls, *args, **kwargs):
names = cls.__init__.func_code.co_varnames[1:]
self = type.__call__(cls, *args, **kwargs)
for name, value in zip(names, args):
setattr(self , name, value)
for name, value in kwargs.iteritems():
setattr(self , name, value)
return self
Here it is in action.
>>> class MyClass(object):
__metaclass__ = MyStruct
def __init__(self, a, b, c):
pass
>>> my_instance = MyClass(1, 2, 3)
>>> my_instance.a
1
>>>
I posted it on reddit and /u/matchu posted a decorator version which is cleaner. I'd encourage you to use it unless you want to expand the metaclass version.
>>> def init_all_args(fn):
#wraps(fn)
def wrapped_init(self, *args, **kwargs):
names = fn.func_code.co_varnames[1:]
for name, value in zip(names, args):
setattr(self, name, value)
for name, value in kwargs.iteritems():
setattr(self, name, value)
return wrapped_init
>>> class Test(object):
#init_all_args
def __init__(self, a, b):
pass
>>> a = Test(1, 2)
>>> a.a
1
>>>
I wrote a decorator which you can use on any method to make it so that all of the arguments passed in, or any defaults, are assigned to the instance.
def argumentsToAttributes(method):
argumentNames = method.func_code.co_varnames[1:]
# Generate a dictionary of default values:
defaultsDict = {}
defaults = method.func_defaults if method.func_defaults else ()
for i, default in enumerate(defaults, start = len(argumentNames) - len(defaults)):
defaultsDict[argumentNames[i]] = default
def newMethod(self, *args, **kwargs):
# Use the positional arguments.
for name, value in zip(argumentNames, args):
setattr(self, name, value)
# Add the key word arguments. If anything is missing, use the default.
for name in argumentNames[len(args):]:
setattr(self, name, kwargs.get(name, defaultsDict[name]))
# Run whatever else the method needs to do.
method(self, *args, **kwargs)
return newMethod
A quick demonstration. Note that I use a positional argument a, use the default value for b, and a named argument c. I then print all 3 referencing self, to show that they've been properly assigned before the method is entered.
class A(object):
#argumentsToAttributes
def __init__(self, a, b = 'Invisible', c = 'Hello'):
print(self.a)
print(self.b)
print(self.c)
A('Why', c = 'Nothing')
Note that my decorator should work with any method, not just __init__.
I don't see this answer here, so I figure I'll add it since I'm leaning Python right now and just discovered it. The Python tutorial (Python 2 in this case) gives the following simple and effective example:
class Employee:
pass
john = Employee() # Create an empty employee record
# Fill the fields of the record
john.name = 'John Doe'
john.dept = 'computer lab'
john.salary = 1000
That is, an empty class object is created, then instantiated, and the fields are added dynamically.
The up-side to this is its really simple. The downside is it isn't particularly self-documenting (the intended members aren't listed anywhere in the class "definition"), and unset fields can cause problems when accessed. Those two problems can be solved by:
class Employee:
def __init__ (self):
self.name = None # or whatever
self.dept = None
self.salary = None
Now at a glance you can at least see what fields the program will be expecting.
Both are prone to typos, john.slarly = 1000 will succeed. Still, it works.
Here is a solution which uses a class (never instantiated) to hold data. I like that this way involves very little typing and does not require any additional packages etc.
class myStruct:
field1 = "one"
field2 = "2"
You can add more fields later, as needed:
myStruct.field3 = 3
To get the values, the fields are accessed as usual:
>>> myStruct.field1
'one'
Personally, I like this variant too. It extends #dF's answer.
class struct:
def __init__(self, *sequential, **named):
fields = dict(zip(sequential, [None]*len(sequential)), **named)
self.__dict__.update(fields)
def __repr__(self):
return str(self.__dict__)
It supports two modes of initialization (that can be blended):
# Struct with field1, field2, field3 that are initialized to None.
mystruct1 = struct("field1", "field2", "field3")
# Struct with field1, field2, field3 that are initialized according to arguments.
mystruct2 = struct(field1=1, field2=2, field3=3)
Also, it prints nicer:
print(mystruct2)
# Prints: {'field3': 3, 'field1': 1, 'field2': 2}
There is a python package exactly for this purpose. see cstruct2py
cstruct2py is a pure python library for generate python classes from C code and use them to pack and unpack data. The library can parse C headres (structs, unions, enums, and arrays declarations) and emulate them in python. The generated pythonic classes can parse and pack the data.
For example:
typedef struct {
int x;
int y;
} Point;
after generating pythonic class...
p = Point(x=0x1234, y=0x5678)
p.packed == "\x34\x12\x00\x00\x78\x56\x00\x00"
How to use
First we need to generate the pythonic structs:
import cstruct2py
parser = cstruct2py.c2py.Parser()
parser.parse_file('examples/example.h')
Now we can import all names from the C code:
parser.update_globals(globals())
We can also do that directly:
A = parser.parse_string('struct A { int x; int y;};')
Using types and defines from the C code
a = A()
a.x = 45
print a
buf = a.packed
b = A(buf)
print b
c = A('aaaa11112222', 2)
print c
print repr(c)
The output will be:
{'x':0x2d, 'y':0x0}
{'x':0x2d, 'y':0x0}
{'x':0x31316161, 'y':0x32323131}
A('aa111122', x=0x31316161, y=0x32323131)
Clone
For clone cstruct2py run:
git clone https://github.com/st0ky/cstruct2py.git --recursive
Here is a quick and dirty trick:
>>> ms = Warning()
>>> ms.foo = 123
>>> ms.bar = 'akafrit'
How does it works? It just re-use the builtin class Warning (derived from Exception) and use it as it was you own defined class.
The good points are that you do not need to import or define anything first, that "Warning" is a short name, and that it also makes clear you are doing something dirty which should not be used elsewhere than a small script of yours.
By the way, I tried to find something even simpler like ms = object() but could not (this last exemple is not working). If you have one, I am interested.
NamedTuple is comfortable. but there no one shares the performance and storage.
from typing import NamedTuple
import guppy # pip install guppy
import timeit
class User:
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserSlot:
__slots__ = ('name', 'uid')
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserTuple(NamedTuple):
# __slots__ = () # AttributeError: Cannot overwrite NamedTuple attribute __slots__
name: str
uid: int
def get_fn(obj, attr_name: str):
def get():
getattr(obj, attr_name)
return get
if 'memory test':
obj = [User('Carson', 1) for _ in range(1000000)] # Cumulative: 189138883
obj_slot = [UserSlot('Carson', 1) for _ in range(1000000)] # 77718299 <-- winner
obj_namedtuple = [UserTuple('Carson', 1) for _ in range(1000000)] # 85718297
print(guppy.hpy().heap()) # Run this function individually.
"""
Index Count % Size % Cumulative % Kind (class / dict of class)
0 1000000 24 112000000 34 112000000 34 dict of __main__.User
1 1000000 24 64000000 19 176000000 53 __main__.UserTuple
2 1000000 24 56000000 17 232000000 70 __main__.User
3 1000000 24 56000000 17 288000000 87 __main__.UserSlot
...
"""
if 'performance test':
obj = User('Carson', 1)
obj_slot = UserSlot('Carson', 1)
obj_tuple = UserTuple('Carson', 1)
time_normal = min(timeit.repeat(get_fn(obj, 'name'), repeat=20))
print(time_normal) # 0.12550550000000005
time_slot = min(timeit.repeat(get_fn(obj_slot, 'name'), repeat=20))
print(time_slot) # 0.1368690000000008
time_tuple = min(timeit.repeat(get_fn(obj_tuple, 'name'), repeat=20))
print(time_tuple) # 0.16006120000000124
print(time_tuple/time_slot) # 1.1694481584580898 # The slot is almost 17% faster than NamedTuple on Windows. (Python 3.7.7)
If your __dict__ is not using, please choose between __slots__ (higher performance and storage) and NamedTuple (clear for reading and use)
You can review this link(Usage of slots
) to get more __slots__ information.
https://stackoverflow.com/a/32448434/159695 does not work in Python3.
https://stackoverflow.com/a/35993/159695 works in Python3.
And I extends it to add default values.
class myStruct:
def __init__(self, **kwds):
self.x=0
self.__dict__.update(kwds) # Must be last to accept assigned member variable.
def __repr__(self):
args = ['%s=%s' % (k, repr(v)) for (k,v) in vars(self).items()]
return '%s(%s)' % ( self.__class__.__qualname__, ', '.join(args) )
a=myStruct()
b=myStruct(x=3,y='test')
c=myStruct(x='str')
>>> a
myStruct(x=0)
>>> b
myStruct(x=3, y='test')
>>> c
myStruct(x='str')
The following solution to a struct is inspired by the namedtuple implementation and some of the previous answers. However, unlike the namedtuple it is mutable, in it's values, but like the c-style struct immutable in the names/attributes, which a normal class or dict isn't.
_class_template = """\
class {typename}:
def __init__(self, *args, **kwargs):
fields = {field_names!r}
for x in fields:
setattr(self, x, None)
for name, value in zip(fields, args):
setattr(self, name, value)
for name, value in kwargs.items():
setattr(self, name, value)
def __repr__(self):
return str(vars(self))
def __setattr__(self, name, value):
if name not in {field_names!r}:
raise KeyError("invalid name: %s" % name)
object.__setattr__(self, name, value)
"""
def struct(typename, field_names):
class_definition = _class_template.format(
typename = typename,
field_names = field_names)
namespace = dict(__name__='struct_%s' % typename)
exec(class_definition, namespace)
result = namespace[typename]
result._source = class_definition
return result
Usage:
Person = struct('Person', ['firstname','lastname'])
generic = Person()
michael = Person('Michael')
jones = Person(lastname = 'Jones')
In [168]: michael.middlename = 'ben'
Traceback (most recent call last):
File "<ipython-input-168-b31c393c0d67>", line 1, in <module>
michael.middlename = 'ben'
File "<string>", line 19, in __setattr__
KeyError: 'invalid name: middlename'
If you don't have a 3.7 for #dataclass and need mutability, the following code might work for you. It's quite self-documenting and IDE-friendly (auto-complete), prevents writing things twice, is easily extendable and it is very simple to test that all instance variables are completely initialized:
class Params():
def __init__(self):
self.var1 : int = None
self.var2 : str = None
def are_all_defined(self):
for key, value in self.__dict__.items():
assert (value is not None), "instance variable {} is still None".format(key)
return True
params = Params()
params.var1 = 2
params.var2 = 'hello'
assert(params.are_all_defined)
The best way I found to do this was to use a custom dictionary class as explained in this post: https://stackoverflow.com/a/14620633/8484485
If iPython autocompletion support is needed, simply define the dir() function like this:
class AttrDict(dict):
def __init__(self, *args, **kwargs):
super(AttrDict, self).__init__(*args, **kwargs)
self.__dict__ = self
def __dir__(self):
return self.keys()
You then define your pseudo struct like so: (this one is nested)
my_struct=AttrDict ({
'com1':AttrDict ({
'inst':[0x05],
'numbytes':2,
'canpayload':False,
'payload':None
})
})
You can then access the values inside my_struct like this:
print(my_struct.com1.inst)
=>[5]
The cleanest way I can think of is to use a class decorator that lets you declare a static class and rewrite it to act as a struct with normal, named properties:
from as_struct import struct
#struct
class Product():
name = 'unknown product'
quantity = -1
sku = '-'
# create instance
p = Product('plush toy', sku='12-345-6789')
# check content:
p.name # plush toy
p.quantity # -1
p.sku # 12-345-6789
Using the following decorator code:
def struct(struct_class):
# create a new init
def struct_init(self, *args, **kwargs):
i = 0 # we really don't need enumerate() here...
for value in args:
name = member_names[i]
default_value = member_values[i]
setattr(self, name, value if value is not None else default_value)
i += 1 # ...we just need to inc an int
for key,value in kwargs.items():
i = member_names.index(key)
default_value = member_values[i]
setattr(self, key, value if value is not None else default_value)
# extract the struct members
member_names = []
member_values = []
for attr_name in dir(struct_class):
if not attr_name.startswith('_'):
value = getattr(struct_class, attr_name)
if not callable(value):
member_names.append(attr_name)
member_values.append(value)
# rebind and return
struct_class.init = struct_init
return struct_class
Which works by taking the class, extracting the field names and their default values, then rewriting the class's __init__ function to set self attributes based on knowing which argument index maps to which property name.
I think Python structure dictionary is suitable for this requirement.
d = dict{}
d[field1] = field1
d[field2] = field2
d[field2] = field3
Extending #gz.'s (generally superior to this one) answer, for a quick and dirty namedtuple structure we can do:
import collections
x = collections.namedtuple('foobar', 'foo bar')(foo=1,bar=2)
y = collections.namedtuple('foobar', 'foo bar')(foo=3,bar=4)
print(x,y)
>foobar(foo=1, bar=2) foobar(foo=3, bar=4)