Related
Is there a way to conveniently define a C-like structure in Python? I'm tired of writing stuff like:
class MyStruct():
def __init__(self, field1, field2, field3):
self.field1 = field1
self.field2 = field2
self.field3 = field3
Update: Data Classes
With the introduction of Data Classes in Python 3.7 we get very close.
The following example is similar to the NamedTuple example below, but the resulting object is mutable and it allows for default values.
from dataclasses import dataclass
#dataclass
class Point:
x: float
y: float
z: float = 0.0
p = Point(1.5, 2.5)
print(p) # Point(x=1.5, y=2.5, z=0.0)
This plays nicely with the new typing module in case you want to use more specific type annotations.
I've been waiting desperately for this! If you ask me, Data Classes and the new NamedTuple declaration, combined with the typing module are a godsend!
Improved NamedTuple declaration
Since Python 3.6 it became quite simple and beautiful (IMHO), as long as you can live with immutability.
A new way of declaring NamedTuples was introduced, which allows for type annotations as well:
from typing import NamedTuple
class User(NamedTuple):
name: str
class MyStruct(NamedTuple):
foo: str
bar: int
baz: list
qux: User
my_item = MyStruct('foo', 0, ['baz'], User('peter'))
print(my_item) # MyStruct(foo='foo', bar=0, baz=['baz'], qux=User(name='peter'))
Use a named tuple, which was added to the collections module in the standard library in Python 2.6. It's also possible to use Raymond Hettinger's named tuple recipe if you need to support Python 2.4.
It's nice for your basic example, but also covers a bunch of edge cases you might run into later as well. Your fragment above would be written as:
from collections import namedtuple
MyStruct = namedtuple("MyStruct", "field1 field2 field3")
The newly created type can be used like this:
m = MyStruct("foo", "bar", "baz")
You can also use named arguments:
m = MyStruct(field1="foo", field2="bar", field3="baz")
You can use a tuple for a lot of things where you would use a struct in C (something like x,y coordinates or RGB colors for example).
For everything else you can use dictionary, or a utility class like this one:
>>> class Bunch:
... def __init__(self, **kwds):
... self.__dict__.update(kwds)
...
>>> mystruct = Bunch(field1=value1, field2=value2)
I think the "definitive" discussion is here, in the published version of the Python Cookbook.
Perhaps you are looking for Structs without constructors:
class Sample:
name = ''
average = 0.0
values = None # list cannot be initialized here!
s1 = Sample()
s1.name = "sample 1"
s1.values = []
s1.values.append(1)
s1.values.append(2)
s1.values.append(3)
s2 = Sample()
s2.name = "sample 2"
s2.values = []
s2.values.append(4)
for v in s1.values: # prints 1,2,3 --> OK.
print v
print "***"
for v in s2.values: # prints 4 --> OK.
print v
How about a dictionary?
Something like this:
myStruct = {'field1': 'some val', 'field2': 'some val'}
Then you can use this to manipulate values:
print myStruct['field1']
myStruct['field2'] = 'some other values'
And the values don't have to be strings. They can be pretty much any other object.
dF: that's pretty cool... I didn't
know that I could access the fields in
a class using dict.
Mark: the situations that I wish I had
this are precisely when I want a tuple
but nothing as "heavy" as a
dictionary.
You can access the fields of a class using a dictionary because the fields of a class, its methods and all its properties are stored internally using dicts (at least in CPython).
...Which leads us to your second comment. Believing that Python dicts are "heavy" is an extremely non-pythonistic concept. And reading such comments kills my Python Zen. That's not good.
You see, when you declare a class you are actually creating a pretty complex wrapper around a dictionary - so, if anything, you are adding more overhead than by using a simple dictionary. An overhead which, by the way, is meaningless in any case. If you are working on performance critical applications, use C or something.
I would also like to add a solution that uses slots:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
Definitely check the documentation for slots but a quick explanation of slots is that it is python's way of saying: "If you can lock these attributes and only these attributes into the class such that you commit that you will not add any new attributes once the class is instantiated (yes you can add new attributes to a class instance, see example below) then I will do away with the large memory allocation that allows for adding new attributes to a class instance and use just what I need for these slotted attributes".
Example of adding attributes to class instance (thus not using slots):
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
print(p1.z)
Output: 8
Example of trying to add attributes to class instance where slots was used:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
Output: AttributeError: 'Point' object has no attribute 'z'
This can effectively works as a struct and uses less memory than a class (like a struct would, although I have not researched exactly how much). It is recommended to use slots if you will be creating a large amount of instances of the object and do not need to add attributes. A point object is a good example of this as it is likely that one may instantiate many points to describe a dataset.
You can subclass the C structure that is available in the standard library. The ctypes module provides a Structure class. The example from the docs:
>>> from ctypes import *
>>> class POINT(Structure):
... _fields_ = [("x", c_int),
... ("y", c_int)]
...
>>> point = POINT(10, 20)
>>> print point.x, point.y
10 20
>>> point = POINT(y=5)
>>> print point.x, point.y
0 5
>>> POINT(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: too many initializers
>>>
>>> class RECT(Structure):
... _fields_ = [("upperleft", POINT),
... ("lowerright", POINT)]
...
>>> rc = RECT(point)
>>> print rc.upperleft.x, rc.upperleft.y
0 5
>>> print rc.lowerright.x, rc.lowerright.y
0 0
>>>
You can also pass the init parameters to the instance variables by position
# Abstract struct class
class Struct:
def __init__ (self, *argv, **argd):
if len(argd):
# Update by dictionary
self.__dict__.update (argd)
else:
# Update by position
attrs = filter (lambda x: x[0:2] != "__", dir(self))
for n in range(len(argv)):
setattr(self, attrs[n], argv[n])
# Specific class
class Point3dStruct (Struct):
x = 0
y = 0
z = 0
pt1 = Point3dStruct()
pt1.x = 10
print pt1.x
print "-"*10
pt2 = Point3dStruct(5, 6)
print pt2.x, pt2.y
print "-"*10
pt3 = Point3dStruct (x=1, y=2, z=3)
print pt3.x, pt3.y, pt3.z
print "-"*10
Whenever I need an "instant data object that also behaves like a dictionary" (I don't think of C structs!), I think of this cute hack:
class Map(dict):
def __init__(self, **kwargs):
super(Map, self).__init__(**kwargs)
self.__dict__ = self
Now you can just say:
struct = Map(field1='foo', field2='bar', field3=42)
self.assertEquals('bar', struct.field2)
self.assertEquals(42, struct['field3'])
Perfectly handy for those times when you need a "data bag that's NOT a class", and for when namedtuples are incomprehensible...
Some the answers here are massively elaborate. The simplest option I've found is (from: http://norvig.com/python-iaq.html):
class Struct:
"A structure that can have any fields defined."
def __init__(self, **entries): self.__dict__.update(entries)
Initialising:
>>> options = Struct(answer=42, linelen=80, font='courier')
>>> options.answer
42
adding more:
>>> options.cat = "dog"
>>> options.cat
dog
edit: Sorry didn't see this example already further down.
You access C-Style struct in python in following way.
class cstruct:
var_i = 0
var_f = 0.0
var_str = ""
if you just want use object of cstruct
obj = cstruct()
obj.var_i = 50
obj.var_f = 50.00
obj.var_str = "fifty"
print "cstruct: obj i=%d f=%f s=%s" %(obj.var_i, obj.var_f, obj.var_str)
if you want to create an array of objects of cstruct
obj_array = [cstruct() for i in range(10)]
obj_array[0].var_i = 10
obj_array[0].var_f = 10.00
obj_array[0].var_str = "ten"
#go ahead and fill rest of array instaces of struct
#print all the value
for i in range(10):
print "cstruct: obj_array i=%d f=%f s=%s" %(obj_array[i].var_i, obj_array[i].var_f, obj_array[i].var_str)
Note:
instead of 'cstruct' name, please use your struct name
instead of var_i, var_f, var_str, please define your structure's member variable.
This might be a bit late but I made a solution using Python Meta-Classes (decorator version below too).
When __init__ is called during run time, it grabs each of the arguments and their value and assigns them as instance variables to your class. This way you can make a struct-like class without having to assign every value manually.
My example has no error checking so it is easier to follow.
class MyStruct(type):
def __call__(cls, *args, **kwargs):
names = cls.__init__.func_code.co_varnames[1:]
self = type.__call__(cls, *args, **kwargs)
for name, value in zip(names, args):
setattr(self , name, value)
for name, value in kwargs.iteritems():
setattr(self , name, value)
return self
Here it is in action.
>>> class MyClass(object):
__metaclass__ = MyStruct
def __init__(self, a, b, c):
pass
>>> my_instance = MyClass(1, 2, 3)
>>> my_instance.a
1
>>>
I posted it on reddit and /u/matchu posted a decorator version which is cleaner. I'd encourage you to use it unless you want to expand the metaclass version.
>>> def init_all_args(fn):
#wraps(fn)
def wrapped_init(self, *args, **kwargs):
names = fn.func_code.co_varnames[1:]
for name, value in zip(names, args):
setattr(self, name, value)
for name, value in kwargs.iteritems():
setattr(self, name, value)
return wrapped_init
>>> class Test(object):
#init_all_args
def __init__(self, a, b):
pass
>>> a = Test(1, 2)
>>> a.a
1
>>>
I wrote a decorator which you can use on any method to make it so that all of the arguments passed in, or any defaults, are assigned to the instance.
def argumentsToAttributes(method):
argumentNames = method.func_code.co_varnames[1:]
# Generate a dictionary of default values:
defaultsDict = {}
defaults = method.func_defaults if method.func_defaults else ()
for i, default in enumerate(defaults, start = len(argumentNames) - len(defaults)):
defaultsDict[argumentNames[i]] = default
def newMethod(self, *args, **kwargs):
# Use the positional arguments.
for name, value in zip(argumentNames, args):
setattr(self, name, value)
# Add the key word arguments. If anything is missing, use the default.
for name in argumentNames[len(args):]:
setattr(self, name, kwargs.get(name, defaultsDict[name]))
# Run whatever else the method needs to do.
method(self, *args, **kwargs)
return newMethod
A quick demonstration. Note that I use a positional argument a, use the default value for b, and a named argument c. I then print all 3 referencing self, to show that they've been properly assigned before the method is entered.
class A(object):
#argumentsToAttributes
def __init__(self, a, b = 'Invisible', c = 'Hello'):
print(self.a)
print(self.b)
print(self.c)
A('Why', c = 'Nothing')
Note that my decorator should work with any method, not just __init__.
I don't see this answer here, so I figure I'll add it since I'm leaning Python right now and just discovered it. The Python tutorial (Python 2 in this case) gives the following simple and effective example:
class Employee:
pass
john = Employee() # Create an empty employee record
# Fill the fields of the record
john.name = 'John Doe'
john.dept = 'computer lab'
john.salary = 1000
That is, an empty class object is created, then instantiated, and the fields are added dynamically.
The up-side to this is its really simple. The downside is it isn't particularly self-documenting (the intended members aren't listed anywhere in the class "definition"), and unset fields can cause problems when accessed. Those two problems can be solved by:
class Employee:
def __init__ (self):
self.name = None # or whatever
self.dept = None
self.salary = None
Now at a glance you can at least see what fields the program will be expecting.
Both are prone to typos, john.slarly = 1000 will succeed. Still, it works.
Here is a solution which uses a class (never instantiated) to hold data. I like that this way involves very little typing and does not require any additional packages etc.
class myStruct:
field1 = "one"
field2 = "2"
You can add more fields later, as needed:
myStruct.field3 = 3
To get the values, the fields are accessed as usual:
>>> myStruct.field1
'one'
Personally, I like this variant too. It extends #dF's answer.
class struct:
def __init__(self, *sequential, **named):
fields = dict(zip(sequential, [None]*len(sequential)), **named)
self.__dict__.update(fields)
def __repr__(self):
return str(self.__dict__)
It supports two modes of initialization (that can be blended):
# Struct with field1, field2, field3 that are initialized to None.
mystruct1 = struct("field1", "field2", "field3")
# Struct with field1, field2, field3 that are initialized according to arguments.
mystruct2 = struct(field1=1, field2=2, field3=3)
Also, it prints nicer:
print(mystruct2)
# Prints: {'field3': 3, 'field1': 1, 'field2': 2}
There is a python package exactly for this purpose. see cstruct2py
cstruct2py is a pure python library for generate python classes from C code and use them to pack and unpack data. The library can parse C headres (structs, unions, enums, and arrays declarations) and emulate them in python. The generated pythonic classes can parse and pack the data.
For example:
typedef struct {
int x;
int y;
} Point;
after generating pythonic class...
p = Point(x=0x1234, y=0x5678)
p.packed == "\x34\x12\x00\x00\x78\x56\x00\x00"
How to use
First we need to generate the pythonic structs:
import cstruct2py
parser = cstruct2py.c2py.Parser()
parser.parse_file('examples/example.h')
Now we can import all names from the C code:
parser.update_globals(globals())
We can also do that directly:
A = parser.parse_string('struct A { int x; int y;};')
Using types and defines from the C code
a = A()
a.x = 45
print a
buf = a.packed
b = A(buf)
print b
c = A('aaaa11112222', 2)
print c
print repr(c)
The output will be:
{'x':0x2d, 'y':0x0}
{'x':0x2d, 'y':0x0}
{'x':0x31316161, 'y':0x32323131}
A('aa111122', x=0x31316161, y=0x32323131)
Clone
For clone cstruct2py run:
git clone https://github.com/st0ky/cstruct2py.git --recursive
Here is a quick and dirty trick:
>>> ms = Warning()
>>> ms.foo = 123
>>> ms.bar = 'akafrit'
How does it works? It just re-use the builtin class Warning (derived from Exception) and use it as it was you own defined class.
The good points are that you do not need to import or define anything first, that "Warning" is a short name, and that it also makes clear you are doing something dirty which should not be used elsewhere than a small script of yours.
By the way, I tried to find something even simpler like ms = object() but could not (this last exemple is not working). If you have one, I am interested.
NamedTuple is comfortable. but there no one shares the performance and storage.
from typing import NamedTuple
import guppy # pip install guppy
import timeit
class User:
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserSlot:
__slots__ = ('name', 'uid')
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserTuple(NamedTuple):
# __slots__ = () # AttributeError: Cannot overwrite NamedTuple attribute __slots__
name: str
uid: int
def get_fn(obj, attr_name: str):
def get():
getattr(obj, attr_name)
return get
if 'memory test':
obj = [User('Carson', 1) for _ in range(1000000)] # Cumulative: 189138883
obj_slot = [UserSlot('Carson', 1) for _ in range(1000000)] # 77718299 <-- winner
obj_namedtuple = [UserTuple('Carson', 1) for _ in range(1000000)] # 85718297
print(guppy.hpy().heap()) # Run this function individually.
"""
Index Count % Size % Cumulative % Kind (class / dict of class)
0 1000000 24 112000000 34 112000000 34 dict of __main__.User
1 1000000 24 64000000 19 176000000 53 __main__.UserTuple
2 1000000 24 56000000 17 232000000 70 __main__.User
3 1000000 24 56000000 17 288000000 87 __main__.UserSlot
...
"""
if 'performance test':
obj = User('Carson', 1)
obj_slot = UserSlot('Carson', 1)
obj_tuple = UserTuple('Carson', 1)
time_normal = min(timeit.repeat(get_fn(obj, 'name'), repeat=20))
print(time_normal) # 0.12550550000000005
time_slot = min(timeit.repeat(get_fn(obj_slot, 'name'), repeat=20))
print(time_slot) # 0.1368690000000008
time_tuple = min(timeit.repeat(get_fn(obj_tuple, 'name'), repeat=20))
print(time_tuple) # 0.16006120000000124
print(time_tuple/time_slot) # 1.1694481584580898 # The slot is almost 17% faster than NamedTuple on Windows. (Python 3.7.7)
If your __dict__ is not using, please choose between __slots__ (higher performance and storage) and NamedTuple (clear for reading and use)
You can review this link(Usage of slots
) to get more __slots__ information.
https://stackoverflow.com/a/32448434/159695 does not work in Python3.
https://stackoverflow.com/a/35993/159695 works in Python3.
And I extends it to add default values.
class myStruct:
def __init__(self, **kwds):
self.x=0
self.__dict__.update(kwds) # Must be last to accept assigned member variable.
def __repr__(self):
args = ['%s=%s' % (k, repr(v)) for (k,v) in vars(self).items()]
return '%s(%s)' % ( self.__class__.__qualname__, ', '.join(args) )
a=myStruct()
b=myStruct(x=3,y='test')
c=myStruct(x='str')
>>> a
myStruct(x=0)
>>> b
myStruct(x=3, y='test')
>>> c
myStruct(x='str')
The following solution to a struct is inspired by the namedtuple implementation and some of the previous answers. However, unlike the namedtuple it is mutable, in it's values, but like the c-style struct immutable in the names/attributes, which a normal class or dict isn't.
_class_template = """\
class {typename}:
def __init__(self, *args, **kwargs):
fields = {field_names!r}
for x in fields:
setattr(self, x, None)
for name, value in zip(fields, args):
setattr(self, name, value)
for name, value in kwargs.items():
setattr(self, name, value)
def __repr__(self):
return str(vars(self))
def __setattr__(self, name, value):
if name not in {field_names!r}:
raise KeyError("invalid name: %s" % name)
object.__setattr__(self, name, value)
"""
def struct(typename, field_names):
class_definition = _class_template.format(
typename = typename,
field_names = field_names)
namespace = dict(__name__='struct_%s' % typename)
exec(class_definition, namespace)
result = namespace[typename]
result._source = class_definition
return result
Usage:
Person = struct('Person', ['firstname','lastname'])
generic = Person()
michael = Person('Michael')
jones = Person(lastname = 'Jones')
In [168]: michael.middlename = 'ben'
Traceback (most recent call last):
File "<ipython-input-168-b31c393c0d67>", line 1, in <module>
michael.middlename = 'ben'
File "<string>", line 19, in __setattr__
KeyError: 'invalid name: middlename'
If you don't have a 3.7 for #dataclass and need mutability, the following code might work for you. It's quite self-documenting and IDE-friendly (auto-complete), prevents writing things twice, is easily extendable and it is very simple to test that all instance variables are completely initialized:
class Params():
def __init__(self):
self.var1 : int = None
self.var2 : str = None
def are_all_defined(self):
for key, value in self.__dict__.items():
assert (value is not None), "instance variable {} is still None".format(key)
return True
params = Params()
params.var1 = 2
params.var2 = 'hello'
assert(params.are_all_defined)
The best way I found to do this was to use a custom dictionary class as explained in this post: https://stackoverflow.com/a/14620633/8484485
If iPython autocompletion support is needed, simply define the dir() function like this:
class AttrDict(dict):
def __init__(self, *args, **kwargs):
super(AttrDict, self).__init__(*args, **kwargs)
self.__dict__ = self
def __dir__(self):
return self.keys()
You then define your pseudo struct like so: (this one is nested)
my_struct=AttrDict ({
'com1':AttrDict ({
'inst':[0x05],
'numbytes':2,
'canpayload':False,
'payload':None
})
})
You can then access the values inside my_struct like this:
print(my_struct.com1.inst)
=>[5]
The cleanest way I can think of is to use a class decorator that lets you declare a static class and rewrite it to act as a struct with normal, named properties:
from as_struct import struct
#struct
class Product():
name = 'unknown product'
quantity = -1
sku = '-'
# create instance
p = Product('plush toy', sku='12-345-6789')
# check content:
p.name # plush toy
p.quantity # -1
p.sku # 12-345-6789
Using the following decorator code:
def struct(struct_class):
# create a new init
def struct_init(self, *args, **kwargs):
i = 0 # we really don't need enumerate() here...
for value in args:
name = member_names[i]
default_value = member_values[i]
setattr(self, name, value if value is not None else default_value)
i += 1 # ...we just need to inc an int
for key,value in kwargs.items():
i = member_names.index(key)
default_value = member_values[i]
setattr(self, key, value if value is not None else default_value)
# extract the struct members
member_names = []
member_values = []
for attr_name in dir(struct_class):
if not attr_name.startswith('_'):
value = getattr(struct_class, attr_name)
if not callable(value):
member_names.append(attr_name)
member_values.append(value)
# rebind and return
struct_class.init = struct_init
return struct_class
Which works by taking the class, extracting the field names and their default values, then rewriting the class's __init__ function to set self attributes based on knowing which argument index maps to which property name.
I think Python structure dictionary is suitable for this requirement.
d = dict{}
d[field1] = field1
d[field2] = field2
d[field2] = field3
Extending #gz.'s (generally superior to this one) answer, for a quick and dirty namedtuple structure we can do:
import collections
x = collections.namedtuple('foobar', 'foo bar')(foo=1,bar=2)
y = collections.namedtuple('foobar', 'foo bar')(foo=3,bar=4)
print(x,y)
>foobar(foo=1, bar=2) foobar(foo=3, bar=4)
I would like to make a deepcopy of a function in Python. The copy module is not helpful, according to the documentation, which says:
This module does not copy types like module, method, stack trace, stack frame, file,
socket, window, array, or any similar types. It does “copy” functions and classes (shallow
and deeply), by returning the original object unchanged; this is compatible with the way
these are treated by the pickle module.
My goal is to have two functions with the same implementation but with different docstrings.
def A():
"""A"""
pass
B = make_a_deepcopy_of(A)
B.__doc__ = """B"""
So how can this be done?
The FunctionType constructor is used to make a deep copy of a function.
import types
def copy_func(f, name=None):
return types.FunctionType(f.func_code, f.func_globals, name or f.func_name,
f.func_defaults, f.func_closure)
def A():
"""A"""
pass
B = copy_func(A, "B")
B.__doc__ = """B"""
My goal is to have two functions with the same implementation but with different docstrings.
Most users will do this, say the original function is in old_module.py:
def implementation(arg1, arg2):
"""this is a killer function"""
and in new_module.py
from old_module import implementation as _implementation
def implementation(arg1, arg2):
"""a different docstring"""
return _implementation(arg1, arg2)
This is the most straightforward way to reuse functionality. It is easy to read and understand the intent.
Nevertheless, perhaps you have a good reason for your main question:
How can I make a deepcopy of a function in Python?
To keep this compatible with Python 2 and 3, I recommend using the function's special __dunder__ attributes. For example:
import types
def copy_func(f, name=None):
'''
return a function with same code, globals, defaults, closure, and
name (or provide a new name)
'''
fn = types.FunctionType(f.__code__, f.__globals__, name or f.__name__,
f.__defaults__, f.__closure__)
# in case f was given attrs (note this dict is a shallow copy):
fn.__dict__.update(f.__dict__)
return fn
And here's an example usage:
def main():
from logging import getLogger as _getLogger # pyflakes:ignore, must copy
getLogger = copy_func(_getLogger)
getLogger.__doc__ += '\n This function is from the Std Lib logging module.\n '
assert getLogger.__doc__ is not _getLogger.__doc__
assert getLogger.__doc__ != _getLogger.__doc__
A commenter says:
This can’t work for built‑in functions
Well I wouldn't do this for a built-in function. I have very little reason to do this for functions written in pure Python, and my suspicion is that if you are doing this, you're probably doing something very wrong (though I could be wrong here).
If you want a function that does what a builtin function does, and reuses the implementation, like a copy would, then you should wrap the function with another function, e.g.:
_sum = sum
def sum(iterable, start=0):
"""sum function that works like the regular sum function, but noisy"""
print('calling the sum function')
return _sum(iterable, start)
from functools import partial
def a():
"""Returns 1"""
return 1
b = partial(a)
b.__doc__ = """Returns 1, OR DOES IT!"""
print help(a)
print help(b)
Wrap it as a partial?
def A():
"""A"""
pass
def B():
"""B"""
return A()
The others answers do not allow for serialization with pickle. Here a code that I am using to clone a function and allow for serialization for python3:
import pickle
import dill
import types
def foo():
print ('a')
oldCode=foo.__code__
name='IAmFooCopied'
newCode= types.CodeType(
oldCode.co_argcount, # integer
oldCode.co_kwonlyargcount, # integer
oldCode.co_nlocals, # integer
oldCode.co_stacksize, # integer
oldCode.co_flags, # integer
oldCode.co_code, # bytes
oldCode.co_consts, # tuple
oldCode.co_names, # tuple
oldCode.co_varnames, # tuple
oldCode.co_filename, # string
name, # string
oldCode.co_firstlineno, # integer
oldCode.co_lnotab, # bytes
oldCode.co_freevars, # tuple
oldCode.co_cellvars # tuple
)
IAmFooCopied=types.FunctionType(newCode, foo.__globals__, name,foo.__defaults__ , foo.__closure__)
IAmFooCopied.__qualname__= name
print ( 'printing foo and the copy', IAmFooCopied, foo )
print ( 'dill output: ', dill.dumps(IAmFooCopied ))
print ( 'pickle Output: ', pickle.dumps (IAmFooCopied) )
Output:
printing foo and the copy <function IAmFooCopied at 0x7f8a6a8159d8> <function foo at 0x7f8a6b5f5268>
dill output: b'\x80\x03cdill._dill\n_create_function\nq\x00(cdill._dill\n_load_type\nq\x01X\x08\x00\x00\x00CodeTypeq\x02\x85q\x03Rq\x04(K\x00K\x00K\x00K\x02KCC\x0ct\x00d\x01\x83\x01\x01\x00d\x00S\x00q\x05NX\x01\x00\x00\x00aq\x06\x86q\x07X\x05\x00\x00\x00printq\x08\x85q\t)X\x10\x00\x00\x00testCloneFunc.pyq\nX\x0c\x00\x00\x00IAmFooCopiedq\x0bK\x05C\x02\x00\x01q\x0c))tq\rRq\x0ec__builtin__\n__main__\nh\x0bNN}q\x0ftq\x10Rq\x11.'
pickle Output: b'\x80\x03c__main__\nIAmFooCopied\nq\x00.'
You may encounter problem with the qualname attribute if you try this snippet with class methods (I think pickle should fail to find your function). I never tried it, however it should be easily fixable. Just check the doc about qualname
It's quite easy to do using lambda and rest parameters:
def my_copy(f):
# Create a lambda that mimics f
g = lambda *args: f(*args)
# Add any properties of f
t = list(filter(lambda prop: not ("__" in prop),dir(f)))
i = 0
while i < len(t):
setattr(g,t[i],getattr(f,t[i]))
i += 1
return g
# Test
def sqr(x): return x*x
sqr.foo = 500
sqr_copy = my_copy(sqr)
print(sqr_copy(5)) # -> 25
print(sqr_copy(6)) # -> 36
print(sqr_copy.foo) # -> 500
print(sqr_copy == sqr) # -> False
Try it online!
put it in a function:
def makefunc( docstring ):
def f():
pass
f.__doc__ = docstring
return f
f = makefunc('I am f')
g = makefunc("I am f too")
Adjusted for python3
import types
def copy_func(f, name=None):
return types.FunctionType(f.__code__, f.__globals__, name or f.__name__,
f.__defaults__, f.__closure__)
def func1(x):
return 2*x
func2=copy_func(func1)
print(func2(7))
I've implemented a general-purpose function copy in haggis, a library which I wrote and maintain (available with pip but probably not conda). haggis.objects.copy_func makes a copy on which you can not only reassign the __doc__ attribute, but also modify the module and __globals__ attributes effectively.
from haggis.objects import copy_func
def a(*args, **kwargs):
"""A docstring"""
a2 = copy_func(a)
a2.__doc__ = """Another docstring"""
>>> a == a2
False
>>> a.__code__ == a2.__code__
True
>>> a.__doc__
'A docstring'
>>> a2.__doc__
'Another docstring'
Does there exist special class in python to create empty objects? I tried object(), but it didn't allow me to add fields.
I want to use it like this:
obj = EmptyObject()
obj.foo = 'far'
obj.bar = 'boo'
Should I each time(in several independent scripts) define new class like this?
class EmptyObject:
pass
I use python2.7
types.SimpleNamespace was introduced with Python 3.3 to serve this exact purpose. The documentation also shows a simple way to implement it yourself in Python, so you can add it to your pre-Python 3.3 setup and use it as if it was there (note that the actual implementation is done in C):
class SimpleNamespace (object):
def __init__ (self, **kwargs):
self.__dict__.update(kwargs)
def __repr__ (self):
keys = sorted(self.__dict__)
items = ("{}={!r}".format(k, self.__dict__[k]) for k in keys)
return "{}({})".format(type(self).__name__, ", ".join(items))
def __eq__ (self, other):
return self.__dict__ == other.__dict__
But of course, if you don’t need its few features, a simple class Empty: pass does just the same.
If you are looking for a place holder object to which you can add arbitrary static members, then the closest I got is an empty lambda function.
obj = lambda: None # Dummy function
obj.foo = 'far'
obj.bar = 'boo'
print obj.foo, obj.bar
# far boo
Remember: obj is not an object of a class, object doesn't mean class instance, because in Python classes and functions are objects at runtime just like class instances
There is no types.SimpleNamespace in Python 2.7, you could use collections.namedtuple() for immutable objects instead:
>>> from collections import namedtuple
>>> FooBar = namedtuple('FooBar', 'foo bar')
>>> FooBar('bar', 'foo')
FooBar(foo='bar', bar='foo')
Or argparse.Namespace:
>>> from argparse import Namespace
>>> o = Namespace(foo='bar')
>>> o.bar = 'foo'
>>> o
Namespace(bar='foo', foo='bar')
See also, How can I create an object and add attributes to it?
You can create a new type dynamically with the fields you want it to have using the type function, like this:
x = type('empty', (object,), {'foo': 'bar'})
x.bar = 3
print(x.foo)
This is not entirely what you want though, since it will have a custom type, not an empty type.
I want to iterate through the methods in a class, or handle class or instance objects differently based on the methods present. How do I get a list of class methods?
Also see:
How can I list the methods in a
Python 2.5 module?
Looping over
a Python / IronPython Object
Methods
Finding the methods an
object has
How do I look inside
a Python object?
How Do I
Perform Introspection on an Object in
Python 2.x?
How to get a
complete list of object’s methods and
attributes?
Finding out which
functions are available from a class
instance in python?
An example (listing the methods of the optparse.OptionParser class):
>>> from optparse import OptionParser
>>> import inspect
#python2
>>> inspect.getmembers(OptionParser, predicate=inspect.ismethod)
[([('__init__', <unbound method OptionParser.__init__>),
...
('add_option', <unbound method OptionParser.add_option>),
('add_option_group', <unbound method OptionParser.add_option_group>),
('add_options', <unbound method OptionParser.add_options>),
('check_values', <unbound method OptionParser.check_values>),
('destroy', <unbound method OptionParser.destroy>),
('disable_interspersed_args',
<unbound method OptionParser.disable_interspersed_args>),
('enable_interspersed_args',
<unbound method OptionParser.enable_interspersed_args>),
('error', <unbound method OptionParser.error>),
('exit', <unbound method OptionParser.exit>),
('expand_prog_name', <unbound method OptionParser.expand_prog_name>),
...
]
# python3
>>> inspect.getmembers(OptionParser, predicate=inspect.isfunction)
...
Notice that getmembers returns a list of 2-tuples. The first item is the name of the member, the second item is the value.
You can also pass an instance to getmembers:
>>> parser = OptionParser()
>>> inspect.getmembers(parser, predicate=inspect.ismethod)
...
There is the dir(theobject) method to list all the fields and methods of your object (as a tuple) and the inspect module (as codeape write) to list the fields and methods with their doc (in """).
Because everything (even fields) might be called in Python, I'm not sure there is a built-in function to list only methods. You might want to try if the object you get through dir is callable or not.
Python 3.x answer without external libraries
method_list = [func for func in dir(Foo) if callable(getattr(Foo, func))]
dunder-excluded result:
method_list = [func for func in dir(Foo) if callable(getattr(Foo, func)) and not func.startswith("__")]
Say you want to know all methods associated with list class
Just Type The following
print (dir(list))
Above will give you all methods of list class
Try the property __dict__.
you can also import the FunctionType from types and test it with the class.__dict__:
from types import FunctionType
class Foo:
def bar(self): pass
def baz(self): pass
def methods(cls):
return [x for x, y in cls.__dict__.items() if type(y) == FunctionType]
methods(Foo) # ['bar', 'baz']
You can list all methods in a python class by using the following code
dir(className)
This will return a list of all the names of the methods in the class
Note that you need to consider whether you want methods from base classes which are inherited (but not overridden) included in the result. The dir() and inspect.getmembers() operations do include base class methods, but use of the __dict__ attribute does not.
If your method is a "regular" method and not a staticmethod, classmethod etc.
There is a little hack I came up with -
for k, v in your_class.__dict__.items():
if "function" in str(v):
print(k)
This can be extended to other type of methods by changing "function" in the if condition correspondingly.
Tested in Python 2.7 and Python 3.5.
Try
print(help(ClassName))
It prints out methods of the class
I just keep this there, because top rated answers are not clear.
This is simple test with not usual class based on Enum.
# -*- coding: utf-8 -*-
import sys, inspect
from enum import Enum
class my_enum(Enum):
"""Enum base class my_enum"""
M_ONE = -1
ZERO = 0
ONE = 1
TWO = 2
THREE = 3
def is_natural(self):
return (self.value > 0)
def is_negative(self):
return (self.value < 0)
def is_clean_name(name):
return not name.startswith('_') and not name.endswith('_')
def clean_names(lst):
return [ n for n in lst if is_clean_name(n) ]
def get_items(cls,lst):
try:
res = [ getattr(cls,n) for n in lst ]
except Exception as e:
res = (Exception, type(e), e)
pass
return res
print( sys.version )
dir_res = clean_names( dir(my_enum) )
inspect_res = clean_names( [ x[0] for x in inspect.getmembers(my_enum) ] )
dict_res = clean_names( my_enum.__dict__.keys() )
print( '## names ##' )
print( dir_res )
print( inspect_res )
print( dict_res )
print( '## items ##' )
print( get_items(my_enum,dir_res) )
print( get_items(my_enum,inspect_res) )
print( get_items(my_enum,dict_res) )
And this is output results.
3.7.7 (default, Mar 10 2020, 13:18:53)
[GCC 9.2.1 20200306]
## names ##
['M_ONE', 'ONE', 'THREE', 'TWO', 'ZERO']
['M_ONE', 'ONE', 'THREE', 'TWO', 'ZERO', 'name', 'value']
['is_natural', 'is_negative', 'M_ONE', 'ZERO', 'ONE', 'TWO', 'THREE']
## items ##
[<my_enum.M_ONE: -1>, <my_enum.ONE: 1>, <my_enum.THREE: 3>, <my_enum.TWO: 2>, <my_enum.ZERO: 0>]
(<class 'Exception'>, <class 'AttributeError'>, AttributeError('name'))
[<function my_enum.is_natural at 0xb78a1fa4>, <function my_enum.is_negative at 0xb78ae854>, <my_enum.M_ONE: -1>, <my_enum.ZERO: 0>, <my_enum.ONE: 1>, <my_enum.TWO: 2>, <my_enum.THREE: 3>]
So what we have:
dir provide not complete data
inspect.getmembers provide not complete data and provide internal keys that are not accessible with getattr()
__dict__.keys() provide complete and reliable result
Why are votes so erroneous? And where i'm wrong? And where wrong other people which answers have so low votes?
There's this approach:
[getattr(obj, m) for m in dir(obj) if not m.startswith('__')]
When dealing with a class instance, perhaps it'd be better to return a list with the method references instead of just names¹. If that's your goal, as well as
Using no import
Excluding private methods (e.g. __init__) from the list
It may be of use. You might also want to assure it's callable(getattr(obj, m)), since dir returns all attributes within obj, not just methods.
In a nutshell, for a class like
class Ghost:
def boo(self, who):
return f'Who you gonna call? {who}'
We could check instance retrieval with
>>> g = Ghost()
>>> methods = [getattr(g, m) for m in dir(g) if not m.startswith('__')]
>>> print(methods)
[<bound method Ghost.boo of <__main__.Ghost object at ...>>]
So you can call it right away:
>>> for method in methods:
... print(method('GHOSTBUSTERS'))
...
Who you gonna call? GHOSTBUSTERS
¹ An use case:
I used this for unit testing. Had a class where all methods performed variations of the same process - which led to lengthy tests, each only a tweak away from the others. DRY was a far away dream.
Thought I should have a single test for all methods, so I made the above iteration.
Although I realized I should instead refactor the code itself to be DRY-compliant anyway... this may still serve a random nitpicky soul in the future.
This also works:
In mymodule.py:
def foo(x):
return 'foo'
def bar():
return 'bar'
In another file:
import inspect
import mymodule
method_list = [ func[0] for func in inspect.getmembers(mymodule, predicate=inspect.isroutine) if callable(getattr(mymodule, func[0])) ]
Output:
['foo', 'bar']
From the Python docs:
inspect.isroutine(object)
Return true if the object is a user-defined or built-in function or method.
def find_defining_class(obj, meth_name):
for ty in type(obj).mro():
if meth_name in ty.__dict__:
return ty
So
print find_defining_class(car, 'speedometer')
Think Python page 210
You can use a function which I have created.
def method_finder(classname):
non_magic_class = []
class_methods = dir(classname)
for m in class_methods:
if m.startswith('__'):
continue
else:
non_magic_class.append(m)
return non_magic_class
method_finder(list)
Output:
['append',
'clear',
'copy',
'count',
'extend',
'index',
'insert',
'pop',
'remove',
'reverse',
'sort']
methods = [(func, getattr(o, func)) for func in dir(o) if callable(getattr(o, func))]
gives an identical list as
methods = inspect.getmembers(o, predicate=inspect.ismethod)
does.
I know this is an old post, but just wrote this function and will leave it here is case someone stumbles looking for an answer:
def classMethods(the_class,class_only=False,instance_only=False,exclude_internal=True):
def acceptMethod(tup):
#internal function that analyzes the tuples returned by getmembers tup[1] is the
#actual member object
is_method = inspect.ismethod(tup[1])
if is_method:
bound_to = tup[1].im_self
internal = tup[1].im_func.func_name[:2] == '__' and tup[1].im_func.func_name[-2:] == '__'
if internal and exclude_internal:
include = False
else:
include = (bound_to == the_class and not instance_only) or (bound_to == None and not class_only)
else:
include = False
return include
#uses filter to return results according to internal function and arguments
return filter(acceptMethod,inspect.getmembers(the_class))
use inspect.ismethod and dir and getattr
import inspect
class ClassWithMethods:
def method1(self):
print('method1')
def method2(self):
print('method2')
obj=ClassWithMethods()
method_names = [attr for attr in dir(obj) if inspect.ismethod(getattr(obj,attr))
print(method_names)
output:
[[('method1', <bound method ClassWithMethods.method1 of <__main__.ClassWithMethods object at 0x00000266779AF388>>), ('method2', <bound method ClassWithMethods.method2 of <__main__.ClassWithMethods object at 0x00000266779AF388>>)]]
None of the above worked for me.
I've encountered this problem while writing pytests.
The only work-around I found was to:
1- create another directory and place all my .py files there
2- create a separate directory for my pytests and then importing the classes I'm interested in
This allowed me to get up-to-dated methods within the class - you can change the method names and then use print(dir(class)) to confirm it.
For my use case, I needed to distinguish between class methods, static methods, properties, and instance methods. The inspect module confuses the issue a bit (particularly with class methods and instance methods), so I used vars based on a comment on this SO question. The basic gist is to use vars to get the __dict__ attribute of the class, then filter based on various isinstance checks. For instance methods, I check that it is callable and not a class method. One caveat: this approach of using vars (or __dict__ for that matter) won't work with __slots__. Using Python 3.6.9 (because it's what the Docker image I'm using as my interpreter has):
class MethodAnalyzer:
class_under_test = None
#classmethod
def get_static_methods(cls):
if cls.class_under_test:
return {
k for k, v in vars(cls.class_under_test).items()
if isinstance(v, staticmethod)
}
return {}
#classmethod
def get_class_methods(cls):
if cls.class_under_test:
return {
k for k, v in vars(cls.class_under_test).items()
if isinstance(v, classmethod)
}
return {}
#classmethod
def get_instance_methods(cls):
if cls.class_under_test:
return {
k for k, v in vars(cls.class_under_test).items()
if callable(v) and not isinstance(v, classmethod)
}
return {}
#classmethod
def get_properties(cls):
if cls.class_under_test:
return {
k for k, v in vars(cls.class_under_test).items()
if isinstance(v, property)
}
return {}
To see it in action, I created this little test class:
class Foo:
#staticmethod
def bar(baz):
print(baz)
#property
def bleep(self):
return 'bloop'
#classmethod
def bork(cls):
return cls.__name__
def flank(self):
return 'on your six'
then did:
MethodAnalyzer.class_under_test = Foo
print(MethodAnalyzer.get_instance_methods())
print(MethodAnalyzer.get_class_methods())
print(MethodAnalyzer.get_static_methods())
print(MethodAnalyzer.get_properties())
which output
{'flank'}
{'bork'}
{'bar'}
{'bleep'}
In this example I'm discarding the actual methods, but if you needed to keep them you could just use a dict comprehension instead of a set comprehension:
{
k, v for k, v in vars(cls.class_under_test).items()
if callable(v) and not isinstance(v, classmethod)
}
This is just an observation. "encode" seems to be a method for string objects
str_1 = 'a'
str_1.encode('utf-8')
>>> b'a'
However, if str1 is inspected for methods, an empty list is returned
inspect.getmember(str_1, predicate=inspect.ismethod)
>>> []
So, maybe I am wrong, but the issue seems to be not simple.
To produce a list of methods put the name of the method in a list without the usual parenthesis. Remove the name and attach the parenthesis and that calls the method.
def methodA():
print("# MethodA")
def methodB():
print("# methodB")
a = []
a.append(methodA)
a.append(methodB)
for item in a:
item()
Just like this
pprint.pprint([x for x in dir(list) if not x.startswith("_")])
class CPerson:
def __init__(self, age):
self._age = age
def run(self):
pass
#property
def age(self): return self._age
#staticmethod
def my_static_method(): print("Life is short, you need Python")
#classmethod
def say(cls, msg): return msg
test_class = CPerson
# print(dir(test_class)) # list all the fields and methods of your object
print([(name, t) for name, t in test_class.__dict__.items() if type(t).__name__ == 'function' and not name.startswith('__')])
print([(name, t) for name, t in test_class.__dict__.items() if type(t).__name__ != 'function' and not name.startswith('__')])
output
[('run', <function CPerson.run at 0x0000000002AD3268>)]
[('age', <property object at 0x0000000002368688>), ('my_static_method', <staticmethod object at 0x0000000002ACBD68>), ('say', <classmethod object at 0x0000000002ACF0B8>)]
If you want to list only methods of a python class
import numpy as np
print(np.random.__all__)
Is there a way to conveniently define a C-like structure in Python? I'm tired of writing stuff like:
class MyStruct():
def __init__(self, field1, field2, field3):
self.field1 = field1
self.field2 = field2
self.field3 = field3
Update: Data Classes
With the introduction of Data Classes in Python 3.7 we get very close.
The following example is similar to the NamedTuple example below, but the resulting object is mutable and it allows for default values.
from dataclasses import dataclass
#dataclass
class Point:
x: float
y: float
z: float = 0.0
p = Point(1.5, 2.5)
print(p) # Point(x=1.5, y=2.5, z=0.0)
This plays nicely with the new typing module in case you want to use more specific type annotations.
I've been waiting desperately for this! If you ask me, Data Classes and the new NamedTuple declaration, combined with the typing module are a godsend!
Improved NamedTuple declaration
Since Python 3.6 it became quite simple and beautiful (IMHO), as long as you can live with immutability.
A new way of declaring NamedTuples was introduced, which allows for type annotations as well:
from typing import NamedTuple
class User(NamedTuple):
name: str
class MyStruct(NamedTuple):
foo: str
bar: int
baz: list
qux: User
my_item = MyStruct('foo', 0, ['baz'], User('peter'))
print(my_item) # MyStruct(foo='foo', bar=0, baz=['baz'], qux=User(name='peter'))
Use a named tuple, which was added to the collections module in the standard library in Python 2.6. It's also possible to use Raymond Hettinger's named tuple recipe if you need to support Python 2.4.
It's nice for your basic example, but also covers a bunch of edge cases you might run into later as well. Your fragment above would be written as:
from collections import namedtuple
MyStruct = namedtuple("MyStruct", "field1 field2 field3")
The newly created type can be used like this:
m = MyStruct("foo", "bar", "baz")
You can also use named arguments:
m = MyStruct(field1="foo", field2="bar", field3="baz")
You can use a tuple for a lot of things where you would use a struct in C (something like x,y coordinates or RGB colors for example).
For everything else you can use dictionary, or a utility class like this one:
>>> class Bunch:
... def __init__(self, **kwds):
... self.__dict__.update(kwds)
...
>>> mystruct = Bunch(field1=value1, field2=value2)
I think the "definitive" discussion is here, in the published version of the Python Cookbook.
Perhaps you are looking for Structs without constructors:
class Sample:
name = ''
average = 0.0
values = None # list cannot be initialized here!
s1 = Sample()
s1.name = "sample 1"
s1.values = []
s1.values.append(1)
s1.values.append(2)
s1.values.append(3)
s2 = Sample()
s2.name = "sample 2"
s2.values = []
s2.values.append(4)
for v in s1.values: # prints 1,2,3 --> OK.
print v
print "***"
for v in s2.values: # prints 4 --> OK.
print v
How about a dictionary?
Something like this:
myStruct = {'field1': 'some val', 'field2': 'some val'}
Then you can use this to manipulate values:
print myStruct['field1']
myStruct['field2'] = 'some other values'
And the values don't have to be strings. They can be pretty much any other object.
dF: that's pretty cool... I didn't
know that I could access the fields in
a class using dict.
Mark: the situations that I wish I had
this are precisely when I want a tuple
but nothing as "heavy" as a
dictionary.
You can access the fields of a class using a dictionary because the fields of a class, its methods and all its properties are stored internally using dicts (at least in CPython).
...Which leads us to your second comment. Believing that Python dicts are "heavy" is an extremely non-pythonistic concept. And reading such comments kills my Python Zen. That's not good.
You see, when you declare a class you are actually creating a pretty complex wrapper around a dictionary - so, if anything, you are adding more overhead than by using a simple dictionary. An overhead which, by the way, is meaningless in any case. If you are working on performance critical applications, use C or something.
I would also like to add a solution that uses slots:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
Definitely check the documentation for slots but a quick explanation of slots is that it is python's way of saying: "If you can lock these attributes and only these attributes into the class such that you commit that you will not add any new attributes once the class is instantiated (yes you can add new attributes to a class instance, see example below) then I will do away with the large memory allocation that allows for adding new attributes to a class instance and use just what I need for these slotted attributes".
Example of adding attributes to class instance (thus not using slots):
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
print(p1.z)
Output: 8
Example of trying to add attributes to class instance where slots was used:
class Point:
__slots__ = ["x", "y"]
def __init__(self, x, y):
self.x = x
self.y = y
p1 = Point(3,5)
p1.z = 8
Output: AttributeError: 'Point' object has no attribute 'z'
This can effectively works as a struct and uses less memory than a class (like a struct would, although I have not researched exactly how much). It is recommended to use slots if you will be creating a large amount of instances of the object and do not need to add attributes. A point object is a good example of this as it is likely that one may instantiate many points to describe a dataset.
You can subclass the C structure that is available in the standard library. The ctypes module provides a Structure class. The example from the docs:
>>> from ctypes import *
>>> class POINT(Structure):
... _fields_ = [("x", c_int),
... ("y", c_int)]
...
>>> point = POINT(10, 20)
>>> print point.x, point.y
10 20
>>> point = POINT(y=5)
>>> print point.x, point.y
0 5
>>> POINT(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: too many initializers
>>>
>>> class RECT(Structure):
... _fields_ = [("upperleft", POINT),
... ("lowerright", POINT)]
...
>>> rc = RECT(point)
>>> print rc.upperleft.x, rc.upperleft.y
0 5
>>> print rc.lowerright.x, rc.lowerright.y
0 0
>>>
You can also pass the init parameters to the instance variables by position
# Abstract struct class
class Struct:
def __init__ (self, *argv, **argd):
if len(argd):
# Update by dictionary
self.__dict__.update (argd)
else:
# Update by position
attrs = filter (lambda x: x[0:2] != "__", dir(self))
for n in range(len(argv)):
setattr(self, attrs[n], argv[n])
# Specific class
class Point3dStruct (Struct):
x = 0
y = 0
z = 0
pt1 = Point3dStruct()
pt1.x = 10
print pt1.x
print "-"*10
pt2 = Point3dStruct(5, 6)
print pt2.x, pt2.y
print "-"*10
pt3 = Point3dStruct (x=1, y=2, z=3)
print pt3.x, pt3.y, pt3.z
print "-"*10
Whenever I need an "instant data object that also behaves like a dictionary" (I don't think of C structs!), I think of this cute hack:
class Map(dict):
def __init__(self, **kwargs):
super(Map, self).__init__(**kwargs)
self.__dict__ = self
Now you can just say:
struct = Map(field1='foo', field2='bar', field3=42)
self.assertEquals('bar', struct.field2)
self.assertEquals(42, struct['field3'])
Perfectly handy for those times when you need a "data bag that's NOT a class", and for when namedtuples are incomprehensible...
Some the answers here are massively elaborate. The simplest option I've found is (from: http://norvig.com/python-iaq.html):
class Struct:
"A structure that can have any fields defined."
def __init__(self, **entries): self.__dict__.update(entries)
Initialising:
>>> options = Struct(answer=42, linelen=80, font='courier')
>>> options.answer
42
adding more:
>>> options.cat = "dog"
>>> options.cat
dog
edit: Sorry didn't see this example already further down.
You access C-Style struct in python in following way.
class cstruct:
var_i = 0
var_f = 0.0
var_str = ""
if you just want use object of cstruct
obj = cstruct()
obj.var_i = 50
obj.var_f = 50.00
obj.var_str = "fifty"
print "cstruct: obj i=%d f=%f s=%s" %(obj.var_i, obj.var_f, obj.var_str)
if you want to create an array of objects of cstruct
obj_array = [cstruct() for i in range(10)]
obj_array[0].var_i = 10
obj_array[0].var_f = 10.00
obj_array[0].var_str = "ten"
#go ahead and fill rest of array instaces of struct
#print all the value
for i in range(10):
print "cstruct: obj_array i=%d f=%f s=%s" %(obj_array[i].var_i, obj_array[i].var_f, obj_array[i].var_str)
Note:
instead of 'cstruct' name, please use your struct name
instead of var_i, var_f, var_str, please define your structure's member variable.
This might be a bit late but I made a solution using Python Meta-Classes (decorator version below too).
When __init__ is called during run time, it grabs each of the arguments and their value and assigns them as instance variables to your class. This way you can make a struct-like class without having to assign every value manually.
My example has no error checking so it is easier to follow.
class MyStruct(type):
def __call__(cls, *args, **kwargs):
names = cls.__init__.func_code.co_varnames[1:]
self = type.__call__(cls, *args, **kwargs)
for name, value in zip(names, args):
setattr(self , name, value)
for name, value in kwargs.iteritems():
setattr(self , name, value)
return self
Here it is in action.
>>> class MyClass(object):
__metaclass__ = MyStruct
def __init__(self, a, b, c):
pass
>>> my_instance = MyClass(1, 2, 3)
>>> my_instance.a
1
>>>
I posted it on reddit and /u/matchu posted a decorator version which is cleaner. I'd encourage you to use it unless you want to expand the metaclass version.
>>> def init_all_args(fn):
#wraps(fn)
def wrapped_init(self, *args, **kwargs):
names = fn.func_code.co_varnames[1:]
for name, value in zip(names, args):
setattr(self, name, value)
for name, value in kwargs.iteritems():
setattr(self, name, value)
return wrapped_init
>>> class Test(object):
#init_all_args
def __init__(self, a, b):
pass
>>> a = Test(1, 2)
>>> a.a
1
>>>
I wrote a decorator which you can use on any method to make it so that all of the arguments passed in, or any defaults, are assigned to the instance.
def argumentsToAttributes(method):
argumentNames = method.func_code.co_varnames[1:]
# Generate a dictionary of default values:
defaultsDict = {}
defaults = method.func_defaults if method.func_defaults else ()
for i, default in enumerate(defaults, start = len(argumentNames) - len(defaults)):
defaultsDict[argumentNames[i]] = default
def newMethod(self, *args, **kwargs):
# Use the positional arguments.
for name, value in zip(argumentNames, args):
setattr(self, name, value)
# Add the key word arguments. If anything is missing, use the default.
for name in argumentNames[len(args):]:
setattr(self, name, kwargs.get(name, defaultsDict[name]))
# Run whatever else the method needs to do.
method(self, *args, **kwargs)
return newMethod
A quick demonstration. Note that I use a positional argument a, use the default value for b, and a named argument c. I then print all 3 referencing self, to show that they've been properly assigned before the method is entered.
class A(object):
#argumentsToAttributes
def __init__(self, a, b = 'Invisible', c = 'Hello'):
print(self.a)
print(self.b)
print(self.c)
A('Why', c = 'Nothing')
Note that my decorator should work with any method, not just __init__.
I don't see this answer here, so I figure I'll add it since I'm leaning Python right now and just discovered it. The Python tutorial (Python 2 in this case) gives the following simple and effective example:
class Employee:
pass
john = Employee() # Create an empty employee record
# Fill the fields of the record
john.name = 'John Doe'
john.dept = 'computer lab'
john.salary = 1000
That is, an empty class object is created, then instantiated, and the fields are added dynamically.
The up-side to this is its really simple. The downside is it isn't particularly self-documenting (the intended members aren't listed anywhere in the class "definition"), and unset fields can cause problems when accessed. Those two problems can be solved by:
class Employee:
def __init__ (self):
self.name = None # or whatever
self.dept = None
self.salary = None
Now at a glance you can at least see what fields the program will be expecting.
Both are prone to typos, john.slarly = 1000 will succeed. Still, it works.
Here is a solution which uses a class (never instantiated) to hold data. I like that this way involves very little typing and does not require any additional packages etc.
class myStruct:
field1 = "one"
field2 = "2"
You can add more fields later, as needed:
myStruct.field3 = 3
To get the values, the fields are accessed as usual:
>>> myStruct.field1
'one'
Personally, I like this variant too. It extends #dF's answer.
class struct:
def __init__(self, *sequential, **named):
fields = dict(zip(sequential, [None]*len(sequential)), **named)
self.__dict__.update(fields)
def __repr__(self):
return str(self.__dict__)
It supports two modes of initialization (that can be blended):
# Struct with field1, field2, field3 that are initialized to None.
mystruct1 = struct("field1", "field2", "field3")
# Struct with field1, field2, field3 that are initialized according to arguments.
mystruct2 = struct(field1=1, field2=2, field3=3)
Also, it prints nicer:
print(mystruct2)
# Prints: {'field3': 3, 'field1': 1, 'field2': 2}
There is a python package exactly for this purpose. see cstruct2py
cstruct2py is a pure python library for generate python classes from C code and use them to pack and unpack data. The library can parse C headres (structs, unions, enums, and arrays declarations) and emulate them in python. The generated pythonic classes can parse and pack the data.
For example:
typedef struct {
int x;
int y;
} Point;
after generating pythonic class...
p = Point(x=0x1234, y=0x5678)
p.packed == "\x34\x12\x00\x00\x78\x56\x00\x00"
How to use
First we need to generate the pythonic structs:
import cstruct2py
parser = cstruct2py.c2py.Parser()
parser.parse_file('examples/example.h')
Now we can import all names from the C code:
parser.update_globals(globals())
We can also do that directly:
A = parser.parse_string('struct A { int x; int y;};')
Using types and defines from the C code
a = A()
a.x = 45
print a
buf = a.packed
b = A(buf)
print b
c = A('aaaa11112222', 2)
print c
print repr(c)
The output will be:
{'x':0x2d, 'y':0x0}
{'x':0x2d, 'y':0x0}
{'x':0x31316161, 'y':0x32323131}
A('aa111122', x=0x31316161, y=0x32323131)
Clone
For clone cstruct2py run:
git clone https://github.com/st0ky/cstruct2py.git --recursive
Here is a quick and dirty trick:
>>> ms = Warning()
>>> ms.foo = 123
>>> ms.bar = 'akafrit'
How does it works? It just re-use the builtin class Warning (derived from Exception) and use it as it was you own defined class.
The good points are that you do not need to import or define anything first, that "Warning" is a short name, and that it also makes clear you are doing something dirty which should not be used elsewhere than a small script of yours.
By the way, I tried to find something even simpler like ms = object() but could not (this last exemple is not working). If you have one, I am interested.
NamedTuple is comfortable. but there no one shares the performance and storage.
from typing import NamedTuple
import guppy # pip install guppy
import timeit
class User:
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserSlot:
__slots__ = ('name', 'uid')
def __init__(self, name: str, uid: int):
self.name = name
self.uid = uid
class UserTuple(NamedTuple):
# __slots__ = () # AttributeError: Cannot overwrite NamedTuple attribute __slots__
name: str
uid: int
def get_fn(obj, attr_name: str):
def get():
getattr(obj, attr_name)
return get
if 'memory test':
obj = [User('Carson', 1) for _ in range(1000000)] # Cumulative: 189138883
obj_slot = [UserSlot('Carson', 1) for _ in range(1000000)] # 77718299 <-- winner
obj_namedtuple = [UserTuple('Carson', 1) for _ in range(1000000)] # 85718297
print(guppy.hpy().heap()) # Run this function individually.
"""
Index Count % Size % Cumulative % Kind (class / dict of class)
0 1000000 24 112000000 34 112000000 34 dict of __main__.User
1 1000000 24 64000000 19 176000000 53 __main__.UserTuple
2 1000000 24 56000000 17 232000000 70 __main__.User
3 1000000 24 56000000 17 288000000 87 __main__.UserSlot
...
"""
if 'performance test':
obj = User('Carson', 1)
obj_slot = UserSlot('Carson', 1)
obj_tuple = UserTuple('Carson', 1)
time_normal = min(timeit.repeat(get_fn(obj, 'name'), repeat=20))
print(time_normal) # 0.12550550000000005
time_slot = min(timeit.repeat(get_fn(obj_slot, 'name'), repeat=20))
print(time_slot) # 0.1368690000000008
time_tuple = min(timeit.repeat(get_fn(obj_tuple, 'name'), repeat=20))
print(time_tuple) # 0.16006120000000124
print(time_tuple/time_slot) # 1.1694481584580898 # The slot is almost 17% faster than NamedTuple on Windows. (Python 3.7.7)
If your __dict__ is not using, please choose between __slots__ (higher performance and storage) and NamedTuple (clear for reading and use)
You can review this link(Usage of slots
) to get more __slots__ information.
https://stackoverflow.com/a/32448434/159695 does not work in Python3.
https://stackoverflow.com/a/35993/159695 works in Python3.
And I extends it to add default values.
class myStruct:
def __init__(self, **kwds):
self.x=0
self.__dict__.update(kwds) # Must be last to accept assigned member variable.
def __repr__(self):
args = ['%s=%s' % (k, repr(v)) for (k,v) in vars(self).items()]
return '%s(%s)' % ( self.__class__.__qualname__, ', '.join(args) )
a=myStruct()
b=myStruct(x=3,y='test')
c=myStruct(x='str')
>>> a
myStruct(x=0)
>>> b
myStruct(x=3, y='test')
>>> c
myStruct(x='str')
The following solution to a struct is inspired by the namedtuple implementation and some of the previous answers. However, unlike the namedtuple it is mutable, in it's values, but like the c-style struct immutable in the names/attributes, which a normal class or dict isn't.
_class_template = """\
class {typename}:
def __init__(self, *args, **kwargs):
fields = {field_names!r}
for x in fields:
setattr(self, x, None)
for name, value in zip(fields, args):
setattr(self, name, value)
for name, value in kwargs.items():
setattr(self, name, value)
def __repr__(self):
return str(vars(self))
def __setattr__(self, name, value):
if name not in {field_names!r}:
raise KeyError("invalid name: %s" % name)
object.__setattr__(self, name, value)
"""
def struct(typename, field_names):
class_definition = _class_template.format(
typename = typename,
field_names = field_names)
namespace = dict(__name__='struct_%s' % typename)
exec(class_definition, namespace)
result = namespace[typename]
result._source = class_definition
return result
Usage:
Person = struct('Person', ['firstname','lastname'])
generic = Person()
michael = Person('Michael')
jones = Person(lastname = 'Jones')
In [168]: michael.middlename = 'ben'
Traceback (most recent call last):
File "<ipython-input-168-b31c393c0d67>", line 1, in <module>
michael.middlename = 'ben'
File "<string>", line 19, in __setattr__
KeyError: 'invalid name: middlename'
If you don't have a 3.7 for #dataclass and need mutability, the following code might work for you. It's quite self-documenting and IDE-friendly (auto-complete), prevents writing things twice, is easily extendable and it is very simple to test that all instance variables are completely initialized:
class Params():
def __init__(self):
self.var1 : int = None
self.var2 : str = None
def are_all_defined(self):
for key, value in self.__dict__.items():
assert (value is not None), "instance variable {} is still None".format(key)
return True
params = Params()
params.var1 = 2
params.var2 = 'hello'
assert(params.are_all_defined)
The best way I found to do this was to use a custom dictionary class as explained in this post: https://stackoverflow.com/a/14620633/8484485
If iPython autocompletion support is needed, simply define the dir() function like this:
class AttrDict(dict):
def __init__(self, *args, **kwargs):
super(AttrDict, self).__init__(*args, **kwargs)
self.__dict__ = self
def __dir__(self):
return self.keys()
You then define your pseudo struct like so: (this one is nested)
my_struct=AttrDict ({
'com1':AttrDict ({
'inst':[0x05],
'numbytes':2,
'canpayload':False,
'payload':None
})
})
You can then access the values inside my_struct like this:
print(my_struct.com1.inst)
=>[5]
The cleanest way I can think of is to use a class decorator that lets you declare a static class and rewrite it to act as a struct with normal, named properties:
from as_struct import struct
#struct
class Product():
name = 'unknown product'
quantity = -1
sku = '-'
# create instance
p = Product('plush toy', sku='12-345-6789')
# check content:
p.name # plush toy
p.quantity # -1
p.sku # 12-345-6789
Using the following decorator code:
def struct(struct_class):
# create a new init
def struct_init(self, *args, **kwargs):
i = 0 # we really don't need enumerate() here...
for value in args:
name = member_names[i]
default_value = member_values[i]
setattr(self, name, value if value is not None else default_value)
i += 1 # ...we just need to inc an int
for key,value in kwargs.items():
i = member_names.index(key)
default_value = member_values[i]
setattr(self, key, value if value is not None else default_value)
# extract the struct members
member_names = []
member_values = []
for attr_name in dir(struct_class):
if not attr_name.startswith('_'):
value = getattr(struct_class, attr_name)
if not callable(value):
member_names.append(attr_name)
member_values.append(value)
# rebind and return
struct_class.init = struct_init
return struct_class
Which works by taking the class, extracting the field names and their default values, then rewriting the class's __init__ function to set self attributes based on knowing which argument index maps to which property name.
I think Python structure dictionary is suitable for this requirement.
d = dict{}
d[field1] = field1
d[field2] = field2
d[field2] = field3
Extending #gz.'s (generally superior to this one) answer, for a quick and dirty namedtuple structure we can do:
import collections
x = collections.namedtuple('foobar', 'foo bar')(foo=1,bar=2)
y = collections.namedtuple('foobar', 'foo bar')(foo=3,bar=4)
print(x,y)
>foobar(foo=1, bar=2) foobar(foo=3, bar=4)