I have some trouble with Python class creations. My task is to create objects using some parse method. But I want to turn off basic class creation using __init__
For example, I have
class A:
#classmethod
def create_from_file(cls, file):
# Create instance form file...
return self
This gives me an opportunity to create object using command like this
a = A.create_from_file()
But this code provides me a chance to create instance using __init__
a = A() won't raise an exception...
When I try to add own __init__ method, my parse function also raises an exception.
class A:
def __init__(self):
raise NotImplementedError
#classmethod
def create_from_file(cls, file):
# Create instance form file...
return self
How I can fix this trouble, and what is the most Pythonic way to write this classes?
__init__ is not responsible for creating a instance. It is a hook method that Python calls for you after the instance is already created. You can't prevent instance creation from there. Besides, you don't want to prevent all instance creation, even your classmethod has to create an instance at some point.
Since all you want to do is raise an exception when your factory method is not used to create the instance, it's still fine to raise an exception in __init__ method. That'll prevent the new instance from being assigned anywhere. What you need to do then is distinguish between direct access, and your factory method being used.
You could achieve this is several different ways. You could use a "secret" token that only the factory method passes in:
_token = object() # unique token to flag factory use
class A:
def __init__(self, data, _from_factory=None):
if _from_factory is not _token:
raise TypeError(f"Can't create {type(self).__name__!r} objects directly")
self._data = data
#classmethod
def create_from_file(cls, file):
data = file.read()
return cls(data, _from_factory=_token)
The classmethod still creates an instance, the __init__ is still called for that instance, and no exception is raised because the right token was passed in.
You could make your class an implementation detail of the module and only provide a public factory function:
def create_from_file(cls, file):
data = file.read()
return _A(data)
class _A:
def __init__(self, data):
self._data = data
Now the public API only gives you create_from_file(), the leading underscore tells developers that _A() is an internal name and should not be relied on outside of the module.
Actual instance creation is the responsibility of the object.__new__ method; you could also use that method to prevent new instances to be created. You could use the same token approach as I showed above, or you could bypass it altogether by using super() to call the original overridden implementation:
class A:
def __new__(cls, *args, **kwargs):
raise TypeError(f"Can't create {cls.__name__!r} objects directly")
def __init__(self, data):
self._data = data
#classmethod
def create_from_file(cls, file):
data = file.read()
# Don't use __new__ *on this class*, but on the next one in the
# MRO. We'll have to manually apply __init__ now.
instance = super().__new__(cls)
instance.__init__(data)
return instance
Here a direct call to A() will raise an exception, but by using super().__new__ in the classmethod we bypass the A.__new__ implementation.
Note: __new__ is implicitly made a staticmethod, so we have to manually pass in the cls argument when we call it from the classmethod.
If you only have one method to create the object, just make that the constructor. That is, instead of
#classmethod
def create_from_file(cls, file):
# Create instance form file...
return self
you would have
def __init__(self, file):
# Create instance form file...
If there are several different methods of creating an object, it's still typically going to be the case that one of them is more fundamental than any other - in other words, that the arguments to other methods can be "converted" into the arguments to the one method. For example, you might have create_from_file(), create_from_url(), and create_from_string(), where the former two methods basically read the content of the file or URL and then do the same thing with it that create_from_string() does. So just turn create_from_string() into __init__(), and have the other two methods read the file or URL and call the constructor with the content.
If you really don't have a single "fundamental" way of creating the object, it's probably worth considering whether you should have different subclasses of a base class.
Related
I have an oddly specific issue related to the overarching NoneType object issues that are frequently found on SE. I am attempting to use a decorator function to open a file while the decorated functions applies specific operations that are file type specific (aka text, rich text, PDF, etc.)
The process parses fine, and works through object creation and class initialization, but throws the exception when I try to call the PDFReader function which currently only has a debug print statement of "Reached".
I have found that it is due to the double defined method I am using as the function call seems to be in the parent function where the actual parameters I am trying to access can only be used in the child function.
My question is: If I wish to have a class bound decorator that uses the object's initialized variables via self, how can I fix my code or is there a better way to achieve what I am trying?
Filehandler.py:
class Fileloader(object):
class _decorators():
#classmethod
def _reader(cls, decorated):
print('File opened in Read-Only Mode')
#---No error if "decorated()" called here.
def decorator(self, *args, **kwargs):
print(self._file) #---Prints to console then throws the exception
decorated(self, *args, **kwargs)
...
def __init__(self, filename, mode='rb'):
self._file = filename
self._mode = mode
print(self._file)
#_decorators._reader
def pdfReader(self):
print("reached") #---never reached
...
Test = Fileloader('test.pdf')
Test.pdfReader()
Super easy fix, I just was staring at the code for to long to see it.
Thanks #martineau for pointing out that my outer decorator does not return anything.
The fixed code is as so
class Fileloader(object):
class _decorators():
#classmethod
def _reader(cls, decorated):
print('File opened in Read-Only Mode')
def decorator(self, *args, **kwargs):
print(self._file)
with open(self._file, mode=self._mode) as self.f:
print('Decorator')
decorated(self, *args, **kwargs)
pass
return decorator #<----- Added this and it works as it should
edit: For any of those after me that need an explanation of what happened, I made an inner class class _decorators() to allow the use of a nested function (#classmethod def_reader(cls, decorated) which takes the required parameters of cls (short for class) and decorated (which is the keyword I used for the function passed). Then an inner function is defined so that I have access to the self dictionary of that object, this is done due to how Python works.
Python will first parse through a portion of the code, so if I only had the inner function decorator then it would pass an exception as at this time there is no self attribute.
After it initializes the object via Test = Fileloader('test.pdf') then it DOES have a self attribute that can be accessed by the inner function decorator at runtime. Hence the need for a nested function as the parser will pass by the inner function during parsetime but will come back and read it during runtime when the object is instantiated.
I'd like a particular function to be callable as a classmethod, and to behave differently when it's called on an instance.
For example, if I have a class Thing, I want Thing.get_other_thing() to work, but also thing = Thing(); thing.get_other_thing() to behave differently.
I think overwriting the get_other_thing method on initialization should work (see below), but that seems a bit hacky. Is there a better way?
class Thing:
def __init__(self):
self.get_other_thing = self._get_other_thing_inst()
#classmethod
def get_other_thing(cls):
# do something...
def _get_other_thing_inst(self):
# do something else
Great question! What you seek can be easily done using descriptors.
Descriptors are Python objects which implement the descriptor protocol, usually starting with __get__().
They exist, mostly, to be set as a class attribute on different classes. Upon accessing them, their __get__() method is called, with the instance and owner class passed in.
class DifferentFunc:
"""Deploys a different function accroding to attribute access
I am a descriptor.
"""
def __init__(self, clsfunc, instfunc):
# Set our functions
self.clsfunc = clsfunc
self.instfunc = instfunc
def __get__(self, inst, owner):
# Accessed from class
if inst is None:
return self.clsfunc.__get__(None, owner)
# Accessed from instance
return self.instfunc.__get__(inst, owner)
class Test:
#classmethod
def _get_other_thing(cls):
print("Accessed through class")
def _get_other_thing_inst(inst):
print("Accessed through instance")
get_other_thing = DifferentFunc(_get_other_thing,
_get_other_thing_inst)
And now for the result:
>>> Test.get_other_thing()
Accessed through class
>>> Test().get_other_thing()
Accessed through instance
That was easy!
By the way, did you notice me using __get__ on the class and instance function? Guess what? Functions are also descriptors, and that's the way they work!
>>> def func(self):
... pass
...
>>> func.__get__(object(), object)
<bound method func of <object object at 0x000000000046E100>>
Upon accessing a function attribute, it's __get__ is called, and that's how you get function binding.
For more information, I highly suggest reading the Python manual and the "How-To" linked above. Descriptors are one of Python's most powerful features and are barely even known.
Why not set the function on instantiation?
Or Why not set self.func = self._func inside __init__?
Setting the function on instantiation comes with quite a few problems:
self.func = self._funccauses a circular reference. The instance is stored inside the function object returned by self._func. This on the other hand is stored upon the instance during the assignment. The end result is that the instance references itself and will clean up in a much slower and heavier manner.
Other code interacting with your class might attempt to take the function straight out of the class, and use __get__(), which is the usual expected method, to bind it. They will receive the wrong function.
Will not work with __slots__.
Although with descriptors you need to understand the mechanism, setting it on __init__ isn't as clean and requires setting multiple functions on __init__.
Takes more memory. Instead of storing one single function, you store a bound function for each and every instance.
Will not work with properties.
There are many more that I didn't add as the list goes on and on.
Here is a bit hacky solution:
class Thing(object):
#staticmethod
def get_other_thing():
return 1
def __getattribute__(self, name):
if name == 'get_other_thing':
return lambda: 2
return super(Thing, self).__getattribute__(name)
print Thing.get_other_thing() # 1
print Thing().get_other_thing() # 2
If we are on class, staticmethod is executed. If we are on instance, __getattribute__ is first to be executed, so we can return not Thing.get_other_thing but some other function (lambda in my case)
I've created a Singleton using a MetaClass as discussed in Method 3 of this answer
class Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
return cls._instances[cls]
class MySing(metaclass=Singleton): ...
I'd like to be able to clear the Singleton in the setUp() method of a unittest.TestCase so that each test starts with a clean Singleton.
I guess I don't really understand what this metaClass is doing because I can't get the correct incantation for a clear() method:
def clear(self):
try:
del(Singleton._instances[type(self)]
except KeyError:
pass #Sometimes we clear before creating
Any thoughts on what I'm doing wrong here? My singleton is not getting cleared.
sing=MySing()
sing.clear()
The type call above returns Singleton not MySing.
Let's walk through a (corrected) definition of Singleton and a class defined using it. I'm replacing uses of cls with Singleton where the lookup is passed through anyway.
class Singleton(type):
_instances = {}
# Each of the following functions use cls instead of self
# to emphasize that although they are instance methods of
# Singleton, they are also *class* methods of a class defined
# with Singleton
def __call__(cls, *args, **kwargs):
if cls not in Singleton._instances:
Singleton._instances[cls] = super().__call__(*args, **kwargs)
return Singleton._instances[cls]
def clear(cls):
try:
del Singleton._instances[cls]
except KeyError:
pass
class MySing(metaclass=Singleton):
pass
s1 = MySing() # First call: actually creates a new instance
s2 = MySing() # Second call: returns the cached instance
assert s1 is s2 # Yup, they are the same
MySing.clear() # Throw away the cached instance
s3 = MySing() # Third call: no cached instance, so create one
assert s1 is not s3 # Yup, s3 is a distinct new instance
First, _instances is a class attribute of the metaclass, meant to map a class to a unique instance of that class.
__call__ is an instance method of the metaclass; its purpose is to make instances of the metaclass (i.e., classes) callable. cls here is the class being defined, not the metaclass. So each time you call MyClass(), that converts to Singleton.__call__(MyClass).
clear is also a instance method of the metaclass, meaning it also takes a instance of the meta class (i.e again, a class) as an argument (not an instance of the class defined with the metaclass.) This means MyClass.clear() is the same as Singleton.clear(MyClass). (This also means you can, but probably shouldn't for clarity, write s1.clear().)
The identification of metaclass instance methods with "regular" class class methods also explains why you need to use __call__ in the meta class where you would use __new__ in the regular class: __new__ is special-cased as a class method without having to decorate it as such. It's slightly tricky for a metaclass to define an instance method for its instances, so we just use __call__ (since type.__call__ doesn't do much, if anything, beyond invoking the correct __new__ method).
I see three useful test cases for this metaclass at first glance.
Test whether a single class creation works properly.
Test whether no new class is created after the initial one.
Test whether multiple classes can use this metaclass in conjunction.
All of these tests can be achieved without a "reset" button. After which you'll have covered most of your bases. (I might have forgotten one).
Simply create a few different TestClasses that use this metaclass and check their Id's and types.
I'm trying to write a class method decorator that modifies its class' state. I'm having troubles implementing it at the moment.
Side question: When does a decorator get called? Does it load when the class is instantiated or on during read time when the class read?
What I'm trying to do is this:
class ObjMeta(object):
methods = []
# This should be a decorator that magically updates the 'methods'
# attribute (or list) of this class that's being read by the proxying
# class below.
def method_wrapper(method):
#functools.wraps(method)
def wrapper(*args, **kwargs):
ObjMeta.methods.append(method.__name__)
return method(*args, **kwargs)
return wrapper
# Our methods
#method_wrapper
def method1(self, *args):
return args
#method_wrapper
def method2(self, *args):
return args
class Obj(object):
klass = None
def __init__(self, object_class=ObjMeta):
self.klass = object_class
self._set_methods(object_class)
# We dynamically load the method proxies that calls to our meta class
# that actually contains the methods. It's actually dependent to the
# meta class' methods attribute that contains a list of names of its
# existing methods. This is where I wanted it to be done automagically with
# the help of decorators
def _set_methods(self, object_class):
for method_name in object_class:
setattr(self, method_name, self._proxy_method(method_name))
# Proxies the method that's being called to our meta class
def _proxy_method(self, method_name):
def wrapper(*fargs, **fkwargs):
return getattr(self.klass(*fargs, **fkwargs), method_name)
return wrapper()
I think it's ugly to write a list of methods manually in the class so perhaps a decorator would fix this.
It's for an open-source project I'm working that ports underscore.js to python. I understand that it says I should just use itertools or something. I'm just doing this just for the love of programming and learning. BTW, project is hosted here
Thanks!
There are a few things wrong here.
Anything inside the inner wrapper is called when the method itself is called. Basically, you're replacing the method with that function, which wraps the original. So, your code as it stands would add the method name to the list each time it is called, which probably isn't what you want. Instead, that append should be at the method_wrapper level, ie outside of the inner wrapper. This is called when the method is defined, which happens the first time the module containing the class is imported.
The second thing wrong is that you never actually call the method - you simply return it. Instead of return method you should be returning the value of calling the method with the supplied args - return method(*args, **kwargs).
Is there a way to circumvent the constructor __init__ of a class in python?
Example:
class A(object):
def __init__(self):
print "FAILURE"
def Print(self):
print "YEHAA"
Now I would like to create an instance of A. It could look like this, however this syntax is not correct.
a = A
a.Print()
EDIT:
An even more complex example:
Suppose I have an object C, which purpose it is to store one single parameter and do some computations with it. The parameter, however, is not passed as such but it is embedded in a huge parameter file. It could look something like this:
class C(object):
def __init__(self, ParameterFile):
self._Parameter = self._ExtractParamterFile(ParameterFile)
def _ExtractParamterFile(self, ParameterFile):
#does some complex magic to extract the right parameter
return the_extracted_parameter
Now I would like to dump and load an instance of that object C. However, when I load this object, I only have the single variable self._Parameter and I cannot call the constructor, because it is expecting the parameter file.
#staticmethod
def Load(file):
f = open(file, "rb")
oldObject = pickle.load(f)
f.close()
#somehow create newObject without calling __init__
newObject._Parameter = oldObject._Parameter
return newObject
In other words, it is not possible to create an instance without passing the parameter file. In my "real" case, however, it is not a parameter file but some huge junk of data I certainly not want to carry around in memory or even store it to disc.
And since I want to return an instance of C from the method Load I do somehow have to call the constructor.
OLD EDIT:
A more complex example, which explains why I am asking the question:
class B(object):
def __init__(self, name, data):
self._Name = name
#do something with data, but do NOT save data in a variable
#staticmethod
def Load(self, file, newName):
f = open(file, "rb")
s = pickle.load(f)
f.close()
newS = B(???)
newS._Name = newName
return newS
As you can see, since data is not stored in a class variable I cannot pass it to __init__. Of course I could simply store it, but what if the data is a huge object, which I do not want to carry around in memory all the time or even save it to disc?
You can circumvent __init__ by calling __new__ directly. Then you can create a object of the given type and call an alternative method for __init__. This is something that pickle would do.
However, first I'd like to stress very much that it is something that you shouldn't do and whatever you're trying to achieve, there are better ways to do it, some of which have been mentioned in the other answers. In particular, it's a bad idea to skip calling __init__.
When objects are created, more or less this happens:
a = A.__new__(A, *args, **kwargs)
a.__init__(*args, **kwargs)
You could skip the second step.
Here's why you shouldn't do this: The purpose of __init__ is to initialize the object, fill in all the fields and ensure that the __init__ methods of the parent classes are also called. With pickle it is an exception because it tries to store all the data associated with the object (including any fields/instance variables that are set for the object), and so anything that was set by __init__ the previous time would be restored by pickle, there's no need to call it again.
If you skip __init__ and use an alternative initializer, you'd have a sort of a code duplication - there would be two places where the instance variables are filled in, and it's easy to miss one of them in one of the initializers or accidentally make the two fill the fields act differently. This gives the possibility of subtle bugs that aren't that trivial to trace (you'd have to know which initializer was called), and the code will be more difficult to maintain. Not to mention that you'd be in an even bigger mess if you're using inheritance - the problems will go up the inheritance chain, because you'd have to use this alternative initializer everywhere up the chain.
Also by doing so you'd be more or less overriding Python's instance creation and making your own. Python already does that for you pretty well, no need to go reinventing it and it will confuse people using your code.
Here's what to best do instead: Use a single __init__ method that is to be called for all possible instantiations of the class that initializes all instance variables properly. For different modes of initialization use either of the two approaches:
Support different signatures for __init__ that handle your cases by using optional arguments.
Create several class methods that serve as alternative constructors. Make sure they all create instances of the class in the normal way (i.e. calling __init__), as shown by Roman Bodnarchuk, while performing additional work or whatever. It's best if they pass all the data to the class (and __init__ handles it), but if that's impossible or inconvenient, you can set some instance variables after the instance was created and __init__ is done initializing.
If __init__ has an optional step (e.g. like processing that data argument, although you'd have to be more specific), you can either make it an optional argument or make a normal method that does the processing... or both.
Use classmethod decorator for your Load method:
class B(object):
def __init__(self, name, data):
self._Name = name
#store data
#classmethod
def Load(cls, file, newName):
f = open(file, "rb")
s = pickle.load(f)
f.close()
return cls(newName, s)
So you can do:
loaded_obj = B.Load('filename.txt', 'foo')
Edit:
Anyway, if you still want to omit __init__ method, try __new__:
>>> class A(object):
... def __init__(self):
... print '__init__'
...
>>> A()
__init__
<__main__.A object at 0x800f1f710>
>>> a = A.__new__(A)
>>> a
<__main__.A object at 0x800f1fd50>
Taking your question literally I would use meta classes :
class MetaSkipInit(type):
def __call__(cls):
return cls.__new__(cls)
class B(object):
__metaclass__ = MetaSkipInit
def __init__(self):
print "FAILURE"
def Print(self):
print "YEHAA"
b = B()
b.Print()
This can be useful e.g. for copying constructors without polluting the parameter list.
But to do this properly would be more work and care than my proposed hack.
Not really. The purpose of __init__ is to instantiate an object, and by default it really doesn't do anything. If the __init__ method is not doing what you want, and it's not your own code to change, you can choose to switch it out though. For example, taking your class A, we could do the following to avoid calling that __init__ method:
def emptyinit(self):
pass
A.__init__ = emptyinit
a = A()
a.Print()
This will dynamically switch out which __init__ method from the class, replacing it with an empty call. Note that this is probably NOT a good thing to do, as it does not call the super class's __init__ method.
You could also subclass it to create your own class that does everything the same, except overriding the __init__ method to do what you want it to (perhaps nothing).
Perhaps, however, you simply wish to call the method from the class without instantiating an object. If that is the case, you should look into the #classmethod and #staticmethod decorators. They allow for just that type of behavior.
In your code you have put the #staticmethod decorator, which does not take a self argument. Perhaps what may be better for the purpose would a #classmethod, which might look more like this:
#classmethod
def Load(cls, file, newName):
# Get the data
data = getdata()
# Create an instance of B with the data
return cls.B(newName, data)
UPDATE: Rosh's Excellent answer pointed out that you CAN avoid calling __init__ by implementing __new__, which I was actually unaware of (although it makes perfect sense). Thanks Rosh!
I was reading the Python cookbook and there's a section talking about this: the example is given using __new__ to bypass __init__()
>>> class A:
def __init__(self,a):
self.a = a
>>> test = A('a')
>>> test.a
'a'
>>> test_noinit = A.__new__(A)
>>> test_noinit.a
Traceback (most recent call last):
File "", line 1, in
test_noinit.a
AttributeError: 'A' object has no attribute 'a'
>>>
However I think this only works in Python3. Below is running under 2.7
>>> class A:
def __init__(self,a):
self.a = a
>>> test = A.__new__(A)
Traceback (most recent call last):
File "", line 1, in
test = A.__new__(A)
AttributeError: class A has no attribute '__new__'
>>>
As I said in my comment you could change your __init__ method so that it allows creation without giving any values to its parameters:
def __init__(self, p0, p1, p2):
# some logic
would become:
def __init__(self, p0=None, p1=None, p2=None):
if p0 and p1 and p2:
# some logic
or:
def __init__(self, p0=None, p1=None, p2=None, init=True):
if init:
# some logic