I'm writing a wrapper for the GMAIL API. In this wrapper, I am trying to include subattributes in the "main class" so it more closely follows the below:
Previously, I was use methods such as:
class Foo:
def __init__(self, ...):
# add some attributes
def get_method(self, ...):
return some_stuff
This allows me to do foo.get_method(...). To follow the GMAIL API, I try to do:
class Foo:
def __init__(self, ...):
# add some attributes
#property
def method(self):
class _Method:
#staticmethod
def get(self, ...):
return some_stuff
return _Method()
Which allows me to do foo.method.get(...). The above has some problems, it redefines the class every time, and I have to add #staticmethod above every method as part of it. I do realise that I could create the class at the outer class level, and set a hidden variable for each which then .method returns or creates, but this seems like too much workaround.
tldr: Is it possible to make the instance passed to the inner class as self be the instance of the outer class (I do not wish to have to pass the attributes of the outer class to each inner class).
Instead of sharing the self parameter between classes, you are probably better off just passing the things you need to the constructor of the class you instantiate.
class Messages:
def __init__(self, name):
self.name = name
def method(self, other_arg):
return self.name + other_arg
class Test:
name = "hi"
def __init__(self):
self.messages = Messages(name=self.name)
If you need to pass a lot of information to the constructor and it starts becoming unwieldy, you can do something like split the shared code into a third class, and then pass that between the Test and Messages classes as a single object.
In Python there are all sorts of clever things that you can do with metaclasses and magic methods, but in 99% of cases just refactoring things into different classes and functions will get you more readable and maintainable code.
Users should have an instance of messages, which allows method get. The scetch for code is:
class Messages:
...
def get()
...
class Users:
...
messages = Messages(...)
allows
users = Users()
users.messages.get()
The bad thing in this API is plural names, which is a bad sign for class. If done from scratch you would rather have classes User and Message, which make more sense.
If you have a closer look at GET/POST calls in the API you link provided, you would notice the urls are like UserId/settings, another hint to implement User class, not Users.
self in the methods reference the self of the outer class
maybe this is what you want factory-method
Although the example code I'll provide bellow might be similar to the already provided answers, and the link above to another answer might satify you wish, because it is slight different formed I'll still provide my vision on what you asked. The code is self explanatory.
class User:
def __init__(self, pk, name):
self.pk = pk
self.name = name
self._messages = None
def messages(self):
if self.messages is None:
self._messages = Messages(self.pk)
return self._messages
class Messages:
def __init__(self, usr):
self.usr = usr
def get(self):
return self._grab_data()
def _grab_data(self):
# grab the data from DB
if self.usr == 1:
print('All messages of usr 1')
elif self.usr == 2:
print('All messages of usr 2')
elif self.usr == 3:
print('All messages of usr 3')
one = User(1, 'One')
two = User(2, 'Two')
three = User(3, 'Three')
one.messages().get()
two.messages().get()
three.messages().get()
The messages method approach practical would be the same for labels, history etc.
Edit: I'll give one more try to myself trying to understand what you want to achieve, even though you said that
I have tried numerous things with defining the classes outside of the container class [...]
. I don't know if you tried inheritance, since your inner class me, despite it quite don't represent nothing here, but still looks like you want to make use of its functionality somehow. You said as well
self in the methods reference the self of the outer class
This sounds to me like you want inheritance at the end.
Then the way to go would be (a proximity idea by using inheritance):
class me(object):
def __init__(self):
self.__other_arg = None # private and hidden variable
# setter and getter methods
def set_other_arg(self, new_other_arg):
self.__other_arg = new_other_arg
def get_other_arg(self):
return self.__other_arg
class Test(me):
name = 'Class Test'
#property
def message(self):
other_arg = self.get_other_arg()
if other_arg is not None:
return '{} {}'.format(self.name, other_arg)
else:
return self.name
t = Test()
t.set_other_arg('said Hello')
print(t.message)
# output >>> Class Test said Hello
I think this could be a preferable way to go rather than your inner class approach, my opinion, you'll decide. Just one side note, look up for getter and setter in python, it might help you if you want to stick with the inheritance idea given.
Related
I have this code:
class CongressApi:
class apiKeyError(Exception):
pass
class Member:
def __init__(self):
print("self.makeRequest()?") # want to call the makeRequest function in the external class
def __init__(self, apiKey):
self.key = apiKey
def makeRequest(self, req):
ret = requests.get(f"https://api.propublica.org/congress/v1/{req}", headers={"X-API-Key": self.key})
return ret.content
I would like to be able to call that makeRequest() function from inside the memeber class. is this possible?
It is not common practice in Python to nest classes like this. I would recommend something like this instead:
class CongressApi:
def __init__(self, apiKey):
self.key = apiKey
def makeRequest(self, req):
ret = requests.get(f"https://api.propublica.org/congress/v1/{req}", headers={"X-API-Key": self.key})
return ret.content
class Member:
def __init__(self, congress_api_key):
self.C = CongressAPI(congress_api_key)
print(f"{self.C.makeRequest()}")
class apiKeyError(Exception):
pass # this is really unnecessary - it's easier just to implement try/except blocks at each point in the code where an exception might be triggered.
In general, it's good practice to separate out your classes.
If you want your internal class's instance methods to be able to access instance methods of the outer class, the internal class's instance needs access to an instance of the external class. For example:
class CongressApi:
class Member:
def __init__(self, api):
api.makeRequest("bar")
def __init__(self, apiKey):
self.key = apiKey
def makeRequest(self, req):
print(f"making request {req} with apiKey {self.key}")
def do_member_thing(self):
member = self.Member(self)
api = CongressApi("foo")
api.do_member_thing() # making request bar with apiKey foo
Note that this is not actually a sensible way to organize your classes -- typically the point of an inner class would be to encapsulate some piece of state that doesn't depend on the outer class, and further to abstract that implementation away from the rest of the outer class's implementation. Passing the inner class a reference to the outer class is permitted, but it also entirely defeats the purpose from an architectural standpoint.
I'm trying to write a tracker class where the instances of the tracker class track the sub-classes of another class that are in the scope of the tracker instance.
More concretely, the following is an example of what I am trying to achieve:
class Foo(object): pass
class FooTracker(object):
def __init__(self):
# use Foo.__subclasses__() or a metaclass to track subclasses
# - but how do I filter this to only get the ones in scope?
self.inscope = <something magic goes here>
ft1 = FooTracker()
assert ft1.inscope == []
class Bar(Foo): pass
ft2 = FooTracker()
assert ft2.inscope == [<class '__main__.Bar'>]
def afunction():
class Baz(Foo): pass # the global definition of Bar is now hidden
class Bar(Foo): pass
ft3 = FooTracker()
assert (set(ft3.inscope) == set([<class '__main__.afunction.<locals>.Baz'>,
<class '__main__.afunction.<locals>.Bar'>])
ft4 = FooTracker() # afunction.Baz and afunction.Bar are no longer in scope
assert ft4.inscope == [<class '__main__.Bar'>]
So I want the instances of FooTracker to track the sub-classes of Foo that were in scope at the time the FooTracker object was created.
I've tried a few different things, such as parsing the qualified names of the Foo sub-classes and using exec() to do the name resolution but the fundamental problem is that it always works out the sub-classes relative to the scope within FooTracker.__init__() and not where it was called.
My only other thought was to try something with inspect.currentframe() but even if this were possible it would probably be too much of a hack and would make the code too brittle (e.g., there is a comment in the docs that not all Python implementations will have frame support in the interpreter").
There's no easy way to do exactly what you're asking for. But you might be able to use some Python features to get something with a roughly similar API, without as much hassle.
One option would be to require each subclass to be decorated with a method of your Tracker class. This would make it really easy to keep track of them, since you'd just append each caller of the method to a list:
class Tracker:
def __init__(self):
self.subclasses = []
def register(self, cls):
self.subclasses.append(cls)
return cls
class Foo(): pass
foo_tracker = Tracker()
#foo_tracker.register
class FooSubclass1(Foo): pass
#foo_tracker.register
class FooSubclass2(Foo): pass
print(foo_tracker.subclasses)
This doesn't actually require that the classes being tracked are subclasses of Foo, all classes (and even non-class objects) can be tracked if you pass them to the register method. Decorator syntax makes it a little nicer than just appending each class to a list after you define it, but not by a whole lot (you still repeat yourself a fair amount, which may be annoying unless you make the tracker and method names very short).
A slightly trickier version might get passed the base class, so that it would detect subclasses automatically (via Foo.__subclasses__). To limit the subclasses it detects (rather than getting all subclasses of the base that have ever existed), you could make it behave as a context manager, and only track new subclasses defined within a with block:
class Tracker:
def __init__(self, base):
self.base = base
self._exclude = set()
self.subclasses = set()
def __enter__(self):
self._exclude = set(self.base.__subclasses__())
return self
def __exit__(self, *args):
self.subclasses = set(self.base.__subclasses__()) - self._exclude
return False
class Foo(): pass
class UntrackedSubclass1(Foo): pass
with Tracker(Foo) as foo_tracker:
class TrackedSubclass1(Foo): pass
class TrackedSubclass2(Foo): pass
class UntrackedSubclass2(Foo): pass
print(foo_tracker.subclasses)
If you're using Python 3.6 or later, you can do the tracking a different way by injecting an __init_subclass__ class method into the tracked base class, rather than relying upon __subclasses__. If you don't need to support class hierarchies that are already using __init_subclass__ for their own purposes (and you don't need to support nested trackers), it can be quite elegant:
class Tracker:
def __init__(self, base):
self.base = base
self.subclasses = []
def __enter__(self):
#classmethod
def __init_subclass__(cls, **kwargs):
self.subclasses.append(cls)
self.base.__init_subclass__ = __init_subclass__
return self
def __exit__(self, *args):
del self.base.__init_subclass__
return False
class Foo(): pass
class UntrackedSubclass1(Foo): pass
with Tracker(Foo) as foo_tracker:
class TrackedSubclass1(Foo): pass
class TrackedSubclass2(Foo): pass
class UntrackedSubclass2(Foo): pass
print(foo_tracker.subclasses)
One nice feature of this version is that it automatically tracks deeper inheritance hierarchies. If a subclass of a subclass is created within the with block, that "grandchild" class will still be tracked. We could make the previous __subclasses__ based version work this way too, if you wanted, by adding another function to recursively expand out the subclasses of each class we find.
If you do want to play nice with existing __init_subclass__ methods, or want to be able to nest trackers, you need to make the code a bit more complicated. Injecting a well behaved classmethod in a reversible way is tricky since you need handle both the case where the base class has its own method, and the case where it's inheriting a version from its parents.
class Tracker:
def __init__(self, base):
self.base = base
self.subclasses = []
def __enter__(self):
if '__init_subclass__' in self.base.__dict__:
self.old_init_subclass = self.base.__dict__['__init_subclass__']
else:
self.old_init_subclass = None
#classmethod
def __init_subclass__(cls, **kwargs):
if self.old_init_subclass is not None:
self.old_init_subclass.__get__(None, cls)(**kwargs)
else:
super(self.base, cls).__init_subclass__(**kwargs)
self.subclasses.append(cls)
self.base.__init_subclass__ = __init_subclass__
return self
def __exit__(self, *args):
if self.old_init_subclass is not None:
self.base.__init_subclass__ = self.old_init_subclass
else:
del self.base.__init_subclass__
return False
class Foo:
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
print("Foo!")
class Bar(Foo): pass # every class definition from here on prints "Foo!" when it runs
with Tracker(Bar) as tracker1:
class Baz(Bar): pass
with Tracker(Foo) as tracker2:
class Quux(Foo): pass
with Tracker(Bar) as tracker3:
class Plop(Bar): pass
# four Foo! lines will have be printed by now by Foo.__init_subclass__
print(tracker1.subclasses) # will describe Baz and Plop, but not Quux
print(tracker2.subclasses) # will describe Quux and Plop
print(tracker3.subclasses) # will describe only Plop
Let me give a brief explanation of the issue:
I have a server object with several functionalities.
all functionalities have some common code, so this warrants a functionalities base class
Each functionality has its own set of constants, defined in a constants class within the functionality.
The functionality base class has a set of common constants as well.
here is a sample code:
class server:
class base_func:
class common_consts:
name = "name"
def validate(self):
pass
def execute(self):
pass
class func1(base_func):
class consts:
new_name = base_func.common_consts.name #this is where the problem occurs
def get_result(self):
self.validate()
self.execute()
so when i try to use the common_consts from base_func, in func1.consts, I get the following error:
NameError: global name 'base_func' is not defined
I do not know why this happens. Can someone help?
Is there a limitation to the scope of nesting in python, especially 2.7
Also if i remove the top level server class, and have the functionality classes as independent classes, everything seems to work fine. The example of the working code is here:
class base_func:
class common_consts:
name = "name"
def validate(self):
pass
def execute(self):
pass
class func1(base_func):
class consts:
new_name = base_func.common_consts.name #this is where the problem occurs
def get_result(self):
self.validate()
self.execute()
This leads me to believe that there definitely exists some limitation to the nesting depth and namespace scopes in python. I just want to be sure before i make changes to my design.
class server:
class base_func:
class common_consts:
name = "name"
def validate(self):
pass
def execute(self):
pass
# base_func and func1 are at same, level. So, here you can directly use base_func and func1 anywhere
# at top level of the server class
class func1(base_func):
class consts:
new_name = base_func.common_consts.name # this is where the problem occurs
def get_result(self):
self.validate()
self.execute
For a class(classes have their own namespace), variable lookup works like this:
While parsing the class body any variable defined inside the class body can be access directly, but once
it is parsed it becomes a class attribute.
As, the class base_func is inside server class which is still being parsed the func1(base_func) will work fine.
But, for class consts base_func is not at the same level. So, after looking the variable in its body it will directly jump
to its enclosing scope, i.e global namespace in this case.
A fix will be to do the assignement like this:
class server:
class base_func:
class common_consts:
name = "name"
def validate(self):
pass
def execute(self):
pass
class func1(base_func):
class consts:
pass
def get_result(self):
self.validate()
self.execute
func1.consts.new_name = base_func.common_consts.name
You've hit a problem with class scope. The class scope is never made available except to operations that directly occur in the class scope. This is why you can't call method within another method without referencing self.
eg.
class A(object):
def f(self):
pass
def g(self):
f() # error, class scope isn't available
The same is true when you create a nested class. The initial class statement class ClassName(Base): has access to the class scope, but once in the nested class scope you lose access to the enclosing class scope.
Generally, there is no good reason to nest classes in python. You should try to create all classes at module level. You can hide internal classes by either placing them in a sub module or all prefixing them with an underscore.
class _BaseFunctionality(object):
# common constants
name = "name"
value = "value"
def execute(self):
return (self.name, self.value)
class _SpecificFunctionality(_BaseFunctionality):
# specific constants
# override value of value attribute
value = "another_value"
def get_result(self):
assert self.name == "name"
return self.execute()
class Server(object):
functionality = _SpecificFunctionality()
assert _BaseFunctionality.value == "value"
assert _SpecificFunctionality.value == "another_value"
assert Server().functionality.get_result() == ("name", "another_value")
I have an instance of a Python class that is creating another, and I'm wondering if the "created" class can access members of the "creator" without passing them in explicitly, or doing something like this:
class Creator(object):
def __init__(self, parameter):
self.parameter = parameter
self.created = Created(self)
class Created(object):
def __init__(self, creator):
self.creator = creator
self.parameter = self.creator.parameter
I guess what I'm trying to do is allow parameter to be accessed by both, except in practice there will be multiple parameters, so passing them all will result in a long list of arguments. My solution right now is to use a Creator as an argument to a Created. Is there a less awkward or smarter way to do this? Maybe put all the parameters in a dictionary?
You can do this, and you've almost done it, with one minor problem:
class Created():
def __init__(self, creator):
self.parameter = self.creator.parameter
There is no self.creator. If you never need to access it again after this __init__ call, just use the creator parameter directly:
class Created():
def __init__(self, creator):
self.parameter = creator.parameter
If you need it later, store it as self.creator, and then of course you can access it there:
class Created():
def __init__(self, creator):
self.creator = creator
self.parameter = self.creator.parameter
… although you probably don't even need to copy over parameter in that case, because you can always get to it as self.creator.parameter.
This is a very common pattern.
The only real downside to this is that if you keep the back-reference around (self.creator = creator), you can't have any __del__ methods. (Or you need to make it a weakref, or an indirect reference, like a key that can be used to look up the creator as needed.)
Another option
class Creator():
def __init__(self, parameter):
self.parameter = parameter
self.created = Created()
self.created.creator = self
IMHO it looks fine. If parameter is a class attribute instead of instance, you can use inheritance, but I don't think it is such an improvement.
def creator_factory(param):
class Creator(object):
parameter = param
return Creator
Creator = creator_factory(something)
class Created(Creator):
pass
>>> Created().parameter
'something'
I'm using base class constructor as factory and changing class in this constructor/factory to select appropriate class -- is this approach is good python practice or there are more elegant ways?
I've tried to read help about metaclasses but without big success.
Here example of what I'm doing.
class Project(object):
"Base class and factory."
def __init__(self, url):
if is_url_local(url):
self.__class__ = ProjectLocal
else:
self.__class__ = ProjectRemote
self.url = url
class ProjectLocal(Project):
def do_something(self):
# do the stuff locally in the dir pointed by self.url
class ProjectRemote(Project):
def do_something(self):
# do the stuff communicating with remote server pointed by self.url
Having this code I can create the instance of ProjectLocal/ProjectRemote via base class Project:
project = Project('http://example.com')
project.do_something()
I know that alternate way is to using fabric function that will return the class object based on url, then code will looks similar:
def project_factory(url):
if is_url_local(url):
return ProjectLocal(url)
else:
return ProjectRemote(url)
project = project_factory(url)
project.do_something()
Is my first approach just matter of taste or it has some hidden pitfalls?
You shouldn't need metaclasses for this. Take a look at the __new__ method. This will allow you to take control of the creation of the object, rather than just the initialisation, and so return an object of your choosing.
class Project(object):
"Base class and factory."
def __new__(cls, url):
if is_url_local(url):
return super(Project, cls).__new__(ProjectLocal, url)
else:
return super(Project, cls).__new__(ProjectRemote, url)
def __init__(self, url):
self.url = url
I would stick with the factory function approach. It's very standard python and easy to read and understand. You could make it more generic to handle more options in several ways such as by passing in the discriminator function and a map of results to classes.
If the first example works it's more by luck than by design. What if you wanted to have an __init__ defined in your subclass?
The following links may be helpful:
http://www.suttoncourtenay.org.uk/duncan/accu/pythonpatterns.html#factory
http://code.activestate.com/recipes/86900/
In addition, as you are using new style classes, using __new__ as the factory function (and not in a base class, a separate class is better) is what is usually done (as far as I know).
A factory function is generally simpler (as other people have already posted)
In addition, it isn't a good idea to set the __class__ attribute the way you have done.
I hope you find the answer and the links helpful.
All the best.
Yeah, as mentioned by #scooterXL, factory function is the best approach in that case, but I like to note a case for factories as classmethods.
Consider the following class hierarchy:
class Base(object):
def __init__(self, config):
""" Initialize Base object with config as dict."""
self.config = config
#classmethod
def from_file(cls, filename):
config = read_and_parse_file_with_config(filename)
return cls(filename)
class ExtendedBase(Base):
def behaviour(self):
pass # do something specific to ExtendedBase
Now you can create Base objects from config dict and from config file:
>>> Base({"k": "v"})
>>> Base.from_file("/etc/base/base.conf")
But also, you can do the same with ExtendedBase for free:
>>> ExtendedBase({"k": "v"})
>>> ExtendedBase.from_file("/etc/extended/extended.conf")
So, this classmethod factory can be also considered as auxiliary constructor.
I usually have a seperate factory class to do this. This way you don't have to use meta classes or assignments to self.__class__
I also try to avoid to put the knowledge about which classes are available for creation into the factory. Rather, I have all the available classes register themselves withe the factory during module import. The give there class and some information about when to select this class to the factory (this could be a name, a regex or a callable (e.g. a class method of the registering class)).
Works very well for me and also implements such things like encapsulation and information hiding.
I think the second approach using a factory function is a lot cleaner than making the implementation of your base class depend on its subclasses.
Adding to #Brian's answer, the way __new__ works with *args and **kwargs would be as follows:
class Animal:
def __new__(cls, subclass: str, name: str, *args, **kwargs):
if subclass.upper() == 'CAT':
return super(Animal, cls).__new__(Dog)
elif subclass.upper() == 'DOG':
return super(Animal, cls).__new__(Cat)
raise NotImplementedError(f'Unsupported subclass: "{subclass}"')
class Dog(Animal):
def __init__(self, name: str, *args, **kwargs):
self.name = name
print(f'Created Dog "{self.name}"')
class Cat(Animal):
def __init__(self, name: str, *args, num_whiskers: int = 5, **kwargs):
self.name = name
self.num_whiskers = num_whiskers
print(f'Created Cat "{self.name}" with {self.num_whiskers} whiskers')
sir_meowsalot = Animal(subclass='Cat', name='Sir Meowsalot')
shadow = Animal(subclass='Dog', name='Shadow')