We Keep Coding

In Python, how do I get the list of classes defined within a particular file?

If a file myfile.py contains:
class A(object):
# Some implementation
class B (object):
# Some implementation
How can I define a method so that, given myfile.py, it returns
[A, B]?
Here, the returned values for A and B can be either the name of the classes or the type of the classes.
(i.e. type(A) = type(str) or type(A) = type(type))

You can get both:
import importlib, inspect
for name, cls in inspect.getmembers(importlib.import_module("myfile"), inspect.isclass):
you may additionally want to check:
if cls.__module__ == 'myfile'

In case it helps someone else. Here is the final solution that I used. This method returns all classes defined in a particular package.
I keep all of the subclasses of X in a particular folder (package) and then, using this method, I can load all the subclasses of X, even if they haven't been imported yet. (If they haven't been imported yet, they cannot be accessible via __all__; otherwise things would have been much easier).
import importlib, os, inspect
def get_modules_in_package(package_name: str):
files = os.listdir(package_name)
for file in files:
if file not in ['__init__.py', '__pycache__']:
if file[-3:] != '.py':
continue
file_name = file[:-3]
module_name = package_name + '.' + file_name
for name, cls in inspect.getmembers(importlib.import_module(module_name), inspect.isclass):
if cls.__module__ == module_name:
yield cls

It's a bit long-winded, but you first need to load the file as a module, then inspect its methods to see which are classes:
import inspect
import importlib.util
# Load the module from file
spec = importlib.util.spec_from_file_location("foo", "foo.py")
foo = importlib.util.module_from_spec(spec)
spec.loader.exec_module(foo)
# Return a list of all attributes of foo which are classes
[x for x in dir(foo) if inspect.isclass(getattr(foo, x))]

Just building on the answers above.
If you need a list of the classes defined within the module (file), i.e. not just those present in the module namespace, and you want the list within that module, i.e. using reflection, then the below will work under both __name__ == __main__ and __name__ == <module> cases.
import sys, inspect
# You can pass a lambda function as the predicate for getmembers()
[name, cls in inspect.getmembers(sys.modules[__name__], lambda x: inspect.isclass(x) and (x.__module__ == __name__))]
In my very specific use case of registering classes to a calling framework, I used as follows:
def register():
myLogger.info(f'Registering classes defined in module {__name__}')
for name, cls in inspect.getmembers(sys.modules[__name__], lambda x: inspect.isclass(x) and (x.__module__ == __name__)):
myLogger.debug(f'Registering class {cls} with name {name}')
<framework>.register_class(cls)

Python dill a dynamic class object inside a module and read it in a new python session

Say I have 2 submodulew inside a package as
|-packageA
|- moduleA
|- moduleB
moduleB has a dynamic class creation method that uses type() to create new class.
def moduleB.fun():
new_cls = type("bar",(obejct,),{})
return new_cls
The class created is always of the namesapce packageA.moduleB.bar. (Instead I need it to be just bar.)
When I dill an object created from this dynamic class, it dumps it along with the namespace name.
def moduleA.fun2():
new_cls = moduleB.fun()
obj = new_cls("Object of new class")
dill.dump(obj, open("test.obj","wb"))
The problem comes when I am in a new python session and trying to load this dill file.
In a new session,
dat = dill.load(open("test.obj","rb"))
The dill complains that Can't get attribute 'bar' on <module 'packageA.moduleB'>
I tried setting it to global()["bar"] = new_cls in moduleB.fun(), but it is the type creation that seems to be a problem. Any help?

Well - looks like dill has 2 class pickling behaviors - in one of then, it tries to bind the class to its declaration in the originating module, and on the other one, it switches to a visible create_type operand inside the Pickle bytestring itself.
I could switch to the second module by overwriting the class'__module__ attribute with "__main__" before pickling - so, give this a try:
def moduleA.fun2():
new_cls = moduleB.fun()
new_cls.__module__ = "__main__"
new_cls.__qualname__ = new_cls.__name__.split(".")[-1]
obj = new_cls("Object of new class")
dill.dump(obj, open("test.obj","wb"))

How to give a class a referencable string name?

The scenerio is I'm using an arg parser to get a command line argument auth_application.
auth_application command can have many values, for example:
cheese
eggs
noodles
pizza
These values are related to a programmable class.
I'd like a way to name the class, possible using a decorator.
So I can say
if auth_application is Cheese.__name__:
return Cheese()
Currently I maintain a tuple of auth_application names and have to expose that to my arg parser class as well as import the classes I need.
Anyways to make this better? Is there a decorator for classes to name them?
I'm looking for a python 2.7 solution, but a python 3 solution might be useful to know.

Easy peasy.
class command(object):
map = {}
def __init__(self, commandname):
self.name = commandname
def __call__(self, cls):
command.map[self.name] = cls
return cls
class NullCommand(object):
pass
#command('cheese')
class Cheese(object):
pass
#command('eggs')
class Eggs(object):
pass
def func(auth_application):
return command.map.get(auth_application, command.NullCommand)()

You can just keep a sinlge list of all of your "allowed classes" and iterate over that to find the class being referred to from the command line.
allow_classes = [Cheese,Eggs,Noodles,Pizza]
for cls in allow_classes:
if auth_application.lower() is cls.__name__.lower():
return cls()

Absolutely you can! You need to understand class attributes.
class NamedClass(object):
name = "Default"
class Cheese(NamedClass):
name = "Cheese"
print(Cheese.name)
> Cheese

You can use the standard Inspect Library to get the real class names, without having to augment your classes with any extra data - and this works for any class, in any module - even if you don't have the source code.
For instance - to list all the classes defined in mymodule :
import mymodule
import inspect
for name, obj in inspect.getmembers(mymodule, inspect.isclass):
print name
the obj variable is a real class object - which you can use to declare an instance, access class methods etc.
To get the definition of a class by it's name string - you can write a simple search function :
import mymodule
import inspect
def find_class(name):
"""Find a named class in mymodule"""
for this_name, _cls_ in inspect.getmembers(mymodule, inspect.isclass):
if this_name = name:
return _cls_
return None
....
# Create an instance of the class named in auth_application
find_class(auth_application)(args, kwargs)
NB: Code snippets not tested

Getting an object by name

Is it possible to create an instance of a class by giving its name, e.g.:
instance = "TestXYZ"()
Assuming that TestXYZ is imported by a class that imports the code above and TestXYZ is defined as:
class TestXYZ(object):
...

Yes, it is possible. The mechanics depend on how you import the class:
>>> globals()["TestXYZ"]()
<__main__.TestXYZ object at 0x10f491090>
or
>>> getattr(sys.modules["test_module"], "TestXYZ")()
<test_module.TestXYZ object at 0x10cf22090>

You can get a reference to an object from current namespace using:
klass = globals()['TestXYZ']
Then you can create an instance of the class:
instance = klass()

I am not sure why do you want to do this, but instead of using globals() I'd suggest you to create a dictionary here:
class Foo:
pass
d = {'Foo':Foo}
ins = d['Foo']()
Or move the class to some other module:
import some_module
ins = getattr(some_module, 'Foo')()

How to pickle an instance of a class which is written inside a function?

Suppose I have this snippet inside a module
def func(params):
class MyClass(object):
pass
How can I pickle an instance of the class MyClass ?

You can't, because picklable object's class definitions must reside in an imported module's scope. Just put your class inside module scope and you are good to go.
That said, in Python there is very little that can't be achieved with a bit of hacking the insides of the machinery (sys.modules in this case), but I wouldn't recommend that.

The MyClass definition is local variable for the func function. You cannot directly create an instance of it, but you can map it's functions to a new class, and then to use the new class as it is the original one. Here's an example:
def func(params):
class MyClass(object):
some_param = 100
def __init__(self, *args):
print "args:", args
def blabla(self):
self.x = 123
print self.some_param
def getme(self):
print self.x
func.func_code is the code of the func function, and func.func_code.co_consts[2] contains the bytecode of the MyClass definition:
In : func.func_code.co_consts
Out:
(None,
'MyClass',
<code object MyClass at 0x164dcb0, file "<ipython-input-35-f53bebe124be>", line 2>)
So we need the bytecode for the MyClass functions:
In : eval(func.func_code.co_consts[2])
Out:
{'blabla': <function blabla at 0x24689b0>,
'__module__': '__main__',
'getme': <function getme at 0x2468938>,
'some_param': 100,
'__init__': <function __init__ at 0x219e398>}
And finally we create a new class with metaclass, that assigns the MyClass functions to the new class:
def map_functions(name, bases, dict):
dict.update(eval(func.func_code.co_consts[2]))
return type(name, bases, dict)
class NewMyClass(object):
__metaclass__ = map_functions
n = NewMyClass(1, 2, 3, 4, 5)
>> args: (1, 2, 3, 4, 5)
n.blabla()
>> 100
n.getme()
>> 123

This is somewhat tough to do because the way Pickle does with objects from user defined classes by default is to create a new instance of the class - using the object's __class__.__name__ attribute to retrieve its type in the object's original module. Which means: pickling and unpickling only works (by default) for classes that have well defined names in the module they are defined.
When one defines a class inside a function, usulay there won't be a module level (i.e. global) variable holding the name of each class that was created inside the function.
The behavior for pickle and npickle can be customized through the __getstate__ and __setstate__ methods on the class - check the docs - but even them, doing it right for dynamic class can be tricky , but I managed to create a working implementation of it for another S.O. question - -check my answer here:
Pickle a dynamically parameterized sub-class

You can work around the pickle requirement that class definitions be importable by including the class definition as a string in the data pickled for the instance and exec()uting it yourself when unpickling by adding a __reduce__() method that passes the class definition to a callable. Here's a trivial example illustrating what I mean:
from textwrap import dedent
# Scaffolding
definition = dedent('''
class MyClass(object):
def __init__(self, attribute):
self.attribute = attribute
def __repr__(self):
return '{}({!r})'.format(self.__class__.__name__, self.attribute)
def __reduce__(self):
return instantiator, (definition, self.attribute)
''')
def instantiator(class_def, init_arg):
""" Create class and return an instance of it. """
exec(class_def)
TheClass = locals()['MyClass']
return TheClass(init_arg)
# Sample usage
import pickle
from io import BytesIO
stream = BytesIO() # use a memory-backed file for testing
obj = instantiator(definition, 'Foo') # create instance of class from definition
print('obj: {}'.format(obj))
pickle.dump(obj, stream)
stream.seek(0) # rewind
obj2 = pickle.load(stream)
print('obj2: {}'.format(obj2))
Output:
obj: MyClass('Foo')
obj2: MyClass('Foo')
Obviously it's inefficient to include the class definition string with every class instance pickled, so that redundancy may make it impractical, depending on the the number of class instances involved.