We Keep Coding

Python How to create method of class in runtime

I am curious how to create a custom method for a class at runtime...
I mean for example with name of method, name of parameters, body of method read from database and assign this method to a class or to an instance.
I have a found possibility to add method that is already written:
class A:
def __init__(self):
pass
def method(self):
return True
A.method = method
a = A()
print(a.method())
but I am interested in completely assembling a new method from scratch:
name = "method"
params = ["self"] # Params in list should be a strings
body = "return True"
# To create method from pieces
Is it possible using __dict__ ? Or how else this be done?

Methods are another attribute on the object that is the class. They can be added like other attributes:
Code:
class A:
def __init__(self):
pass
def method(self):
return True
def another_method(self):
return False
setattr(A, 'another_method', another_method)
Test Code:
a = A()
print(a.another_method())
Results:
False
Methods from a string:
Add if you really need to get your methods from a database or such you can use exec like:
method_string = """
def yet_another_method(self):
return type(self).__name__
"""
exec(method_string)
setattr(A, 'yet_another_method', yet_another_method)
a = A()
print(a.yet_another_method())
Results:
A

This answer has to be treated with care, using exec or eval can run arbitary code and may compromise your system. So if you rely on user-input to create the function you mustn't use this!!!
The warning aside you can simply create anything using exec:
exec("""
def method():
return True
""")
>>> method()
True
So what you basically need is just a way to get your requirements in there:
functionname = 'funfunc'
parameters = ['a', 'b']
body = 'return a + b'
exec("""
def {functionname}({parameters}):
{body}
""".format(
functionname=functionname,
parameters=', '.join(parameters),
body='\n'.join([' {line}'.format(line=line) for line in body.split('\n')])))
The body will be indented so that it's valid syntax and the parameter list will be joined using ,. And the test:
>>> funfunc(1, 2)
3

One of the best solutions that I have found is the following:
def import_code(code, name, add_to_sys_modules=0):
"""
Import dynamically generated code as a module. code is the
object containing the code (a string, a file handle or an
actual compiled code object, same types as accepted by an
exec statement). The name is the name to give to the module,
and the final argument says wheter to add it to sys.modules
or not. If it is added, a subsequent import statement using
name will return this module. If it is not added to sys.modules
import will try to load it in the normal fashion.
import foo
is equivalent to
foofile = open("/path/to/foo.py")
foo = importCode(foofile,"foo",1)
Returns a newly generated module.
"""
import sys,imp
module = imp.new_module(name)
exec(code,module.__dict__)
if add_to_sys_modules:
sys.modules[name] = module
return module
class A:
def __init__(self):
pass
name = "method"
params = ["self"] # Params in list should be a strings
body = "return True"
scratch = "def {0}({1}):\n\t{2}".format(name, ','.join(params), body)
new_module = import_code(scratch, "test")
A.method = new_module.method
a = A()
print(a.method())
Original function import_code by the following link http://code.activestate.com/recipes/82234-importing-a-dynamically-generated-module/
Using this solution I can dynamically create methods, load them in runtime and link to whatever I want object !!

Is there a magic function which is used to parse raw string into object? [duplicate]

Given a string as user input to a Python function, I'd like to get a class object out of it if there's a class with that name in the currently defined namespace. Essentially, I want the implementation for a function which will produce this kind of result:
class Foo:
pass
str_to_class("Foo")
==> <class __main__.Foo at 0x69ba0>
Is this, at all, possible?

This could work:
import sys
def str_to_class(classname):
return getattr(sys.modules[__name__], classname)

Warning: eval() can be used to execute arbitrary Python code. You should never use eval() with untrusted strings. (See Security of Python's eval() on untrusted strings?)
This seems simplest.
>>> class Foo(object):
... pass
...
>>> eval("Foo")
<class '__main__.Foo'>

You could do something like:
globals()[class_name]

You want the class Baz, which lives in module foo.bar. With Python 2.7,
you want to use importlib.import_module(), as this will make transitioning to Python 3 easier:
import importlib
def class_for_name(module_name, class_name):
# load the module, will raise ImportError if module cannot be loaded
m = importlib.import_module(module_name)
# get the class, will raise AttributeError if class cannot be found
c = getattr(m, class_name)
return c
With Python < 2.7:
def class_for_name(module_name, class_name):
# load the module, will raise ImportError if module cannot be loaded
m = __import__(module_name, globals(), locals(), class_name)
# get the class, will raise AttributeError if class cannot be found
c = getattr(m, class_name)
return c
Use:
loaded_class = class_for_name('foo.bar', 'Baz')

I've looked at how django handles this
django.utils.module_loading has this
def import_string(dotted_path):
"""
Import a dotted module path and return the attribute/class designated by the
last name in the path. Raise ImportError if the import failed.
"""
try:
module_path, class_name = dotted_path.rsplit('.', 1)
except ValueError:
msg = "%s doesn't look like a module path" % dotted_path
six.reraise(ImportError, ImportError(msg), sys.exc_info()[2])
module = import_module(module_path)
try:
return getattr(module, class_name)
except AttributeError:
msg = 'Module "%s" does not define a "%s" attribute/class' % (
module_path, class_name)
six.reraise(ImportError, ImportError(msg), sys.exc_info()[2])
You can use it like import_string("module_path.to.all.the.way.to.your_class")

import sys
import types
def str_to_class(field):
try:
identifier = getattr(sys.modules[__name__], field)
except AttributeError:
raise NameError("%s doesn't exist." % field)
if isinstance(identifier, (types.ClassType, types.TypeType)):
return identifier
raise TypeError("%s is not a class." % field)
This accurately handles both old-style and new-style classes.

If you really want to retrieve classes you make with a string, you should store (or properly worded, reference) them in a dictionary. After all, that'll also allow to name your classes in a higher level and avoid exposing unwanted classes.
Example, from a game where actor classes are defined in Python and you want to avoid other general classes to be reached by user input.
Another approach (like in the example below) would to make an entire new class, that holds the dict above. This would:
Allow multiple class holders to be made for easier organization (like, one for actor classes and another for types of sound);
Make modifications to both the holder and the classes being held easier;
And you can use class methods to add classes to the dict. (Although the abstraction below isn't really necessary, it is merely for... "illustration").
Example:
class ClassHolder:
def __init__(self):
self.classes = {}
def add_class(self, c):
self.classes[c.__name__] = c
def __getitem__(self, n):
return self.classes[n]
class Foo:
def __init__(self):
self.a = 0
def bar(self):
return self.a + 1
class Spam(Foo):
def __init__(self):
self.a = 2
def bar(self):
return self.a + 4
class SomethingDifferent:
def __init__(self):
self.a = "Hello"
def add_world(self):
self.a += " World"
def add_word(self, w):
self.a += " " + w
def finish(self):
self.a += "!"
return self.a
aclasses = ClassHolder()
dclasses = ClassHolder()
aclasses.add_class(Foo)
aclasses.add_class(Spam)
dclasses.add_class(SomethingDifferent)
print aclasses
print dclasses
print "======="
print "o"
print aclasses["Foo"]
print aclasses["Spam"]
print "o"
print dclasses["SomethingDifferent"]
print "======="
g = dclasses["SomethingDifferent"]()
g.add_world()
print g.finish()
print "======="
s = []
s.append(aclasses["Foo"]())
s.append(aclasses["Spam"]())
for a in s:
print a.a
print a.bar()
print "--"
print "Done experiment!"
This returns me:
<__main__.ClassHolder object at 0x02D9EEF0>
<__main__.ClassHolder object at 0x02D9EF30>
=======
o
<class '__main__.Foo'>
<class '__main__.Spam'>
o
<class '__main__.SomethingDifferent'>
=======
Hello World!
=======
0
1
--
2
6
--
Done experiment!
Another fun experiment to do with those is to add a method that pickles the ClassHolder so you never lose all the classes you did :^)
UPDATE: It is also possible to use a decorator as a shorthand.
class ClassHolder:
def __init__(self):
self.classes = {}
def add_class(self, c):
self.classes[c.__name__] = c
# -- the decorator
def held(self, c):
self.add_class(c)
# Decorators have to return the function/class passed (or a modified variant thereof), however I'd rather do this separately than retroactively change add_class, so.
# "held" is more succint, anyway.
return c
def __getitem__(self, n):
return self.classes[n]
food_types = ClassHolder()
#food_types.held
class bacon:
taste = "salty"
#food_types.held
class chocolate:
taste = "sweet"
#food_types.held
class tee:
taste = "bitter" # coffee, ftw ;)
#food_types.held
class lemon:
taste = "sour"
print(food_types['bacon'].taste) # No manual add_class needed! :D

Yes, you can do this. Assuming your classes exist in the global namespace, something like this will do it:
import types
class Foo:
pass
def str_to_class(s):
if s in globals() and isinstance(globals()[s], types.ClassType):
return globals()[s]
return None
str_to_class('Foo')
==> <class __main__.Foo at 0x340808cc>

In terms of arbitrary code execution, or undesired user passed names, you could have a list of acceptable function/class names, and if the input matches one in the list, it is eval'd.
PS: I know....kinda late....but it's for anyone else who stumbles across this in the future.

Using importlib worked the best for me.
import importlib
importlib.import_module('accounting.views')
This uses string dot notation for the python module that you want to import.

How to do yaml.safe_dump() and .safe_load() of Python object without yaml.YAMLObject?

I want to serialize some object with yaml.safe_dump(). How can I serialize Python objects with add_representer() and add_constructor() ...
I can not add yaml.YAMLObject to Thing (third party module) and not want use.
I do such dump:
import yaml
class Thing(object):
def __init__(self, name):
self.name = name
def Thing_representer(dumper, data):
return dumper.represent_mapping('!Thing', data.__dict__)
yaml.SafeDumper.add_representer(Thing, Thing_representer)
safe_dump = yaml.safe_dump(t)
print safe_dump
It works fine but I have no idea how to do constructor?
def Thing_constructor(loader, data):
thing = Thing()
return thing.__dict__.update(loader.construct_mapping(data))
yaml.SafeLoader.add_constructor('!Thing', Thing_constructor)
yaml.safe_load(safe_dump)
It throws exception TypeError: __init__() takes exactly 2 arguments (1 given) and should throw since constructor requires parameters. Maybe there is another option to construct object skipping constructor?

You cannot construct Thing() without handing in the name. You can solve that
in various ways, but the following should work.
def thing_constructor(self, node):
name = None
for x in node.value:
if x[0].value == 'name':
name = x[1].value
return Thing(name)
yaml.SafeLoader.add_constructor('!Thing', thing_constructor)
res = yaml.safe_load(safe_dump)
print res.name
You can simplify the setting of the name parameter, but this way it is more extensible if Thing would have taken more parameters.

Python :TypeError: 'str' object is not callable

I am using a function to instansiate the python classes .
Hers is the class structure
from DB.models import ApiKey,ServiceProvider
class SMSMrg( object ):
_instance = None
class Singleton:
def __init__(self):
self.username = None
self.password = None
self.allsp = []
self.classnames = {}
def __new__(cls, *args, **kwargs):
if not cls._instance:
cls._instance = super(SMSMrg, cls).__new__(
cls, *args, **kwargs)
return cls._instance
def loadsettings(self):
get_all_sp = ServiceProvider.objects.filter(status = False)
for (options,obj) in enumerate(get_all_sp):
cla = str(obj.class_Name)
self.classnames[cla] = cla
print self.classnames
for (options,obj) in enumerate(get_all_sp):
cla = str(obj.class_Name)
class_object = self.classnames[cla](obj.userName,obj.password,obj.sendingurl)
# self.allsp = get_all_sp
def send(self):
print "+++++++++++++++++++== Global send "
if __name__ == "__main__":
b = SMSMrg()
b.loadsettings()
I have stored the classnames in database and I have defined each class structures on different files .
Like cla will contain a class name .
But when i am calling above function i am getting the type error .
Traceback (most recent call last):
File "allsms.py", line 30, in <module>
b.loadsettings()
File "allsms.py", line 21, in loadsettings
class_object = cla(obj.userName,obj.password,obj.sendingurl)
TypeError: 'str' object is not callable
Please tell me how can instansiate all the classes which names are present in my db .

On the line cla = str(SERVIVEPROVIDER) you convert SERVIVEPROVIDER to string. And on the next line you are trying to call it, thus you get an error...

# Means `cla` is pointing to a string
cla = str(SERVIVEPROVIDER)
# there is no function called `cla` now it contains a string
cla(obj.userName,obj.password,obj.sendingurl)

As you said cla contains the name of the class, which means that you can't use it as a callable.
You can build a dict and take the class object from there:
from somemodule import SomeClass
class TheClass(object):
def __init__(self, username, password, url):
#do stuff
class AnOtherClass(object):
def __init__(self, username, password, url):
# do stuff
CLASS_NAMES_TO_CLASSES = {
# Note: TheClass is *not* a string, is the class!!!
'FirstName': TheClass,
'SecondName': AnOtherClass,
'SomeClass': SomeClass,
}
class SMSMrg(object):
#do stuff
def loadsettings(self):
get_all_sp = ServiceProvider.objects.filter(status = True)
for obj in get_all_sp:
SERVIVEPROVIDER = obj.class_Name
cla = str(SERVIVEPROVIDER)
class_object = CLASS_NAMES_TO_CLASSES[cla](obj.userName,obj.password,obj.sendingurl)
This method requires you to be able to build such a dict, so either you know ahead which classes could end up in the db or you can't use this method.
Note that CLASS_NAMES_TO_CLASSES is not a dictionary that maps strings to strings. It maps strings to class objects. If you import the class SomeClass from a module then you have to put it inside the dictionary.
An other method could be to use eval to evaluate the class name, but you should avoid this if the db contains data from users(which is not safe).
An other option that might turn out useful is to avoid saving the class names and instead use pickle to save the instances directly.

Please tell me how can instansiate all the classes which names are
present in my db .
Try this:
class A(object): pass
class B(object): pass
class_names = {'first': A, 'second': B}
obj = class_names['first']()
type(obj)
<class 'yourmodule.A'>
Or, if your classes are stored somewhere else, say in a module called mymodule:
import mymodule
obj = getattr(mymodule, 'A')()

Getting an instance name inside class __init__() [duplicate]

This question already has answers here:
Getting the name of a variable as a string
(32 answers)
Closed 3 years ago.
While building a new class object in python, I want to be able to create a default value based on the instance name of the class without passing in an extra argument. How can I accomplish this? Here's the basic pseudo-code I'm trying for:
class SomeObject():
defined_name = u""
def __init__(self, def_name=None):
if def_name == None:
def_name = u"%s" % (<INSTANCE NAME>)
self.defined_name = def_name
ThisObject = SomeObject()
print ThisObject.defined_name # Should print "ThisObject"

Well, there is almost a way to do it:
#!/usr/bin/env python
import traceback
class SomeObject():
def __init__(self, def_name=None):
if def_name == None:
(filename,line_number,function_name,text)=traceback.extract_stack()[-2]
def_name = text[:text.find('=')].strip()
self.defined_name = def_name
ThisObject = SomeObject()
print ThisObject.defined_name
# ThisObject
The traceback module allows you to peek at the code used to call SomeObject().
With a little string wrangling, text[:text.find('=')].strip() you can
guess what the def_name should be.
However, this hack is brittle. For example, this doesn't work so well:
ThisObject,ThatObject = SomeObject(),SomeObject()
print ThisObject.defined_name
# ThisObject,ThatObject
print ThatObject.defined_name
# ThisObject,ThatObject
So if you were to use this hack, you have to bear in mind that you must call SomeObject()
using simple python statement:
ThisObject = SomeObject()
By the way, as a further example of using traceback, if you define
def pv(var):
# stack is a list of 4-tuples: (filename, line number, function name, text)
# see http://docs.python.org/library/traceback.html#module-traceback
#
(filename,line_number,function_name,text)=traceback.extract_stack()[-2]
# ('x_traceback.py', 18, 'f', 'print_var(y)')
print('%s: %s'%(text[text.find('(')+1:-1],var))
then you can call
x=3.14
pv(x)
# x: 3.14
to print both the variable name and its value.

Instances don't have names. By the time the global name ThisObject gets bound to the instance created by evaluating the SomeObject constructor, the constructor has finished running.
If you want an object to have a name, just pass the name along in the constructor.
def __init__(self, name):
self.name = name

You can create a method inside your class that check all variables in the current frame and use hash() to look for the self variable.
The solution proposed here will return all the variables pointing to the instance object.
In the class below, isinstance() is used to avoid problems when applying hash(), since some objects like a numpy.array or a list, for example, are unhashable.
import inspect
class A(object):
def get_my_name(self):
ans = []
frame = inspect.currentframe().f_back
tmp = dict(frame.f_globals.items() + frame.f_locals.items())
for k, var in tmp.items():
if isinstance(var, self.__class__):
if hash(self) == hash(var):
ans.append(k)
return ans
The following test has been done:
def test():
a = A()
b = a
c = b
print c.get_my_name()
The result is:
test()
#['a', 'c', 'b']

This cannot work, just imagine this: a = b = TheMagicObjet(). Names have no effect on Values, they just point to them.

One horrible, horrible way to accomplish this is to reverse the responsibilities:
class SomeObject():
def __init__(self, def_name):
self.defined_name = def_name
globals()[def_name] = self
SomeObject("ThisObject")
print ThisObject.defined_name
If you wanted to support something other than global scope, you'd have to do something even more awful.

In Python, all data is stored in objects. Additionally, a name can be bound with an object, after which that name can be used to look up that object.
It makes no difference to the object what names, if any, it might be bound to. It might be bound to dozens of different names, or none. Also, Python does not have any "back links" that point from an object to a name.
Consider this example:
foo = 1
bar = foo
baz = foo
Now, suppose you have the integer object with value 1, and you want to work backwards and find its name. What would you print? Three different names have that object bound to them, and all are equally valid.
print(bar is foo) # prints True
print(baz is foo) # prints True
In Python, a name is a way to access an object, so there is no way to work with names directly. You could search through various name spaces until you find a name that is bound with the object of interest, but I don't recommend this.
How do I get the string representation of a variable in python?
There is a famous presentation called "Code Like a Pythonista" that summarizes this situation as "Other languages have 'variables'" and "Python has 'names'"
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html#other-languages-have-variables

If you want an unique instance name for a class, try __repr__() or id(self)
class Some:
def __init__(self):
print(self.__repr__()) # = hex(id(self))
print(id(self))
It will print the memory address of the instance, which is unique.

Inspired by the answers of unutbu and Saullo Castro, I have created a more sophisticated class that can even be subclassed. It solves what was asked for in the question.
"create a default value based on the instance name of the class
without passing in an extra argument."
Here's what it does, when an instance of this class or a subclass is created:
Go up in the frame stack until the first frame which does not belong to a method of the current instance.
Inspect this frame to get the attributes self.creation_(name/file/module/function/line/text).
Perform an an additional check whether an object with name self.creation_name was actually defined in the frame's locals() namespace to make 100% sure the found creation_name is correct or raise an error otherwise.
The Code:
import traceback, threading, time
class InstanceCreationError(Exception):
pass
class RememberInstanceCreationInfo:
def __init__(self):
for frame, line in traceback.walk_stack(None):
varnames = frame.f_code.co_varnames
if varnames is ():
break
if frame.f_locals[varnames[0]] not in (self, self.__class__):
break
# if the frame is inside a method of this instance,
# the first argument usually contains either the instance or
# its class
# we want to find the first frame, where this is not the case
else:
raise InstanceCreationError("No suitable outer frame found.")
self._outer_frame = frame
self.creation_module = frame.f_globals["__name__"]
self.creation_file, self.creation_line, self.creation_function, \
self.creation_text = \
traceback.extract_stack(frame, 1)[0]
self.creation_name = self.creation_text.split("=")[0].strip()
super().__init__()
threading.Thread(target=self._check_existence_after_creation).start()
def _check_existence_after_creation(self):
while self._outer_frame.f_lineno == self.creation_line:
time.sleep(0.01)
# this is executed as soon as the line number changes
# now we can be sure the instance was actually created
error = InstanceCreationError(
"\nCreation name not found in creation frame.\ncreation_file: "
"%s \ncreation_line: %s \ncreation_text: %s\ncreation_name ("
"might be wrong): %s" % (
self.creation_file, self.creation_line, self.creation_text,
self.creation_name))
nameparts = self.creation_name.split(".")
try:
var = self._outer_frame.f_locals[nameparts[0]]
except KeyError:
raise error
finally:
del self._outer_frame
# make sure we have no permament inter frame reference
# which could hinder garbage collection
try:
for name in nameparts[1:]: var = getattr(var, name)
except AttributeError:
raise error
if var is not self: raise error
def __repr__(self):
return super().__repr__()[
:-1] + " with creation_name '%s'>" % self.creation_name
A simple example:
class MySubclass(RememberInstanceCreationInfo):
def __init__(self):
super().__init__()
def print_creation_info(self):
print(self.creation_name, self.creation_module, self.creation_function,
self.creation_line, self.creation_text, sep=", ")
instance = MySubclass()
instance.print_creation_info()
#out: instance, __main__, <module>, 68, instance = MySubclass()
If the creation name cannot be determined properly an error is raised:
variable, another_instance = 2, MySubclass()
# InstanceCreationError:
# Creation name not found in creation frame.
# creation_file: /.../myfile.py
# creation_line: 71
# creation_text: variable, another_instance = 2, MySubclass()
# creation_name (might be wrong): variable, another_instance

I think that names matters if they are the pointers to any object..
no matters if:
foo = 1
bar = foo
I know that foo points to 1 and bar points to the same value 1 into the same memory space.
but supose that I want to create a class with a function that adds a object to it.
Class Bag(object):
def __init__(self):
some code here...
def addItem(self,item):
self.__dict__[somewaytogetItemName] = item
So, when I instantiate the class bag like below:
newObj1 = Bag()
newObj2 = Bag()
newObj1.addItem(newObj2)I can do this to get an attribute of newObj1:
newObj1.newObj2

The best way is really to pass the name to the constructor as in the chosen answer. However, if you REALLY want to avoid asking the user to pass the name to the constructor, you can do the following hack:
If you are creating the instance with 'ThisObject = SomeObject()' from the command line, you can get the object name from the command string in command history:
import readline
import re
class SomeObject():
def __init__(self):
cmd = readline.get_history_item(readline.get_current_history_length())
self.name = re.split('=| ',cmd)[0]
If you are creating the instance using 'exec' command, you can handle this with:
if cmd[0:4] == 'exec': self.name = re.split('\'|=| ',cmd)[1] # if command performed using 'exec'
else: self.name = re.split('=| ',cmd)[0]