Question about my codes when using the Tokenizer.__init__ in a class - python

I wish to understand more about why we can use the below code and what does it do?
If possible can anyone explain to me how does Tokenizer.__init__ work? Since we can attach the "__init__" to a library, can we do it with other way too? If possible can someone give me some example so that I could understand, it will be glad if can do another simple example.
My codes:
from tensorflow.python.keras.preprocessing.text import Tokenizer
// we import the tokenizer library
//Then we build a class
class DataWrapper(Tokenizer):
def __init__(self, num_words):
Tokenizer.__init__(self, num_words=num_words) // I dont understand how this part work.
self.fit_on_texts(texts)
self.tokens = self.texts_to_sequences(texts)
self.reverse_tokens = dict(zip(self.word_index.values(),
self.word_index.keys()))
if reverse:
self.tokens = [list(reversed(token)) for token in self.tokens]
self.num_tokens = [len(token) for token in self.tokens]
self.max_num = np.mean(self.num_tokens) + 2 * np.std(self.num_tokens)
self.max_num = int(self.max_num)
self.sequences_padded = pad_sequences(self.tokens, maxlen=self.max_num,
padding=padding,
truncating=truncating)

In python, all functions with self can be called in two ways: obj.method(params) or, cls.method(obj, params). See this example:
class A:
def foo(self):
print('foo')
a = A()
# now we can call foo function in two ways:
a.foo()
A.foo(a)
This rule works also for function with double underscore like __init__. In your case:
Tokenizer.__init__(self, num_words=num_words)
cls = Tokenizer
method = __init__
obj = self # object of class DataWrapper
params = num_words

Related

name 'generateRandom' is not defined"

I'm trying to run this code but I'm getting an error msg ---> "NameError: name 'generateRandom' is not defined"
Can anyone help me please?
`import numpy as np
class Mul:
def __init__ (self,ra_result=None,rb_result=None):
self.ra_result = ra_result
self.rb_result = rb_result
def generateRandom():
return np.random.randint(0,2**32), np.random.randint(0,2**32)
def Multi_test(self):
self.ra_result,self.rb_result= generateRandom()
print("expected_output (python) = ")
print("ra=",self.ra_result,"rb=",self.rb_result," (ra*rb)=")
return self.ra_result*self.rb_result
object = Mul()
object.Multi_test()`
You have to use keyword self to let your class know you are using a method of itself.
You have to specify the self when calling the function self.generateRandom() and also you need to either specify when creating the function that it takes self as an argument, but since you have a static method, you can wrap it in #staticmethod
This is a working versio of your class Mul
class Mul:
def __init__ (self,ra_result=None,rb_result=None):
self.ra_result = ra_result
self.rb_result = rb_result
#staticmethod
def generateRandom():
return np.random.randint(0,2**32), np.random.randint(0,2**32)
def Multi_test(self):
self.ra_result,self.rb_result = self.generateRandom()
print("expected_output (python) = ")
print("ra=",self.ra_result,"rb=",self.rb_result," (ra*rb)=")
return self.ra_result*self.rb_result
I am not going to discuss why you have written your code in this way and directly talk about your error. You are getting this error because generateRandom is a method within Mul class. To refer to the other methods of the same class within a method, you should use 'self'. Additionally add static method since generateRandom doesn't use self. It would also be good if you could improve how you name variables, methods etc. The code will look like below
\
import numpy as np
class Mul:
def __init__ (self,ra_result=None,rb_result=None):
self.ra_result = ra_result
self.rb_result = rb_result
#staticmethod
def generateRandom():
return np.random.randint(0,2**32), np.random.randint(0,2**32)
def Multi_test(self):
self.ra_result,self.rb_result= self.generateRandom()
print("expected_output (python) = ")
print("ra=",self.ra_result,"rb=",self.rb_result," (ra*rb)=")
return self.ra_result*self.rb_result
my_object = Mul()
my_object.Multi_test()
The function you're trying to call, is itself a class function, I.E. it needs to be called like this:
self.generateRandom()
Edit:
You could make the generateRandom() an internal private function, by changing it to
def __generateRandom(self):
...
rng = self.__generateRandom()
In this case you can easily access it and it's still out of reach

Python How to create method of class in runtime

I am curious how to create a custom method for a class at runtime...
I mean for example with name of method, name of parameters, body of method read from database and assign this method to a class or to an instance.
I have a found possibility to add method that is already written:
class A:
def __init__(self):
pass
def method(self):
return True
A.method = method
a = A()
print(a.method())
but I am interested in completely assembling a new method from scratch:
name = "method"
params = ["self"] # Params in list should be a strings
body = "return True"
# To create method from pieces
Is it possible using __dict__ ? Or how else this be done?
Methods are another attribute on the object that is the class. They can be added like other attributes:
Code:
class A:
def __init__(self):
pass
def method(self):
return True
def another_method(self):
return False
setattr(A, 'another_method', another_method)
Test Code:
a = A()
print(a.another_method())
Results:
False
Methods from a string:
Add if you really need to get your methods from a database or such you can use exec like:
method_string = """
def yet_another_method(self):
return type(self).__name__
"""
exec(method_string)
setattr(A, 'yet_another_method', yet_another_method)
a = A()
print(a.yet_another_method())
Results:
A
This answer has to be treated with care, using exec or eval can run arbitary code and may compromise your system. So if you rely on user-input to create the function you mustn't use this!!!
The warning aside you can simply create anything using exec:
exec("""
def method():
return True
""")
>>> method()
True
So what you basically need is just a way to get your requirements in there:
functionname = 'funfunc'
parameters = ['a', 'b']
body = 'return a + b'
exec("""
def {functionname}({parameters}):
{body}
""".format(
functionname=functionname,
parameters=', '.join(parameters),
body='\n'.join([' {line}'.format(line=line) for line in body.split('\n')])))
The body will be indented so that it's valid syntax and the parameter list will be joined using ,. And the test:
>>> funfunc(1, 2)
3
One of the best solutions that I have found is the following:
def import_code(code, name, add_to_sys_modules=0):
"""
Import dynamically generated code as a module. code is the
object containing the code (a string, a file handle or an
actual compiled code object, same types as accepted by an
exec statement). The name is the name to give to the module,
and the final argument says wheter to add it to sys.modules
or not. If it is added, a subsequent import statement using
name will return this module. If it is not added to sys.modules
import will try to load it in the normal fashion.
import foo
is equivalent to
foofile = open("/path/to/foo.py")
foo = importCode(foofile,"foo",1)
Returns a newly generated module.
"""
import sys,imp
module = imp.new_module(name)
exec(code,module.__dict__)
if add_to_sys_modules:
sys.modules[name] = module
return module
class A:
def __init__(self):
pass
name = "method"
params = ["self"] # Params in list should be a strings
body = "return True"
scratch = "def {0}({1}):\n\t{2}".format(name, ','.join(params), body)
new_module = import_code(scratch, "test")
A.method = new_module.method
a = A()
print(a.method())
Original function import_code by the following link http://code.activestate.com/recipes/82234-importing-a-dynamically-generated-module/
Using this solution I can dynamically create methods, load them in runtime and link to whatever I want object !!

Using getattr function on self in python

I am trying to write call multiple functions through a loop using the getattr(...). Snippet below:
class cl1(module):
I =1
Name= 'name'+str(I)
Func= 'func'+str(I)
Namecall = gettattr(self,name)
Namecall = getattr(self,name)()
This is when get the following code: self.name1 = self.func1()
The desire is to loop multiple of these but the code is not working. Can you please advise?
Firstly, do use CapitalLetters for Classes and lowercase_letters for variables as it is easier to read for other Python programmers :)
Now, you don't need to use getattr() inside the class itself
Just do :
self.attribute
However, an example will be:
class Foo(object): # Class Foo inherits from 'object'
def __init__(self, a, b): # This is the initialize function. Add all arguments here
self.a = a # Setting attributes
self.b = b
def func(self):
print('Hello World!' + str(self.a) + str(self.b))
>>> new_object = Foo(a=1, b=2) # Creating a new 'Foo' object called 'new_object'
>>> getattr(new_object, 'a') # Getting the 'a' attribute from 'new_object'
1
However, an easier way would just be referencing the attribute directly
>>> new_object.a
1
>>> new_object.func()
Hello World!12
Or, by using getattr():
>>> getattr(new_object, 'func')()
Hello World!12
Although I explained the getattr() function,
I don't seem to understand what you want to achieve, do post a sample output.

Python extension methods

OK, in C# we have something like:
public static string Destroy(this string s) {
return "";
}
So basically, when you have a string you can do:
str = "This is my string to be destroyed";
newstr = str.Destroy()
# instead of
newstr = Destroy(str)
Now this is cool because in my opinion it's more readable. Does Python have something similar? I mean instead of writing like this:
x = SomeClass()
div = x.getMyDiv()
span = x.FirstChild(x.FirstChild(div)) # so instead of this
I'd like to write:
span = div.FirstChild().FirstChild() # which is more readable to me
Any suggestion?
You can just modify the class directly, sometimes known as monkey patching.
def MyMethod(self):
return self + self
MyClass.MyMethod = MyMethod
del(MyMethod)#clean up namespace
I'm not 100% sure you can do this on a special class like str, but it's fine for your user-defined classes.
Update
You confirm in a comment my suspicion that this is not possible for a builtin like str. In which case I believe there is no analogue to C# extension methods for such classes.
Finally, the convenience of these methods, in both C# and Python, comes with an associated risk. Using these techniques can make code more complex to understand and maintain.
You can do what you have asked like the following:
def extension_method(self):
#do stuff
class.extension_method = extension_method
I would use the Adapter pattern here. So, let's say we have a Person class and in one specific place we would like to add some health-related methods.
from dataclasses import dataclass
#dataclass
class Person:
name: str
height: float # in meters
mass: float # in kg
class PersonMedicalAdapter:
person: Person
def __init__(self, person: Person):
self.person = person
def __getattr__(self, item):
return getattr(self.person, item)
def get_body_mass_index(self) -> float:
return self.person.mass / self.person.height ** 2
if __name__ == '__main__':
person = Person('John', height=1.7, mass=76)
person_adapter = PersonMedicalAdapter(person)
print(person_adapter.name) # Call to Person object field
print(person_adapter.get_body_mass_index()) # Call to wrapper object method
I consider it to be an easy-to-read, yet flexible and pythonic solution.
You can change the built-in classes by monkey-patching with the help of forbidden fruit
But installing forbidden fruit requires a C compiler and unrestricted environment so it probably will not work or needs hard effort to run on Google App Engine, Heroku, etc.
I changed the behaviour of unicode class in Python 2.7 for a Turkish i,I uppercase/lowercase problem by this library.
# -*- coding: utf8 -*-
# Redesigned by #guneysus
import __builtin__
from forbiddenfruit import curse
lcase_table = tuple(u'abcçdefgğhıijklmnoöprsştuüvyz')
ucase_table = tuple(u'ABCÇDEFGĞHIİJKLMNOÖPRSŞTUÜVYZ')
def upper(data):
data = data.replace('i',u'İ')
data = data.replace(u'ı',u'I')
result = ''
for char in data:
try:
char_index = lcase_table.index(char)
ucase_char = ucase_table[char_index]
except:
ucase_char = char
result += ucase_char
return result
curse(__builtin__.unicode, 'upper', upper)
class unicode_tr(unicode):
"""For Backward compatibility"""
def __init__(self, arg):
super(unicode_tr, self).__init__(*args, **kwargs)
if __name__ == '__main__':
print u'istanbul'.upper()
You can achieve this nicely with the following context manager that adds the method to the class or object inside the context block and removes it afterwards:
class extension_method:
def __init__(self, obj, method):
method_name = method.__name__
setattr(obj, method_name, method)
self.obj = obj
self.method_name = method_name
def __enter__(self):
return self.obj
def __exit__(self, type, value, traceback):
# remove this if you want to keep the extension method after context exit
delattr(self.obj, self.method_name)
Usage is as follows:
class C:
pass
def get_class_name(self):
return self.__class__.__name__
with extension_method(C, get_class_name):
assert hasattr(C, 'get_class_name') # the method is added to C
c = C()
print(c.get_class_name()) # prints 'C'
assert not hasattr(C, 'get_class_name') # the method is gone from C
I'd like to think that extension methods in C# are pretty much the same as normal method call where you pass the instance then arguments and stuff.
instance.method(*args, **kwargs)
method(instance, *args, **kwargs) # pretty much the same as above, I don't see much benefit of it getting implemented in python.
After a week, I have a solution that is closest to what I was seeking for. The solution consists of using getattr and __getattr__. Here is an example for those who are interested.
class myClass:
def __init__(self): pass
def __getattr__(self, attr):
try:
methodToCall = getattr(myClass, attr)
return methodToCall(myClass(), self)
except:
pass
def firstChild(self, node):
# bla bla bla
def lastChild(self, node):
# bla bla bla
x = myClass()
div = x.getMYDiv()
y = div.firstChild.lastChild
I haven't test this example, I just gave it to give an idea for who might be interested. Hope that helps.
C# implemented extension methods because it lacks first class functions, Python has them and it is the preferred method for "wrapping" common functionality across disparate classes in Python.
There are good reasons to believe Python will never have extension methods, simply look at the available built-ins:
len(o) calls o.__len__
iter(o) calls o.__iter__
next(o) calls o.next
format(o, s) calls o.__format__(s)
Basically, Python likes functions.

Lambda function for classes in python?

There must be an easy way to do this, but somehow I can wrap my head around it. The best way I can describe what I want is a lambda function for a class. I have a library that expects as an argument an uninstantiated version of a class to work with. It then instantiates the class itself to work on. The problem is that I'd like to be able to dynamically create versions of the class, to pass to the library, but I can't figure out how to do it since the library expects an uninstantiated version. The code below describes the problem:
class Double:
def run(self,x):
return x*2
class Triple:
def run(self,x):
return x*3
class Multiply:
def __init__(self,mult):
self.mult = mult
def run(self,x):
return x*self.mult
class Library:
def __init__(self,c):
self.c = c()
def Op(self,val):
return self.c.run(val)
op1 = Double
op2 = Triple
#op3 = Multiply(5)
lib1 = Library(op1)
lib2 = Library(op2)
#lib3 = Library(op3)
print lib1.Op(2)
print lib2.Op(2)
#print lib3.Op(2)
I can't use the generic Multiply class, because I must instantiate it first which breaks the library "AttributeError: Multiply instance has no call method". Without changing the Library class, is there a way I can do this?
Does the library really specify that it wants an "uninitialized version" (i.e. a class reference)?
It looks to me as if the library actually wants an object factory. In that case, it's acceptable to type:
lib3 = Library(lambda: Multiply(5))
To understand how the lambda works, consider the following:
Multiply5 = lambda: Multiply(5)
assert Multiply5().run(3) == Multiply(5).run(3)
There's no need for lambda at all. lambda is just syntatic sugar to define a function and use it at the same time. Just like any lambda call can be replaced with an explicit def, we can solve your problem by creating a real class that meets your needs and returning it.
class Double:
def run(self,x):
return x*2
class Triple:
def run(self,x):
return x*3
def createMultiplier(n):
class Multiply:
def run(self,x):
return x*n
return Multiply
class Library:
def __init__(self,c):
self.c = c()
def Op(self,val):
return self.c.run(val)
op1 = Double
op2 = Triple
op3 = createMultiplier(5)
lib1 = Library(op1)
lib2 = Library(op2)
lib3 = Library(op3)
print lib1.Op(2)
print lib2.Op(2)
print lib3.Op(2)
This is sort of cheating, but you could give your Multiply class a __call__ method that returns itself:
class Multiply:
def __init__(self,mult):
self.mult = mult
def __call__(self):
return self
def run(self,x):
return x*self.mult
That way when the library calls c() it actually calls c.__call__() which returns the object you want.
def mult(x):
def f():
return Multiply(x)
return f
op3 = mult(5)
lib3 = Library(op3)
print lib3.Op(2)
If I understand your problem space correctly, you have a general interface that takes 1 argument which is called using the Library class. Unfortunately, rather than calling a function, Library assumes that the function is wrapped in a class with a run method.
You can certainly create these classes programatically. Classes may be returned by methods, and thanks to the concept of closures you should be able to wrap any function in a Class that meets your needs. Something like:
def make_op(f):
class MyOp(object):
def run(self, x):
return f(x)
return MyOp
op1 = make_op(lambda x: return x*2)
op2 = make_op(lambda x: return x*3)
def multiply_op(y):
return make_op(lambda x: return x*y)
op3 = multiply_op(3)
lib1 = Library(op1)
lib2 = Library(op2)
lib3 = Library(op3)
print( lib1.Op(2) )
print( lib2.Op(2) )
print( lib3.Op(2) )
That being said, changing Library to take a function and then providing functions is probably the stronger way to do this.
Since type is the default class of a python class object, and calling a class creates a new instance of that class, calling type with the correct arguments will result in a new class.
my_class = type("my_class", (object,), {"an_attribute": 1})
my_class now refers to a new class named "my_class", which is a subclass of object, with an attribute called "an_attribute", whose value is 1. Since methods are also just class attributes pointing to a function object, you can add them to the dictionary of attributes as well:
{"an_attribute": 1, "a_method": lambda self: print("Hello")}
This is how it works. I do not recommend doing it this way, unless you absolutely need to. In 99% of all cases, you don't. Refer to #Parker Coates' answer for the clean way to achieve your goal.

Categories