I have a simple class as:
connection has db connection
import pandas as pd
from animal import kettle
class cat:
def foo(connection):
a=pd.read_sql('select * from zoo',connection)
return1= kettle.boo1(a)
return2= kettle.boo2(a)
return return1,return2
Now I want to pass a to both boo1 and boo2 of kettle, am I passing it the correct way in above foo()?
I thought above way is correct and I tried this way , but is this correct way to pass?
animal.py:
class kettle:
def boo1(return1):
print(return1)
def boo2(return2):
print(return2)
sorry if this doesn't make any sense,
my intention is passing a to both boo1 and boo2 of kettle class
This looks like the correct approach to me: by assigning the return value of pd.read_sql('select * from zoo', connection) to a first and then passing a to kettle.boo1 and kettle.boo2 you ensure you only do the potentially time-consuming database IO only once.
One thing to keep in mind with this design pattern when you are passing objects such as lists/dicts/dataframes is the question of whether kettle.boo1 changes the value that is in a. If it does, kettle.boo2 will receive the modified version of a as an input, which can lead to unexpected behavior.
A very minimal example is the following:
>>> def foo(x):
... x[0] = 'b'
...
>>> x = ['a'] # define a list of length 1
>>> foo(x) # call a function that modifies the first element in x
>>> print(x) # the value in x has changed
['b']
There are (many) possible solutions for your problem, whatever that might be. I assume you just start out object oriented programming in Python, and get errors along the lines of
unbound method boo1() must be called with kettle instance as first argument
and probably want this solution:
Give your class methods an instance parameter:
def boo1(self, return1):
Instantiate the class kettle in cat.foo:
k = kettle()
Then use it like:
k.boo1(a)
Same for the boo2 method.
Also you probably want to:
return return1 # instead of or after print(return1)
as your methods return None at the moment.
Related
I apologize if I'm butchering the terminology. I'm trying to understand the code in this example on how to chain a custom function onto a PySpark dataframe. I'd really want to understand exactly what it's doing, and if it is not awful practice before I implement anything.
From the way I'm understanding the code, it:
defines a function g with sub-functions inside of it, that returns a copy of itself
assigns the sub-functions to g as attributes
assigns g as a property of the DataFrame class
I don't think at any step in the process do any of them become a method (when I do getattr, it always says "function")
When I run a (as best as I can do) simplified version of the code (below), it seems like only when I assign the function as a property to a class, and then instantiate at least one copy of the class, do the attributes on the function become available (even outside of the class). I want to understand what and why that is happening.
An answer [here(https://stackoverflow.com/a/17007966/19871699) indicates that this is a behavior, but doesn't really explain what/why it is. I've read this too but I'm having trouble seeing the connection to the code above.
I read here about the setattr part of the code. He doesn't mention exactly the use case above. this post has some use cases where people do it, but I'm not understanding how it directly applies to the above, unless I've missed something.
The confusing part is when the inner attributes become available.
class SampleClass():
def __init__(self):
pass
def my_custom_attribute(self):
def inner_function_one():
pass
setattr(my_custom_attribute,"inner_function",inner_function_one)
return my_custom_attribute
[x for x in dir(my_custom_attribute) if x[0] != "_"]
returns []
then when I do:
SampleClass.custom_attribute = property(my_custom_attribute)
[x for x in dir(my_custom_attribute) if x[0] != "_"]
it returns []
but when I do:
class_instance = SampleClass()
class_instance.custom_attribute
[x for x in dir(my_custom_attribute) if x[0] != "_"]
it returns ['inner_function']
In the code above though, if I do SampleClass.custom_attribute = my_custom_attribute instead of =property(...) the [x for x... code still returns [].
edit: I'm not intending to access the function itself outside of the class. I just don't understand the behavior, and don't like implementing something I don't understand.
So, setattr is not relevant here. This would all work exactly the same without it, say, by just doing my_custom_attribute.inner_function = inner_function_one etc. What is relevant is that the approach in the link you showed (which your example doesn't exactly make clear what the purpose is) relies on using a property, which is a descriptor. But the function won't get called unless you access the attribute corresponding to the property on an instance. This comes down to how property works. For any property, given a class Foo:
Foo.attribute_name = property(some_function)
Then some_function won't get called until you do Foo().attribute_name. That is the whole point of property.
But this whole solution is very confusingly engineered. It relies on the above behavior, and it sets attributes on the function object.
Note, if all you want to do is add some method to your DataFrame class, you don't need any of this. Consider the following example (using pandas for simplicity):
>>> import pandas as pd
>>> def foobar(self):
... print("in foobar with instance", self)
...
>>> pd.DataFrame.baz = foobar
>>> df = pd.DataFrame(dict(x=[1,2,3], y=['a','b','c']))
>>> df
x y
0 1 a
1 2 b
2 3 c
>>> df.baz()
in foobar with instance x y
0 1 a
1 2 b
2 3 c
That's it. You don't need all that rigamarole. Of course, if you wanted to add a nested accessor, df.custom.whatever, you would need something a bit more complicated. You could use the approach in the OP, but I would prefer something more explicit:
import pandas as pd
class AccessorDelegator:
def __init__(self, accessor_type):
self.accessor_type = accessor_type
def __get__(self, instance, cls=None):
return self.accessor_type(instance)
class CustomMethods:
def __init__(self, instance):
self.instance = instance
def foo(self):
# do something with self.instance as if this were your `self` on the dataframe being augmented
print(self.instance.value_counts())
pd.DataFrame.custom = AccessorDelegator(CustomMethods)
df = pd.DataFrame(dict(a=[1,2,3], b=['a','b','c']))
df.foo()
The above will print:
a b
1 a 1
2 b 1
3 c 1
Because when you call a function the attributes within that function aren't returned only the returned value is passed back.
In other words the additional attributes are only available on the returned function and not with 'g' itself.
Try moving setattr() outside of the function.
How can I pass an integer by reference in Python?
I want to modify the value of a variable that I am passing to the function. I have read that everything in Python is pass by value, but there has to be an easy trick. For example, in Java you could pass the reference types of Integer, Long, etc.
How can I pass an integer into a function by reference?
What are the best practices?
It doesn't quite work that way in Python. Python passes references to objects. Inside your function you have an object -- You're free to mutate that object (if possible). However, integers are immutable. One workaround is to pass the integer in a container which can be mutated:
def change(x):
x[0] = 3
x = [1]
change(x)
print x
This is ugly/clumsy at best, but you're not going to do any better in Python. The reason is because in Python, assignment (=) takes whatever object is the result of the right hand side and binds it to whatever is on the left hand side *(or passes it to the appropriate function).
Understanding this, we can see why there is no way to change the value of an immutable object inside a function -- you can't change any of its attributes because it's immutable, and you can't just assign the "variable" a new value because then you're actually creating a new object (which is distinct from the old one) and giving it the name that the old object had in the local namespace.
Usually the workaround is to simply return the object that you want:
def multiply_by_2(x):
return 2*x
x = 1
x = multiply_by_2(x)
*In the first example case above, 3 actually gets passed to x.__setitem__.
Most cases where you would need to pass by reference are where you need to return more than one value back to the caller. A "best practice" is to use multiple return values, which is much easier to do in Python than in languages like Java.
Here's a simple example:
def RectToPolar(x, y):
r = (x ** 2 + y ** 2) ** 0.5
theta = math.atan2(y, x)
return r, theta # return 2 things at once
r, theta = RectToPolar(3, 4) # assign 2 things at once
Not exactly passing a value directly, but using it as if it was passed.
x = 7
def my_method():
nonlocal x
x += 1
my_method()
print(x) # 8
Caveats:
nonlocal was introduced in python 3
If the enclosing scope is the global one, use global instead of nonlocal.
Maybe it's not pythonic way, but you can do this
import ctypes
def incr(a):
a += 1
x = ctypes.c_int(1) # create c-var
incr(ctypes.ctypes.byref(x)) # passing by ref
Really, the best practice is to step back and ask whether you really need to do this. Why do you want to modify the value of a variable that you're passing in to the function?
If you need to do it for a quick hack, the quickest way is to pass a list holding the integer, and stick a [0] around every use of it, as mgilson's answer demonstrates.
If you need to do it for something more significant, write a class that has an int as an attribute, so you can just set it. Of course this forces you to come up with a good name for the class, and for the attribute—if you can't think of anything, go back and read the sentence again a few times, and then use the list.
More generally, if you're trying to port some Java idiom directly to Python, you're doing it wrong. Even when there is something directly corresponding (as with static/#staticmethod), you still don't want to use it in most Python programs just because you'd use it in Java.
Maybe slightly more self-documenting than the list-of-length-1 trick is the old empty type trick:
def inc_i(v):
v.i += 1
x = type('', (), {})()
x.i = 7
inc_i(x)
print(x.i)
A numpy single-element array is mutable and yet for most purposes, it can be evaluated as if it was a numerical python variable. Therefore, it's a more convenient by-reference number container than a single-element list.
import numpy as np
def triple_var_by_ref(x):
x[0]=x[0]*3
a=np.array([2])
triple_var_by_ref(a)
print(a+1)
output:
7
The correct answer, is to use a class and put the value inside the class, this lets you pass by reference exactly as you desire.
class Thing:
def __init__(self,a):
self.a = a
def dosomething(ref)
ref.a += 1
t = Thing(3)
dosomething(t)
print("T is now",t.a)
In Python, every value is a reference (a pointer to an object), just like non-primitives in Java. Also, like Java, Python only has pass by value. So, semantically, they are pretty much the same.
Since you mention Java in your question, I would like to see how you achieve what you want in Java. If you can show it in Java, I can show you how to do it exactly equivalently in Python.
class PassByReference:
def Change(self, var):
self.a = var
print(self.a)
s=PassByReference()
s.Change(5)
class Obj:
def __init__(self,a):
self.value = a
def sum(self, a):
self.value += a
a = Obj(1)
b = a
a.sum(1)
print(a.value, b.value)// 2 2
In Python, everything is passed by value, but if you want to modify some state, you can change the value of an integer inside a list or object that's passed to a method.
integers are immutable in python and once they are created we cannot change their value by using assignment operator to a variable we are making it to point to some other address not the previous address.
In python a function can return multiple values we can make use of it:
def swap(a,b):
return b,a
a,b=22,55
a,b=swap(a,b)
print(a,b)
To change the reference a variable is pointing to we can wrap immutable data types(int, long, float, complex, str, bytes, truple, frozenset) inside of mutable data types (bytearray, list, set, dict).
#var is an instance of dictionary type
def change(var,key,new_value):
var[key]=new_value
var =dict()
var['a']=33
change(var,'a',2625)
print(var['a'])
How can I assign the results of a function call to multiple variables when the results are stored by name (not index-able), in python.
For example (tested in Python 3),
import random
# foo, as defined somewhere else where we can't or don't want to change it
def foo():
t = random.randint(1,100)
# put in a dummy class instead of just "return t,t+1"
# because otherwise we could subscript or just A,B = foo()
class Cat(object):
x = t
y = t + 1
return Cat()
# METHOD 1
# clearly wrong; A should be 1 more than B; they point to fields of different objects
A,B = foo().x, foo().y
print(A,B)
# METHOD 2
# correct, but requires two lines and an implicit variable
t = foo()
A,B = t.x, t.y
del t # don't really want t lying around
print(A,B)
# METHOD 3
# correct and one line, but an obfuscated mess
A,B = [ (t.x,t.y) for t in (foo(),) ][0]
print(A,B)
print(t) # this will raise an exception, but unless you know your python cold it might not be obvious before running
# METHOD 4
# Conforms to the suggestions in the links below without modifying the initial function foo or class Cat.
# But while all subsequent calls are pretty, but we have to use an otherwise meaningless shell function
def get_foo():
t = foo()
return t.x, t.y
A,B = get_foo()
What we don't want to do
If the results were indexable ( Cat extended tuple/list, we had used a namedtuple, etc.), we could simply write A,B = foo() as indicated in the comment above the Cat class. That's what's recommended here , for example.
Let's assume we have a good reason not to allow that. Maybe we like the clarity of assigning from the variable names (if they're more meaningful than x and y) or maybe the object is not primarily a container. Maybe the fields are properties, so access actually involves a method call. We don't have to assume any of those to answer this question though; the Cat class can be taken at face value.
This question already deals with how to design functions/classes the best way possible; if the function's expected return value are already well defined and does not involve tuple-like access, what is the best way to accept multiple values when returning?
I would strongly recommend either using multiple statements, or just keeping the result object without unpacking its attributes. That said, you can use operator.attrgetter for this:
from operator import attrgetter
a, b, c = attrgetter('a', 'b', 'c')(foo())
I have a class (list of dicts) and I want it to sort itself:
class Table(list):
…
def sort (self, in_col_name):
self = Table(sorted(self, key=lambda x: x[in_col_name]))
but it doesn't work at all. Why? How to avoid it? Except for sorting it externally, like:
new_table = Table(sorted(old_table, key=lambda x: x['col_name'])
Isn't it possible to manipulate the object itself? It's more meaningful to have:
class Table(list):
pass
than:
class Table(object):
l = []
…
def sort (self, in_col_name):
self.l = sorted(self.l, key=lambda x: x[in_col_name])
which, I think, works.
And in general, isn't there any way in Python which an object is able to change itself (not only an instance variable)?
You can't re-assign to self from within a method and expect it to change external references to the object.
self is just an argument that is passed to your function. It's a name that points to the instance the method was called on. "Assigning to self" is equivalent to:
def fn(a):
a = 2
a = 1
fn(a)
# a is still equal to 1
Assigning to self changes what the self name points to (from one Table instance to a new Table instance here). But that's it. It just changes the name (in the scope of your method), and does affect not the underlying object, nor other names (references) that point to it.
Just sort in place using list.sort:
def sort(self, in_col_name):
super(Table, self).sort(key=lambda x: x[in_col_name])
Python is pass by value, always. This means that assigning to a parameter will never have an effect on the outside of the function. self is just the name you chose for one of the parameters.
I was intrigued by this question because I had never thought about this. I looked for the list.sort code, to see how it's done there, but apparently it's in C. I think I see where you're getting at; what if there is no super method to invoke? Then you can do something like this:
class Table(list):
def pop_n(self, n):
for _ in range(n):
self.pop()
>>> a = Table(range(10))
>>> a.pop_n(3)
>>> print a
[0, 1, 2, 3, 4, 5, 6]
You can call self's methods, do index assignments to self and whatever else is implemented in its class (or that you implement yourself).
I have a class, and I would like to be able to create multiple objects of that class and place them in an array. I did it like so:
rooms = []
rooms.append(Object1())
...
rooms.append(Object4())
I then have a dict of functions, and I would like to pass the object to the function. However, I'm encountering some problems..For example, I have a dict:
dict = {'look': CallLook(rooms[i])}
I'm able to pass it into the function, however; in the function if I try to call an objects method it gives me problems
def CallLook(current_room)
current_room.examine()
I'm sure that there has to be a better way to do what I'm trying to do, but I'm new to Python and I haven't seen a clean example on how to do this. Anyone have a good way to implement a list of objects to be passed into functions? All of the objects contain the examine method, but they are objects of different classes. (I'm sorry I didn't say so earlier)
The specific error states: TypeError: 'NoneType' object is not callable
Anyone have a good way to implement a list of objects to be passed into functions? All of the objects contain the examine method, but they are objects of different classes. (I'm sorry I didn't say so earlier)
This is Python's plain duck-typing.
class Room:
def __init__(self, name):
self.name = name
def examine(self):
return "This %s looks clean!" % self.name
class Furniture:
def __init__(self, name):
self.name = name
def examine(self):
return "This %s looks comfortable..." % self.name
def examination(l):
for item in l:
print item.examine()
list_of_objects = [ Room("Living Room"), Furniture("Couch"),
Room("Restrooms"), Furniture("Bed") ]
examination(list_of_objects)
Prints:
This Living Room looks clean!
This Couch looks comfortable...
This Restrooms looks clean!
This Bed looks comfortable...
As for your specific problem: probably you have forgotten to return a value from examine()? (Please post the full error message (including full backtrace).)
I then have a dict of functions, and I would like to pass the object to the function. However, I'm encountering some problems..For example, I have a dict:
my_dict = {'look': CallLook(rooms[i])} # this is no dict of functions
The dict you have created may evaluate to {'look': None} (assuming your examine() doesn't return a value.) Which could explain the error you've observed.
If you wanted a dict of functions you needed to put in a callable, not an actual function call, e.g. like this:
my_dict = {'look': CallLook} # this is a dict of functions
if you want to bind the 'look' to a specific room you could redefine CallLook:
def CallLook(current_room)
return current_room.examine # return the bound examine
my_dict = {'look': CallLook(room[i])} # this is also a dict of functions
Another issue with your code is that you are shadowing the built-in dict() method by naming your local dictionary dict. You shouldn't do this. This yields nasty errors.
Assuming you don't have basic problems (like syntax errors because the code you have pasted is not valid Python), this example shows you how to do what you want:
>>> class Foo():
... def hello(self):
... return 'hello'
...
>>> r = [Foo(),Foo(),Foo()]
>>> def call_method(obj):
... return obj.hello()
...
>>> call_method(r[1])
'hello'
Assuming you have a class Room the usual way to create a list of instances would be using a list comprehension like this
rooms = [Room() for i in range(num_rooms)]
I think there are some things you may not be getting about this:
dict = {'look': CallLook(rooms[i])}
This creates a dict with just one entry: a key 'look', and a value which is the result of evaluating CallLook(rooms[i]) right at the point of that statement. It also then uses the name dict to store this object, so you can no longer use dict as a constructor in that context.
Now, the error you are getting tells us that rooms[i] is None at that point in the programme.
You don't need CallLook (which is also named non-standardly) - you can just use the expression rooms[i].examine(), or if you want to evaluate the call later rooms[i].examine.
You probably don't need the dict at all.
That is not a must, but in some cases, using hasattr() is good... getattr() is another way to get an attribute off an object...
So:
rooms = [Obj1(),Obj2(),Obj3()]
if hasattr(rooms[i], 'examine'):#First check if our object has selected function or attribute...
getattr(rooms[i], 'examine') #that will just evaluate the function do not call it, and equals to Obj1().examine
getattr(rooms[i], 'examine')() # By adding () to the end of getattr function, we evalute and then call the function...
You may also pass parameters to examine function like:
getattr(rooms[i], 'examine')(param1, param2)
I'm not sure of your requirement, but you can use dict to store multiple object of a class.
May be this will help,
>>> class c1():
... print "hi"
...
hi
>>> c = c1()
>>> c
<__main__.c1 instance at 0x032165F8>
>>> d ={}
>>> for i in range (10):
... d[i] = c1()
...
>>> d[0]
<__main__.c1 instance at 0x032166E8>
>>> d[1]
<__main__.c1 instance at 0x032164B8>
>>>
It will create a object of c1 class and store it in dict. Obviously, in this case you can use list instead of dict.