I'm trying to figure out how to organize app engine code with transactions. Currently I have a separate python file with all my transaction functions. For transactions that are closely related to entities, I was wondering if it made sense to use a #staticmethod for the transaction.
Here is a simple example:
class MyEntity(ndb.Model):
n = ndb.IntegerProperty(default=0)
#staticmethod
#ndb.transactional # does the order of the decorators matter?
def increment_n(my_entity_key):
my_entity = my_entity_key.get()
my_entity.n += 1
my_entity.put()
def do_something(self):
MyEntity.increment_n(self.key)
It would be nice to have increment_n associated with the entity definition, but I have never seen anyone do this so I was wondering if this would be a bad idea.
MY SOLUTION:
Following Brent's answer, I've implemented this:
class MyEntity(ndb.Model):
n = ndb.IntegerProperty(default=0)
#staticmethod
#ndb.transactional
def increment_n_transaction(my_entity_key):
my_entity = my_entity_key.get()
my_entity.increment_n()
def increment_n(self):
self.n += 1
self.put()
This way I can keep entity related code all in one place and I can easily use the transactional version or not as needed.
Yes, it makes sense to use a #staticmethod in this case, since the function doesn't use a class or an instance (self).
And yes, the order of decorators is important, as noted in #Kekito's later answer.
I later came across the accepted answer to this question. Here is a quote:
A decorator would wrap the function it is decorating. Here, your
add_cost function is wrapped by ndb.transactional so everything thing
within the function happens in the context of a transaction and then
the method returned by that is wrapped by classmethod which returns a
descriptor object.
So, when you apply multiple decorators in a class, then decorators
such as classmethod or staticmethod should be the top ones. If you
change the order you would receive an TypeError: unbound method ....
type of error if the other decorator doesn't accept descriptors.
So it looks like the order of decorators is very important. By luck, I had put mine in the right order, but updating this for others who come across this question.
Related
TLDR: What's best practice for accessing and altering attributes of a class instance?
Let's say we have a class to generate an object that is intended to hold data about a product (e.g. maybe a product master dataset).
class StoreProduct(object):
def __init__(self,ProductID,ProductPrice,ProductDescription):
self.ProductID = ProductID
self.ProductPrice = ProductPrice
self.ProductDescription = ProductDescription
def ChangeProductPrice(self, newProductPrice):
self.ProductPrice = newProductPrice
And we have another class we may be able to use to access those instances generated by StoreProduct(), with methods for making adjustments / changes.
class ChangeProductData(object):
def __init__(self):
pass
#staticmethod
def ChangeProductObjectPrice(newProductPrice,ProductObject):
ProductObject.ProductPrice = newProductPrice
So we generate an instance named Product1:
Product1 = StoreProduct(
ProductID="Product1",
ProductPrice=4,
ProductDescription="A nice lamp"
)
What's best practice coding for reading and/or altering class instances?
If I wanted to alter an attribute in Product 1 (in this case the price) is something like this acceptable in general, or is it bad code?
Method 1
ChangeProductData().ChangeProductObjectPrice(8,Product1)
Or is this the preferred way to do it?
Method 2
Product1.ChangeProductPrice(2)
When might there be exceptions?
While the above are simplified situations, what I've currently read seems to indicate that Method 2 might be better practice. However, wouldn't Method 1 provide greater flexibility in future (e.g. if you need to change how a method works, or add new methods).
I have also been reading into getattr() and setattr(), but people seem to be mixed on whether its better than using dot (e.g. Product1.ProductPrice to get the price).
Definitely method 2. As #user2357112 mentions, method 1 doesn't make sense.
As you rightly pointed out, this is a simple scenario and the pythonic way would be use property
class StoreProduct(object): # minor: please notice the PEP styling
def __init__(self, product_price):
self._price = produce_price
# other attributes, omitted for brevity
#property
def product_price(self): # pythonic getter
return self._price
#product_price.setter
def product_price(self, new_price): # python setter
# you could do any custom validation before setting
self._price = new_price
Coming to the question of
I'm specifically tackling the issue of whether accessing and altering an class' attribute with another class is a big no-no.
Accessing: Accessing is okay here. In fact many design patterns that rely on composition heavily do this. e.g., adapter, strategy, decorator, command etc patterns
Altering: You want the class owning the attribute to be in charge of "altering". Foreign classes should only request to alter.
P.S. This is treading a bit along the lines of CQRS(Command Query Responsibility Segregation) pattern
I just can't see why do we need to use #staticmethod. Let's start with an exmaple.
class test1:
def __init__(self,value):
self.value=value
#staticmethod
def static_add_one(value):
return value+1
#property
def new_val(self):
self.value=self.static_add_one(self.value)
return self.value
a=test1(3)
print(a.new_val) ## >>> 4
class test2:
def __init__(self,value):
self.value=value
def static_add_one(self,value):
return value+1
#property
def new_val(self):
self.value=self.static_add_one(self.value)
return self.value
b=test2(3)
print(b.new_val) ## >>> 4
In the example above, the method, static_add_one , in the two classes do not require the instance of the class(self) in calculation.
The method static_add_one in the class test1 is decorated by #staticmethod and work properly.
But at the same time, the method static_add_one in the class test2 which has no #staticmethod decoration also works properly by using a trick that provides a self in the argument but doesn't use it at all.
So what is the benefit of using #staticmethod? Does it improve the performance? Or is it just due to the zen of python which states that "Explicit is better than implicit"?
The reason to use staticmethod is if you have something that could be written as a standalone function (not part of any class), but you want to keep it within the class because it's somehow semantically related to the class. (For instance, it could be a function that doesn't require any information from the class, but whose behavior is specific to the class, so that subclasses might want to override it.) In many cases, it could make just as much sense to write something as a standalone function instead of a staticmethod.
Your example isn't really the same. A key difference is that, even though you don't use self, you still need an instance to call static_add_one --- you can't call it directly on the class with test2.static_add_one(1). So there is a genuine difference in behavior there. The most serious "rival" to a staticmethod isn't a regular method that ignores self, but a standalone function.
Today I suddenly find a benefit of using #staticmethod.
If you created a staticmethod within a class, you don't need to create an instance of the class before using the staticmethod.
For example,
class File1:
def __init__(self, path):
out=self.parse(path)
def parse(self, path):
..parsing works..
return x
class File2:
def __init__(self, path):
out=self.parse(path)
#staticmethod
def parse(path):
..parsing works..
return x
if __name__=='__main__':
path='abc.txt'
File1.parse(path) #TypeError: unbound method parse() ....
File2.parse(path) #Goal!!!!!!!!!!!!!!!!!!!!
Since the method parse is strongly related to the classes File1 and File2, it is more natural to put it inside the class. However, sometimes this parse method may also be used in other classes under some circumstances. If you want to do so using File1, you must create an instance of File1 before calling the method parse. While using staticmethod in the class File2, you may directly call the method by using the syntax File2.parse.
This makes your works more convenient and natural.
I will add something other answers didn't mention. It's not only a matter of modularity, of putting something next to other logically related parts. It's also that the method could be non-static at other point of the hierarchy (i.e. in a subclass or superclass) and thus participate in polymorphism (type based dispatching). So if you put that function outside the class you will be precluding subclasses from effectively overriding it. Now, say you realize you don't need self in function C.f of class C, you have three two options:
Put it outside the class. But we just decided against this.
Do nothing new: while unused, still keep the self parameter.
Declare you are not using the self parameter, while still letting other C methods to call f as self.f, which is required if you wish to keep open the possibility of further overrides of f that do depend on some instance state.
Option 2 demands less conceptual baggage (you already have to know about self and methods-as-bound-functions, because it's the more general case). But you still may prefer to be explicit about self not being using (and the interpreter could even reward you with some optimization, not having to partially apply a function to self). In that case, you pick option 3 and add #staticmethod on top of your function.
Use #staticmethod for methods that don't need to operate on a specific object, but that you still want located in the scope of the class (as opposed to module scope).
Your example in test2.static_add_one wastes its time passing an unused self parameter, but otherwise works the same as test1.static_add_one. Note that this extraneous parameter can't be optimized away.
One example I can think of is in a Django project I have, where a model class represents a database table, and an object of that class represents a record. There are some functions used by the class that are stand-alone and do not need an object to operate on, for example a function that converts a title into a "slug", which is a representation of the title that follows the character set limits imposed by URL syntax. The function that converts a title to a slug is declared as a staticmethod precisely to strongly associate it with the class that uses it.
I am struggling to understand when it makes sense to use an instance method versus a static method. Also, I don't know if my functions are static since there is not a #staticmethod decorator. Would I be able to access the class functions when I make a call to one of the methods?
I am working on a webscraper that sends information to a database. It’s setup to run once a week. The structure of my code looks like this
import libraries...
class Get:
def build_url(url_paramater1, url_parameter2, request_date):
return url_with_parameters
def web_data(request_date, url_parameter1, url_parameter2): #no use of self
# using parameters pull the variables to look up in the database
for a in db_info:
url = build_url(a, url_parameter2, request_date)
x = requests.Session().get(url, proxies).json()
#save data to the database
return None
#same type of function for pulling the web data from the database and parsing it
if __name__ == ‘__main__’:
Get.web_data(request_date, url_parameter1, url_parameter2)
Parse.web_data(get_date, parameter) #to illustrate the second part of the scrapper
That is the basic structure. The code is functional but I don’t know if I am using the methods (functions?) correctly and potentially missing out on ways to use my code in the future. I may even be writing bad code that will cause errors down the line that are impossibly hard to debug only because I didn’t follow best practices.
After reading about when class and instance methods are used. I cannot see why I would use them. If I want the url built or the data pulled from the website I call the build_url or get_web_data function. I don’t need an instance of the function to keep track of anything separate. I cannot imagine when I would need to keep something separate either which I think is part of the problem.
The reason I think my question is different than the previous questions is: the conceptual examples to explain the differences don't seem to help me when I am sitting down and writing code. I have not run into real world problems that are solved with the different methods that show when I should even use an instance method, yet instance methods seem to be mandatory when looking at conceptual examples of code.
Thank you!
Classes can be used to represent objects, and also to group functions under a common namespace.
When a class represents an object, like a cat, anything that this object 'can do', logically, should be an instance method, such as meowing.
But when you have a group of static functions that are all related to each other or are usually used together to achieve a common goal, like build_url and web_data, you can make your code clearer and more organized by putting them under a static class, which provides a common namespace, like you did.
Therefore in my opinion the structure you chose is legitimate. It is worth considering though, that you'd find static classes more in more definitively OOP languages, like Java, while in python it is more common to use modules for namespace separation.
This code doesn't need to be a class at all. It should just be a pair of functions. You can't see why you would need an instance method because you don't have a reason to instantiate the object in the first place.
The functions you have wrote in your code are instance methods but they were written incorrectly.
An instance method must have self as first parameter
i.e def build_url(self, url_paramater1, url_parameter2, request_date):
Then you call it like that
get_inst = Get()
get_inst.build_url(url_paramater1, url_parameter2, request_date)
This self parameter is provided by python and it allow you to access all properties and functions - static or not - of your Get class.
If you don't need to access other functions or properties in your class then you add #staticmethod decorator and remove self parameter
#staticmethod
def build_url(url_paramater1, url_parameter2, request_date):
And then you can call it directly
Get.build_url(url_paramater1, url_parameter2, request_date)
or call from from class instance
get_inst = Get()
get_inst.build_url(url_paramater1, url_parameter2, request_date)
But what is the problem with your current code you might ask?
Try calling it from an instance like this and u will see the problem
get_inst = Get()
get_inst.build_url(url_paramater1, url_parameter2, request_date)
Example where creating an instance is useful:
Let's say you want to make a chat client.
You could write code like this
class Chat:
def send(server_url, message):
connection = connect(server_url)
connection.write(message)
connection.close()
def read(server_url):
connection = connect(server_url)
message = connection.read()
connection.close()
return message
But a much cleaner and better way to do it:
class Chat:
def __init__(server_url):
# Initialize connection only once when instance is created
self.connection = connect(server_url)
def __del__()
# Close connection only once when instance is deleted
self.connection.close()
def send(self, message):
self.connection.write(message)
def read(self):
return self.connection.read()
To use that last class you do
# Create new instance and pass server_url as argument
chat = Chat("http://example.com/chat")
chat.send("Hello")
chat.read()
# deleting chat causes __del__ function to be called and connection be closed
delete chat
From given example, there is no need to have Get class after all, since you are using it just like a additional namespace. You do not have any 'state' that you want to preserve, in either class or class instance.
What seems like a good thing is to have separate module and define these functions in it. This way, when importing this module, you get to have this namespace that you want.
I need to combine Classes from two separate Python modules (which are similar in purpose but with different Methods) into a single Class so that the Methods can be accessed from the same object in a natural way both in code and for automatic documentation generation.
I am currently accomplishing the former but not the latter with the following code (this is not verbatim, as I can't share my actual source, but there's nothing different here that would impact the conversation).
Basically, I am creating the new class via a function which combines the __dict__ attributes of the two child Classes and returns a new Class.
def combine(argone, argtwo):
"""
Combine Classes
"""
_combined_arg = "some_string_%s_%s" % argone, argtwo
_temp = type('Temp', (ModuleOne, ModuleTwo), dict())
self = _temp(_combined_arg) # Calling the constructor with our combined arg
# The two classes have an identical constructor method within their __init__() methods
# Return the object we've instantiated off of the combined class
return self
This method works fine for producing an object that lets me call Methods from either of the original Classes, but my IDE can't auto-complete Method names nor can documentation generators (like pdoc) produce any documentation beyond our combine() function.
This process is necessary because we are generating code off of other code (descriptive, I know, sorry!) and it isn't practical to combine them upstream (ie, by hand).
Any ideas?
Thank you in advance!!!
ADDENDUM:
What I can say about what we are doing here is that we're just combining client Methods generated off of REST API endpoints that happen to be split into two, non-overlapping, namespaces for practical reasons that aren't important to this discussion. So that's why simply dropping the methods from ModuleTwo into ModuleOne would be all that needs doing.
If there are suggestions on an automatable and clean way to do this before shipping either module, I am definitely open to hearing them. Not having to do this work would be far preferable. Thanks!
There is no need for combine to define a new class every time it is called.
class CombinedAPI(APIOne, APITwo):
#classmethod
def combine(cls, arg_one, arg_two):
arg = "some_string_%s_%s" % (argone, argtwo)
return cls(arg)
obj = CombinedAPI.combine(foo, bar)
I have a class like below
class TestClass(object):
def __init__(self, data):
self.data = data
def method_a(self, data):
self.data += data/2
return self
def method_b(self, data):
self.data += data
return self
def method_c(self, data):
self.data -= data
return self
Every method returns self. I wrote it that way to be albe to call few metohds in a chain, ie. object.method_a(10).method_b(12).method_c(11). I was told that return self in method doesn't return current object, but creates new one. Is it really how it works? Is it good practice to use return self in python methods?
Is it really how it works?
No, it won't create a new object, it will return the same instance. You can check it by using is keyword, which checks if two objects are the same:
t = TestClass(3)
c = t.method_a(4)
print t is c
>>> True
Is it good practice to use return self in python methods?
Yes, it's often used to allow chaining.
Returning self does not create a new object.
I'm sure some people will tell you that method chaining is bad. I don't agree that it is necessarily bad. Writing an API that is designed for method chaining is a choice, and if you decide to do it you need to make sure everything works as people expect. But if you decide to do it, then returning self is often the right thing to do. (Sometimes, you may want to return a newly-created object instead. It depends whether you want the chained methods to cumulatively modify the original object or not.)
return self does not create a new object, you were told wrong.
If you want to build an API that supports chaining, returning self is a fine practice. You may want to build a new object instead for a chaining API, but that's not a requirement either.
See Python class method chaining for how the SQLAlchemy library handles chaining with new instances per change. Creating a new object when mutating objects in a chain allows you to re-use partial transformations as a starting point for several new objects with different chained transformations applied.