I would like to extend the Decimal class to add some helpful methods to it, specially for handling money.
The problem when I go do this:
from decimal import Decimal
class NewDecimal(Decimal):
def new_str(self):
return "${}".format(self)
d1 = NewDecimal(1)
print d1.new_str() # prints '$1'
d2 = NewDecimal(2)
d3 = NewDecimal(3)
d5 = d2 + d3
print d5.new_str() #exception happens here
It throws an exception:
AttributeError: 'Decimal' object has no attribute 'new_str'
This is because of the way Decimal does arithmetic, it always returns a new Decimal object, by literally calling Decimal(new value) at the end of the computation.
Does anyone no a workaround for this other than completely reimplementing all the arithmetic?
You probably don't actually want to do this just to have an extra method for printing Decimal objects in an alternate way. A top-level function or monkeypatched method is a whole lot simpler, and cleaner. Or, alternatively, a Money class that has a Decimal member that it delegates arithmetic to.
But what you want is doable.
To make NewDecimal(1) + NewDecimal(2) return NewDecimal(3), you can just override __add__:
def __add__(self, rhs):
return NewDecimal(super().__add__(rhs))
And of course you'll want to override __iadd__ as well. And don't forget mul and all the other numeric special methods.
But that still won't help for Decimal(2) + NewDecimal(3). To make that work, you need to define NewDecimal.__radd__. You also need to ensure that NewDecimal.__radd__ will get called instead of Decimal.__add__, but when you're using inheritance, that's easy, because Python has a rule specifically to make this easy:
Note: If the right operand’s type is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations.
You may want to read the section Implementing the arithmetic operations in the numbers module docs, and the implementation of fractions.Fraction (which was intended to serve as sample code for creating new numeric types, which is why the docs link directly to the source). Your life is easier than Fraction's because you can effectively fall back to Decimal for every operation and then convert (since NewDecimal doesn't have any different numeric behavior from Decimal), but it's worth seeing all the issues, and understanding which ones are and aren't relevant and why.
The quick way to what you want, would be like this:
from decimal import Decimal
class NewDecimal(Decimal):
def __str__(self):
return "${}".format(self)
def __add__(self,b):
return NewDecimal( Decimal.__add__(self,b) )
d1 = NewDecimal(1)
print d1 # prints '$1'
d2 = NewDecimal(2)
d3 = NewDecimal(3)
d5 = d2 + d3
print d5
> $5
Related
I'm learning Python on my own and I found a task that requires using a decorator #dataclass to create a class with basic arithmetic operations.
from dataclasses import dataclass
from numbers import Number
#dataclass
class MyClass:
x: float
y: float
def __add__(self, other):
match other:
case Number():
return MyClass(float(other) + self.x, self.y)
case MyClass(ot_x, ot_y):
return MyClass(self.x + ot_x, self.y + ot_y)
__radd__ = __add__
I have implemented the addition operation. But I also need to do the operations of subtraction __sub__, multiplication __mul__, division __truediv__, negation __neg__, also __mod__ and __pow__. But I couldn't realize these operations. The main thing for me is to use the construction match/case. Maybe there are simpler ways to create it.
I will be glad of your help.
If you're trying to make a complete numeric type, I strongly suggest checking out the implementation of the fractions.Fraction type in the fractions source code. The class was intentionally designed as a model for how you'd overload all the pairs of operators needed to implement a numeric type at the Python layer (it's explicitly pointed out in the numbers module's guide to type implementers).
The critical parts for minimizing boilerplate begin with the definition of the _operator_fallbacks utility function within the class (which is used to take a single implementation of the operation and the paired operator module function representing it, and generate the associated __op__ and __rop__ operators, being type strict for the former and relaxed for the latter, matching the intended behavior of each operator based on whether it's the first chance or last chance to implement the method).
It's far too much code to include here, but to show how you'd implement addition using it, I'll adapt your code to call it (you'd likely use a slightly different implementation of _operator_fallbacks, but the idea is the same):
import operator
# Optional, but if you want to act like built-in numeric types, you
# should be immutable, and using slots (if you can rely on Python 3.10+)
# dramatically reduce per-instance memory overhead
# Pre-3.10, since x and y don't have defaults, you could define __slots__ manually
#dataclass(frozen=True, slots=True)
class MyClass:
x: float
y: float
# _operator_fallbacks defined here
# When it received a non-MyClass, it would construct a MyClass from it, e.g.
# to match your original code it would construct it as MyClass(float(val), 0)
# and then invoke the monomorphic_operator, e.g. the _add passed to it below
# or, if the type did not make sense to convert to MyClass, but it made sense
# to convert the MyClass instance to the other type, it would do so, then use the
# provided fallback operator to perform the computation
# For completely incompatible types, it just returns NotImplemented
def _add(a, b):
"""a + b"""
return MyClass(a.x + b.x, a.y + b.y) # _operator_fallback has already coerced the types appropriately
__add__, __radd__ = _operator_fallbacks(_add, operator.add)
By putting the ugliness of type-checking and coercion in common code found in _operator_fallbacks, and putting only the real work of addition in _add, it avoids a lot of every-operator-overload boilerplate (as you can see here; _operator_fallbacks will be a page of code to make the forward and reverse functions and return them, but each new operator is only a few lines, defining the monomorphic operator and calling _operator_fallbacks to generate the __op__/__rop__ pair.
I was playing around with the python3k interpreter and came across the Decimal class in the decimal.py module. Documentation on the class indicates that objects created from the class are immutable. I tried to look into the python library code to see exactly how the programmers coded the class to make it immutable. Unfortunately, I was unable to discover exactly what they had done (mainly due to the fact that the class itself is quite large).
I am curious to know if anyone understands how this class achieves its immutable state. I am thinking it may have something to do with overriding the getattr() and setattr() methods within the class, but I could not find any evidence of this.
I also noticed that this class is able to hide the attributes it defines in its slots variable somehow. Perhaps the class uses some type of metaclass technique to achieve this as well as its immutability.
If anyone has any ideas I would appreciate your feedback. Thanks
Good question. Doesn't seem invincibly immutable to me. Python 2.7.2:
>>> from decimal import Decimal
>>> d = Decimal('1.23')
>>> d
Decimal('1.23')
>>> f = d
>>> f
Decimal('1.23')
>>> d._exp = 1
>>> f
Decimal('1.23E+3')
I do see that the documentation says it's immutable. I guess they mean if you use the documented interface, it's immutable. E.g.
>>> d += 100
>>> f
Decimal('1.23E+3')
>>> d
Decimal('1330')
The way this works is simply that Decimal does not define operator overloads for ?= operators: functions like __i[op]__. In that case, d += 100 translates to d = d.__add__(100), which returns a new object and changes the identity of d while not affecting the original object.
In Python, what is the difference between add and __add__ methods?
A method called add is just that - a method with that name. It has no special meaning whatsoever to the language or the interpreter. The only other thing that could be said about it is that sets have a method with the same name. That's it, nothing special about it.
The method __add__ is called internally by the + operator, so it gets special attention in the language spec and by the interpreter and you override it to define addition for object of a class. You don't call it directly (you can - they're still normal methods, they only get called implicitly in some circumstances and have some extra restrictions - but there's rarely if ever a reason - let alone a good reason). See the docs on "special" methods for details and a complete list of other "special" methods.
If you just went through this doc https://docs.python.org/3/library/operator.html and was curious about the differences between e.g.
operator.add(a, b)
operator.__add__(a, b)
Check the source code https://github.com/python/cpython/blob/3.10/Lib/operator.py :
def add(a, b):
"Same as a + b."
return a + b
...
# All of these "__func__ = func" assignments have to happen after importing
# from _operator to make sure they're set to the right function
...
__add__ = add
So
print(3+3) # call `operator.__add__` which is `operator.add`
import operator
print(operator.add(3, 3)) # call `operator.add` directory
To add to the earlier posts, __*__ are often discouraged as names for identifiers in own-classes unless one is doing some hacking on core-python functionality, like modifying / over-loading standard operators, etc. And also, often such names are linked with magical behavior, so it might be wise to avoid using them in own-namespaces unless the magical nature of a method is implied.
See this post for an elaborate argument
[Python 3.1]
I'm following up on this answer:
class prettyfloat(float):
def __repr__(self):
return "%0.2f" % self
I know I need to keep track of my float literals (i.e., replace 3.0 with prettyfloat(3.0), etc.), and that's fine.
But whenever I do any calculations, prettyfloat objects get converted into float.
What's the easiest way to fix it?
EDIT:
I need exactly two decimal digits; and I need it across the whole code, including where I print a dictionary with float values inside. That makes any formatting functions hard to use.
I can't use Decimal global setting, since I want computations to be at full precision (just printing at 2 decimal points).
#Glenn Maynard: I agree I shouldn't override __repr__; if anything, it would be just __str__. But it's a moot point because of the following point.
#Glenn Maynard and #singularity: I won't subclass float, since I agree it will look very ugly in the end.
I will stop trying to be clever, and just call a function everywhere a float is being printed. Though I am really sad that I can't override __str__ in the builtin class float.
Thank you!
I had a look at the answer you followed up on, and I think you're confusing data and its representation.
#Robert Rossney suggested to subclass float so you could map() an iterable of standard, non-adulterated floats into prettyfloats for display purposes:
# Perform all our computations using standard floats.
results = compute_huge_numbers(42)
# Switch to prettyfloats for printing.
print(map(prettyfloat, results))
In other words, you were not supposed to (and you shouldn't) use prettyfloat as a replacement for float everywhere in your code.
Of course, inheriting from float to solve that problem is overkill, since it's a representation problem and not a data problem. A simple function would be enough:
def prettyfloat(number):
return "%0.2f" % number # Works the same.
Now, if it's not about representation after all, and what you actually want to achieve is fixed-point computations limited to two decimal places everywhere in your code, that's another story entirely.
that because prettyfloat (op) prettyfloat don't return a prettyfloat
example:
>>> prettyfloat(0.6)
0.60 # type prettyfloat
>>> prettyfloat(0.6) + prettyfloat(4.4)
5.0 # type float
solution if you don't want to cast every operation result manually to prettyfloat and if you still want to use prettyfloat is to override all operators.
example with operator __add__ (which is ugly)
class prettyfloat(float):
def __repr__(self):
return "%0.2f" % self
def __add__(self, other):
return prettyfloat(float(self) + other)
>>> prettyfloat(0.6) + prettyfloat(4.4)
5.00
by doing this i think you will have also to change the name from prettyfloat to uglyfloat :) , Hope this will help
Use decimal. This is what it's for.
>>> import decimal
>>> decimal.getcontext().prec = 2
>>> one = decimal.Decimal("1.0")
>>> three = decimal.Decimal("3.0")
>>> one / three
Decimal('0.33')
...unless you actually want to work with full-precision floats everywhere in your code but print them rounded to two decimal places. In that case, you need to rewrite your printing logic.
From what I remember from my C++ class, the professor said that operator overloading is cool, but since it takes relatively a lot of thought and code to cover all end-cases (e.g. when overloading + you probably also want to overload ++ and +=, and also make sure to handle end cases like adding an object to itself etc.), you should only consider it in those cases where this feature will have a major impact on your code, like overloading the operators for the matrix class in a math application.
Does the same apply to python? Would you recommend overriding operator behavior in python? And what rules of thumb can you give me?
Operator overloading is mostly useful when you're making a new class that falls into an existing "Abstract Base Class" (ABC) -- indeed, many of the ABCs in standard library module collections rely on the presence of certain special methods (and special methods, one with names starting and ending with double underscores AKA "dunders", are exactly the way you perform operator overloading in Python). This provides good starting guidance.
For example, a Container class must override special method __contains__, i.e., the membership check operator item in container (as in, if item in container: -- don't confuse with the for statement, for item in container:, which relies on __iter__!-).
Similarly, a Hashable must override __hash__, a Sized must override __len__, a Sequence or a Mapping must override __getitem__, and so forth. (Moreover, the ABCs can provide your class with mixin functionality -- e.g., both Sequence and Mapping can provide __contains__ on the basis of your supplied __getitem__ override, and thereby automatically make your class a Container).
Beyond the collections, you'll want to override special methods (i.e. provide for operator overloading) mostly if your new class "is a number". Other special cases exist, but resist the temptation of overloading operators "just for coolness", with no semantic connection to the "normal" meanings, as C++'s streams do for << and >> and Python strings (in Python 2.*, fortunately not in 3.* any more;-) do for % -- when such operators do not any more mean "bit-shifting" or "division remainder", you're just engendering confusion. A language's standard library can get away with it (though it shouldn't;-), but unless your library gets as widespread as the language's standard one, the confusion will hurt!-)
I've written software with significant amounts of overloading, and lately I regret that policy. I would say this:
Only overload operators if it's the natural, expected thing to do and doesn't have any side effects.
So if you make a new RomanNumeral class, it makes sense to overload addition and subtraction etc. But don't overload it unless it's natural: it makes no sense to define addition and subtraction for a Car or a Vehicle object.
Another rule of thumb: don't overload ==. It makes it very hard (though not impossible) to actually test if two objects are the same. I made this mistake and paid for it for a long time.
As for when to overload +=, ++ etc, I'd actually say: only overload additional operators if you have a lot of demand for that functionality. It's easier to have one way to do something than five. Sure, it means sometimes you'll have to write x = x + 1 instead of x += 1, but more code is ok if it's clearer.
In general, like with many 'fancy' features, it's easy to think that you want something when you don't really, implement a bunch of stuff, not notice the side effects, and then figure it out later. Err on the conservative side.
EDIT: I wanted to add an explanatory note about overloading ==, because it seems various commenters misunderstand this, and it's caught me out. Yes, is exists, but it's a different operation. Say I have an object x, which is either from my custom class, or is an integer. I want to see if x is the number 500. But if you set x = 500, then later test x is 500, you will get False, due to the way Python caches numbers. With 50, it would return True. But you can't use is, because you might want x == 500 to return True if x is an instance of your class. Confusing? Definitely. But this is the kind of detail you need to understand to successfully overload operators.
Here is an example that uses the bitwise or operation to simulate a unix pipeline. This is intended as a counter example to most of the rules of thumb.
I just found Lumberjack which uses this syntax in real code
class pipely(object):
def __init__(self, *args, **kw):
self._args = args
self.__dict__.update(kw)
def __ror__(self, other):
return ( self.map(x) for x in other if self.filter(x) )
def map(self, x):
return x
def filter(self, x):
return True
class sieve(pipely):
def filter(self, x):
n = self._args[0]
return x==n or x%n
class strify(pipely):
def map(self, x):
return str(x)
class startswith(pipely):
def filter(self, x):
n=str(self._args[0])
if x.startswith(n):
return x
print"*"*80
for i in xrange(2,100) | sieve(2) | sieve(3) | sieve(5) | sieve(7) | strify() | startswith(5):
print i
print"*"*80
for i in xrange(2,100) | sieve(2) | sieve(3) | sieve(5) | sieve(7) | pipely(map=str) | startswith(5):
print i
print"*"*80
for i in xrange(2,100) | sieve(2) | sieve(3) | sieve(5) | sieve(7) | pipely(map=str) | pipely(filter=lambda x: x.startswith('5')):
print i
Python's overloading is "safer" in general than C++'s -- for example, the assignment operator can't be overloaded, and += has a sensible default implementation.
In some ways, though, overloading in Python is still as "broken" as in C++. Programmers should restrain the desire to "re-use" an operator for unrelated purposes, such as C++ re-using the bitshifts to perform string formatting and parsing. Don't overload an operator with different semantics from your implementation just to get prettier syntax.
Modern Python style strongly discourages "rogue" overloading, but many aspects of the language and standard library retain poorly-named operators for backwards compatibility. For example:
%: modulus and string formatting
+: addition and sequence concatenation
*: multiplication and sequence repetition
So, rule of thumb? If your operator implementation will surprise people, don't do it.