We Keep Coding

Return type of slice for a user-made container in python

I am creating a custom container that returns an instance of itself when sliced:
from typing import Union, List
class CustomContainer:
def __init__(self, values: List[int]):
self.values = values
def __getitem__(self, item: Union[int, slice]) -> Union[int, CustomContainer]:
if isinstance(item, slice):
return CustomContainer(self.values[item])
return self.values[item]
This works but comes with the following problem:
a = CustomContainer([1, 2])
b = a[0] # is always int, but recognized as both int and CustomContainer
c = a[:] # is always CustomContainer, but recognized as both int and CustomContainer
# Non-scalable solution: Forced type hint
d: int = a[0]
e: CustomContainer = a[:]
If I change the return type of __getitem__ to only int (my original approach), then a[0] correctly shows type int, but a[:] is considered a list instead of a CustomContainer.
As far as I understand, there used to be a function in python2 to define how slices are created, but it was removed in python3.
Is there a way to give the proper type hint without having to force the type hint every time I use my container?

You want to use typing.overload, which allows you to register multiple different signatures of a function with a type checker. Functions decorated with #overload are ignored at runtime, so you'll typically just fill the body with a literal ellipsis ..., pass, or a docstring. This also means that you have to keep at least one version of the function that isn't decorated with #overload, which will be the actual function used at runtime.
If you take a look at typeshed, the repository of stub files used by most major type-checkers for checking the standard library, you'll see this is the technique they use for annotating __getitem__ methods in custom containers such as collections.UserList. In your case, you'd annotate your method like this:
from typing import overload, Union, List
class CustomContainer:
def __init__(self, values: List[int]):
self.values = values
#overload
def __getitem__(self, item: int) -> int:
"""Signature when the function is passed an int"""
#overload
def __getitem__(self, item: slice) -> CustomContainer:
"""Signature when the function is passed a slice"""
def __getitem__(self, item: Union[int, slice]) -> Union[int, CustomContainer]:
"""Actual runtime implementation"""
if isinstance(item, slice):
return CustomContainer(self.values[item])
return self.values[item]
a = CustomContainer([1, 2])
b = a[0]
c = a[:]
reveal_type(b)
reveal_type(c)
Run it through MyPy, and it tells us:
main.py:24: note: Revealed type is "builtins.int"
main.py:25: note: Revealed type is "__main__.CustomContainer"
Further reading
The mypy docs for #overload can be found here.

How to type hint a return value that's defined in the function's local scope?

Given the following example:
class A:
def __init__(self, num: int):
self.num = num
def bar(self, eggs: int):
if eggs == self.num:
raise ValueError
def foo(spam: bool) -> B:
class B(A):
def bar(self, eggs: int):
try:
super().bar(eggs)
except ValueError as e:
if not spam:
raise e
return B
The base class A has a method named bar that raises a ValueError if eggs equals to self.num. I also have a function named foo that accepts an argument spam, it returns a subclass of A and overrides the bar method so that no ValueError will be raised if spam is non Falsey.
I'm trying to type hint the return value for the foo function. If I do -> B, B is undefined. If I do -> A, type B isn't exactly A. If I do -> "B", using future hinting, B is still not defined since it's in a local scope.
Is there anyway to type hint this? If not, is there a better way to rewrite my code?

Here's what I could think of from a quick scan of the docs. This isn't a good solution at all (I think), but it should do the job for you.
There's something called typing.TypeVar (docs). Its basically a generic type. So what you could possibly do is:
At the global level (after defining class A) define:
from typing import TypeVar
B = TypeVar('B', bound=A)
Now in the function signature of def foo you can say:
def foo(spam: bool) -> B:
Then inside the function you can continue to create a new class with the name B and return it.
I'm not even sure if this would work properly in all cases. If you find any problems with it, definitely correct me and post here.

Function signature in python

In C++ two functions with the same name can be created as long as the signature is different.
So for example myfunc(int x) is different from myfunc(float x).
In python you cannot do this, so, do you need to define functions with different names, or is there a better way to handle this situation?

In Python3.4+ you can use the functools.singledispatch decorator, which allows you to define a generic function and then register typed implementations against it.
From the docs
Generic function:
>>> from functools import singledispatch
>>> #singledispatch
... def fun(arg, verbose=False):
... if verbose:
... print("Let me just say,", end=" ")
... print(arg)
Typed functions:
>>> #fun.register(int)
... def _(arg, verbose=False):
... if verbose:
... print("Strength in numbers, eh?", end=" ")
... print(arg)
...
>>> #fun.register(list)
... def _(arg, verbose=False):
... if verbose:
... print("Enumerate this:")
... for i, elem in enumerate(arg):
... print(i, elem)
There's no built-in solution for earlier releases of Python, but Guido van Rossum blogged about a solution for python2 using decorators. (Edit: there is also a backport of the 3.4 functionality for pythons 2.6 - 3.3 on pypi)
Edit:
Of course, one of the advantages of using Python is that the the same code can often handle ints and floats without explicit dispatching on type, which is one of the reasons why the functionality has only recently been added.

Python doesn't really care whether an argument is an integer or a float. It's dynamically typed. You can do, for example, this:
def SquareMe(num):
return num**2
And you can call this function with any number (int, float, complex, ...).
It's also possible to do this:
def MultMe(data):
return data*2
This will work with numbers, strings (!), lists (!!), NumPy arrays and anything that can be multiplied by a number (if some class provides a method for this).

In python, you have to create only one method, but you can check what arguments can get passed, and if they are different arguments (ie: one is a float and another is an int) then you can differentiate two functions. In code this would look like:
def myfunc(*args):
# do something
# when you call the method
myfunc(a1, a2, k1=a3, k2=a4)
# you get:
args = (a1, a2)
kwds = {'k1':a3, 'k2':a4}
#So now lets recreate myfunc to check arguments
def myfunc(*args):
if isinstance(args[0], str): #This is where you determine argument type
# do what you want to do if argument is string
elif isinstance(args[1], int):
# do what you want to do if argument is an int

As ForceBru said Python dosen't realy care about parameter type , so if you do , you can handle it yourself:
def myfunc(x):
if(isinstance(x,int)):
print (x, 'int') # as myfunc(int x)
if(isinstance(x,float)):
print (x, 'float') # as myfunc(float x)
myfunc(10) # 10 int
myfunc(10.2) # 10.2 float
myfunc ("A") #

You could have the function itself do different things based on the types and number of parameters.
def f (a):
if type (a) == 'float' or type (a) == 'int':
...
if type (a) == 'list':
...
if type (a) == 'dict':
...

Check if an object is order-able in python?

How can I check if an object is orderable/sortable in Python?
I'm trying to implement basic type checking for the __init__ method of my binary tree class, and I want to be able to check if the value of the node is orderable, and throw an error if it isn't. It's similar to checking for hashability in the implementation of a hashtable.
I'm trying to accomplish something similar to Haskell's (Ord a) => etc. qualifiers. Is there a similar check in Python?

If you want to know if an object is sortable, you must check if it implements the necessary methods of comparison.
In Python 2.X there were two different ways to implement those methods:
cmp method (equivalent of compareTo in Java per example)
__cmp__(self, other): returns >0, 0 or <0 wether self is more, equal or less than other
rich comparison methods
__lt__, __gt__, __eq__, __le__, __ge__, __ne__
The sort() functions call this method to make the necessary comparisons between instances (actually sort only needs the __lt__ or __gt__ methods but it's recommended to implement all of them)
In Python 3.X the __cmp__ was removed in favor of the rich comparison methods as having more than one way to do the same thing is really against Python's "laws".
So, you basically need a function that check if these methods are implemented by a class:
# Python 2.X
def is_sortable(obj):
return hasattr(obj, "__cmp__") or \
hasattr(obj, "__lt__") or \
hasattr(obj, "__gt__")
# Python 3.X
def is_sortable(obj):
cls = obj.__class__
return cls.__lt__ != object.__lt__ or \
cls.__gt__ != object.__gt__
Different functions are needed for Python 2 and 3 because a lot of other things also change about unbound methods, method-wrappers and other internal things in Python 3.
Read this links you want better understanding of the sortable objects in Python:
http://python3porting.com/problems.html#unorderable-types-cmp-and-cmp
http://docs.python.org/2/howto/sorting.html#the-old-way-using-the-cmp-parameter
PS: this was a complete re-edit of my first answer, but it was needed as I investigated the problem better and had a cleaner idea about it :)

While the explanations in answers already here address runtime type inspection, here's how the static types are annotated by typeshed. They start by defining a collection of comparison Protocols, e.g.
class SupportsDunderLT(Protocol):
def __lt__(self, __other: Any) -> bool: ...
which are then collected into rich comparison sum types, such as
SupportsRichComparison = Union[SupportsDunderLT, SupportsDunderGT]
SupportsRichComparisonT = TypeVar("SupportsRichComparisonT", bound=SupportsRichComparison)
then finally these are used to type e.g. the key functions of list.sort:
#overload
def sort(self: list[SupportsRichComparisonT], *, key: None = ..., reverse: bool = ...) -> None: ...
#overload
def sort(self, *, key: Callable[[_T], SupportsRichComparison], reverse: bool = ...) -> None: ...
and sorted:
#overload
def sorted(
__iterable: Iterable[SupportsRichComparisonT], *, key: None = ..., reverse: bool = ...
) -> list[SupportsRichComparisonT]: ...
#overload
def sorted(__iterable: Iterable[_T], *, key: Callable[[_T], SupportsRichComparison], reverse: bool = ...) -> list[_T]: ...

Regrettably it is not enough to check that your object implements lt.
numpy uses the '<' operator to return an array of Booleans, which has no truth value. SQL Alchemy uses it to return a query filter, which again no truth value.
Ordinary sets uses it to check for a subset relationship, so that
set1 = {1,2}
set2 = {2,3}
set1 == set2
False
set1 < set2
False
set1 > set2
False
The best partial solution I could think of (starting from a single object of unknown type) is this, but with rich comparisons it seems to be officially impossible to determine orderability:
if hasattr(x, '__lt__'):
try:
isOrderable = ( ((x == x) is True) and ((x > x) is False)
and not isinstance(x, (set, frozenset)) )
except:
isOrderable = False
else:
isOrderable = False

Edited
As far as I know, all lists are sortable, so if you want to know if a list is "sortable", the answer is yes, no mather what elements it has.
class C:
def __init__(self):
self.a = 5
self.b = "asd"
c = C()
d = True
list1 = ["abc", "aad", c, 1, "b", 2, d]
list1.sort()
print list1
>>> [<__main__.C instance at 0x0000000002B7DF08>, 1, True, 2, 'aad', 'abc', 'b']
You could determine what types you consider "sortable" and implement a method to verify if all elements in the list are "sortable", something like this:
def isSortable(list1):
types = [int, float, str]
res = True
for e in list1:
res = res and (type(e) in types)
return res
print isSortable([1,2,3.0, "asd", [1,2,3]])

Automatically Type Cast Parameters In Python

Background:
I mostly run python scripts from the command line in pipelines and so my arguments are always strings that need to be type casted to the appropriate type. I make a lot of little scripts each day and type casting each parameter for every script takes more time than it should.
Question:
Is there a canonical way to automatically type cast parameters for a function?
My Way:
I've developed a decorator to do what I want if there isn't a better way. The decorator is the autocast fxn below. The decorated fxn is fxn2 in the example. Note that at the end of the code block I passed 1 and 2 as strings and if you run the script it will automatically add them. Is this a good way to do this?
def estimateType(var):
#first test bools
if var == 'True':
return True
elif var == 'False':
return False
else:
#int
try:
return int(var)
except ValueError:
pass
#float
try:
return float(var)
except ValueError:
pass
#string
try:
return str(var)
except ValueError:
raise NameError('Something Messed Up Autocasting var %s (%s)'
% (var, type(var)))
def autocast(dFxn):
'''Still need to figure out if you pass a variable with kw args!!!
I guess I can just pass the dictionary to the fxn **args?'''
def wrapped(*c, **d):
print c, d
t = [estimateType(x) for x in c]
return dFxn(*t)
return wrapped
#autocast
def fxn2(one, two):
print one + two
fxn2('1', '2')
EDIT: For anyone that comes here and wants the updated and concise working version go here:
https://github.com/sequenceGeek/cgAutoCast
And here is also quick working version based on above:
def boolify(s):
if s == 'True' or s == 'true':
return True
if s == 'False' or s == 'false':
return False
raise ValueError('Not Boolean Value!')
def estimateType(var):
'''guesses the str representation of the variables type'''
var = str(var) #important if the parameters aren't strings...
for caster in (boolify, int, float):
try:
return caster(var)
except ValueError:
pass
return var
def autocast(dFxn):
def wrapped(*c, **d):
cp = [estimateType(x) for x in c]
dp = dict( (i, estimateType(j)) for (i,j) in d.items())
return dFxn(*cp, **dp)
return wrapped
######usage######
#autocast
def randomFunction(firstVar, secondVar):
print firstVar + secondVar
randomFunction('1', '2')

If you want to auto-convert values:
def boolify(s):
if s == 'True':
return True
if s == 'False':
return False
raise ValueError("huh?")
def autoconvert(s):
for fn in (boolify, int, float):
try:
return fn(s)
except ValueError:
pass
return s
You can adjust boolify to accept other boolean values if you like.

You could just use plain eval to input string if you trust the source:
>>> eval("3.2", {}, {})
3.2
>>> eval("True", {}, {})
True
But if you don't trust the source, you could use literal_eval from ast module.
>>> ast.literal_eval("'hi'")
'hi'
>>> ast.literal_eval("(5, 3, ['a', 'b'])")
(5, 3, ['a', 'b'])
Edit:
As Ned Batchelder's comment, it won't accept non-quoted strings, so I added a workaround, also an example about autocaste decorator with keyword arguments.
import ast
def my_eval(s):
try:
return ast.literal_eval(s)
except ValueError: #maybe it's a string, eval failed, return anyway
return s #thanks gnibbler
def autocaste(func):
def wrapped(*c, **d):
cp = [my_eval(x) for x in c]
dp = {i: my_eval(j) for i,j in d.items()} #for Python 2.6+
#you can use dict((i, my_eval(j)) for i,j in d.items()) for older versions
return func(*cp, **dp)
return wrapped
#autocaste
def f(a, b):
return a + b
print(f("3.4", "1")) # 4.4
print(f("s", "sd")) # ssd
print(my_eval("True")) # True
print(my_eval("None")) # None
print(my_eval("[1, 2, (3, 4)]")) # [1, 2, (3, 4)]

I'd imagine you can make a type signature system with a function decorator, much like you have, only one that takes arguments. For example:
#signature(int, str, int)
func(x, y, z):
...
Such a decorator can be built rather easily. Something like this (EDIT -- works!):
def signature(*args, **kwargs):
def decorator(fn):
def wrapped(*fn_args, **fn_kwargs):
new_args = [t(raw) for t, raw in zip(args, fn_args)]
new_kwargs = dict([(k, kwargs[k](v)) for k, v in fn_kwargs.items()])
return fn(*new_args, **new_kwargs)
return wrapped
return decorator
And just like that, you can now imbue functions with type signatures!
#signature(int, int)
def foo(x, y):
print type(x)
print type(y)
print x+y
>>> foo('3','4')
<type: 'int'>
<type: 'int'>
7
Basically, this is an type-explicit version of #utdemir's method.

If you're parsing arguments from the command line, you should use the argparse module (if you're using Python 2.7).
Each argument can have an expected type so knowing what to do with it should be relatively straightforward. You can even define your own types.
...quite often the command-line string should instead be interpreted as another type, like a float or int. The type keyword argument of add_argument() allows any necessary type-checking and type conversions to be performed. Common built-in types and functions can be used directly as the value of the type argument:
parser = argparse.ArgumentParser()
parser.add_argument('foo', type=int)
parser.add_argument('bar', type=file)
parser.parse_args('2 temp.txt'.split())
>>> Namespace(bar=<open file 'temp.txt', mode 'r' at 0x...>, foo=2)

There are couple of problems in your snippet.
#first test bools
if var == 'True':
return True
elif var == 'False':
return False
This would always check for True because you are testing against the strings 'True' and 'False'.
There is not an automatic coercion of types in python. Your arguments when you receive via *args and **kwargs can be anything. First will look for list of values (each of which can be any datatype, primitive and complex) and second will look for a mapping (with any valid mapping possible). So if you write a decorator, you will end up with a good list of error checks.
Normally, if you wish to send in str, just when the function is invoked, typecast it to string via (str) and send it.

I know I arrived late at this game, but how about eval?
def my_cast(a):
try:
return eval(a)
except:
return a
or alternatively (and more safely):
from ast import literal_eval
def mycast(a):
try:
return literal_eval(a)
except:
return a