Defining typed dictionaries in Python [duplicate] - python

How do I check if an object is of a given type, or if it inherits from a given type?
How do I check if the object o is of type str?
Beginners often wrongly expect the string to already be "a number" - either expecting Python 3.x input to convert type, or expecting that a string like '1' is also simultaneously an integer. This is the wrong canonical for those questions. Please carefully read the question and then use How do I check if a string represents a number (float or int)?, How can I read inputs as numbers? and/or Asking the user for input until they give a valid response as appropriate.

Use isinstance to check if o is an instance of str or any subclass of str:
if isinstance(o, str):
To check if the type of o is exactly str, excluding subclasses of str:
if type(o) is str:
See Built-in Functions in the Python Library Reference for relevant information.
Checking for strings in Python 2
For Python 2, this is a better way to check if o is a string:
if isinstance(o, basestring):
because this will also catch Unicode strings. unicode is not a subclass of str; both str and unicode are subclasses of basestring. In Python 3, basestring no longer exists since there's a strict separation of strings (str) and binary data (bytes).
Alternatively, isinstance accepts a tuple of classes. This will return True if o is an instance of any subclass of any of (str, unicode):
if isinstance(o, (str, unicode)):

The most Pythonic way to check the type of an object is... not to check it.
Since Python encourages Duck Typing, you should just try...except to use the object's methods the way you want to use them. So if your function is looking for a writable file object, don't check that it's a subclass of file, just try to use its .write() method!
Of course, sometimes these nice abstractions break down and isinstance(obj, cls) is what you need. But use sparingly.

isinstance(o, str) will return True if o is an str or is of a type that inherits from str.
type(o) is str will return True if and only if o is a str. It will return False if o is of a type that inherits from str.

After the question was asked and answered, type hints were added to Python. Type hints in Python allow types to be checked but in a very different way from statically typed languages. Type hints in Python associate the expected types of arguments with functions as runtime accessible data associated with functions and this allows for types to be checked. Example of type hint syntax:
def foo(i: int):
return i
foo(5)
foo('oops')
In this case we want an error to be triggered for foo('oops') since the annotated type of the argument is int. The added type hint does not cause an error to occur when the script is run normally. However, it adds attributes to the function describing the expected types that other programs can query and use to check for type errors.
One of these other programs that can be used to find the type error is mypy:
mypy script.py
script.py:12: error: Argument 1 to "foo" has incompatible type "str"; expected "int"
(You might need to install mypy from your package manager. I don't think it comes with CPython but seems to have some level of "officialness".)
Type checking this way is different from type checking in statically typed compiled languages. Because types are dynamic in Python, type checking must be done at runtime, which imposes a cost -- even on correct programs -- if we insist that it happen at every chance. Explicit type checks may also be more restrictive than needed and cause unnecessary errors (e.g. does the argument really need to be of exactly list type or is anything iterable sufficient?).
The upside of explicit type checking is that it can catch errors earlier and give clearer error messages than duck typing. The exact requirements of a duck type can only be expressed with external documentation (hopefully it's thorough and accurate) and errors from incompatible types can occur far from where they originate.
Python's type hints are meant to offer a compromise where types can be specified and checked but there is no additional cost during usual code execution.
The typing package offers type variables that can be used in type hints to express needed behaviors without requiring particular types. For example, it includes variables such as Iterable and Callable for hints to specify the need for any type with those behaviors.
While type hints are the most Pythonic way to check types, it's often even more Pythonic to not check types at all and rely on duck typing. Type hints are relatively new and the jury is still out on when they're the most Pythonic solution. A relatively uncontroversial but very general comparison: Type hints provide a form of documentation that can be enforced, allow code to generate earlier and easier to understand errors, can catch errors that duck typing can't, and can be checked statically (in an unusual sense but it's still outside of runtime). On the other hand, duck typing has been the Pythonic way for a long time, doesn't impose the cognitive overhead of static typing, is less verbose, and will accept all viable types and then some.

In Python 3.10, you can use | in isinstance:
>>> isinstance('1223', int | str)
True
>>> isinstance('abcd', int | str)
True

isinstance(o, str)
Link to docs

You can check for type of a variable using __name__ of a type.
Ex:
>>> a = [1,2,3,4]
>>> b = 1
>>> type(a).__name__
'list'
>>> type(a).__name__ == 'list'
True
>>> type(b).__name__ == 'list'
False
>>> type(b).__name__
'int'

For more complex type validations I like typeguard's approach of validating based on python type hint annotations:
from typeguard import check_type
from typing import List
try:
check_type('mylist', [1, 2], List[int])
except TypeError as e:
print(e)
You can perform very complex validations in very clean and readable fashion.
check_type('foo', [1, 3.14], List[Union[int, float]])
# vs
isinstance(foo, list) and all(isinstance(a, (int, float)) for a in foo)

I think the cool thing about using a dynamic language like Python is you really shouldn't have to check something like that.
I would just call the required methods on your object and catch an AttributeError. Later on this will allow you to call your methods with other (seemingly unrelated) objects to accomplish different tasks, such as mocking an object for testing.
I've used this a lot when getting data off the web with urllib2.urlopen() which returns a file like object. This can in turn can be passed to almost any method that reads from a file, because it implements the same read() method as a real file.
But I'm sure there is a time and place for using isinstance(), otherwise it probably wouldn't be there :)

The accepted answer answers the question in that it provides the answers to the asked questions.
Q: What is the best way to check whether a given object is of a given type? How about checking whether the object inherits from a given type?
A: Use isinstance, issubclass, type to check based on types.
As other answers and comments are quick to point out however, there's a lot more to the idea of "type-checking" than that in python. Since the addition of Python 3 and type hints, much has changed as well. Below, I go over some of the difficulties with type checking, duck typing, and exception handling. For those that think type checking isn't what is needed (it usually isn't, but we're here), I also point out how type hints can be used instead.
Type Checking
Type checking is not always an appropriate thing to do in python. Consider the following example:
def sum(nums):
"""Expect an iterable of integers and return the sum."""
result = 0
for n in nums:
result += n
return result
To check if the input is an iterable of integers, we run into a major issue. The only way to check if every element is an integer would be to loop through to check each element. But if we loop through the entire iterator, then there will be nothing left for intended code. We have two options in this kind of situation.
Check as we loop.
Check beforehand but store everything as we check.
Option 1 has the downside of complicating our code, especially if we need to perform similar checks in many places. It forces us to move type checking from the top of the function to everywhere we use the iterable in our code.
Option 2 has the obvious downside that it destroys the entire purpose of iterators. The entire point is to not store the data because we shouldn't need to.
One might also think that checking if checking all of the elements is too much then perhaps we can just check if the input itself is of the type iterable, but there isn't actually any iterable base class. Any type implementing __iter__ is iterable.
Exception Handling and Duck Typing
An alternative approach would be to forgo type checking altogether and focus on exception handling and duck typing instead. That is to say, wrap your code in a try-except block and catch any errors that occur. Alternatively, don't do anything and let exceptions rise naturally from your code.
Here's one way to go about catching an exception.
def sum(nums):
"""Try to catch exceptions?"""
try:
result = 0
for n in nums:
result += n
return result
except TypeError as e:
print(e)
Compared to the options before, this is certainly better. We're checking as we run the code. If there's a TypeError anywhere, we'll know. We don't have to place a check everywhere that we loop through the input. And we don't have to store the input as we iterate over it.
Furthermore, this approach enables duck typing. Rather than checking for specific types, we have moved to checking for specific behaviors and look for when the input fails to behave as expected (in this case, looping through nums and being able to add n).
However, the exact reasons which make exception handling nice can also be their downfall.
A float isn't an int, but it satisfies the behavioral requirements to work.
It is also bad practice to wrap the entire code with a try-except block.
At first these may not seem like issues, but here's some reasons that may change your mind.
A user can no longer expect our function to return an int as intended. This may break code elsewhere.
Since exceptions can come from a wide variety of sources, using the try-except on the whole code block may end up catching exceptions you didn't intend to. We only wanted to check if nums was iterable and had integer elements.
Ideally we'd like to catch exceptions our code generators and raise, in their place, more informative exceptions. It's not fun when an exception is raised from someone else's code with no explanation other than a line you didn't write and that some TypeError occured.
In order to fix the exception handling in response to the above points, our code would then become this... abomination.
def sum(nums):
"""
Try to catch all of our exceptions only.
Re-raise them with more specific details.
"""
result = 0
try:
iter(nums)
except TypeError as e:
raise TypeError("nums must be iterable")
for n in nums:
try:
result += int(n)
except TypeError as e:
raise TypeError("stopped mid iteration since a non-integer was found")
return result
You can kinda see where this is going. The more we try to "properly" check things, the worse our code is looking. Compared to the original code, this isn't readable at all.
We could argue perhaps this is a bit extreme. But on the other hand, this is only a very simple example. In practice, your code is probably much more complicated than this.
Type Hints
We've seen what happens when we try to modify our small example to "enable type checking". Rather than focusing on trying to force specific types, type hinting allows for a way to make types clear to users.
from typing import Iterable
def sum(nums: Iterable[int]) -> int:
result = 0
for n in nums:
result += n
return result
Here are some advantages to using type-hints.
The code actually looks good now!
Static type analysis may be performed by your editor if you use type hints!
They are stored on the function/class, making them dynamically usable e.g. typeguard and dataclasses.
They show up for functions when using help(...).
No need to sanity check if your input type is right based on a description or worse lack thereof.
You can "type" hint based on structure e.g. "does it have this attribute?" without requiring subclassing by the user.
The downside to type hinting?
Type hints are nothing more than syntax and special text on their own. It isn't the same as type checking.
In other words, it doesn't actually answer the question because it doesn't provide type checking. Regardless, however, if you are here for type checking, then you should be type hinting as well. Of course, if you've come to the conclusion that type checking isn't actually necessary but you want some semblance of typing, then type hints are for you.

To Hugo:
You probably mean list rather than array, but that points to the whole problem with type checking - you don't want to know if the object in question is a list, you want to know if it's some kind of sequence or if it's a single object. So try to use it like a sequence.
Say you want to add the object to an existing sequence, or if it's a sequence of objects, add them all
try:
my_sequence.extend(o)
except TypeError:
my_sequence.append(o)
One trick with this is if you are working with strings and/or sequences of strings - that's tricky, as a string is often thought of as a single object, but it's also a sequence of characters. Worse than that, as it's really a sequence of single-length strings.
I usually choose to design my API so that it only accepts either a single value or a sequence - it makes things easier. It's not hard to put a [ ] around your single value when you pass it in if need be.
(Though this can cause errors with strings, as they do look like (are) sequences.)

If you have to check for the type of str or int please use instanceof. As already mentioned by others the explanation is to also include sub classes. One important example for sub classes from my perspective are Enums with data type like IntEnum or StrEnum. Which are a pretty nice way to define related constants. However, it is kind of annoying if libraries do not accept those as such types.
Example:
import enum
class MyEnum(str, enum.Enum):
A = "a"
B = "b"
print(f"is string: {isinstance(MyEnum.A, str)}") # True
print(f"is string: {type(MyEnum.A) == str}") # False!!!
print(f"is string: {type(MyEnum.A.value) == str}") # True

In Python, you can use the built-in isinstance() function to check if an object is of a given type, or if it inherits from a given type.
To check if the object o is of type str, you would use the following code:
if isinstance(o, str):
# o is of type str
You can also use type() function to check the object type.
if type(o) == str:
# o is of type str
You can also check if the object is a sub class of a particular class using issubclass() function.
if issubclass(type(o),str):
# o is sub class of str

A simple way to check type is to compare it with something whose type you know.
>>> a = 1
>>> type(a) == type(1)
True
>>> b = 'abc'
>>> type(b) == type('')
True

I think the best way is to typing well your variables. You can do this by using the "typing" library.
Example:
from typing import NewType
UserId = NewType ('UserId', int)
some_id = UserId (524313`)
See https://docs.python.org/3/library/typing.html.

Related

I'm trying to validate a python class property, but it isn't raising the error I tried to raise. How should I correct this? [duplicate]

How do I check if an object is of a given type, or if it inherits from a given type?
How do I check if the object o is of type str?
Beginners often wrongly expect the string to already be "a number" - either expecting Python 3.x input to convert type, or expecting that a string like '1' is also simultaneously an integer. This is the wrong canonical for those questions. Please carefully read the question and then use How do I check if a string represents a number (float or int)?, How can I read inputs as numbers? and/or Asking the user for input until they give a valid response as appropriate.
Use isinstance to check if o is an instance of str or any subclass of str:
if isinstance(o, str):
To check if the type of o is exactly str, excluding subclasses of str:
if type(o) is str:
See Built-in Functions in the Python Library Reference for relevant information.
Checking for strings in Python 2
For Python 2, this is a better way to check if o is a string:
if isinstance(o, basestring):
because this will also catch Unicode strings. unicode is not a subclass of str; both str and unicode are subclasses of basestring. In Python 3, basestring no longer exists since there's a strict separation of strings (str) and binary data (bytes).
Alternatively, isinstance accepts a tuple of classes. This will return True if o is an instance of any subclass of any of (str, unicode):
if isinstance(o, (str, unicode)):
The most Pythonic way to check the type of an object is... not to check it.
Since Python encourages Duck Typing, you should just try...except to use the object's methods the way you want to use them. So if your function is looking for a writable file object, don't check that it's a subclass of file, just try to use its .write() method!
Of course, sometimes these nice abstractions break down and isinstance(obj, cls) is what you need. But use sparingly.
isinstance(o, str) will return True if o is an str or is of a type that inherits from str.
type(o) is str will return True if and only if o is a str. It will return False if o is of a type that inherits from str.
After the question was asked and answered, type hints were added to Python. Type hints in Python allow types to be checked but in a very different way from statically typed languages. Type hints in Python associate the expected types of arguments with functions as runtime accessible data associated with functions and this allows for types to be checked. Example of type hint syntax:
def foo(i: int):
return i
foo(5)
foo('oops')
In this case we want an error to be triggered for foo('oops') since the annotated type of the argument is int. The added type hint does not cause an error to occur when the script is run normally. However, it adds attributes to the function describing the expected types that other programs can query and use to check for type errors.
One of these other programs that can be used to find the type error is mypy:
mypy script.py
script.py:12: error: Argument 1 to "foo" has incompatible type "str"; expected "int"
(You might need to install mypy from your package manager. I don't think it comes with CPython but seems to have some level of "officialness".)
Type checking this way is different from type checking in statically typed compiled languages. Because types are dynamic in Python, type checking must be done at runtime, which imposes a cost -- even on correct programs -- if we insist that it happen at every chance. Explicit type checks may also be more restrictive than needed and cause unnecessary errors (e.g. does the argument really need to be of exactly list type or is anything iterable sufficient?).
The upside of explicit type checking is that it can catch errors earlier and give clearer error messages than duck typing. The exact requirements of a duck type can only be expressed with external documentation (hopefully it's thorough and accurate) and errors from incompatible types can occur far from where they originate.
Python's type hints are meant to offer a compromise where types can be specified and checked but there is no additional cost during usual code execution.
The typing package offers type variables that can be used in type hints to express needed behaviors without requiring particular types. For example, it includes variables such as Iterable and Callable for hints to specify the need for any type with those behaviors.
While type hints are the most Pythonic way to check types, it's often even more Pythonic to not check types at all and rely on duck typing. Type hints are relatively new and the jury is still out on when they're the most Pythonic solution. A relatively uncontroversial but very general comparison: Type hints provide a form of documentation that can be enforced, allow code to generate earlier and easier to understand errors, can catch errors that duck typing can't, and can be checked statically (in an unusual sense but it's still outside of runtime). On the other hand, duck typing has been the Pythonic way for a long time, doesn't impose the cognitive overhead of static typing, is less verbose, and will accept all viable types and then some.
In Python 3.10, you can use | in isinstance:
>>> isinstance('1223', int | str)
True
>>> isinstance('abcd', int | str)
True
isinstance(o, str)
Link to docs
You can check for type of a variable using __name__ of a type.
Ex:
>>> a = [1,2,3,4]
>>> b = 1
>>> type(a).__name__
'list'
>>> type(a).__name__ == 'list'
True
>>> type(b).__name__ == 'list'
False
>>> type(b).__name__
'int'
For more complex type validations I like typeguard's approach of validating based on python type hint annotations:
from typeguard import check_type
from typing import List
try:
check_type('mylist', [1, 2], List[int])
except TypeError as e:
print(e)
You can perform very complex validations in very clean and readable fashion.
check_type('foo', [1, 3.14], List[Union[int, float]])
# vs
isinstance(foo, list) and all(isinstance(a, (int, float)) for a in foo)
I think the cool thing about using a dynamic language like Python is you really shouldn't have to check something like that.
I would just call the required methods on your object and catch an AttributeError. Later on this will allow you to call your methods with other (seemingly unrelated) objects to accomplish different tasks, such as mocking an object for testing.
I've used this a lot when getting data off the web with urllib2.urlopen() which returns a file like object. This can in turn can be passed to almost any method that reads from a file, because it implements the same read() method as a real file.
But I'm sure there is a time and place for using isinstance(), otherwise it probably wouldn't be there :)
The accepted answer answers the question in that it provides the answers to the asked questions.
Q: What is the best way to check whether a given object is of a given type? How about checking whether the object inherits from a given type?
A: Use isinstance, issubclass, type to check based on types.
As other answers and comments are quick to point out however, there's a lot more to the idea of "type-checking" than that in python. Since the addition of Python 3 and type hints, much has changed as well. Below, I go over some of the difficulties with type checking, duck typing, and exception handling. For those that think type checking isn't what is needed (it usually isn't, but we're here), I also point out how type hints can be used instead.
Type Checking
Type checking is not always an appropriate thing to do in python. Consider the following example:
def sum(nums):
"""Expect an iterable of integers and return the sum."""
result = 0
for n in nums:
result += n
return result
To check if the input is an iterable of integers, we run into a major issue. The only way to check if every element is an integer would be to loop through to check each element. But if we loop through the entire iterator, then there will be nothing left for intended code. We have two options in this kind of situation.
Check as we loop.
Check beforehand but store everything as we check.
Option 1 has the downside of complicating our code, especially if we need to perform similar checks in many places. It forces us to move type checking from the top of the function to everywhere we use the iterable in our code.
Option 2 has the obvious downside that it destroys the entire purpose of iterators. The entire point is to not store the data because we shouldn't need to.
One might also think that checking if checking all of the elements is too much then perhaps we can just check if the input itself is of the type iterable, but there isn't actually any iterable base class. Any type implementing __iter__ is iterable.
Exception Handling and Duck Typing
An alternative approach would be to forgo type checking altogether and focus on exception handling and duck typing instead. That is to say, wrap your code in a try-except block and catch any errors that occur. Alternatively, don't do anything and let exceptions rise naturally from your code.
Here's one way to go about catching an exception.
def sum(nums):
"""Try to catch exceptions?"""
try:
result = 0
for n in nums:
result += n
return result
except TypeError as e:
print(e)
Compared to the options before, this is certainly better. We're checking as we run the code. If there's a TypeError anywhere, we'll know. We don't have to place a check everywhere that we loop through the input. And we don't have to store the input as we iterate over it.
Furthermore, this approach enables duck typing. Rather than checking for specific types, we have moved to checking for specific behaviors and look for when the input fails to behave as expected (in this case, looping through nums and being able to add n).
However, the exact reasons which make exception handling nice can also be their downfall.
A float isn't an int, but it satisfies the behavioral requirements to work.
It is also bad practice to wrap the entire code with a try-except block.
At first these may not seem like issues, but here's some reasons that may change your mind.
A user can no longer expect our function to return an int as intended. This may break code elsewhere.
Since exceptions can come from a wide variety of sources, using the try-except on the whole code block may end up catching exceptions you didn't intend to. We only wanted to check if nums was iterable and had integer elements.
Ideally we'd like to catch exceptions our code generators and raise, in their place, more informative exceptions. It's not fun when an exception is raised from someone else's code with no explanation other than a line you didn't write and that some TypeError occured.
In order to fix the exception handling in response to the above points, our code would then become this... abomination.
def sum(nums):
"""
Try to catch all of our exceptions only.
Re-raise them with more specific details.
"""
result = 0
try:
iter(nums)
except TypeError as e:
raise TypeError("nums must be iterable")
for n in nums:
try:
result += int(n)
except TypeError as e:
raise TypeError("stopped mid iteration since a non-integer was found")
return result
You can kinda see where this is going. The more we try to "properly" check things, the worse our code is looking. Compared to the original code, this isn't readable at all.
We could argue perhaps this is a bit extreme. But on the other hand, this is only a very simple example. In practice, your code is probably much more complicated than this.
Type Hints
We've seen what happens when we try to modify our small example to "enable type checking". Rather than focusing on trying to force specific types, type hinting allows for a way to make types clear to users.
from typing import Iterable
def sum(nums: Iterable[int]) -> int:
result = 0
for n in nums:
result += n
return result
Here are some advantages to using type-hints.
The code actually looks good now!
Static type analysis may be performed by your editor if you use type hints!
They are stored on the function/class, making them dynamically usable e.g. typeguard and dataclasses.
They show up for functions when using help(...).
No need to sanity check if your input type is right based on a description or worse lack thereof.
You can "type" hint based on structure e.g. "does it have this attribute?" without requiring subclassing by the user.
The downside to type hinting?
Type hints are nothing more than syntax and special text on their own. It isn't the same as type checking.
In other words, it doesn't actually answer the question because it doesn't provide type checking. Regardless, however, if you are here for type checking, then you should be type hinting as well. Of course, if you've come to the conclusion that type checking isn't actually necessary but you want some semblance of typing, then type hints are for you.
To Hugo:
You probably mean list rather than array, but that points to the whole problem with type checking - you don't want to know if the object in question is a list, you want to know if it's some kind of sequence or if it's a single object. So try to use it like a sequence.
Say you want to add the object to an existing sequence, or if it's a sequence of objects, add them all
try:
my_sequence.extend(o)
except TypeError:
my_sequence.append(o)
One trick with this is if you are working with strings and/or sequences of strings - that's tricky, as a string is often thought of as a single object, but it's also a sequence of characters. Worse than that, as it's really a sequence of single-length strings.
I usually choose to design my API so that it only accepts either a single value or a sequence - it makes things easier. It's not hard to put a [ ] around your single value when you pass it in if need be.
(Though this can cause errors with strings, as they do look like (are) sequences.)
If you have to check for the type of str or int please use instanceof. As already mentioned by others the explanation is to also include sub classes. One important example for sub classes from my perspective are Enums with data type like IntEnum or StrEnum. Which are a pretty nice way to define related constants. However, it is kind of annoying if libraries do not accept those as such types.
Example:
import enum
class MyEnum(str, enum.Enum):
A = "a"
B = "b"
print(f"is string: {isinstance(MyEnum.A, str)}") # True
print(f"is string: {type(MyEnum.A) == str}") # False!!!
print(f"is string: {type(MyEnum.A.value) == str}") # True
In Python, you can use the built-in isinstance() function to check if an object is of a given type, or if it inherits from a given type.
To check if the object o is of type str, you would use the following code:
if isinstance(o, str):
# o is of type str
You can also use type() function to check the object type.
if type(o) == str:
# o is of type str
You can also check if the object is a sub class of a particular class using issubclass() function.
if issubclass(type(o),str):
# o is sub class of str
A simple way to check type is to compare it with something whose type you know.
>>> a = 1
>>> type(a) == type(1)
True
>>> b = 'abc'
>>> type(b) == type('')
True
I think the best way is to typing well your variables. You can do this by using the "typing" library.
Example:
from typing import NewType
UserId = NewType ('UserId', int)
some_id = UserId (524313`)
See https://docs.python.org/3/library/typing.html.

Should I type something as Optional if None breaks the logic of the function, but I do check for it inside the body?

Sometimes at the beginning of my Python functions I check whether the correct variables types were used, or whether something was passed as None. For example:
def fails_with_none(x: int):
if x is None:
raise TypeError('Function fails with None!')
return x + 1
I am hesitating whether x should be typed as int or as Optional[int].
The reason for using just int is that, semantically, the function requires an int.
However, if I think about this from a programming perspective, the function handles both integer and None inputs.
Is there a recommended way?
For example, according to this answer the Optional hint means "either an object of the specific type is required, or None is required".
However, the question still remains: Required by what? If we take it to mean "required by the logic of the function", then it should be typed as int. If we take it to mean "required by the code being executed", then since we check whether x is None, it should be included as a possible type hint.
Precisely because the type hint won't stop None from being passed at runtime is why you should use int instead of Optional[int].
Using Optional[int] prevents someone who uses mypy from catching what should be a static type error, while using int won't affect anyone not using a static type checker anyway.
int both documents your intent and provides the ability to catch the error prior to running your code.
Once you do that, it is my opinion that you should drop the explicit instance check. If the check fails, that means the caller has already ignored the documentation and potentially a static type checker and passed None as an argument anyway. At this point, it's really not your job to protect them from the consequences of None + 1 anymore.
TL;DR: If x being None has meaning in your function, then annotate x accordingly. Otherwise don't and don't check it.
Sometimes at the beginning of my python functions I check whether the correct variables types were used, or whether something was passed as None.
I am not saying this is absolutely wrong, but there is no doubt that this goes hardcore against the philosophy of Python.
A main pillar of the paradigm of the language is dynamic typing. This is also why type annotations in Python are commonly referred to as hints. This is to signal that the language has no built in mechanism for enforcing this and that by convention functions do not force type conformity on users.
The contract is this:
"Hey user, my function f accepts an argument of type int. That is how I designed it. You may try and use it differently, but at your own peril. I don't vouch for it working as intended, if you pass anything other than int to it."
If you insist on enforcing types, then the only consistent way IMHO is to do a negative check against the type you annotated with. If you have f(x: int), then it would be consistent to check like this:
def f(x: int) -> None:
if not isinstance(x, int):
raise TypeError
print(x**2)
Because the alternative is arbitrarily checking against any other type that x might also be. Specifically, why would you check for it being None? Why not also check against it being a str? Or an empty tuple? Or literally anything else?
Since you provided no additional context, I have to assume from your example that None has no special semantic meaning in your function other than that it triggers the TypeError.
I am hesitating whether x should be typed as int or as Optional[int]. The reason for using just int is that, semantically, the function requires an int.
There is your answer then. Semantics is king. If None has no other meaning in your function, then you should completely ignore its existence. It's up to the user of the function to adhere to your type annotations or disregard them at his own peril.
In this concrete case, this would not even be particularly useful because the default built-in Python error for misusing numeric operands is this:
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'
So you don't even change the error type, just the message in this particular case.
Don't get me wrong, there are obviously cases, where passing None as an argument is meaningful. If an argument being None has meaning in a function, then obviously you should type it accordingly. Here is a crude example:
def f(x: int | None) -> None:
if x is None:
print("Ok then.")
else:
print(f"{x**2=}")
In this case the function behaves differently depending on whether you pass it an int or a NoneType. Thus, a union of those is the proper type annotation for x. And Optional[int] is just a (arguably poorly named) equivalent of that union.
Thus, to me, your question title is a non-starter:
Should I type something as Optional if None breaks the logic of the function, but I do check for it inside the body?
If you check for None, then None does not break the logic of your function; it is specifically part of the logic and thus should be accounted for in the parameter type annotation.
The bigger question (as I laid out above) is, whether you should check for it. If you only check, so you can yell at the programmer in your own customized way for using None, I don't see the value in that.

How to correctly determine a type?

this has probably been asked a hundred times before, but you are a product of your own success!
In Python I see this in other peoples code
if type(number) != type(1):
Is there any good reason to do this, should not
if type(number) != int
be the better way to do this?
Is it a historical thing?
Neither is a the correct way.
The correct way is to use isinstance():
if not isinstance(number, int):
This makes sure that subclasses are not allowed either.
In the extremely rare case you must absolutely only bar int and allow subclasses, you use type() with not is; types are singletons:
if type(number) is not int:
Still more pythonic is to use ask for forgiveness rather than for permission; treat the value as an integer until proven otherwise (via an exception for example).
There is no good reason to do any of this. It is probably a symptom of someone used to other strongly typed languages writing Python code.
If you must (but again, think very hard about this) you should use isinstance():
>>> isinstance('1', str)
True
Python is a language for "consenting adults", which is a fancy way of saying we expect you to know what you are doing; just like that famous "with great power comes great responsibility" quote from your favorite superhero movie.
In Python, objects are expected to behave nicely and you are not supposed to have such surprises to have to do explicit type checking.
Normally such type checking is done to make sure you can do something to the value. A naive example would be to add two things (use the + sign), you might want to check that they are both some sort of number. Obviously 'hello' + 5 is not going to work.
So you might be tempted to write code like this:
def add_things(first, second):
if type(first) == int and type(second) == int:
print(first+second)
Then after reading these answers you'd change the above to:
def add_things_smarter(first, second):
if isinstance(first, int) and isinstance(second, int):
print(first+second)
However, its probably best to try to do the operation, then handle the error:
def add_things_better(first, second):
try:
print(first+second)
except ValueError:
print("I can't add {} + {}".format(first, second))
This is known as:
EAFP
Easier to ask for forgiveness than permission. This common Python
coding style assumes the existence of valid keys or attributes and
catches exceptions if the assumption proves false. This clean and fast
style is characterized by the presence of many try and except
statements. The technique contrasts with the LBYL style common to many
other languages such as C.
This is from the glossary section of the documentation, where you will find the definition of LBYL (Look Before You Leap).
Martijn Pieters and Burhan Khalid have given good answers, but neither are the whole story.
In Python, normally the best thing to do with a value is to assume it's the right type. You should avoid using type-based dispatch (that's what the type(x) is type(y) check is) unless it's important.
So instead of doing:
if type(number) is int:
do_int_stuff
else:
not_an_int
you should do:
try:
do_int_stuff
except NotTheRightTypeForThatException:
not_an_int
Better yet, don't get into that position in the first place; just have the right type. Although languages like Java let you overload function by their type, in Python it's normally best to assume that a function takes a mandatory non-optional interface (like requiring addition or being iterable) and never give them something they don't support.
But there are times where it's appropriate to do type-based dispatch, and both kinds (isinstance and type(x) is type(y)) have their place, although the former is far more common.
One time to use isinstance is when implementing operators (like +, -, *, /) for a new type. You want
X() * X()
to work, but you don't want to say what
X() * Y()
means unless you know you're doing the right thing. The reasons for this are nontrivial, but involve the fact that both types need to cooperate to decide what gets run. You would normally do an isinstance check, because if you implement
int(4) * int(2)
and someone subclasses int:
class MyInt(int): ...
you don't want * to break!
The uses of type(X) is type(Y) are much rarer. One practical case is where you are interfacing two type-systems; when converting one type to another language's analogue (or serializing it in something like JSON), it can make more sense to require exact types. In particular, this keeps the round-trip (converting and converting back) lossless.
Normally you'll know when you need this, and it's also far rarer than the others.

How do you set a conditional in python based on datatypes?

This question seems mind-boggling simple, yet I can't figure it out. I know you can check datatypes in python, but how can you set a conditional based on the datatype? For instance, if I have to write a code that sorts through a dictionary/list and adds up all the integers, how do I isolate the search to look for only integers?
I guess a quick example would look something like this:
y = []
for x in somelist:
if type(x) == <type 'int'>: ### <--- psuedo-code line
y.append(x)
print sum(int(z) for z in y)
So for line 3, how would I set such a conditional?
How about,
if isinstance(x, int):
but a cleaner way would simply be
sum(z for z in y if isinstance(z, int))
TLDR:
Use if isinstance(x, int): unless you have a reason not to.
Use if type(x) is int: if you need exact type equality and nothing else.
Use try: ix = int(x) if you are fine with converting to the target type.
There is a really big "it depends" to type-checking in Python. There are many ways to deal with types, and all have their pros and cons. With Python3, several more have emerged.
Explicit type equality
Types are first-class objects, and you can treat them like any other value.
So if you want the type of something to be equal to int, just test for it:
if type(x) is int:
This is the most restrictive type of testing: it requires exact type equality. Often, this is not what you want:
It rules out substitute types: a float would not be valid, even though it behaves like an int for many purposes.
It rules out subclasses and abstract types: a pretty-printing int subclass or enum would be rejected, even though they are logically Integers.
This severely limits portability: Python2 Strings can be either str or unicode, and Integers can be either int or long.
Note that explicit type equality has its uses for low-level operations:
Some types cannot be subclassed, such as slice. An explicit check is, well, more explicit here.
Some low-level operations, such as serialisation or C-APIs, require specific types.
Variants
A comparison can also be performed against the __class__ attribute:
if x.__class__ is int:
Note if a class defines a __class__ property, this is not the same as type(x).
When there are several classes to check for, using a dict to dispatch actions is more extensible and can be faster (≥5-10 types) than explicit checks.
This is especially useful for conversions and serialisation:
dispatch_dict = {float: round, str: int, int: lambda x: x}
def convert(x):
converter = self.dispatch_dict[type(x)] # lookup callable based on type
return converter(x)
Instance check on explicit types
The idiomatic type test uses the isinstance builtin:
if isinstance(x, int):
This check is both exact and performant. This is most often what people want for checking types:
It handles subtypes properly. A pretty-printing int subclass would still pass this test.
It allows checking multiple types at once. In Python2, doing isinstance(x, (int, long)) gets you all builtin integers.
Most importantly, the downsides are negligible most of the time:
It still accepts funky subclasses that behave in weird ways. Since anything can be made to behave in weird ways, this is futile to guard against.
It can easily be too restrictive: many people check for isinstance(x, list) when any sequence (e.g. tuple) or even iterable (e.g. a generator) would do as well. This is more of a concern for general purpose libraries than scripts or applications.
Variant
If you already have a type, issubclass behaves the same:
if issubclass(x_type, int):
Instance check on abstract type
Python has a concept of abstract base classes. Loosely speaking, these express the meaning of types, not their hierarchy:
if isinstance(x, numbers.Real): # accept anything you can sum up like a number
In other words, type(x) does not necessarily inherit from numbers.Real but must behave like it.
Still, this is a very complex and difficult concept:
It is often overkill if you are looking for basic types. An Integer is simply an int most of the time.
People coming from other languages often confuse its concepts.
Distinguishing it from e.g. C++, the emphasis is abstract base class as opposed to abstract base class.
ABCs can be used like Java interfaces, but may still have concrete functionality.
However, it is incredibly useful for generic libraries and abstractions.
Many functions/algorithms do not need explicit types, just their behaviour.
If you just need to look up things by key, dict restricts you to a specific in-memory type. By contrast, collections.abc.Mapping also includes database wrappers, large disk-backed dictionaries, lazy containers, ... - and dict.
It allows expressing partial type constraints.
There is no strict base type implementing iteration. But if you check objects against collections.abc.Iterable, they all work in a for loop.
It allows creating separate, optimised implementations that appear as the same abstract type.
While it is usually not needed for throwaway scripts, I would highly recommend using this for anything that lives beyond a few python releases.
Tentative conversion
The idiomatic way of handling types is not to test them, but to assume they are compatible. If you already expect some wrong types in your input, simply skip everything that is not compatible:
try:
ix = int(x)
except (ValueError, TypeError):
continue # not compatible with int, try the next one
else:
a.append(ix)
This is not actually a type check, but usually serves the same intention.
It guarantees you have the expected type in your output.
It has some limited leeway in converting wrong types, e.g. specialising float to int.
It works without you knowing which types conform to int.
The major downside is that it is an explicit transformation.
You can silently accept "wrong" values, e.g. converting a str containing a literal.
It needlessly converts even types that would be good enough, e.g. float to int when you just need numbers.
Conversion is an effective tool for some specific use cases. It works best if you know roughly what your input is, and must make guarantees about your output.
Function dispatch
Sometimes the goal of type checking is just to select an appropriate function. In this case, function dispatch such as functools.singledispatch allows specialising function implementations for specific types:
#singledispatch
def append_int(value, sequence):
return
#append_int.register
def _(value: int, sequence):
sequence.append(value)
This is a combination of isinstance and dict dispatch. It is most useful for larger applications:
It keeps the site of usage small, regardless of the number of dispatched types.
It allows registering specialisations for additional types later, even in other modules.
Still, it doesn't come without its downsides:
Originating in functional and strongly typed languages, many Python programmers are not familiar with single- or even multiple-dispatch.
Dispatches require separate functions, and are therefore not suitable to be defined at the site of usage.
Creating the functions and "warming up" the dispatch cache takes notable runtime overhead. Dispatch functions should be defined once and re-used often.
Even a warmed up dispatch table is slower than a hand-written if/else or dict lookup.
Controlling the input
The best course of action is to ensure you never have to check for type in the first place. This is a bit of a meta-topic, as it depends strongly on the use case.
Here, the source of somelist should never have put non-numbers into it.
You can simply use type and equal operator like this
if (type(x) == int):
let me declare variable x of type int
x = 2
if type(x) == type(1) or isinstance(x, int):
# do something
Both works fine.
Easy - use types.
import types
k = 5
if(type(k)==types.IntType):
print "int"
Here's a quick dir(types):
['BooleanType', 'BufferType', 'BuiltinFunctionType', 'BuiltinMethodType', 'ClassType', 'CodeType', 'ComplexType', 'DictProxyType', 'DictType', 'DictionaryType', 'EllipsisType', 'FileType', 'FloatType', 'FrameType', 'FunctionType', 'GeneratorType', 'GetSetDescriptorType', 'InstanceType', 'IntType', 'LambdaType', 'ListType', 'LongType', 'MemberDescriptorType', 'MethodType', 'ModuleType', 'NoneType', 'NotImplementedType', 'ObjectType', 'SliceType', 'StringType', 'StringTypes', 'TracebackType', 'TupleType', 'TypeType', 'UnboundMethodType', 'UnicodeType', 'XRangeType', '__builtins__', '__doc__', '__file__', '__name__', '__package__']
You can use the type function on both sides of the operator. Like this:
if type(x) == type(1):

How can I avoid type checking a python object if its attributes aren't used?

I've come across answers here for type checking in general, type checking for numbers, and type checking for strings. Most people seem to respond by saying that type checking should never be performed in python (< 2.6) due to duck typing. My (limited) understanding of duck typing is that type is determined by use of an object's attributes. What do I do if I'm not using any attributes?
I have a simple function that determines constants based on the argument, which should be a number. I raise an exception defined by
class outOfBoundsError(ValueError):
"""
Specified value is outside the domain.
"""
with a message telling them the number they gave me is too big. I would like to keep this message specific. But if the argument is a string (like 'charlie'), it still considers the argument to be greater than my specified number (and raises my exception). Should I just add a dummy line to the code like argument + 2 so that a TypeError is raised?
Note: I don't know anything about ABCs but I don't think they're available to me since the latest python version we have access to is 2.5 : (.
A common duck-typish Python solution to this problem is to (try to) convert what you got to what you need. For example, if you need an integer:
def getconstants(arg):
try:
arg = int(arg)
except:
raise TypeError("expected integer, or something that can be converted to one, but got " + repr(arg))
The int type constructor knows how to deal with many built-in types. Additionally, types that are convertible to an int type can implement the __int__() special method, which would allow them to be used here. If the user passed in a string, that would work fine too as long as it contained only digits.
Your idea of performing some operation that could only be performed on a numeric type (such as adding 2) is similar, and would work great in many cases, but would fail with strings that can be converted to the desired type, so I like the type conversion better.
You could probably get by without the try/except here, since int() will raise either TypeError or ValueError if it can't do the conversion, but I think TypeError is more appropriate since you are mainly interested in the object's type, so I chose to catch the exception and always raise TypeError.
My honest answer to
Most people seem to respond by saying
that type checking should never be
performed in python (< 2.6) due to
duck typing
is: nonsense.
Type-checking - were needed - is common practice.
Don't listen to all and everything and don't accept every statement unfiltered.
You can check to make sure you're expecting a type you support initially. i.e.
def foo(arg):
if not isinstance(arg, int):
raise TypeError
...
Nothing wrong with that if you're only supporting integers.
Introspection works for this...primitive types have class names too:
>>> i=2
>>> i.__class__.__name__
'int'
>>> s="two"
>>> s.__class__.__name__
'str'
>>>

Categories