I found out that Python is intending to support static typing, but it still in beta phase.
I tried the following code with python 3.4.3:
def somme(a: int, b: int) -> int:
return a + b
the syntax is supported but I did not get the expected result. if I type somme('1', '3') I get 13, while I should get TypeError exception saying int variable expected.
Has anyone an idea why it's not working ?
The function annotations there are annotations, nothing more. They're documentation about intended usage. Like it says in PEP 3107, the mechanism provides a single, standard way to specify function parameters and return values, replacing a variety of ad-hoc tools and libraries.
But as it goes on to say:
Function annotations are nothing more than a way of associating
arbitrary Python expressions with various parts of a function at
compile-time.
By itself, Python does not attach any particular meaning or
significance to annotations. Left to its own, Python simply makes
these expressions available as described in Accessing Function
Annotations below.
PEP 484 adds more conventions and tools for using these annotations to mark types, adopted in Python 3.5. But still there is no runtime checking in the language itself: it "includes support for off-line type checkers such as mypy".
That is, after using these annotations, you can run third-party type checkers to check if your code adheres to the annotated types. Building these annotations into the language should make it easier for various IDEs to provide this service.
In reality, you aren't supposed to get static typing here. The code you wrote is actually a form of easy documentation and "type hinting", as explained here. To quote that PEP:
While these annotations are available at runtime through the usual annotations attribute, no type checking happens at runtime . Instead, the proposal assumes the existence of a separate off-line type checker which users can run over their source code voluntarily. Essentially, such a type checker acts as a very powerful linter. (While it would of course be possible for individual users to employ a similar checker at run time for Design By Contract enforcement or JIT optimization, those tools are not yet as mature.)
So in other words, it was developed to support a third party library that will actually do the checking for you. I've never bothered with static typing in Python, although a friend of mine often uses MyPy to achieve it. This might be what you're looking for.
EDIT:
installation : MyPy can be installed using pip:
pip install mypy-lang
Usage : you can type-check your program as follow:
~$ mypy my_script.py
Related
I've created the class CustomErr (inherits from Python's Exception) long time ago and I'm using it on a few hundred places in code. It has a member method which is declared with def logErr(self).
After some time I found that this function should have mandatory keyword argument logger, so I've changed declaration of the function to def logErr(self, *, logger).
After this change I've expected a few hundred compiler errors, or at least warnings (so it will be easy for me to make needed changes), but it didn't happen - there is not a single error reported. I only get runtime exception when this function is called.
Did I expect too much from Python compiler and/or my working IDE (I'm using Eclipse with PyDev 8.1, Python 3.8.7)?
Do I need a Python linter ?
PyDev by default doesn't really provide this checking -- it does have code-analysis builtin, but it focuses on being very fast and on having very few false positives and it doesn't really delve in the type-checking territory.
In PyDev the recommended way to work with this is through an external linter or type-checker.
PyDev has integrations with Flake8, Mypy and PyLint (the related configuration in the preferences is under PyDev > Editor > Code Analysis > Flake8/Mypy/PyLint).
If you're using type hints in your code, enabling Mypy is probably a no-brainer and actually required to make use of such typing for code analysis in PyDev... out of the box PyDev will use that info for code-completion, but there aren't plans to add it for code analysis as I'm a bit worried about duplicating what's already available in Mypy anyways and I don't see much positives in having yet another type checker.
PyLint may also be interesting as it does get some interesting errors (but in my experience it usually also has a higher false positive ratio, so, it may need to be tweaked a bit more) -- if you don't use type-hints (and thus wouldn't really get much from Mypy), I think that using PyLint is interesting as it does try to cover more ground than PyDev.
As a note, if you're not using type-checking/MyPy, what you want is probably not achievable with 100% certainty by any tool because it's difficult to find out that something as parameter.logErr means the logErr in your CustomErr and not really some other logErr somewhere else (because discovering that the type of parameter is CustomErr is really hard).
p.s.: automatically running the type-checker/linter is done for each file as you save it, but you can also run it for the whole project by right-clicking a project/file and then selecting PyDev > Code Analysis.
I'm working with a large existing Python codebase and would like to start adding in type annotations so I can get some level of static checking. I'm imagining something like Erlang, Strongtalk, or Typed Scheme/Racket.
I've seen quick-and-dirty decorators that insert dynamic checks based on function parameter and return type annotations, but I'm looking for something that is more robust and that performs checks at compile-time.
What tools are available right now for this kind of thing? I'm familiar with compilers and type checking and am definitely willing to improve an incomplete tool if it has a good foundation.
(Note: I'm not interested in a discussion of the pros/cons of static typing.)
EDIT: An example:
def put(d, k, v):
d[k] = v
I'd like to be able to annotate the put function as having type put<K,V>(dict<K,V>, K, V) -> None.
UPDATE: The new PEP 484 (Sep 2014) defines a standard for static typing and type annotations in Python 3.5+. There's a type-checking tool called mypy that is compatible with PEP 484.
Edit 2016-11-11: Just use mypy. Type hints can be added gradually. In Python 3 source code, it verifies standard PEP 484 type hints. Types can still be expressed in Python 2 using special comments. Guido likes it.
This post was originally written a long time ago before mypy was a thing. I've preserved the post's original content below, even though it isn't quite accurate.
Original post:
You might want to check out some of the projects mentioned in this related StackOverflow post on static analysis for Python.
In summary:
pylint
PyFlakes
PyChecker
Since Python uses duck typing extensively, things that might be called "type errors" in other languages might end up being "object X doesn't support method Y" in Python.
Edit 2011-05-17:
I agree with delnan that static typing is not possible for Python [apparently wrong]. But since our skepticism doesn't seem to deter you, I can only give you more information on the subject. I present:
A discussion of type inference for Python. (Other links are from here.)
Guido van van Rossum's articles on adding optional static typing: part 1 and part 2.
RPython, a subset of Python that might stand a chance of being statically analyzed enough to do some form of type checking.
You may find mypy interesting. It has been proposed for inclusion in Python 3.5 by Guido.
Check out this post: PySonar: a Static Analyzer for Python. PySonar is a tool that infers types using abstract interpretation (partially executing) of code. It finds all possible execution paths of your program and finds all possible types of all variables.
There are basically three versions of PySonar:
Open-sourced Java (Jython indexer)
Closed-sourced Java (Hidden in Google)
Open-sourced Python (mini-pysonar)
None of them (except of closed source one) is fully implemented. But the basic idea is that you can use it as a basis for your work.
I don't know if this helps but for what it's worth, Jeremy Siek at U of Colorado did some work on gradual typing, and I found this doing a quick search.
http://www.wiki.jvmlangsummit.com/pdf/28_Siek_gradual.pdf
My guess (I might be wrong) is there isn't any promising open source tools you can find at the moment since his research looks relatively new.
Your best bet might be to contact the authors and ask if they can release their code to you.
There is the 'gradual' package for Python 3; see PIP or the Bitbucket Repo
Apparently this is an implementation by the group around Jeremy Siek who seems to be quite an authority in the field of gradual typing.
Some annotations are apparently necessary, here is an example:
from gradual import *
#typed
def calculate_total(a:int, b:int) -> int:
return a + b//100
As far as annotations go, this is not so bad. I have not used the package so I cannot speak to its quality, but the syntax (and the people behind it) certainly make it look promising.
I like prospector, backend of landscape.io. It combines output of existing analyzers, such as pylint, pyflakes, pep8, frosted..., into one report. Neat.
I had a similar need some time ago.
All the existing solutions that I've found had some problem or does not have a feature that I'd like to have, so I've made my own.
Here's how you use it:
from requiretype import require
#require(name=str, age=(int, float, long))
def greet_person(name, age):
print "Hello {0} ({1})".format(name, age)
>>> greet_person("John", 42)
Hello John (42)
>>> greet_person("John", "Doe")
# [...traceback...]
TypeError: Doe is not a valid type.
Valid types: <type 'int'>, <type 'float'>, <type 'long'>
>>> greet_person(42, 43)
# [...traceback...]
TypeError: 42 is not a <type 'str'> type
I hope this is useful for you.
For more details look at:
https://pypi.python.org/pypi/RequireType
https://github.com/ivanalejandro0/RequireType
P.S.: (quoting myself from github repo)
For most cases I'd recommend using tests instead of type checking since it's more natural to do that in Python. But, there are some cases where you want/need to specify a specific type to use and since python does not have type checks for parameters here's where this is useful.
I want to create a class that requires a specific method, with specifically typed arguments and return values.
I can inherit from an abstract class that requires the method to be implemented - but I do not have the ability to force specific argument values and return values ala a static language like Java (I could throw an error at runtime if I wanted to). What is the best way of approaching this with Python? Have looked into type hinting a bit but I don't think it solves this problem.
This seems fundamentally un-pythonic. There's no typing of function parameters in python, so there's no way to restrict the argument types to a function.
Type hinting is useful for documentation or code linters, but python doesn't use that information to enforce anything at runtime.
If you really want to ensure the validity of an interface (even beyond just argument types), the way to do that would be with functional unittests.
Unittesting and Test-Driven Development are so prevalent in the python community that type-hinting doesn't really add much when it comes to testing and finding bugs. And while it's a debatable point, there are many who believe that any benefit from type-hinting is immediately destroyed by making python code harder to read. There are some promising possibilities with type-hinting of being able to compile python out to C or Java, but they don't exist, yet.
Both epydoc and Sphinx document generators permit the coder to annotate what the types should be of any/all function parameter.
My question is: Is there a way (or module) that enforces these types (at run-time) when documented in the docstring. This wouldn't be strong-typing (compile-time checking), but (more likely) might be called firm-typing (run-time checking). Maybe raising a "ValueError", or even better still... raising a "SemanticError"
Ideally there would already be something (like a module) similar to the "import antigravity" module as per xkcd, and this "firm_type_check" module would already exist somewhere handy for download.
FYI: The docstring for epydoc and sphinz are as follows:
epydoc:
Functions and Methods parameters:
#param p: ... # A description of the parameter p for a function or method.
#type p: ... # The expected type for the parameter p.
#return: ... # The return value for a function or method.
#rtype: ... # The type of the return value for a function or method.
#keyword p: ... # A description of the keyword parameter p.
#raise e: ... # A description of the circumstances under which a function or method
raises exception e.
Sphinx: Inside Python object description directives, reST field lists with these fields are recognized and formatted nicely:
param, parameter, arg, argument, key, keyword: Description of a parameter.
type: Type of a parameter.
raises, raise, except, exception: That (and when) a specific exception is raised.
var, ivar, cvar: Description of a variable.
returns, return: Description of the return value.
rtype: Return type.
The closest I could find was a mention by Guido in mail.python.org and created by Jukka Lehtosalo at Mypy Examples. CMIIW: mypy cannot be imported as a py3 module.
Similar stackoverflow questions that do not use the docstring per se:
Pythonic Way To Check for A Parameter Type
What's the canonical way to check for type in python?
To my knowledge, nothing of the sort exists, for a few important reasons:
First, docstrings are documentation, just like comments. And just like comments, people will expect them to have no effect on the way your program works. Making your program's behavior depend on its documentation is a major antipattern, and a horrible idea all around.
Second, docstrings aren't guaranteed to be preserved. If you run python with -OO, for example, all docstrings are removed. What then?
Finally, Python 3 introduced optional function annotations, which would serve that purpose much better: http://legacy.python.org/dev/peps/pep-3107/ . Python currently does nothing with them (they're documentation), but if I were to write such a module, I'd use those, not docstrings.
My honest opinion is this: if you're gonna go through the (considerable) trouble of writing a (necessarily half-baked) static type system for Python, all the time it will take you would be put to better use by learning another programming language that supports static typing in a less insane way:
Clojure (http://clojure.org/) is incredibly dynamic and powerful (due to its nature as a Lisp) and supports optional static typing through core.typed (https://github.com/clojure/core.typed). It is geared towards concurrency and networking (it has STM and persistent data structures <3 ), has a great community, and is one of the most elegantly-designed languages I've seen. That said, it runs on the JVM, which is both a good and a bad thing.
Golang (http://golang.org/) feels sort-of Pythonic (or at least, it's attracting a lot of refugees from Python), is statically typed and compiles to native code.
Rust (http://www.rust-lang.org/) is lower-level than that, but it has one of the best type systems I've seen (type inference, pattern matching, traits, generics, zero-sized types...) and enforces memory and resource safety at compile time. It is being developed by Mozilla as a language to write their next browser (Servo) in, so performance and safety are its main goals. You can think of it as a modern take on C++. It compiles to native code, but hasn't hit 1.0 yet and as such, the language itself is still subject to change. Which is why I wouldn't recommend writing production code in it yet.
I'm working with a large existing Python codebase and would like to start adding in type annotations so I can get some level of static checking. I'm imagining something like Erlang, Strongtalk, or Typed Scheme/Racket.
I've seen quick-and-dirty decorators that insert dynamic checks based on function parameter and return type annotations, but I'm looking for something that is more robust and that performs checks at compile-time.
What tools are available right now for this kind of thing? I'm familiar with compilers and type checking and am definitely willing to improve an incomplete tool if it has a good foundation.
(Note: I'm not interested in a discussion of the pros/cons of static typing.)
EDIT: An example:
def put(d, k, v):
d[k] = v
I'd like to be able to annotate the put function as having type put<K,V>(dict<K,V>, K, V) -> None.
UPDATE: The new PEP 484 (Sep 2014) defines a standard for static typing and type annotations in Python 3.5+. There's a type-checking tool called mypy that is compatible with PEP 484.
Edit 2016-11-11: Just use mypy. Type hints can be added gradually. In Python 3 source code, it verifies standard PEP 484 type hints. Types can still be expressed in Python 2 using special comments. Guido likes it.
This post was originally written a long time ago before mypy was a thing. I've preserved the post's original content below, even though it isn't quite accurate.
Original post:
You might want to check out some of the projects mentioned in this related StackOverflow post on static analysis for Python.
In summary:
pylint
PyFlakes
PyChecker
Since Python uses duck typing extensively, things that might be called "type errors" in other languages might end up being "object X doesn't support method Y" in Python.
Edit 2011-05-17:
I agree with delnan that static typing is not possible for Python [apparently wrong]. But since our skepticism doesn't seem to deter you, I can only give you more information on the subject. I present:
A discussion of type inference for Python. (Other links are from here.)
Guido van van Rossum's articles on adding optional static typing: part 1 and part 2.
RPython, a subset of Python that might stand a chance of being statically analyzed enough to do some form of type checking.
You may find mypy interesting. It has been proposed for inclusion in Python 3.5 by Guido.
Check out this post: PySonar: a Static Analyzer for Python. PySonar is a tool that infers types using abstract interpretation (partially executing) of code. It finds all possible execution paths of your program and finds all possible types of all variables.
There are basically three versions of PySonar:
Open-sourced Java (Jython indexer)
Closed-sourced Java (Hidden in Google)
Open-sourced Python (mini-pysonar)
None of them (except of closed source one) is fully implemented. But the basic idea is that you can use it as a basis for your work.
I don't know if this helps but for what it's worth, Jeremy Siek at U of Colorado did some work on gradual typing, and I found this doing a quick search.
http://www.wiki.jvmlangsummit.com/pdf/28_Siek_gradual.pdf
My guess (I might be wrong) is there isn't any promising open source tools you can find at the moment since his research looks relatively new.
Your best bet might be to contact the authors and ask if they can release their code to you.
There is the 'gradual' package for Python 3; see PIP or the Bitbucket Repo
Apparently this is an implementation by the group around Jeremy Siek who seems to be quite an authority in the field of gradual typing.
Some annotations are apparently necessary, here is an example:
from gradual import *
#typed
def calculate_total(a:int, b:int) -> int:
return a + b//100
As far as annotations go, this is not so bad. I have not used the package so I cannot speak to its quality, but the syntax (and the people behind it) certainly make it look promising.
I like prospector, backend of landscape.io. It combines output of existing analyzers, such as pylint, pyflakes, pep8, frosted..., into one report. Neat.
I had a similar need some time ago.
All the existing solutions that I've found had some problem or does not have a feature that I'd like to have, so I've made my own.
Here's how you use it:
from requiretype import require
#require(name=str, age=(int, float, long))
def greet_person(name, age):
print "Hello {0} ({1})".format(name, age)
>>> greet_person("John", 42)
Hello John (42)
>>> greet_person("John", "Doe")
# [...traceback...]
TypeError: Doe is not a valid type.
Valid types: <type 'int'>, <type 'float'>, <type 'long'>
>>> greet_person(42, 43)
# [...traceback...]
TypeError: 42 is not a <type 'str'> type
I hope this is useful for you.
For more details look at:
https://pypi.python.org/pypi/RequireType
https://github.com/ivanalejandro0/RequireType
P.S.: (quoting myself from github repo)
For most cases I'd recommend using tests instead of type checking since it's more natural to do that in Python. But, there are some cases where you want/need to specify a specific type to use and since python does not have type checks for parameters here's where this is useful.