I have a library module-wrapper that recursively wraps objects. I want to determine if an object has a function-like type. I can check almost all function-like object using:
inspect.isbuiltin(object=obj) or
inspect.isfunction(object=obj) or
inspect.ismethod(object=obj) or
inspect.ismethoddescriptor(object=obj)
The problem is that some bound methods are not detected with this code, for example:
s = "Hello, world!"
type(s.__add__)
# method-wrapper
I guess I cannot check objects for being method-wrapper using inspect module. But how do I import this type? I didn't find it.
Now I have an ugly hack in my code:
MethodWrapper = type(''.__add__)
isinstance(obj, MethodWrapper)
UPD0:
I don't want to use callable because it detects classes and objects, that implement __call__, but I want those classes and objects to be handled separately.
The types module provides names for many “implementation” types, including several for functions implemented in C. The specific example of type("".__str__), MethodWrapperType, was just added in CPython 3.7.
Because this area is subtle and the number of types is large, callable may really be the best choice. You can easily check for type objects first to “exclude” classes, and various heuristics (e.g., trying to call vars on an object or examining type(x).__module__) can be used to recognize typical “callable objects” (not that those are fundamentally different from the built-in function types).
I have a function that accepts a class that derives from NamedTuple and converts it into a schema. However when I run MyPy on the following code it fails with Argument 1 to "to_schema" has incompatible type "Type[Foo]"; expected "Type[NamedTuple]"
from typing import NamedTuple, Type
def to_schema(named_tuple: Type[NamedTuple]):
pass
class Foo(NamedTuple):
pass
to_schema(Foo)
Is there a way to properly type the code so that it typechecks with MyPy?
Edit:
Python documentation states that Type[Foo] accepts any subclasses of Foo (https://docs.python.org/3/library/typing.html#typing.Type). I have multiple subclasses of NamedTuple, for entities in our data model, so I'm looking for a way to annotate the function in a way that would typecheck.
The root issue with your code is that NamedTuple is not an actual type -- it's actually just a special sort of "type constructor" that synthesizes an entirely new class and type. E.g. if you try printing out the value of Foo.__mro__, you'll see (<class '__main__.Foo'>, <class 'tuple'>, <class 'object'>) -- NamedTuple is not present there at all.
That means that NamedTuple isn't actually a valid type to use at all -- in that regard, it's actually a little surprising to me that mypy just silently lets you construct Type[NamedTuple] to begin with.
To work around this, you have several potential approaches:
Rather then using Type[NamedTuple], use either Type[tuple] or Type[Tuple[Any]].
Your Foo genuinely is a subtype of a tuple, after all.
If you need methods or fields that are specifically present only in namedtuples, use a custom protocol. For example, if you particularly need the _asdict method in namedtuples, you could do:
from typing_extensions import Protocol
class NamedTupleProto(Protocol):
def _asdict(self) -> Dict[str, Any]: ...
def to_schema(x: Type[NamedTupleProto]) -> None: pass
class Foo(NamedTuple):
pass
to_schema(Foo)
Note that you will need to install the typing_extensions third party library to use this, though there are plans to formalize Protocols and add it to Python at some point. (I forget if the plan was Python 3.7 or 3.8).
Add a type ignore or a cast on the call to to_schema to silence mypy. This isn't the greatest solution, but is also the quickest.
For related discussion, see this issue. Basically, there's consensus on the mypy team that somebody ought to do something about this NamedTuple thing, whether it's by adding an error message or by adding an officially sanctioned protocol, but I think people are too busy with other tasks/bugs to push this forward. (So if you're bored and looking for something to do...)
I'm trying to understand why should I use types annotation in python. For exemple I can write function like:
def some_function(a: int, b: int) -> int:
return a + b
When I use it with int all gone good:
some_function(1, 2) # return 3, type int
But when I run for exemple
some_function(1, 2.0) # return 3.0, type float
I have result without any notes that types are wrong. So what is the reason to use types annotation?
Type hints are there for other tools to check your code, they are not enforced at runtime. The goal is enable static analysis tools to detect invalid argument use.
Use an IDE like PyCharm, or the commandline code checker mypy to be told that 2.0 is not a valid argument type.
From the Type Hinting PEP (484):
This PEP aims to provide a standard syntax for type annotations, opening up Python code to easier static analysis and refactoring, potential runtime type checking, and (perhaps, in some contexts) code generation utilizing type information.
Emphasis mine. Runtime type checking is left to third-party tools. Note that such runtime checks would come with a performance downside, your code will likely run slower if you were to check for types on every call.
As one can read in the PEP 484 that introduces type hints:
(...)
This PEP aims to provide a standard syntax for type annotations,
opening up Python code to easier static analysis and refactoring,
potential runtime type checking, and (perhaps, in some contexts) code
generation utilizing type information.
Of these goals, static analysis is the most important. This includes
support for off-line type checkers such as mypy, as well as providing
a standard notation that can be used by IDEs for code completion and
refactoring.
IDE's (static analysis)
So the main use is in static analysis: your IDE can detect that something is wrong when you call a function and can provide a list of functions you can call on the result of function.
For instance if you write:
some_function(1,2).
your IDE can provide a list with real as a possible option so you can easily write:
some_function(1,2).real
and if you write:
some_function('foo',2).bar
It will hint that 'foo' is not an acceptable parameter nor is .bar a good call on that object.
Dynamic inspection
You can also use it for dynamic inspection with inspect.getfulargspec like:
>>> import inspect
>>> inspect.getfullargspec(some_function).annotations
{'return': <class 'int'>, 'a': <class 'int'>, 'b': <class 'int'>}
Now we know that some_function returns an int and can be feeded two ints. This can be used for arbitrary tests (which are popular in Haskell): you simply feed the some_function random integers and looks that it always returns an int (and does not raises an exception for instance).
I've seen somewhere that there was a way to change some object functions in python
def decorable(cls):
cls.__lshift__ = lambda objet, fonction: fonction(objet)
return cls
I wondered if you could do things like in ruby, with the :
number.times
Can we actually change some predefined classes by applying the function above to the class int for example? If so, any ideas how I could manage to do it? And could you link me the doc of python showing every function (like lshift) that can be changed?
Ordinarily not -
as a rule, Python types defined in native code -in CPython can't be monkey patched to have new methods. Although there are means to do that with direct memory access and changing the C object structures, using CPython - that is not considered "clever", "beautiful", much less usable. (check https://github.com/clarete/forbiddenfruit)
That said, for class hierarchies you define on your own packages, that pretty much works - any magic "dunder" method that is set changes the behavior for all objects of that class, in all the process.
So, you can't do that to Python's "int" - but you can have a
class MyInt(int):
pass
a = MyInt(10)
MyInt.__rshift__ = lambda self, other: MyInt(str(self) + str(other))
print(a >> 20)
Will result in 1020 being printed.
The Python document thta tells about all the magic methods taht are used by the language is the Data Model:
https://docs.python.org/3/reference/datamodel.html
When you call the object.__repr__() method in Python you get something like this back:
<__main__.Test object at 0x2aba1c0cf890>
Is there any way to get a hold of the memory address if you overload __repr__(), other then calling super(Class, obj).__repr__() and regexing it out?
The Python manual has this to say about id():
Return the "identity'' of an object.
This is an integer (or long integer)
which is guaranteed to be unique and
constant for this object during its
lifetime. Two objects with
non-overlapping lifetimes may have the
same id() value. (Implementation note:
this is the address of the object.)
So in CPython, this will be the address of the object. No such guarantee for any other Python interpreter, though.
Note that if you're writing a C extension, you have full access to the internals of the Python interpreter, including access to the addresses of objects directly.
You could reimplement the default repr this way:
def __repr__(self):
return '<%s.%s object at %s>' % (
self.__class__.__module__,
self.__class__.__name__,
hex(id(self))
)
Just use
id(object)
There are a few issues here that aren't covered by any of the other answers.
First, id only returns:
the “identity” of an object. This is an integer (or long integer) which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same id() value.
In CPython, this happens to be the pointer to the PyObject that represents the object in the interpreter, which is the same thing that object.__repr__ displays. But this is just an implementation detail of CPython, not something that's true of Python in general. Jython doesn't deal in pointers, it deals in Java references (which the JVM of course probably represents as pointers, but you can't see those—and wouldn't want to, because the GC is allowed to move them around). PyPy lets different types have different kinds of id, but the most general is just an index into a table of objects you've called id on, which is obviously not going to be a pointer. I'm not sure about IronPython, but I'd suspect it's more like Jython than like CPython in this regard. So, in most Python implementations, there's no way to get whatever showed up in that repr, and no use if you did.
But what if you only care about CPython? That's a pretty common case, after all.
Well, first, you may notice that id is an integer;* if you want that 0x2aba1c0cf890 string instead of the number 46978822895760, you're going to have to format it yourself. Under the covers, I believe object.__repr__ is ultimately using printf's %p format, which you don't have from Python… but you can always do this:
format(id(spam), '#010x' if sys.maxsize.bit_length() <= 32 else '#18x')
* In 3.x, it's an int. In 2.x, it's an int if that's big enough to hold a pointer—which is may not be because of signed number issues on some platforms—and a long otherwise.
Is there anything you can do with these pointers besides print them out? Sure (again, assuming you only care about CPython).
All of the C API functions take a pointer to a PyObject or a related type. For those related types, you can just call PyFoo_Check to make sure it really is a Foo object, then cast with (PyFoo *)p. So, if you're writing a C extension, the id is exactly what you need.
What if you're writing pure Python code? You can call the exact same functions with pythonapi from ctypes.
Finally, a few of the other answers have brought up ctypes.addressof. That isn't relevant here. This only works for ctypes objects like c_int32 (and maybe a few memory-buffer-like objects, like those provided by numpy). And, even there, it isn't giving you the address of the c_int32 value, it's giving you the address of the C-level int32 that the c_int32 wraps up.
That being said, more often than not, if you really think you need the address of something, you didn't want a native Python object in the first place, you wanted a ctypes object.
Just in response to Torsten, I wasn't able to call addressof() on a regular python object. Furthermore, id(a) != addressof(a). This is in CPython, don't know about anything else.
>>> from ctypes import c_int, addressof
>>> a = 69
>>> addressof(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: invalid type
>>> b = c_int(69)
>>> addressof(b)
4300673472
>>> id(b)
4300673392
You can get something suitable for that purpose with:
id(self)
With ctypes, you can achieve the same thing with
>>> import ctypes
>>> a = (1,2,3)
>>> ctypes.addressof(a)
3077760748L
Documentation:
addressof(C instance) -> integer
Return the address of the C instance internal buffer
Note that in CPython, currently id(a) == ctypes.addressof(a), but ctypes.addressof should return the real address for each Python implementation, if
ctypes is supported
memory pointers are a valid notion.
Edit: added information about interpreter-independence of ctypes
I know this is an old question but if you're still programming, in python 3 these days... I have actually found that if it is a string, then there is a really easy way to do this:
>>> spam.upper
<built-in method upper of str object at 0x1042e4830>
>>> spam.upper()
'YO I NEED HELP!'
>>> id(spam)
4365109296
string conversion does not affect location in memory either:
>>> spam = {437 : 'passphrase'}
>>> object.__repr__(spam)
'<dict object at 0x1043313f0>'
>>> str(spam)
"{437: 'passphrase'}"
>>> object.__repr__(spam)
'<dict object at 0x1043313f0>'
You can get the memory address/location of any object by using the 'partition' method of the built-in 'str' type.
Here is an example of using it to get the memory address of an object:
Python 3.8.3 (default, May 27 2020, 02:08:17)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> object.__repr__(1)
'<int object at 0x7ca70923f0>'
>>> hex(int(object.__repr__(1).partition('object at ')[2].strip('>'), 16))
0x7ca70923f0
>>>
Here, I am using the built-in 'object' class' '__repr__' method with an object/item such as 1 as an argument to return the string and then I am partitioning that string which will return a tuple of the string before the string that I provided, the string that I provided and then the string after the string that I provided, and as the memory location is positioned after 'object at', I can get the memory address as it has partitioned it from that part.
And then as the memory address was returned as the third item in the returned tuple, I can access it with index 2 from the tuple. But then, it has a right angled bracket as a suffix in the string that I obtained, so I use the 'strip' function to remove it, which will return it without the angled bracket. I then transformed the resulted string into an integer with base 16 and then turn it into a hex number.
While it's true that id(object) gets the object's address in the default CPython implementation, this is generally useless... you can't do anything with the address from pure Python code.
The only time you would actually be able to use the address is from a C extension library... in which case it is trivial to get the object's address since Python objects are always passed around as C pointers.
If the __repr__ is overloaded, you may consider __str__ to see the memory address of the variable.
Here is the details of __repr__ versus __str__ by Moshe Zadka in StackOverflow.
There is a way to recovery the value from the 'id' command, here it the TL;DR.
ctypes.cast(memory_address,ctypes.py_object).value
source