How do you figure out what methods are available within a class?
Example :
I am trying to learn about urllib.request. I found urlopen() in the docs :
http://docs.python.org/3.0/library/urllib.request.html
So I have :
response = urllib.request.urlopen(url)
What does this return? I know it will probably be an object, but what are this objects methods? Eventually I get to the examples (stuck at the bottom of the page far from where the idea was introduced) and discover read(), but I had to look outside the docs to find .decode(), which finally lets you do what the whole purpose of this library is.
Im having this type of problem with much of the docs.python.org pages. Is there a better documentation somewhere else, or am I going about learning this all the wrong way?
The real problem here is that you're using a very old version of the documentation. I have no idea how you found it, but it should be pretty clear from the URL http://docs.python.org/3.0/library/urllib.request.html, and the header on the top of that page ("Python v3.0.1 documentation"), and so on that you're not looking at the documentation for your version.*
If you were looking at the 3.3 documentation, you would have seen this:
For http and https urls, this function returns a http.client.HTTPResponse object which has the following HTTPResponse Objects methods.
The first link takes you to the reference for the exact class, the second to the reference for the abstract type. Which gives you exactly what you were looking for—the read method, and everything else.
In older versions of Python, the term "file-like object" was thrown around loosely. This was always a vague term (sometimes it means "an iterable of lines", "has a read() and/or write() as appropriate", "has a fileno()", …), and became much more so in Python 3 (because you have to distinguish a binary file from a text file). So, over the years, they've phased this out in favor of more specific documentation. But if you're looking at the very earliest Python 3 documentation, you're not getting the benefit.
* Note that if you visit any recent-ish version of the docs, like the 3.3 linked above, there's a pulldown menu in the header that lets you switch to a different version if you've found the wrong one. And, the default will always be 2.7 or the latest stable 3.x, and those are also the most common search results, and the easiest things to link to, so you will usually be on one of those unless you're really trying to make things hard for yourself. If you do find yourself on ancient docs like 3.0 or 2.4 or something, you can often just edit the URL to 3.3 or 2.7, or just 3 or 2; if not, a quick search should work.
The key phrase in the documentation of urlopen is:
This function returns a file-like object with two additional methods from the urllib.response module
The fact that this is a file-like object indicates that all the normal file operation functions (read, close) apply to the returned object also.
The document you linked to says this:
This function returns a file-like object with two additional methods
from the urllib.response module
geturl() — return the URL of the resource retrieved, commonly used to determine if a redirect was followed
info() — return the meta-information of the page, such as headers, in the form of an http.client.HTTPMessage instance (see Quick Reference to HTTP Headers)
Emphasis mine. A "file-like object" is defined as follows:
file object
An object exposing a file-oriented API (with methods such as read() or write()) to an underlying resource. Depending on the way it was created, a file object can mediate access to a real on-disk file or to another type of storage or communication device (for example standard input/output, in-memory buffers, sockets, pipes, etc.). File objects are also called file-like objects or streams.
As #abarnert points out, you're probably just better off using the new docs, which explicitly link to the methods of the returned object.
If you aren't using Ipython already you should be. Then you can just type urllib. and press tab to see all the available options and do the same for whatever it returns. If you type a question mark after the function it will bring up the documentation, two question marks often brings up the source.
For what its worth though I have always found the online documentation to be very helpful. It does say that urllib.request.urlopen is going to return a file like object early in the explanation
Related
In the below code I'm attempting to use the, 'RSA_V1_5_SHA256', algorithm to create a signature for connecting to an API. However, despite the documentation having it written this way, it doesn't seem to recognize algorithm as a valid argument.
Here's the documentation linked as well,
https://pyauth.github.io/requests-http-signature/#asymmetric-key-algorithms
auth = HTTPSignatureAuth(
algorithm=algorithms.RSA_V1_5_SHA256, key=preshared_secret, key_id=signature_key_id
)
This is the specific error that's being returned,
TypeError: __init__() got an unexpected keyword argument 'algorithm'.
I assume perhaps the documentation may be outdated, as I need to pass a passphrase that was used when I created the public and private keys, to properly authenticate my requests to said API... and passphrase/password isn't an acceptable argument in the other available algorithms based on what I've seen in the source code for this library.
Any help is appreciated!
You're totally right and the docs aren't updated, by checking the source code we can see that the __init__ method for the HTTPSignatureAuth class has no algorithm argument. Instead we find a signature_algorithm parameter (also we can see that this is the parameter that we need - source) so I would go with that:
auth = HTTPSignatureAuth(
signature_algorithm=algorithms.RSA_V1_5_SHA256, key=preshared_secret, key_id=signature_key_id
)
The changes were made in version v0.3.0 so if you use a previous version you will be able to use algorithm as the argument. But I would go with the first approach. (Note that version v0.3.0 was released 23 days ago so it's pretty recent and they may update the docs soon!
I have installed package from https://pypi.org/project/trading-calendars/
When i run get_calendar('XNYS') it produces and object file. How can find out what method does this object have for viewing data? The reason i am asking is because in this case in the documentation shows no examples on its usage.
You must read the documentation. If there is none, the module is ... well ... crap. In such case do print(obj.__dict__) to see all the properties of the object, and if anything is of use to you. Worst come to worst, read the source code of the module on Github.
Here is the documentation as mentioned in trading_calendars#107 (found by going to Issues and searching "documentation", for future reference).
I would like, given a python module, to monkey patch all functions, classes and attributes it defines. Simply put, I would like to log every interaction a script I do not directly control has with a module I do not directly control. I'm looking for an elegant solution that will not require prior knowledge of either the module or the code using it.
I found several high-level tools that help wrapping, decorating, patching etc... and i've went over the code of some of them, but I cannot find an elegant solution to create a proxy of any given module and automatically proxy it, as seamlessly as possible, except for appending logic to every interaction (record input arguments and return value, for example).
in case someone else is looking for a more complete proxy implementation
Although there are several python proxy solutions similar to those OP is looking for, I could not find a solution that will also proxy classes and arbitrary class objects, as well as automatically proxy functions return values and arguments. Which is what I needed.
I've got some code written for that purpose as part of a full proxying/logging python execution and I may make it into a separate library in the future. If anyone's interested you can find the core of the code in a pull request. Drop me a line if you'd like this as a standalone library.
My code will automatically return wrapper/proxy objects for any proxied object's attributes, functions and classes. The purpose is to log and replay some of the code, so i've got the equivalent "replay" code and some logic to store all proxied objects to a json file.
I'm learning how to use Qt with PyQt, and I have a QTabelView with a StandardItemModel I've populated the model successfully and hooked up the itemChanged signal to a slot. I'd l'd like to mess around with whatever object is returned in IPython, so currently I have the line:
def itemChangedSlot(epw, item):
new_data = item.data()
print new_data
print item
which prints
<PyQt4.QtGui.QStandardItem object at 0x07C5F930>
<PyQt4.QtCore.QVariant object at 0x07D331F0>
In the IPython session is it possible to get the object using this memory address? I'm not seeing anything on Google, maybe I don't have my terminology right?
You need to hold a reference to an object (i.e. assign it to a variable or store it in a list).
There is no language support for going from an object address directly to an object (i.e. pointer dereferencing).
You're almost certainly asking the wrong question, and Raymond Hettinger's answer is almost certainly what you really want.
Something like this might be useful trying to dig into the internals of the CPython interpreter for learning purposes or auditing it for security holes or something… But even then, you're probably better off embedding the Python interpreter into a program and writing functions that expose whatever you want into the Python interpreter, or at least writing a C extension module that lets you manipulate CPython objects.
But, on the off chance that you really do need to do this…
First, there is no reliable way to even get the address from the repr. Most objects with a useful eval-able representation will give you that instead. For example, the repr of ('1', 1) is "('1', 1)", not <tuple at 0x10ed51908>. Also, even for objects that have no useful representation, returning <TYPE at ADDR> is just an unstated convention that many types follow (and a default for user-defined classes), not something you can rely on.
However, since you presumably only care about CPython, you can rely on id:
CPython implementation detail: This is the address of the object in memory.
(Of course if you have the object to call id (or repr) on, you don't need to dereference it via pointer, and if you don't have the object, it's probably been garbage collected so there's nothing to dereference, but maybe you still have it and just can't remember where you put it…)
Next, what do you do with this address? Well, Python doesn't expose any functions to do the opposite of id. But the Python C API is well documented—and, if your Python is built around a shared library, that C API can be accessed via ctypes, just by loading it up. In fact, ctypes provides a special variable that automatically loads the right shared library to call the C API on, ctypes.pythonapi.
In very old versions of ctypes, you may have to find and load it explicitly, like pydll = ctypes.cdll.LoadLibrary('/usr/lib/libpython2.5.so') (This is for linux with Python 2.5 installed into /usr/lib; obviously if any of those details differ, the exact command line will differ.)
Of course it's much easier to crash the Python interpreter doing this than to do anything useful, but it's not impossible to do anything useful, and you may have fun experimenting with it.
I think the answer is here. This way, I accessed an object using its address:
import ctypes
myObj = ctypes.cast(0x249c8c6cac0, ctypes.py_object).value
I just spent two hours debugging some django code because the API was not clear, in my opinion. Here is what the Django documentation says about the "render" method: https://docs.djangoproject.com/en/1.3/topics/http/shortcuts/#render
render(request, template[, dictionary][, context_instance][,
content_type][, status][, current_app])
Combines a given template with a given context dictionary and returns
an HttpResponse object with that rendered text.
render() is the same as a call to render_to_response() with a
context_instance argument that forces the use of a RequestContext.
Now, seeing that the second argument that the render method expects is a template, I passed it a Template object. It turns out that what the method really wants is the name of the template given as a String. How am I supposed to know this? And what is "status" -- an integer? a string? a Status object? a list? Where can I get clear documentation to refer to?
Also, I would love to find an IDE that can detect these kinds of things while I'm programming -- when I worked with Netbeans and Java this sort of error would have been caught by the editor with a squiggly red underline before I'd even started typing the next line of code.
I can't at all see how this could be any better explained. As miku points out in the comment, the documentation clearly says "the full name of a template to use". Plus, the example given directly underneath shows the use of a string.
Python is not Java, nor any statically typed language. The documentation doesn't say it wants a string, because it doesn't: it wants something string-like that contains the name.
And don't forget, you've got immediate access to the code. Don't understand the function? Well, look at https://code.djangoproject.com/browser/django/trunk/django/shortcuts/__init__.py and there it is.
Even better, in Python you have an option that you simply don't have in Java: look in the shell. Do from django.shortcuts import render, then do help(render) to see what it says, and if that doesn't help try calling it with a few different arguments right there to see what it does.
If you insist on programming Python like it is Java, you will find it difficult. If you program Python like it's Python, you will find it much simpler.
While I agree with Daniel Roseman's answer, I do sympathise with your bewilderment, and I do think the interface and its parameters could have been better explained.
Django's interface is very committed to the idea of dynamic typing, or "duck typing" — as in, if it walks like a duck, if it swims like a duck, then call it a duck. Django's interface doesn't require you to pass a string type, it requires you to pass an entity which behaves like a string.
Look carefully, in the documentation to which you linked, at the specific wording for each parameter to render(). request is "the request object". template is "the full name... or sequence of names." dictionary is "a dictionary of values...". status is "the status code for the response. Defaults to 200" [an integer].
I've added emphasis to words which hint at which dynamic type the interface expects or returns. object means an object type, with a set of methods. name implies string. The example of 200 implies that status code is an integer.
I think the Django documentation would be clearer if it were more explicit about its vocabulary of the dynamic types it expects or returns. It could say something like, "request is an object implementing the Request Object Interface", and then define exactly what methods an object needs to implement to qualify as that interface. The documentation could also be explicit instead of implicit in cases like status, where it could say "integer" literally.
But those caveats aside, Django's interface is designed to be "pythonic", to embrace duck typing. Daniel is right, "if you insist on programming Python like it is Java, you will find it difficult." This might be a good time to read over Django's Design Philosophies and Python's PEP 20 -- The Zen of Python. Note that neither mentions "compile-time syntax checking thanks to strong typing" as a goal. Toto, I don't think we're in Java anymore.