We Keep Coding

Sphinx: Autodocs for classes inheriting from base classes implemented in C

I am running into an error/warning when inheriting from collections.deque. Sphinx complains:
utils.py:docstring of utils.Test.count:1: WARNING: py:class reference target not found: integer -- return number of occurrences of value
For this example, utils.py only contains
from collections import deque
class Test(deque):
pass
My best guess is that this might be caused by deque being implemented in C, since integer is not a valid Python type. Looking around a bit, I found this line in the CPython source of collections, which could be exactly that part, should my hypothesis be correct. But I have no clue about CPython, the inner workings of Sphinx autodoc and the boundary between those worlds.
Is there a good solution to correcting this, whether in the annotation of CPython or the autodoc plugin?

It seems I was more or less on the money. integer is used elsewhere and seems to work, but more importantly Sphinx expects docstrings to follow the example outlined in PEP 7, which suggests signatures to be placed on the first line (followed by a line break before any other documentation). So it looks for a class named by the whole string between -> and \n, in my case being integer -- return number of occurrences of value.
The workaround is to put all those strings into nitpick_ignore:
# Wrong docstrings in the CPython source of collections.queue
("py:class", "-- reverse *IN PLACE*"),
("py:class", "integer -- return first index of value."),
("py:class", "integer -- return number of occurrences of value"),
("py:class", "-- insert object before index"),
("py:class", "-- remove first occurrence of value."),
("py:class", "-- size of D in memory, in bytes"),
("py:class", "-- return a reverse iterator over the deque"),
A proper fix would be to change the documentation in the CPython code accordingly.
I was about to do that, when I found out my coworker beat me to it by 2 weeks. But on the issue he opened, a core developer argues that this scheme is nothing but an example and that the documentation is only required to be human readable as opposed to specifying the exact types. So it doesn't look like there is going to be a fix anytime soon.

When (and why) has been `__name__` magic variable removed from `ConfigParser` interpolation?

In Python 2.7 ConfigParser.ConfigParser interpolates pattern %(__name__)s as name of the section.
In Python 3.6 (as well as 3.3) configparser.ConfigParser fails at the same interpolation with InterpolationMissingOptionError exception.
When has the behaviour of the interpolation changed? What is the rationale of the decision (as it is harming backward-compatibility)?
In contrast to how to interpolate the section-name with configparser I am not asking how can I get that interpolation in Python 3.x.

In python 3.2 the old ConfigParser class (which implemented __name__ interpolation) was removed and replaced by what had previously been the SafeConfigParser class. From the What's New file:
The configparser module was modified to improve usability and predictability of the default parser and its supported INI syntax. The old ConfigParser class was removed in favor of SafeConfigParser which has in turn been renamed to ConfigParser.
The detailed motivation seems to be described in this bug report:
I want to sum up all strange things about the behaviour of __name__,
a special key present in every section of a parser instance.
There is a special __name__ key in every section.
Except for the DEFAULTSECT.
__name__ key is set for every section read from a file.
and not when adding by add_section().
if __name__ does exist, it's not visible in parser.options('section')
but it is visible here: parser.has_option('section', '__name__') == True
and can be obtained by parser.get('section', '__name__')
and can be changed by parser.set('section', '__name__', 'ANY VALUE')
and can be removed by parser.remove_option('section', '__name__')
even if the value is changed by parser.set(), it won't be written back to a file with parser.write()
All this looks like a feature that was not particularly complete and
well defined when it was first created. Or possibly, it became rotten
with time and now nobody is using it anyway. That way or the other, I
couldn't come up with a valid use case for __name__ with the current
implementation. It doesn't serve any internal purpose and the only
way you can actually get it is to parser.get('section', '__name__')
which returns 'section' anyway. About as useless as it gets. Of
course, one can go and take the internal parser._sections data
structure of the parser but that's evil.
I want simply remove all mentions of a special __name__ key in
configparser.py. Backwards compatibility is not a concern here because
in this case we have a concept that is so broken that you can't
actually use it.

How to document a module constant in Python?

I have a module, errors.py in which several global constants are defined (note: I understand that Python doesn't have constants, but I've defined them by convention using UPPERCASE).
"""Indicates some unknown error."""
API_ERROR = 1
"""Indicates that the request was bad in some way."""
BAD_REQUEST = 2
"""Indicates that the request is missing required parameters."""
MISSING_PARAMS = 3
Using reStructuredText how can I document these constants? As you can see I've listed a docstring above them, but I haven't found any documentation that indicates to do that, I've just done it as a guess.

Unfortunately, variables (and constants) do not have docstrings. After all, the variable is just a name for an integer, and you wouldn't want to attach a docstring to the number 1 the way you would to a function or class object.
If you look at almost any module in the stdlib, like pickle, you will see that the only documentation they use is comments. And yes, that means that help(pickle) only shows this:
DATA
APPEND = b'a'
APPENDS = b'e'
…
… completely ignoring the comments. If you want your docs to show up in the built-in help, you have to add them to the module's docstring, which is not exactly ideal.
But Sphinx can do more than the built-in help can. You can configure it to extract the comments on the constants, or use autodata to do it semi-automatically. For example:
#: Indicates some unknown error.
API_ERROR = 1
Multiple #: lines before any assignment statement, or a single #: comment to the right of the statement, work effectively the same as docstrings on objects picked up by autodoc. Which includes handling inline rST, and auto-generating an rST header for the variable name; there's nothing extra you have to do to make that work.
As a side note, you may want to consider using an enum instead of separate constants like this. If you're not using Python 3.4 (which you probably aren't yet…), there's a backport.enum package for 3.2+, or flufl.enum (which is not identical, but it is similar, as it was the main inspiration for the stdlib module) for 2.6+.
Enum instances (not flufl.enum, but the stdlib/backport version) can even have docstrings:
class MyErrors(enum.Enum):
"""Indicates some unknown error."""
API_ERROR = 1
"""Indicates that the request was bad in some way."""
BAD_REQUEST = 2
"""Indicates that the request is missing required parameters."""
MISSING_PARAMS = 3
Although they unfortunately don't show up in help(MyErrors.MISSING_PARAMS), they are docstrings that Sphinx autodoc can pick up.

If you put a string after the variable, then sphinx will pick it up as the variable's documentation. I know it works because I do it all over the place. Like this:
FOO = 1
"""
Constant signifying foo.
Blah blah blah...
""" # pylint: disable=W0105
The pylint directive tells pylint to avoid flagging the documentation as being a statement with no effect.

This is an older question, but I noted that a relevant answer was missing.
Or you can just include a description of the constants in the docstring of the module via .. py:data::. That way the documentation is also made available via the interactive help. Sphinx will render this nicely.
"""
Docstring for my module.
.. data:: API_ERROR
Indicates some unknown error.
.. data:: BAD_REQUEST
Indicates that the request was bad in some way.
.. data:: MISSING_PARAMS
Indicates that the request is missing required parameters.
"""

You can use hash + colon to document attributes (class or module level).
#: Use this content as input for moo to do bar
MY_CONSTANT = "foo"
This will be picked up by some document generators.
An example here, could not find a better one: Sphinx document module properties

the following worked for me with Sphinx 2.4.4:
in foo.py :
API_ERROR = 1
"""int: Indicates some unknown error."""
then to document it:
.. automodule:: foo.py
:members:

I think you're out of luck here.
Python don't support directly docstrings on variables: there is no attribute that can be attached to variables and retrieved interactively like the __doc__ attribute on modules, classes and functions.
Source.

The Sphinx Napoleon Python documentation extension allows to document module-level variables in an Attributes section.
Per https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html :
Attributes
----------
module_level_variable1 : int
Module level variables may be documented in either the ``Attributes``
section of the module docstring, or in an inline docstring immediately
following the variable.
Either form is acceptable, but the two should not be mixed. Choose
one convention to document module level variables and be consistent
with it.

Writing only because I haven't seen this option in the answers so far:
You can also define your constants as functions that simply return the desired constant value when called, so for example:
def get_const_my_const() -> str:
"""Returns 'my_const'."""
return "my_const"
This way they'll be a bit "more constant" on one hand (less worrying about reassignment) and they'll also provide the opportunity for regular documentation, as with any other function.

python "string" module?

So I'm reading this old module from I think around 2002 and it has this line "import string". Did Python require you to import a string module explicitly before to be able to use string type variables or something? I don't see it used like this in the code:
string.something

The string module contains a set of useful constants, such as ascii_letters and digits, and the module is often still imported for that reason.

If you see a import string but never see string.something, someone just forgot to remove an unused import.
While there did use to be some things in string that are now standard methods of str objects, you still had to either
prefix them with string. after importing the library, or
use from string import <whatever> syntax.
Typically, the only times you'll see something properly imported but never "explicitly used" are from __future__ import with_statement or the like - the forwards/backwards compatability triggers used by Python for new language features.

Like Ambar said, it seems to be a redundant import, and RoeeeK is also right in saying that most of the string module's functions are meanwhile string methods, i.e. you can do "foobar".method() instead of string.function("foobar"). However, sometimes it is still useful to explicitely import the module; for instance, in the case of callbacks:
map(string.strip, [' foo ', ' bar ']).
Note that the above can also be achieved by [chunk.strip() for chunk in [' foo ', ' bar ']], so importing string is actually not required in this case.

well, in older versions the string module was indeed much more useful, but in the recent versions most of the string module functions are available also as string methods..
this page will give you a better look:
http://effbot.org/librarybook/string.htm

Pylint best practices

Pylint looks like a good tool for running analysis of Python code.
However, our main objective is to catch any potential bugs and not coding conventions. Enabling all Pylint checks seems to generate a lot of noise. What is the set of Pylint features you use and is effective?

You can block any warnings/errors you don't like, via:
pylint --disable=error1,error2
I've blocked the following (description from http://www.logilab.org/card/pylintfeatures):
W0511: Used when a warning note as FIXME or XXX is detected
W0142: Used * or * magic*. Used when a function or method is called using *args or **kwargs to dispatch arguments. This doesn't improve readability and should be used with care.
W0141: Used builtin function %r. Used when a black listed builtin function is used (see the bad-function option). Usual black listed functions are the ones like map, or filter, where Python offers now some cleaner alternative like list comprehension.
R0912: Too many branches (%s/%s). Used when a function or method has too many branches, making it hard to follow.
R0913: Too many arguments (%s/%s). Used when a function or method takes too many arguments.
R0914: Too many local variables (%s/%s). Used when a function or method has too many local variables.
R0903: Too few public methods (%s/%s). Used when class has too few public methods, so be sure it's really worth it.
W0212: Access to a protected member %s of a client class. Used when a protected member (i.e. class member with a name beginning with an underscore) is access outside the class or a descendant of the class where it's defined.
W0312: Found indentation with %ss instead of %ss. Used when there are some mixed tabs and spaces in a module.
C0111: Missing docstring. Used when a module, function, class or method has no docstring. Some special methods like __init__ don't necessarily require a docstring.
C0103: Invalid name "%s" (should match %s). Used when the name doesn't match the regular expression associated to its type (constant, variable, class...).

To persistently disable warnings and conventions:
Create a ~/.pylintrc file by running pylint --generate-rcfile > ~/.pylintrc
Edit ~/.pylintrc
Uncomment disable= and change that line to disable=W,C

Pyflakes should serve your purpose well.

-E will only flag what Pylint thinks is an error (i.e., no warnings, no conventions, etc.)

Using grep like:
pylint my_file.py | grep -v "^C"
Edit :
As mentionned in the question, to remove the conventions advices from pylint output, you remove the lines that start with an uppercase C.
From the doc of pylint, the output consists in lines that fit the format
MESSAGE_TYPE: LINE_NUM:[OBJECT:] MESSAGE
and the message type can be:
[R]efactor for a “good practice” metric violation
[C]onvention for coding standard violation
[W]arning for stylistic problems, or minor programming issues
[E]rror for important programming issues (i.e. most probably bug)
[F]atal for errors which prevented further processing
Only the first letter is displayed, so you can play with grep to select/remove the level of message type you want.
I didn't use Pylint recently, but I would probably use a parameter inside Pylint to do so.