Sphinx: Use method docstring from abstract base class if derived class has none

[alexvorndran]

Let's say I have the following two classes

# -*- coding: utf-8 -*
import abc


class Superclass(object):
    """The one to rule them all"""

    @abc.abstractmethod
    def give(self, ring):
        """Give out a ring"""
        pass


class Derived(Superclass):
    """Somebody has to do the work"""

    def give(self, ring):
        print("I pass the ring {} to you".format(ring))

Is it possible to instruct Sphinx (or even better sphinx-apidoc) to use the docstring of Superclass.give(...) for Derived.give(...) without copying it manually?

The Help widget of Spyder IDE does something similar but as far as I understand, fetching the parent's docstring is not handled by Sphinx there.

[shimizukawa]

For now, sphinx doesn't fetch docstring from superclass.
This is a _ugly_ workaround..

class Derived(Superclass):
    """Somebody has to do the work"""

    def give(self, ring):
        print("I pass the ring {} to you".format(ring))
    give.__doc__ = Superclass.give.__doc__

[brechtm]

You could make this a little more automatic by employing a metaclass for the abstract base class. The following is a very basic implementation of such a metaclass. It needs to be extended to properly handle multiple base classes and corner cases.

# -*- coding: utf-8 -*
import abc


class SuperclassMeta(type):
    def __new__(mcls, classname, bases, cls_dict):
        cls = super().__new__(mcls, classname, bases, cls_dict)
        for name, member in cls_dict.items():
            if not getattr(member, '__doc__'):
                member.__doc__ = getattr(bases[-1], name).__doc__
        return cls


class Superclass(object, metaclass=SuperclassMeta):
    """The one to rule them all"""

    @abc.abstractmethod
    def give(self, ring):
        """Give out a ring"""
        pass


class Derived(Superclass):
    """Somebody has to do the work"""

    def give(self, ring):
        print("I pass the ring {} to you".format(ring))

This may be a better solution than having Sphinx do this, because this will also work when calling help() on the derived classes.

[alexvorndran]

After playing a little bit with @brechtm's solution, I came up with the following:

import abc

class DocstringMeta(abc.ABCMeta):
    """Metaclass that allows docstring 'inheritance'"""

    def __new__(mcls, classname, bases, cls_dict):
        cls = abc.ABCMeta.__new__(mcls, classname, bases, cls_dict)
        mro = cls.__mro__[1:]
        for name, member in cls_dict.iteritems():
            if not getattr(member, '__doc__'):
                for base in mro:
                    try:
                        member.__doc__ = getattr(base, name).__doc__
                        break
                    except AttributeError:
                        pass
        return cls

Note: Some changes were necessary to get it working in Python 2.7.

This works for my use case, but I would nevertheless like to help to find a solution to that problem using Sphinx.

[alexvorndran]

After having a more or less thorough look at how this problem is handled in Spyder, I found that Rope (especially rope.contrib.codeassist.get_doc) does most of the heavy lifting when it comes to finding docstrings.
A little drawback of rope is (in my opinion) that it will always fetch the documentation from base classes even if the derived classes provide their own.

[alexvorndran]

Some playing around with code lead me to the following two functions. Especially the Python 3 parts could need a second pair of eyes as I'm normally coding for Python 2. From what I could see the code should be fairly robust and should fail gracefully.

import six
import inspect
from sphinx.util.inspect import safe_getattr


def get_mro(meth):
    """Get MRO of method"""
    cls = None
    if inspect.ismethod(meth):
        if six.PY2:
            cls = meth.im_class
        else:
            cls = meth.__self__.__class__
    if inspect.isfunction(meth) and six.PY3:
        # based on http://stackoverflow.com/a/25959545/5682996
        cls = getattr(inspect.getmodule(meth),
                      meth.__qualname__.split('.<locals>', 1)[0].rsplit('.', 1)[0])
    cls = cls if isinstance(cls, type) else None
    return inspect.getmro(cls) if cls else []


def find_docstring(meth, greedy=False):
    """Get docstring of method by searching the class hierarchy

    If *greedy* is False, the first and only the first non-empty docstring
    in the class hierarchy will be used.
    """
    docstrings = []
    for cls in get_mro(meth):
        # try to find method in current class
        try:
            cls_meth = getattr(cls, meth.__name__)
        except AttributeError:
            continue

        # try to fetch doc
        doc = safe_getattr(cls_meth, '__doc__', None)
        # doc = inspect.getdoc(cls_meth)

        # None: not documented at all, empty docstring is useless too
        doc = doc.rstrip() if doc is not None else ""
        if not doc:
            continue

        # valid docstring found
        docstrings.append(doc)
        if not greedy:
            break
    return '\n'.join(docstrings) if docstrings else safe_getattr(meth, '__doc__', None)

Replacing

docstring = self.get_attr(self.object, '__doc__', None)

with

docstring = find_docstring(self.object)

in sphinx.ext.autodoc.Documenter.get_doc brought me the result I wanted for my project. I'm aware that this introduces a large amount of overhead for simple cases, so it may be better to treat this as a fallback if no docstring could be found using the standard method.

@shimizukawa Do you think that might be a viable way to go?

A few simple test cases:

class A(object):

    def foo():
        """What a wonder-foo-l world"""
        print("A.foo")


class B(A):

    def foo():
        """        """
        print("B.foo")

    def bar():
        """Bar rules"""
        print("B.bar")


class C(B):

    def foo():
        """What a wonderful world"""
        print("C.foo")

    def bar():
        print("C.bar")


if __name__ == '__main__':
    assert find_docstring(B.foo) == 'What a wonder-foo-l world'
    assert find_docstring(C.foo) == 'What a wonderful world'
    assert find_docstring(C.bar) == 'Bar rules'
    assert find_docstring(C.foo, True) == ('What a wonderful world\n'
                                           'What a wonder-foo-l world')
    assert find_docstring(C.__str__) == 'x.__str__() <==> str(x)'
    assert find_docstring(safe_getattr) == ('A getattr() that turns all '
                                            'exceptions into AttributeErrors.')

[alexvorndran]

@brechtm I asked the same question on Stack Overflow (link) before posting it here. If you also contribute to Stack Overflow, I would really like to honor your idea there. Please consider to add it as an answer which I will be happy to accept.

[avalentino]

It seems that in Python >= 3.5 inspect.getdoc already provides the functionality of retrieving the docstring form the correct base class if __doc__ is None