Can't override __init__ of class from Cython extension

cython, inheritance, python

Solution

The documentation is a little confusing here, in that it assumes that you're familiar with using `__new__` and `__init__`.

The `__cinit__` method is roughly equivalent to a `__new__` method in Python.*

Like `__new__`, `__cinit__` is not called by your `super().__init__`; it's called before Python even gets to your subclass's `__init__` method. The reason `__cinit__` needs to handle the signature of your subclass `__init__` methods is the exact same reason `__new__` does.

If your subclass does explicitly call `super().__init__`, that looks for an `__init__` method in a superclass—again, like `__new__`, a `__cinit__` is not an `__init__`. So, unless you've also defined an `__init__`, it will pass through to `object`.

You can see the sequence with the following code.

cinit.pyx:

cdef class Foo:
    def __cinit__(self, a, b, *args, **kw):
        print('Foo.cinit', a, b, args, kw)
    def __init__(self, *args, **kw):
        print('Foo.init', args, kw)

init.py:

import pyximport; pyximport.install()
import cinit

class Bar(cinit.Foo):
    def __new__(cls, *args, **kw):
        print('Bar.new', args, kw)
        return super().__new__(cls, *args, **kw)
    def __init__(self, a, b, c, d):
        print('Bar.init', a, b, c, d)
        super().__init__(a, b, c, d)

b = Bar(1, 2, 3, 4)

When run, you'll see something like:

Bar.new (1, 2, 3, 4) {}
Foo.cinit 1 2 (3, 4) {}
Bar.init 1 2 3 4
Foo.init (1, 2, 3, 4) {}

So, the right fix here depends on what you're trying to do, but it's one of these:

- Add an `__init__` method to the Cython base class.

- Remove the `super().__init__` call entirely.

- Change the `super().__init__` to not pass any params.

- Add an appropriate `__new__` method to the Python subclass.

I suspect in this case it's #2 you want.

* It's worth noting that `__cinit__` definitely isn't identical to `__new__`. Instead of getting a `cls` parameter, you get a partially-constructed `self` object (where you can trust `__class__` and C attributes but not Python attributes or methods), the `__new__` methods of all classes in the MRO have already been called before any `__cinit__`; the `__cinit__` of your bases gets called automatically instead of manually; you don't get to return a different object besides the one that's been requested; etc. It's just that it's called before the `__init__`, and expected to take pass-through parameters, in the same way as `__new__` is.

Problem

I am trying to subclass pysam's `Tabixfile` class and add additional attributes on instantiation. ``` class MyTabixfile(pysam.Tabixfile): def __init__(self, filename, mode='r', *args, **kwargs): super().__init__(filename, mode=mode, *args, **kwargs) self.x = 'foo' ``` When I try to instantiate my `MyTabixfile` subclass, I get a `TypeError: object.__init__() takes no parameters`: ``` >>> mt = MyTabixfile('actn2-oligos-forward.tsv.gz') Traceback (most recent call last): File "<ipython-input-11-553015ac7d43>", line 1, in <module> mt = MyTabixfile('actn2-oligos-forward.tsv.gz') File "mytabix.py", line 4, in __init__ super().__init__(filename, mode=mode, *args, **kwargs) TypeError: object.__init__() takes no parameters ``` I also tried calling the `Tabixfile` constructor explicitly: ``` class MyTabixfile(pysam.Tabixfile): def __init__(self, filename, mode='r', *args, **kwargs): pysam.Tabixfile.__init__(self, filename, mode=mode, *args, **kwargs) self.x = 'foo' ``` but this still raises `TypeError: object.__init__() takes no parameters`. This class is actually implemented in Cython; the constructor code is below: ``` cdef class Tabixfile: '''*(filename, mode='r')* opens a :term:`tabix file` for reading. A missing index (*filename* + ".tbi") will raise an exception. ''' def __cinit__(self, filename, mode = 'r', *args, **kwargs ): self.tabixfile = NULL self._open( filename, mode, *args, **kwargs ) ``` I read through the Cython documentation on `__cinit__` and `__init__` which says Any arguments passed to the constructor will be passed to both the `__cinit__()` method and the `__init__()` method. If you anticipate subclassing your extension type in Python, you may find it useful to give the `__cinit__()` method `*` and `**` arguments so that it can accept and ignore extra arguments. Otherwise, any Python subclass which has an `__init__()` with a different signature will have to override `__new__()` 1 as well as `__init__()`, which the writer of a Python class wouldn’t expect to have to do. The pysam developers did take the care to add `*args` and `**kwargs` to the `Tabixfile.__cinit__` method, and my subclass `__init__` matches the signature of `__cinit__` so I do not understand why I'm unable to override the initialization of `Tabixfile`. I'm developing with Python 3.3.1, Cython v.0.19.1, and pysam v.0.7.5.

Original source