Why is __init__ not called after __new__ SOMETIMES

genshi, python

Solution

From the documentation:

If `__new__()` does not return an instance of cls, then the new instance’s `__init__()` method will not be invoked.

This is to allow `__new__()` to return a new instance of a different class, which has its own `__init__()` to be called instead. You will need to detect if you're creating a new cls, and call the appropriate constructor instead if not.

Problem

Let me start with this is not a repeat of Why does __init__ not get called if __new__ called with no args. I have tried to carefully construct some sample code for `__new__` and `__init__` that has no explanation I can find. The basic parameters: - There is a base class called NotMine as it comes from another library (I'll disclose at the end, not important here) - That class has an `__init__` method that in turn calls a `_parse` method - I need to override the `_parse` method in subclasses - which subclass I'm creating is not known until invocation - I know there are factory design methods but I cannot use them here (More at the end) - I have tried to make careful use of `super` to avoid the problems in Python logging: Why is __init__ called twice? - I know this is also 'kind of' an AbstractBaseMehtod opportunity but that did not help Anyway, `__init__` should be called after `__new__` and for every explanation of why SOME samples below don't work I seem to be able to point to other cases that do work and rule out the explanation. ``` class NotMine(object): def __init__(self, *args, **kwargs): print "NotMine __init__" self._parse() def _parse(self): print "NotMine _parse" class ABC(NotMine): def __new__(cls,name,*args, **kwargs): print "-"*80 print "Entered through the front door ABC.__new__(%s,%s,*%s,**%s)"%(cls,name,args,kwargs) if name == 'AA': obj = super(NotMine,ABC).__new__(AA,*args,**kwargs) print "Exiting door number 1 with an instance of: %s"%type(obj) return obj elif name == 'BB': obj = super(NotMine,ABC).__new__(BB,*args,**kwargs) print "Exiting door number 2 with an instance of: %s"%type(obj) return obj else: obj = super(NotMine,ABC).__new__(cls,*args,**kwargs) print "Exiting door number 3 with an instance of: %s"%type(obj) return obj class AA(ABC): def _parse(self): print "AA _parse" class BB(ABC): def __init__(self, *args, **kw): print "BB_init:*%s, **%s"%(args,kw) super(BB,self).__init__(self,*args,**kw) def _parse(self): print "BB _parse" class CCC(AA): def _parse(self): print "CCCC _parse" print("########### Starting with ABC always calls __init__ ############") ABC("AA") # case 1 ABC("BB") # case 2 ABC("NOT_AA_OR_BB") # case 3 print("########### These also all call __init__ ############") AA("AA") # case 4 BB("BB") # case 5 AA("NOT_AA_OR_BB") # case 6 BB("NOT_AA_OR_BB") # case 7 CCC("ANYTHING") # case 8 print("########### WHY DO THESE NOT CALL __init__ ############") AA("BB") # case 9 BB("AA") # case 10 CCC("BB") # case 11 ``` If you execute the code, you can see that for each call to `__new__` it announces "which door" it is exiting through and with what type. I can exit the same "door" with the same "type" object and have `__init__` called in one case and not the other. I've looked at the mro of the "calling" class and that offers no insight since I can invoke that class ( or a subcass as in CCC ) and have `__init__` called. End Notes: The `NotMine` library I'm using is the Genshi MarkupTemplate and the reason for not using a Factory design method is that their TemplateLoader needs a defaultClass to construct. I don't know until I start parsing, which I do in `__new__`. There is a lot of cool voodoo magic that genshi loaders and templates do that make this worth the effort. I can run an unmodified instance of their loader and currently everything works as long as I ONLY pass the ABC (abstract sort-of-factory) class as the default. Things are working well but this unexplained behavior is an almost certain bug later. UPDATE: Ignacio, nailed the top line question, if the returned object is not an "instance of" cls then `__init__` is not called. I do find that calling the "constructor" (e.g. `AA(args..)` is wrong as it will call `__new__` again and you are right back where you started. You could modify an arg to take a different path. That just means you call `ABC.__new__` twice rather than infinitely. A working solution is to edit `class ABC` above as: ``` class ABC(NotMine): def __new__(cls,name,*args, **kwargs): print "-"*80 print "Entered through the front door ABC.__new__(%s,%s,*%s,**%s)"%(cls,name,args,kwargs) if name == 'AA': obj = super(NotMine,ABC).__new__(AA,*args,**kwargs) print "Exiting door number 1 with an instance of: %s"%type(obj) elif name == 'BB': obj = super(NotMine,ABC).__new__(BB,*args,**kwargs) print "Exiting door number 2 with an instance of: %s"%type(obj) elif name == 'CCC': obj = super(NotMine,ABC).__new__(CCC,*args,**kwargs) print "Exiting door number 3 with an instance of: %s"%type(obj) else: obj = super(NotMine,ABC).__new__(cls,*args,**kwargs) print "Exiting door number 4 with an instance of: %s"%type(obj) ## Addition to decide who calls __init__ ## if isinstance(obj,cls): print "this IS an instance of %s So call your own dam __init__"%cls return obj print "this is NOT an instance of %s So __new__ will call __init__ for you"%cls obj.__init__(name,*args, **kwargs) return obj print("########### now, these DO CALL __init__ ############") AA("BB") # case 9 BB("AA") # case 10 CCC("BB") # case 11 ``` Notice the last few lines. Not calling `__init__` if it's a "different" class does not make sense to me, ESPECIALLY when the "different" class is still a subclass of the class calling `__init__`. I don't like the above edit but least I get the rules a little better now.

Original source

Related problems