Attrs: init hooks

[glyph]

Sometimes I want to write an attr.s class that has some behavior in its constructor. Particularly:

• I want to validate between/among different related values passed to the c'tor
• I want to establish a bidirectional relationship to one of the arguments passed to the c'tor (for example: protocol/transport hook-up)
• I want to do some gross I/O because it's a handle for some external gross I/O object (i.e. I want to open something, for example) (sorry)

Arguably, many of these things could be expressed as a function, but some of them involve preserving valid structural relationships between the object and its dependencies, so I'd really prefer to be able to use __init__ in some cases.

I know, I can write my own __init__. I know! But if I do that, I give up not just on the convenience of attr.s writing the __init__ for me, but also the useful _behavior_ of attr.s: i.e. validators, default factories.

All of my use-cases here would be easily accomplished with either a pre-__init__ and post-__init__ hook. Most of the ones I'm _actually_ interested in would all be achievable with a post-__init__ hook; I think pre-__init__ might be pointless given the availability of convert=.

This has been discussed already, but some of the other issues (#33 #24 #38 #38 #58) which have raised the possibility of a post-__init__ hook have been somewhat conflated with better support for inheritance. As we all know, inheritance is deprecated and will be removed in python 3.7 so we don't need to focus on that.

[Insoleet]

As we all know, inheritance is deprecated and will be removed in python 3.7 so we don't need to focus on that.

I think the problem is that many libraries are based on inheritance. Gtk-python, PyQt, Django... I think it would reduce too much the use cases of attr if inheritance was totally not supported.

[wearpants]

Re: __pre_init__ could be useful if you need to set up some state on the instance before validating for some reason

[hynek]

@Insoleet I’m not sure what you’re trying to say; is there anything about this proposal contrary to that?

[Insoleet]

I'm not sure, I would just like to be sure that you guys takes inheritance into account and not just don't care about it :)

The __pre_init__ hook is important for PyQt for example. It is the first thing to do before doing any PyQt related thing in the constructor (see for example : http://stackoverflow.com/questions/12280371/python-runtimeerror-super-class-init-of-s-was-never-called )

[hynek]

Random idea time! What do y’all think about:

import attr

class C:
    x = attr.ib()

   @attr.hooks.post_init
   def i_dont_care_how_its_called(self):
       # ...

Positive:

• It’s much cleaner than adding another argument to attr.s
• More explicit. Someone who never heard of attrs can deduce the meaning immediately.
• you can have multiple of them which is useful in subclassing

Negative:

• can be rather confusing if multiple hooks are registered?

[wearpants]

+1

I'd say just call them in order of definition within class... For
subclasses, after super().init()

On Aug 22, 2016 4:49 AM, "Hynek Schlawack" [email protected] wrote:

Random idea time! What do y’all think about:

import attr
class C:
x = attr.ib()

@attr.hooks.post_init
def i_dont_care_how_its_called(self):
# ...

Positive:

• It’s much cleaner than adding another argument to attr.s
• you can have multiple of them which is useful in subclassing

Negative:

• can be rather confusing if multiple hooks are registered?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/hynek/attrs/issues/68#issuecomment-241350493, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAe2AJABwv2cDbn6tlD6dPimcn-9FhCrks5qiWKzgaJpZM4JpO43
.

[Tinche]

I like the cleanness of the decorator idea, although I'd alias it attr.post_init.

attrs has never played too well with non-attrs superclasses (since we never call super), so I'm not too worried there, and this feature might even improve the situation by allowing users to manually do it themselves.

What about attrs superclasses? Do we call their hooks, and if so in which order? (Presumably MRO or reverse MRO. My gut feeling is reverse MRO, so the basest class first.)

[glyph]

What about attrs superclasses? Do we call their hooks, and if so in which order? (Presumably MRO or reverse MRO. My gut feeling is reverse MRO, so the basest class first.)

MRO. The only right way to cooperatively call an overriden method is super(MyClass, self).__init__(). In py3 this is built in to the language; it's what super().__init__(...) does. Reversing the MRO is just a mistake; the basest class is always object and you never want to upcall to its __init__.

[glyph]

I very much like the idea of this hook decorator, though! I don't like typing underscores, so my suggestion for the spelling would be @attr.after.init and (maybe later) @attr.before.init ("pre" and "post" are more jargon-y than "before" and "after", and are also prefixes rather than words, which means preinit would be closer to valid, but looks weird).

[glyph]

Regarding "if multiple hooks are registered" - my suggestion for the first iteration of this would be to just make registering a second hook into an immediate exception at registration time. If we come up with a non-confusing way to register multiple hooks, it's always fine to add it later.

[Tinche]

MRO. The only right way to cooperatively call an overriden method is super(MyClass, self).init(). In py3 this is built in to the language; it's what super().init(...) does. Reversing the MRO is just a mistake; the basest class is always object and you never want to upcall to its init.

Just to clarify my gut feeling: usually when you have a class hierarchy, the first line in your __init__ will be a call to super() (in Java, for example, not doing this on the first line is a compile error if I remember correctly from a past life), then the business logic of that particular __init__. So __init__s get called in the MRO, but the business logic gets executed reverse MRO, with the most base class setting itself up first. Then subclasses can count on the superclass invariants holding.

Since attrs will be in charge of invoking hooks, the user doesn't get to have a choice.

class A:
    x = attr.ib()
    y = attr.ib()

    @after_init
    def after(self):
        # Set me up.
        # If I'm an instance of B, B.after has been called already.
        # Maybe I want to make sure y is greater than x.
        if self.x > self.y:
            self.x, self.y = self.y, self.x

class B(A):
    z = attr.ib()

    @after_init
    def after(self):
        # This gets called before A.after
        # Has the superclass set itself up?
        # Maybe I want to make sure z is greater than y, but I can't yet, because A.after might flip them.

Of course, even in Java you can call super() with the result of an expression or function IIRC, so the truth is you get to execute some code before the call to super anyway. And of course Python isn't Java.
¯_(ツ)_/¯

[max-sixty]

Have you guys thought about using __new__ for the attr logic, and then letting users use __init__ for themselves?

I've built a similar library (not nearly as impressive, but prior to this coming along), and that technique serves us very well

[glyph]

For serialization and deserialization, the expected protocol for __new__ is that it never _requires_ any arguments. Violating this expectation is possible, but ugly. Not that you should use Pickle for most things, but it would be annoying to break the ability to use attrs with multiprocessing, for example, without a bunch of additional explicit work.

[max-sixty]

For serialization and deserialization, the expected protocol for __new__ is that it never requires any arguments

I don't have much context, so potentially I'm missing something
But serialization & deserialization generally doesn't call __new__ at all. Serialization pickles the contents of __dict__ and then deserialization puts the contents back on __dict__.

We use __getstate__ to only pickle what we want to retain. __new__ still isn't called

[wearpants]

There are a few variants of the pickle protocol that do call __new__.
It's also useful if you're writing a __setstate__. Finally, this
wouldn't work with inheriting from anything with its own __new__ like
native C extensions (like PyQt) or classes with slots; if memory serves,
you can't have more than one __new__ in an inheritance hierarchy

I may be wrong on some of the details above, it's been quite a while, but
__new__ is not the way to do this IMO

On Aug 23, 2016 12:44 AM, "Maximilian Roos" [email protected]
wrote:

For serialization and deserialization, the expected protocol for new
is that it never requires any arguments

I don't have much context, so potentially I'm missing something
But serialization & deserialization generally doesn't call new at
all. Serialization pickles the contents of dict and then
deserialization puts the contents back on dict.

We use getstate to only pickle what we want to retain. new still
isn't called

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/hynek/attrs/issues/68#issuecomment-241625587, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAe2AOj_WFFqjdoBH3pb8fXH24Seu6sKks5qinrCgaJpZM4JpO43
.

[hynek]

Yeah so any type of classic meta programming is out of the question. attrs classes will always be regular Python classes with some generated methods attached to it. I think there’s enough alternatives that stretch Python’s meta possibilities but for me it was always a big design goal to spit out normal classes without surprises.

[max-sixty]

This isn't my battle, so I'll leave this here unless really needed. But I'd really encourage you guys to have a read up on how __new__ / __init__ works. The words above aren't correct and I think you'll find it much more pythonic and an easier fit with existing classes (The _post_init stuff makes me wince a bit!).

if memory serves, you can't have more than one __new__ in an inheritance hierarchy

Nope! It's just like __init__

Finally, this wouldn't work with inheriting from anything with its own __new__ like native C extensions (like PyQt) or classes with slots

It would need to super up, in the same way that you'd need to super up if using __init__ when inheriting from a class that relied on __init__. By using __new__, you get out of the way of the much more common user usage of __init__

any type of classic meta programming is out of the question

No meta programming here!

Overall I would love to transition to this library, and you guys have done some amazing work...

[wearpants]

On Aug 23, 2016 10:01 AM, "Maximilian Roos" [email protected]
wrote:

This isn't my battle, so I'll leave this here unless really needed. But
I'd really encourage you guys to have a read up on how new / init
works.

Likewise.

Let's drop it.

[Tinche]

Ok looks like we got bogged down in discussing inheritance stuff, which is exactly what @glyph warned against in the OP.

A suggestion:

• one after_init per class. Like __init__.
• if your class has an after_init, attrs calls it and only it. If you're inheriting from a class that has an after_init too, call it yourself. Like __init__.
• if your class doesn't have an after_init, at class creation time we will walk the MRO (in proper MRO, not reverse), find the first class that has it, and insert a call to that. Basically like __init__.

[hynek]

Removing milestone, this bikeshed will take a while to pain.

[glyph]

@hynek - Small PRs, small releases :)

[glyph]

It sounds like we're converging on after_init/after.init though. @Tinche's summary is more or less exactly what I think should happen - do you have any objections?

[hynek]

I feel there has to be the word hook somewhere. attr.hook.after[._]init looks good to me. I know you hate typing (except e-mails ;P) but shit should make _some_ sense when looking at it, agreed?

[glyph]

@hynek I specifically wanted to avoid the word "hook", not for brevity, but it sounds like jargon. The word "after" is very clear (this is "after init") and "hook" is something some users might not have heard of, and for the users who have heard of it, it's redundant.

[glyph]

@hynek trust me, if I'm going for brevity, you'll know. import a; @a.tt.rs; @a.ft.rr for example :)

[glyph]

(I am not strongly opposed though, if you really think it's more readable it's fine with me.)

[hynek]

How about call.after.init?

[wearpants]

Like this?

@attr.call.after.init
def custom_setup(self, **whatev):
    pass

Are there are great many more .call.s (besides before_init of course)? Just wondering

[glyph]

I stil don't understand why anything beyond "after" is required, but "call" certainly reads more nicely to me than "hook" :)

[hynek]

OK, volunteers?

[wbolster]

stumbled upon this via https://twitter.com/wbolster/status/774257616910290944 and @hynek's response to it.

the main point here is the inversion of the problem:

what if attrs also exposed the generated __init__() as an __attrs_init__()? i could simply write my own __init__() and call self.__attrs_init__() how i see fit. no hooks, no magic.

@attr.s
class Foo:
    bar = attr.ib(convert=..., validate=..., default=..., hash=...)
    baz = attr.ib(convert=..., validate=..., default=..., hash=...)

    def __init__(self, bar, baz):
        my_custom_multi_attribute_validate(bar, baz)
        self.__attrs_init__(bar, baz)
        even_more_setup()  # after attrs has done its magic

looks like this would be almost a one line addition:
https://github.com/hynek/attrs/blob/master/src/attr/_make.py#L432

[hynek]

yep I find the inversion better too. and it solves the super() problem too.

opinions y’all?

[wearpants]

that's so clever!
it's much better!
attrs forever!

On Fri, Sep 9, 2016 at 11:07 AM, Hynek Schlawack [email protected]
wrote:

yep I find the inversion better too. and it solves the super() problem too.

opinions y’all?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/hynek/attrs/issues/68#issuecomment-245941213, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAe2AJ1CiKwHOH0ZK_6WVKvsscBeRUX2ks5qoXYxgaJpZM4JpO43
.

Pete Fein | wearpants.org | @wearpants

[Insoleet]

I like the idea of never writing __init__ anymore. I don't like the idea of having to write it again.

For empty inits, we will have to write everywhere the same useless code even when we don't need it :

@attr.s
def Foo:
  aaa = attr.ib()
  aab = attr.ib()
  aac = attr.ib()
  aad = attr.ib()
  aae = attr.ib()

  def __init__(aaa, aab, aac, aad, aae):
    self.__attrs_init__(aaa, aab, aac, aad, aae)

[hynek]

nonono, that’s not what we mean. it would be an option of course. Maybe we’ll expand the init argument to also take a magic value that makes attrs attach an __attrs_init__ instead of an __init__.

[wbolster]

@Insoleet i think you misread my proposal. i said

what if attrs also exposed the generated __init__() as an __attrs_init__()

i think you missed the word "also". it's an EXTRA way of reaching into attrs magic __init__() for those cases where you really want your own __init__(), which, i agree with you, should not be the default. :)

[Tinche]

Not too thrilled. I like the fact I declare my class fields only once, in the attr.ib()s, and __init__ gets generated magically.

If I add another attribute, I need to remember to change my __init__ too. If I add, remove or change a default, I need to change my __init__, too, etc.

Then I need to repeat myself again to invoke the __attr_init__. (Or use _args *_kwargs, but I bet there's a performance penalty there and it probably messes with the introspection.)

[Insoleet]

Ok. An option is better ! But still, even if it is an option, if I need to call super, I have to write this really annoying code which is obviously not the goal of attrs :

@attr.s
def Foo(PyQt5.QObject): # Damnit, a QObject, super() is mandatory !
  aaa = attr.ib()
  aab = attr.ib()
  aac = attr.ib()
  aad = attr.ib()
  aae = attr.ib()

  def __init__(aaa, aab, aac, aad, aae):
    self.super()
    self.__attrs_init__(aaa, aab, aac, aad, aae)

Also @Tinche got most of the problems with maintaing this code !

I really prefer the idea of having post/pre init hooks.

[hynek]

If I add another attribute, I need to remember to change my init too. If I add, remove or change a default, I need to change my init, too, etc.

That’s is a problem we’d have with pre-hooks too and which would make them generally inconsistent. So we have to weight those downsides against each other. As soon as you want both pre and post, things get much more natural to read. And it looks a lot less magicy.

[wearpants]

On Fri, Sep 9, 2016 at 11:21 AM, Tin Tvrtković [email protected]
wrote:

If I add another attribute, I need to remember to change my init too.
If I add, remove or change a default, I need to change my init, too,
etc.

Then I need to repeat myself again to invoke the attr_init. (Or use _args
*_kwargs, but I bet there's a performance penalty there and it probably
messes with the introspection.)

Actually, this could be a benefit in some cases - not all attribs need to
be arguments to __init__ - perhaps user init generates some values itself
and passes those __attr_init__

[hynek]

Yeah it’s generally more flexible and Pythonic. Although it may encourage anti-patterns of side-effecty __init__s.

[wbolster]

my main use cases for custom __init__() are usually:

• multi attribute validation
• additional setup, e.g. opening a file or socket or whatever (let's not get philosophical about it, attrs is also great for code that wraps file-like objects and also takes file names)
• deriving some attr values from other attributes, e.g. making one attrib default to the value of another (e.g. local_name defaults to remote_name, and so on)

all of these would be very explicitly supported with a custom __init__() in which i delegate to __attrs_init__().

the "downside" is repeating argument lists for the __init__(), but in my experience i usually want a _different_ argument list, not just all attributes enumerated, so i really don't consider that a downside. explicit is better than implicit: if you override something, better make it explicit what exactly you're overriding. this point is the same as what @wearpants said in https://github.com/hynek/attrs/issues/68#issuecomment-245947150.

[Tinche]

We haven't talked about pre hooks much so I haven't thought about them, nor have I ever needed one. Post hooks I have needed, and I always figured would be called with just self.

If you only need a post hook, this is objectively worse in my opinion. Also all cases mentioned by @wbolster in https://github.com/hynek/attrs/issues/68#issuecomment-245948831 can be done with post hooks.

Also if you need an __init__ with different params, I'd argue a classmethod would be the way to go.

Anyway, I'm gonna be -1 on this particular proposal for now, but of course it's not my call. :)

[Insoleet]

Also if you need an init with different params, I'd argue a classmethod would be the way to go.

Could not have said it better. attr forcing the list of parameters of __init__ is good because it involves consistency and it forces the usage of good patterns (reverse-dependencies).
Most of pre-init stuff can be done via a class method. I think Tinche got it right : we only need a post-init hook. (Which can involves usage of anti-patterns, thought... )

[wbolster]

Also if you need an init with different params, I'd argue a classmethod would be the way to go.

disagreed.

attrs is perfect for straight-forward data containers, but that's not all there is to it. rich classes with a bunch of attributes and a nice api to create instances is also where attrs should shine.

providing nice apis through __init__() is quite pythonic (no need for factories). enforcing classmethods for those "custom" constructors implies that the default should be that "all attribs can be explicitly specified", which may not be a sensible nice api at all, especially if you want to do the things that this ticket is about: deriving attrib values from each other, doing multi-attrib validation, and so on.

[hynek]

I would generally tend to say that deriving params should be done in classmethods too but it’s my impression that we def need pre-hooks to satisfy the super() crowd, right? And if we have a pre-hook, ISTM ugly to have two hooks with two different signatures.

---

I know y’all hate typing but the inversion seems like the clearest API to me. The intent is clear and the execution too: because it’s right there in front of your nose.

---

Only counterpoint to me is that people are probably gonna forget to call __attrs_init__. :|

---

I guess I’ll collect more opinions (especially from the whoever opened this ticket
😜 although he’s the laziest typist AFAICT 😞) and sleep a night over it.

[glyph]

This is a very promising direction - more explicit than what we were discussing. It's a bit more typing, but I think I like it anyway; it supports other use-cases like changing the value of arguments first. However, one of the things that I like out of attrs is that things don't appear "out of nowhere" - only special methods (which you probably already had an implementation of) get implemented. Having __attrs_init__ get added as an attribute is not ideal. There's also the issue of inheritance, which while it shouldn't be a priority, we probably shouldn't break unnecessarily.

What if the public interface for this were just attr.init? i.e.: attr.init(self, aaa, aab, aac, aad, aae)?

This has a downside as well - it won't have the right argument signature from introspection because it's a generic method and not specific to the class. However, in this sense it is symmetrical with attr.assoc, which is generic but restricts acceptable arguments passed to it.

In any case, I like either of these proposals better than the hooks stuff we were discussing. They're terser but have more implicit behavior.

[Tinche]

we def need pre-hooks to satisfy the super() crowd, right?

You can call super() from post_init().

I know y’all hate typing

It's not just about the amount of typing, but about DRY. Now instead of naming an attribute once, you need to name it three times. (Once in attr.ib(), once in the init signature, and once when calling attr.init.) Default values of attributes get more complex.

it supports other use-cases like changing the value of arguments first

Like convert=?

On the other hand, going this way is very simple. We're halfway there - @attr.s(init=False) lets you write your own init today. Implementing attr.init() is also fairly simple, with the caveat that it's going to be significantly slower than a tailored, generated init.

If you need to do super special stuff in your init, I get you might be willing to put up with all of this, and that's fine. If you just need to call super() or compare the values of two attributes post convert, this is kind of a raw deal.

[Tinche]

It occured to me we could stick an optimized, generated init somewhere, like in cls.__attrs_init__, but still expose attr.init as a nice API that would just call it. Just like attr.fields is basically just a nicer gateway to cls.__attrs_attrs__. That would actually be less work and have faster execution.

[hynek]

You can call super() from post_init().

I can but aren’t there cases where you need to call super() before you touch the classes? Some kind of metaclasses bullshit or something? Pretty sure those who need to call super() need it for really nasty super classes?

[20after4]

Sounds kind of like aspect oriented programming... which is really what @decorators are all about in python, isn't it?

If you wanna be fancy then this syntax would be flexible and easy to understand just by reading what's plainly in front of you:

@attr.s
class BikeShed(object):
  @attr.before('__init_')
  def my_init(self, *initargs, **kwargs):
    # before hooks get the same args as the method they are attached to
    pass

  @attr.after('__init__')
  def post_init(self):
    # should an after hook receive the method's args? I think probably not
    pass

[hynek]

Accepting strings suggests that you can intercept any method which is out of scope and I'm not a fan of strings in APIs in general. I prefer a good old enum/constant.

[hynek]

After thinking it through I think I like Tin's suggestion best. For pickling it's important to keep the methods on the class but since it's a dunder, there shouldn't be any issues with subclassing @glyph?

Are we gonna pass a class or an instance?

attr.init(C)(self, a, b, c)
Or
attr.init(self)(a, b, c)

First one is cleaner, second one more pragmatic.

one thing to keep in mind: attr.init() has to work _always_.

[20after4]

How about this:

somewhere in the attr namespace:

AFTER='after'
class call(object):
    @staticmethod
    def after(*args, **kwargs):
        def wrapper(func):
            func._attr_call=AFTER
            return func

@attr.s
class BikeShed(object):
    @attr.call.after
    def __init__(self, *args):
        pass

Then when attr.s() encounters the AFTER marker, it could save the existing __init__ as __after__init__ and replace it with the normal generated __init__ which would call __after_init__ if such a method exists.

EDIT: Sorry, I didn't notice the last reply here. I will stop bikeshedding now ;)

[moshez]

Like __attrs_init__ but saner:

@attr.s
class BikeShed(object):
    color = attr.ib()
    def __init__(self, color):
          print("Before init, color={}".format(color))
          attr.init(self, color=color)
          print("After init, color={}".format(color))

Everything is obvious (you need to go read the docs for attr.init, though). No magical calls. Downside: spelling out the signature for init.

[hynek]

Like attrs_init but saner:

Is that a typo? The last proposals were about attr.init() – no underscores involved. :)

@attr.s
class BikeShed(object):
color = attr.ib()
def init(self, color):
print("Before init, color={}".format(color))
attr.init(self, color=color)
print("After init, color={}".format(color))
Everything is obvious (you need to go read the docs for attr.init, though). No magical calls. Downside: spelling out the signature for init.

This is mostly what I'm suggesting except that in this case the signature would be attr.init(inst, *args, **kw) which I don't like TBH. I'd prefer init just retrieving the init method; hence the second pair parens to preserve all signatures.

I just wonder if I'm preferring purity over pragmatism here...

[wbolster]

attr.init(...) without extra parens certainly looks cleaner.

ideas to get the init function method:

• attach it to __attrs_init__ which would allow accessing it directly
• make attr.init() distinguish on the first arg:
  
  
  
  • if an instance (self) it invokes the init method
  
  
  • if a class (type(self) or just spell out the name) it returns the method itself
  
  

example of the second idea:

@attr.s
class Foo():
    def __init__(self, ...):
        attr.init(self, ...)  # invokes init method
        attr.init(Foo)  # returns init method without invoking it

[hynek]

Oi, that seems like confusing magic that I _really_ try to avoid in attrs. I’m not a fan of implicit type dispatch.

[wbolster]

having attr.init() as a friendly way to invoke the "magic" __attrs_init__() method mimicks how python dispatches from built-ins to magic methods: e.g. len(obj) calling obj.__len__(), str(obj) calling obj.__str__(), and so on. attr.init(obj, ...) would call obj.__attrs_init__(...).

what's exactly the use case for getting the initialiser method without invoking it? and if it's a non-standard use case, what's wrong with accessing it as "magic" attribute?

[hynek]

I personally wouldn’t mind. But @glyph thought something might go wrong with subclassing (which I don’t follow which is why I asked for clarification before) and @Tinche mentioned something about performance which I also don’t understand. :)

[glyph]

The problem with subclassing is that the naive implementation of attr.init would only call the proximal __attrs_init__ method. I suppose if we pass the class to attr.init as well (i.e. attr.init(self, MyClass, a=b, c=d, etc)) it resolves the issue of which initializer to invoke.

[hynek]

But what about self.__attrs_init__(self, a, b, c)? The only case it could catch a method from a super-class is if one passes init=False, no?

[glyph]

Oh right. The inheritance issue with __attrs_init__ as an attribute is that you expect to call self.__attrs_init__ in __init__, except subclasses will change the signature of self.__attrs_init__, so any subclassing will instantly break.

attr.init(self, MyClass, a=b, c=d, etc) actually neatly resolves this issue, although then you pollute every call site with this extra sad class argument.

[wearpants]

It's Python 2 super() all over again. :-(

On Thu, Sep 15, 2016 at 2:41 PM, Glyph [email protected] wrote:

Oh right. The inheritance issue with attrs_init as an attribute is
that you expect to call self.attrs_init in init, except
subclasses will change the signature of self.attrs_init, so any
subclassing will instantly break.

attr.init(self, MyClass, a=b, c=d, etc) actually neatly resolves this
issue, although then you pollute every call site with this extra sad class
argument.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hynek/attrs/issues/68#issuecomment-247414415, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAe2ADg_eo-triJXDPvtPFdYSxlW2o0Gks5qqZE-gaJpZM4JpO43
.

Pete Fein | wearpants.org | @wearpants

[glyph]

And this is exactly why language designers should not reserve special features (i.e. the containing-class detection of super()) for themselves, it turns out users need them too :).

[wearpants]

Hmm, it almost seems like we could use super() here, but we need the
current class, not the parent... Might be able to do some clever
introspection with a super() object and the __mro__ to find it...

Would spelling this MyCurrentClass.__attrs_init__(self, a, b, c) be any
better than attr.init(self, MyClass, ...)?

On Thu, Sep 15, 2016 at 3:04 PM, Glyph [email protected] wrote:

And this is exactly why language designers should not reserve special
features (i.e. the containing-class detection of super()) for themselves,
it turns out users need them too :).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hynek/attrs/issues/68#issuecomment-247421189, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAe2AD6Vg0FAlmho0VnBQBN5PVVGADY8ks5qqZa0gaJpZM4JpO43
.

Pete Fein | wearpants.org | @wearpants

[hynek]

Dumb question, but what’s wrong with self.__class__?

[glyph]

@hynek Same thing as is wrong with it with super().

@attr.s
class A(object):
 a = attr.ib()
 b = attr.ib()
 def __init__(self, a, b):
  a, b = mutually_customize(a, b)
  self.__class__.__attrs_init__(a, b)
@attr.s
class B(object):
 c = attr.ib()
 d = attr.ib()
 ???

B's __attrs_init__ takes a, b, c, d, presumably, which is now wrong for A.__init__ to call. And B.__init__ has to super-call to A.__init__ in order to get the logic it encapsulates.

[wearpants]

It seems you can use the __self_class__ and __thisclass__ attributes on
a super() instance to find out where you are in the MRO...

Sorry, not enough brains on a Friday afternoon to mock up the code to paint
this bike shed myself.

On Fri, Sep 16, 2016 at 3:38 PM, Glyph [email protected] wrote:

@hynek https://github.com/hynek Same thing as is wrong with it with
super().

@attr.sclass A(object):
a = attr.ib()
b = attr.ib()
def init(self, a, b):
a, b = mutually_customize(a, b)
self.class.attrs_init(a, b)@attr.sclass B(object):
c = attr.ib()
d = attr.ib()
???

B's attrs_init takes a, b, c, d, presumably, which is now wrong for
A.init to call. And B.init has to super-call to A.init in
order to get the logic it encapsulates.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hynek/attrs/issues/68#issuecomment-247689112, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAe2AB8ki4cJPJcDi6C0CM2np6O6cCYLks5qqvAbgaJpZM4JpO43
.

Pete Fein | wearpants.org | @wearpants

[hynek]

I'm just not smart enough to subclass.

[vortec]

What's wrong with class methods?

@attr.s
class Object:
    id = attr.ib()
    position = attr.ib(validator=instance_of(int))

    @classmethod
    def create(cls, *args, **kwargs):
        obj = cls(*args, **kwargs)
        obj.position = obj.position + 1
        return obj

obj = Object.create(123, position=2)
print(obj.position)

[wearpants]

Because then the standard constructor is callable but gives an unitialized
instance. See previous comments

On Sep 20, 2016 9:03 AM, "Fabian Kochem" [email protected] wrote:

What's wrong with class methods?

@attr.sclass Object:
id = attr.ib()
position = attr.ib(validator=instance_of(int))

@classmethod
def create(cls, *args, **kwargs):
    obj = cls(*args, **kwargs)
    obj.position = obj.position + 1
    return obj

obj = Object.create(123, position=2)print(obj.position)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hynek/attrs/issues/68#issuecomment-248294899, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAe2AOBIeKhht6Iib-eGItRqvz_XAl4Eks5qr9mLgaJpZM4JpO43
.

[vortec]

Sorry, I just searched for "class method", not for "classmethod".

[wearpants]

No worries, I wouldn't read the whole thread either. :-)

[wearpants]

Here's SO on how to find yourself: http://stackoverflow.com/posts/9934299/revisions

[mathieulongtin]

Let's step back and talk about the use cases:

1. You need pre-processing before the attributes are even set
2. You need post-processing after the attributes are set
3. A combination of both cases

In case 1 and 3, you might as well write __init__ completely, including attribute assignments. attr is still useful because it generates comparison and representation methods. However, this:

    self.a = a
    self.b = b

is not very different amount of typing, and has a clearer intent than this:

    attr.init(self, a, b)

In case 2, any of the post-init hook solution mentioned above would work. And there is no issue with inheritance in that case.

[glyph]

is not very different amount of typing, and has a clearer intent than this:

It's a potentially _wrong_ intent, though. attr.init(self, a, b) will apply coercers and validators to a and b, but manually setting the attributes won't.

[mathieulongtin]

My thinking is by the time you write def __init__(self, a, b, c):, you might as well do all the coercing and validating and defaulting in init.

[mathieulongtin]

attr is mostly about not repeating yourself and avoiding boilerplate. attr.init makes for a lot of repeating.

How about this. Kind of like @argh.expect_obj.

@attr.s(wrap_init=True):
class Xyz(object):
  a = attr.ib(default="a"); b = attr.ib(default="b")
  def __init__(self, args):
    # preprocessing, submitted values for a and b are in args.a and args.b
    # they can be modified
    args.set_attr()  # default, type-checks, coerces and set attributes
    # postprocessing goes here, attributes are fully set

1. attr wraps the given init with a generated one that passes the arguments in the args object.
2. No need to repeat the list of arguments, just put them in the attr.ib declarations.
3. args.set_attr() may be broken down in args.set_default(), args.coerce(), args.validate() and args.assign() if fine control is needed.

From the outside, Abc.__init__ takes an optional a and b argument and looks perfectly normal.

[glyph]

From the outside, Abc.init takes an optional a and b argument and looks perfectly normal.

Personally (and I suspect @hynek would agree) I find the idea of writing an __init__ but lying about its signature bad. Potentially a better suggestion would be our own special method name (I could have sworn that someone had suggested __post_init__ above). Perhaps __attrs_init__(args)?

[hynek]

__post_init__ was my original idea heh.

Problem is that it's gonna need some way to signal that it's there (checking using getattr seems gross and not fitting the explicit design of attrs) which brings us back to the decorator solution... 🎠

[mathieulongtin]

Wouldn't attr_init be able to use type(self).__dict__['__attr_init__'] to grab the class specific init, therefore skipping inheritance issues?

[ambv]

Two ideas.

1. The order of field declarations is known. So, you could initialize your fields deterministically with a factory, if that factory were given a self attribute.
2. But since "1." isn't entirely generic, having the generated __init__() call __post_init__() makes sense to me. It is explicit enough already that you use the class decorator so forcing users to put another decorator every time is counter-productive. Isn't attrs all about avoiding boilerplate?

Subclassing and the custom super() ideas all sound inferior to me. They are far less obvious which means I couldn't recommend using them to regular Joes (like myself).

[hynek]

A new painter appears! :D

At this I think the question is whether we go:

• magic dunder method, or
• decorator.

I’m not sure what you mean by subclassing/custom super, because it’s all just about people wanting to subclass non-attrs classes and call super on them. It’s not meant as a mean to solve the problem here but a goal to enable. :)

Thoughts

Magic Dunder

• It’s more error prone since mistyping the name will result in nothing happening.
• One hook per class. Or do we traverse the MRO to collect them all? I guess we have to, if the super classes depend on it?

Decorator

• More explicit.
• Allows for multiple hooks even throughout hierarchies. Meaning: attrs can collect all hooks and execute them.
• That has the downside of possible conflicts though?

---

So personally, I feel like a decorator would be more attrs-style because it’s more explicit and less error prone and plays well WRT to intra-attrs subclassing.

I’d love thoughts on these two approaches.

I guess if someone offered me a good PR for Lukasz’s 1. point I’d begrudgingly merge it.

[ambv]

Well, a counter-argument to the typos in the name is that you can also mistype init, str, etc. and they will all happily do nothing.

As for allowing multiple hooks with a decorator, then you run into issues with ordering. I'd rather only have to override one hook, possibly calling super() inside it. While this also has problems, it's less custom and magical than decorators.

[hynek]

Yes but what about subclassing? What if someone relies on the hook being run to initialize an attribute? :-/

[glyph]

If you go with __post_init__ I think it's actually a lot more straightforward to deal with subclassing.

_…reads through the whole previous discussion to make sure nobody else has raised this already…_

The problem with __init__ and subclassing is that it doesn't have a fixed signature. So if the MRO might get reordered by diamond inheritance or what have you, you don't know what to call super().__init__ with; it's literally impossible to know in the general case. attrs actually has more information in __init__ than regular old Python code does, so it might actually be able to solve this problem, especially if the other superclasses are also attrs classes.

Let me posit such a solution, actually. Instead of trying to make __init__ cooperative, let's say attrs just makes all inherited attr.ibs part of your constructor. (Maybe it does this already? I dunno.) The important thing is that attrs defines an __init__ that doesn't upcall other __init__s _at all_, sidestepping the inheritance problem entirely. This uber-__init__ then calls __post_init__ once, at the end of the __init__.

Making __post_init__ _itself_ cooperative is then very easy: it _does_ have a fixed signature. Just super().__post_init__() at the beginning or end of your __post_init__ (depending on your behavioral dependencies); this will correctly invoke __post_init__ up and down the MRO, and you can either depend on your superclasses's post-init behavior, in which case you invoke it early, or you know you're dealing only with your own subclass's state, in which case you put it at the end.

[glyph]

After typing that all out and swapping the whole previous conversation into my brain again, I'm coming down more and more in favor of just a __post_init__ special method. In many ways it's less magical than the decorator-based solution.

[mathieulongtin]

attrs initializes parents attrs class's attributes without ever calling super(). It just inspects the list of arguments, including the parents.

So should pre-init and post-init follow the same rule as attrs' __init__? It gets called magically in the parent class by the generated __init__?

Should attr's generated init be like this:

def __init__(self, ...):
  self.__pre_init__()
  super(Ba, self).__pre_init__()  # do this for all parent classes
  # init attributes
  super(Ba, self).__post_init__()  # do this for all parent classes
  self.__post_init__()

Or should it be like this:

def __init__(self, ...):
  self.__pre_init__()
  # init attributes
  self.__post_init__()

And the defined __pre_init__ and __post_init__ have to call super().__pre_init__ and super().__post_init__, if needed?

[glyph]

@mathieulongtin Okay so I just re-invented what it already does? :) (I have _very_ lightly used inheritance with attrs once or twice, but never had a complex hierarchy and never inherited anything that required an __init__, or one attrs class from another.)

[glyph]

__pre_init__ is a separate problem; I'm not entirely sure we need it.

__post_init__, though, should be called exactly once at the end of the generated __init__. super() works fine for methods _other_ than __init__; you either rely on your superclass's behavior or not, you either call it first or call it last. So let's not do anything magic, just let users implement inheritance with the regular semantics that are usually used for method overloading.

Honestly, the more I think about it, the more I think that with this hook attrs might actually fix the second biggest problem with subclassing that is specific to Python. (The first biggest being confusion about who owns the self namespace when inheriting across library boundaries...)

[hynek]

Hm how do you mean “_other_ than __init__”? It would be nice if the feature could stop people from opening issues, that they want to use attrs with PyQT etc.

[hynek]

Fixed in #111! 🚲 🏠 🌈

[Bananaman]

Semi-related: Here's a ticket (and comment) describing how to run all post-init hooks in a full class inheritance hierarchy with attrs:

https://github.com/python-attrs/attrs/issues/167#issuecomment-536244615

[Bananaman]

PS, for anyone as confused as I was about the end of the OP's post:

This has been discussed already, but some of the other issues (#33 #24 #38 #38 #58) which have raised the possibility of a post-__init__ hook have been somewhat conflated with better support for inheritance. As we all know, inheritance is deprecated and will be removed in python 3.7 so we don't need to focus on that.

^ That comment is a joke. Class inheritance is not deprecated in python and is not being removed. In fact the python docs at https://docs.python.org/3/tutorial/classes.html#inheritance say "Python classes provide all the standard features of Object Oriented Programming: the class inheritance mechanism allows multiple base classes, a derived class can override any methods of its base class or classes, and a method can call the method of a base class with the same name." and "Of course, a language feature would not be worthy of the name "class" without supporting inheritance."

Python is not removing class inheritance.

[glyph]

Python is not removing class inheritance.

The joke is only funny because it should, though.

Looks like we missed the boat for 3.7 but there's always Python 4!