Problem-solving: The Problem Solving Process is not working

[lizmat]

We're now at 100+ open problem solving issues. There is no progress, there is hardly any discussion.

I'm trying to move things forward by just doing, but that is clearly also not working: https://github.com/rakudo/rakudo/commit/8b94d8d279

Having @jnthn being responsible for a decision on all tickets, is not working. To be clear, I'm not blaming @jnthn, I'm blaming the process.

We need to move forward on these issues. Consensus is hard, I know, but stagnation is killing.

[AlexDaniel]

Well, this ticket almost makes it sound like this repo is making things worse, when in fact it's the opposite. Despite all the naysaying, we did manage to resolve at least one issue that truly required some sort of structured consensus. Sure, it's not like this repo does magic and of course there were a bunch of other things that helped, but I think it played its small (but important) role.

One of the motivations for this repo was this ticket, and the process is working, it makes sure that similar situations won't happen again. Or so I thought. Now that I'm leaving for weirdly related reasons I'm no longer sure.

I don't think this ticket presents enough basis to suggest that the process itself is at fault. IMO you should simply discuss with @jnthn how different kinds of issues can be delegated and how people can make proper decisions on their own. As for why all things go to @jnthn, it's simply because we never had enough people specializing on things. Again, that's something you can discuss with @jnthn and fix.

Yes, I'm still listed under the meta label but that's temporary. There are just a few wording tweaks I have to commit, but that's it.

[niner]

Well I do think that the process needs improvement, too. problem-solving has become a bit of a dumping ground. It sometimes feels like "here's some issue, discuss!". Yes, the issues are real and they do need solutions.

The trouble is that Raku is a huge language. Not all parts are interesting to all people here or not at the same time. There are interesting questions about the meta model and native call and mathematics and network programming and some very specialized ones like finding a good way to get the highest n elements of a list. Personally I could contribute something to the pretty much all of these discussions. But I'm not even one of the reviewers and I've never asked to be put on the list. The sole reason for that is that I don't have the time and energy to dig into this diverse range of topics. I have barely enough time and energy to work on the issues that are important to me and my company right now.

Of course, jnthn can contribute to all of these issues as well and give his usual stunning insights and fabulous solutions. But how can we expect him to do so and keep working on the major structural changes our runtime needs?

My impression is that the problem-solving process currently maximizes the workload put on jnthn when it should "use" him as a resource as little as possible (but no less). It actually reminds me very much of the life as a manager at work. As there's someone in charge, it's much easier to delegate your problems upwards and let them find a solution and make decisions. But that just doesn't scale. It leads to burned out managers and disgruntled employees because they won't get service on the level they're expecting.

At work we've installed the 5-3-1 rule for that reason. If you need a decision from your manager, you describe the problem in 5 sentences, provide 3 options for solving and make 1 recommendation for which solution you'd prefer. The problem-solving repo lacks the solution part, so for each problem posted, jnthn would have to think about possible solutions first, which is loads of work and sucks much energy. Having the poster include 3 options shows that they considered the problem from different angles and reduces the cognitive load on the manager tremendously. Having 3 different ways to solve something already there makes it much easier to find other ways and to ponder the consequences.

More often than not it turns out that the decision is clear anyway and management insight is not even needed. Most of the time my answer is a simple "I follow your suggestion" because the process forced the employee to think things through and consider multiple angles. This process is actually empowering employees and encouraging growth. That's something I experience myself as a rakudo developer. Can't tell you how often I'd love to take the shortcut of asking jnthn how to solve some issue. But getting a hold on him is hard and I'm always aware that this would steal time from the work I'd also like him to do. So more often than not I'm forced to dig in myself and find my answers on my own and learn a ton on the way.

I'm absolutely positive that we need the oversight of a language designer and architect like jnthn and it's good to have him hin the loop. But for the process to scale the questions that reach him must all be the kind of "Is it OK if we make this change to solve these problems or would it destroy the language/architecture"? I.e. it should be about a final chance for a veto when given all the information in a digestible way instead of including him in broad discussions that have barely started exploring the problem space. Then and only then I can imagine the workload to be manageable for a single highly sought after person.

[lizmat]

To make this process more manageable, we also need to enforce the time constraints more quickly: if there is no reaction by 2 weeks, then the suggested way forward is taken by the committer. Still at the risk of being reverted, but moving forward nonetheless.

[patrickbkr]

Maybe one of the ancients of Raku language design is willing to join the effort?

@pmichaud @thoughtstream Raku development currently suffers from a deficit of language designers. There is is a need for a Raku language designer willing to regularly participate in, lead and rule on Raku language design discussions. Can you imagine taking such a leader role?

[lizmat]

I can imagine, but I don't think it will happen :-(

[jnthn]

My impression is that the problem-solving process currently maximizes the workload put on jnthn when it should "use" him as a resource as little as possible (but no less). It actually reminds me very much of the life as a manager at work. As there's someone in charge, it's much easier to delegate your problems upwards and let them find a solution and make decisions. But that just doesn't scale. It leads to burned out managers and disgruntled employees because they won't get service on the level they're expecting.

This reflects my feelings too, and thanks for putting it so plainly. The quality of the input I receive from this process is often too low for my job to simply be to "decide". For instance:

• Too often, folks aren't even bothering to clearly split problem and solution (#194 proposed a solution right in its subject, #187 the same, #196 puts the problem and solution right in the first post). This matters a lot, because bluntly, people are much better at spotting legitimate problems than they are at suggesting good solutions.
• If a change could have ecosystem fallout, it's very rare to see it quantified (for example, I requested further information about this in #177 and in #161, and didn't get it, but really anyone proposing a potentially breaking change should be proactively working this stuff out).
• Some issues don't really motivate why there's a problem to solve; for example, #146 suggests something to add, but doesn't illustrate a single ecosystem module that could benefit from it in the problem description. It doesn't explain why "use the feature via the regex engine" won't suffice. We already introduced the accident-prone contains by following this path in the past.
• Some, like #121, are along the lines of "it'd be cool if somebody did X", but making a problem-solving issue doesn't magic up additional resources to do stuff. #69 is much the same, and there are more.

Another issue is that there's a significant asymmetry in the time needed for somebody to make an issue and the time needed to try and work out a good way forward. Some issues raised here are genuinely difficult, and don't have any easy solutions. Others are obvious enough to me, but that doesn't always help; when I did issue a quick decision in #194, it ended up with a bunch more discussion and me needing to explain basic Raku semantics. That sure won't scale, but at the same time I like helping people to understand stuff. But really, taking the time to meant I didn't instead spend that time on a different issue here. Or was trying to look thoughtful to somebody new to the community more important that that? Maybe.

Also, things like this:

we also need to enforce the time constraints more quickly: if there is no reaction by 2 weeks, then the suggested way forward is taken by the committer

Imply movement in any direction now is better than movement in a good direction in 6 months time, and that movement in any direction now is risk-free, but it's not, because we're a production language with a userbase, and things often end up getting relied on quickly. Some of the hardest issues here are those that involve dealing with behaviors that we basically agree aren't good, but that are really hard to change without breaking people's working code, which we try quite hard to avoid. (And we are doing quite good at that, overall.) I think most people would consider me quite conservative on these matters, yet I've still been the target of at least one grumpy blog post when I did decide a change (all the way down in the MOP, on something we had no spectests for) was just one that had to be lived with. :-) Anyway, #196 is an example of something which is already in "if the ecosystem permits" status, and pretty much a case of: even if we don't find IO::Handle.new a useful thing, it may be in wide enough use we can't just drop it.

Another question is exactly what the lifecycle of an issue here should be, and we should define that better. For example, so far as I'm concerned, all the decision making on #149 is done, but there's still plenty of work to do on the implementation side. Ditto (though with much easier implementation work) for #73. So should we close it? Tag it in some way, so folks looking on already-agreed-on-stuff to work on can find it?

I'm absolutely positive that we need the oversight of a language designer and architect like jnthn and it's good to have him hin the loop. But for the process to scale the questions that reach him must all be the kind of "Is it OK if we make this change to solve these problems or would it destroy the language/architecture"? I.e. it should be about a final chance for a veto when given all the information in a digestible way instead of including him in broad discussions that have barely started exploring the problem space. Then and only then I can imagine the workload to be manageable for a single highly sought after person.

Yes, it's generally been the case in Raku that there is somebody playing the language design role and trying to ensure a level of consistency, and for better or worse that currently seems to be me. I hope we are at least doing a little better than if we'd just waited around to see if somebody else might show up to do the job.

One of the challenges for me is how to set expectations on others to make things scale better, in the context where others are volunteers. That's where the manager analogy breaks down. At $dayjob, I sure as heck try to make sure I give people the tasks they'll find most enjoyment in, but at the end of the day everyone understands that, well, it's a job, and sometimes we all have to grunt our way through the grunt work that we'd rather not have to do.

Ultimately, if this process is going to work better, then it will at least partly depend on asking others to do more to get changes they want to see through to a decision - not just in terms of better stating their own ideas, but also by evaluating the identified problems and solutions submitted by other folks, and bringing them to a concrete proposal where the relevant data/arguments (ecosystem impact, references to industry standards, arguments for/against potential solutions, consideration of performance impact, interaction with existing features, etc.) are to hand. In doing so, I suspect a bunch of issues will reach a consensus position (or consensus rejection) naturally anyway, in which case they really will need little more than oversight.

[jnthn]

I was curious how true this claim is:

There is no progress

Because I was sure that I'd spent no small amount of time considering issues filed here in recent months. It turns out that if I just count language ones (including a few Rakudo ones that really should have been marked language), a total of 21 issues were resolved so far this year. It's currently week 22, thus on language issues alone, it's moving along at 1 resolution a week. That ain't a blistering pace, but analyzing it as "no progress" is harsh at best.

[lizmat]

I'm going to think long and hard about the points made in @jnthn responses, and how they relate to my past, current and future involvement with Raku. In the meantime, I have closed all issues that I've added to this repo (except this one), so that one would not have to worry about them anymore.

[patrickbkr]

I'd like to give my view of the situation we are facing.

Language design is a difficult topic. When backwards compatibility is involved every decision brings a new liability that has the potential to be disadvantageous in the future.

We want to enable people to participate in Raku development. To do so it is necessary to support them in their endeavours. This includes leading the path and pointing out where to go and where not to go.

So it is necessary to aim for a well thought out solution in language design questions. This is work that takes time. It's not easy. It requires to think things through.

We are in a situation where there is a shortage of manpower to implement things and to design things. We have to be clever in distributing the work. We do not want people to burn out nor to be blocked.

I think an important thing to recognize is, that often people are blocked, because they think it's someone elses responsibility to work on a particular problem and no one else is allowed to. Examples are design issues, which "need to be put in problem-solving for some language designer to work out". People often do not realize, that it is very well allowed to do the design work oneself. I believe people are often willing to invest time given they understand they are allowed and encouraged and some good pointers on what needs to happen for a good design to emerge (research, look what other languages are doing, evaluating different solutions, ...). The most important bit is being very clear about what the next step is and who is responsible / allowed to do it.

I think failure to be clear about the above is what turned problem-solving into this "dump a problem and forget" pile it became. "We can't code up some solution and put it in rakudo any more, we now have to throw the issue in problem-solving instead and wait for jnthn to provide the design. Then I can implement that." That view is boosted by the fact that problem-solving issues are assigned to jnthn and that there is a perceived distinction between coding problems and design problems. People often think that a design issue is usually solved by some mythic designer just deciding. With that (wrong) assumption it's obvious why one thinks one can't help with such issues. "We are not allowed to. We were even told so! Design problems are not to be decided and implemented right away but put into problem-solving (so the mythic designer can decide). That's what problem-solving was created for."

To be fair, the problem-solving repo description is quite explicit about the process. But more than once people showed up creating normal bug reports assuming problem-solving is just the new place to report bugs. So clarity and making people understand is a weak spot.

Mentoring people into doing design work would be the optimum. We don't have the man power to do that at large. But the most important bit is making sure people understand what needs to be done and who is allowed to do it. We need to make extra sure people don't think jnthn (or whoever) is the only one allowed to do the work. That assumption leads to frustration as it then blocks people.

Let us try to be very clear, that there are limited resources and people are welcome to help and give pointers to how to help. The first step to resolving a bug is debugging it. The first step to resolving a design issue is researching it. People are invited and welcome to do so.

Our huge PR backlog is a problem that's blocking people, but that's a different story unrelated to problem-solving.

[AlexDaniel]

Actually, I'm reading this again with a fresher mind, and it makes me wonder…

We're now at 100+ open problem solving issues. There is no progress, there is hardly any discussion.

But is it really a bad thing? Many tickets in the problem-solving repo are asking to add more stuff to the language or modify it in very questionable ways. .child change that prompted the issue we're discussing right now is a good case in point. If we rushed to have all these tickets resolved, wouldn't it make maintaining the compiler even harder than it already is now? Keep in mind that we have over 1000 open tickets in rakudo, and another 1200 migrated from RT.

Maybe we're better off not resolving most of the language tickets in this repo for now. That is until we are able to change the direction and start thinking how to simplify the language instead of piling even more features on top. Either way my hope was that any solution would always resolve more than one problem, so having a backlog of tickets is a good thing for getting a better picture and coming up with something nice.

Yeah, I understand that not being able to change the language on a whim is less fun for the devs, but what would be best for the users?

[lizmat]

In light of #203 I'm closing this ticket. Please re-open if you disagree.