Triplea: Glitch (Bug) Reporting - Where? Who?

[DanVanAtta]

Following up from testing group forum thread: https://forums.triplea-game.org/topic/268/volunteers-needed-early-release-testing-group/13

The question is raised: where to report game glitches? Candidate choices are to keep using github issues or to use the nodeBB forum.

---

Requirements from a game dev point of view:

• Need to be able to see which bugs are open and be able to 'close' bugs to remove them from that view. Ideally would be able to track status as well (assignee + in-progress)

---

Problems to Solve

• Lack of process clarity for bug reports, where to report them.
• Lack of decisive bug reporting tool/instructions. We're close to having this, but the bug reporting links are not all working across all game versions, the instructions are not necessarily decisive with a simple and minimal: "this is how you submit bug reports".

---

Some Considerations

• Minimal process overhead, we don't want anyone responsible for hand moving content. Ideally we would be able to enlist additional support and not just rely on admins alone for managing bugs.
• Tooling integration, github issues and PRs link to each other

---

To discuss: if github issues should be the final home for where we track bugs, or if we want to consolidate to the NodeBB forum, and we should discuss the pro's/cons and implications of the migration so we know we are doing something worthwhile.

[RoiEXLab]

I believe it's easier for a user to report something to the forum. A Github account is very uncommon to have if the user is not a developer or related to development in any way, while users can sign up to the forum using their twitter or Facebook credentials.
I wouldn't mind having users report bugs and feature requests to the forum and moving them to github issues if they are not already present, although I must admit I haven't been very active on the forum in the past months.
Somebody would have to regularly check the forum for such reports manually - a dedicated category would be very helpful, but still this would be additional overhead.

[DanVanAtta]

Keeping options open, I was looking more at how we could use forum for bug tracking. NodeBB does support tags (I just enabled it for the navigation bar). We can create an "open-bug" and a "fixed-bug" tag to add/remove as appropriate to bug threads. That would give us the critical open/close filtering capability.

[DanVanAtta]

Given tags exist on NodeBB, it is feasible to migrate. I re-wrote the overview, we should answer if we do, and I think be decisive about moving to one or the other (partial hybrids

I re-wrote the overview - given tags on NodeBB it is more feasible we could move bug tracking out of Github issues to NodeBB.

I like the idea that we could enlist forum moderators to help with bugs, potentially training them to do triage, fix verification and repro step minimization. If it works, that is quite the advantage considering Github issues requires write permissions to do issue tagging.

[ron-murhammer]

I think to some extent we need an area in the forum for reporting bugs, issues, etc. I think once we decide there is a true 'bug' that requires some kind of technical change then we open a github issue and put some kind of tag or edit to the original forum post so we know its being tracked in github now for the official fix.

[DanVanAtta]

I've been trying to find other projects that use forums for bug tracking, NodeBB itself has an interesting approach with the first post clearly directing people to github: https://community.nodebb.org/category/6/bug-reports. We could do similar, and also lock the bug reporting section as it does not look like it was 100% successf for nodeBB.

[DanVanAtta]

It's pretty important I think we get this right, whatever we decide we will have to live with for some time (5~10 years+).

I do get that people want to go back to the SF bug tracker plus forum type of setup, that everyday users don't want to see the technical posts. Looking at how well it worked in practice in practice, I'm not convinced entirely it was that effective of a model:

• bug tickets spiralled out of control, ticket queue grew faster than it could be worked on
• bugs were not always moved over to tracker, the reports lingered in forums and users gave up on reporting them.
• often technical notes were kept in forum posts, stack traces or line numbers. I'm not sure what extra value was gained from having yet another ticket in SF bug tracker. It's another place where communication would get lost, potentially doubling our work so we need to summarize status in 3 similar yet different places, now 4 with PRs (commit comment, PR overview comment, ticket commentary, forum commentary).

If we funnel users to one place or the other, I really think we gain a lot in terms of efficiency and simply not making our lives more difficult than need be : )

It seems workable to have one person move bugs from forum to github issues, but I'm not sure that would work in an optimistic year or two with a larger team and testers thrown into the mix.

Thinking to bug reports in Github issues and how that has gone so far - on the whole we've been very responsive and that has been excellent. Our follow up has been quite good. I do think that is largely because we do not have to cross reference the "technical" tickets with a forum post. We can tag forum posts now, I think that gives us the option now to maybe track bugs in forum, but I'm not convinced that would make a hybrid forum + bug tracker model that more successful (the problem is in the two lists and the need for human labor to manage and those, tags does not fix that).

Summarizing, it seems possible now to track bugs in forums alone, which is new. But, if we go back to a hybrid approach with additinal human labor and coordination requirements, I think we should do so very consciously and because of compelling reasons.

I would suggest to move forward we identify if it is an option to even use forum as the one bug tracker. If we rule that out then we can focus on whether to use github issues alone as a bug tracker, or do a hybrid model.

[DanVanAtta]

TBH I'm newly very concerned now that we have feature requests back in forums. It seems it has opened the flood gates for bugs to be reported there. There never was any change in how bugs were to be reported, yet there they are now appearing in the forum. We've very casually reverted back to an old process that was overly labor intensive and frankly broken. I'm tempted to say we should move official requests and reminders for the game engine to change to be back in github issues if it means there is no confusion on where bugs should be reported.

I really do not want to see us having admins hand manage bug reports and moving them, it did not work well when we did that for nabble forums + SF bug tracker.

[DanVanAtta]

Hmm, few extra thoughts, we had the hybrid process pretty much exactly when we first transitioned to github. People were using nabble forum to report problems and we were cutting tickets. I really do not recall that working out that well, we got buried and more problems were reported in forum than we could create tickets. The link between the two was always weak and just higher overhead/confusion. I do recall spending full days simply paying overhead tax reconciling status and updating the same topic in multiple places.

[prastle]

I suck at Tech ;) but i am sure i can find some mods to use your tags if you set it up in forum for the regular users to be able to report easily. MOST DON'T WANT MULTIPLE SITES ETC! Thus they don't bother to post. They just complain and/or quit.

Also @DanVanAtta I dont think you should change how git hub works. Two sites will be fine. We just need the ability for the regular guys that play live to be able to easily report issues without having to join github. The forum mods can tag issues.

Edited in ... BTW to explain further its the Regular Players that will find your glitches. They are the ones that play daily.

Just my two cents worth.

[DanVanAtta]

play live to be able to easily report issues without having to join github.

Would be great to resolve that problem, it is what we are running into. It does seem there is less resistance to creating a NodeBB login...

TBH, ideally we woudl build in to the game a bug reporting software, then any crashes would pop-up ina dialog and be sent automatically and recorded in lobby DB for us to look at. Then it'd not matter too much if users report issues to forums and not directly to bug tracker.

Edited in ... BTW to explain further its the Regular Players that will find your glitches. They are the ones that play daily.

Indeed. I suspect a few conversations/documents on how to best report bugs and where, and it won't really matter where we track bugs. It would make sense in that framework to get the players to submit directly to bug tracker for minimized overhead.

So ideally we can convince players it is easy and worthwhile to submit directly to our bug tarcker (github issues). Failing that, we need to look at the next best options short and long term.

[prastle]

So can the tags auto show in git hub if a player reports something in the forum?

Second as you mentioned in "lobby login no password needed thread" wouldn't it be easier if we could achieve the goal of one site? One site one login?

Or is github impossible to achieve in the new forum? If that is the case then Github has to stay!!

[RoiEXLab]

@prastle It would be possible to share the login between the lobby and the forum in theory.
The problem is that this would require quite some reworking of the lobby software (which is planned on the long term anyway), so we should definitely keep that in mind.

[DanVanAtta]

So can the tags auto show in git hub if a player reports something in the forum?

No, but it does allow you to filter/search for tagged threads in forum. EG: https://forums.triplea-game.org/tags/improvements

So we can add/update/remove tags to help us find open bugs.

Second as you mentioned in "lobby login no password needed thread" wouldn't it be easier if we could achieve the goal of one site? One site one login?

Not quite, we need the capability to mask passwords and save them. If we further consolidate user databases, the switch ideally can happen behind the scenes.

Or is github impossible to achieve in the new forum? If that is the case then Github has to stay!!

It is interesting, the new forum has tags which gives us most of what github provides. The old forum did not have tags, so the ability to see open issues and 'close' them was not there. The only thing forum does not have is the ability to hard 'close' an issue, but we can fake it by using a "Fixed" tag.

[panther2]

Being used to bug-reporting on Github I am not a fan of using a forum for this purpose instead.
But I see that users not familiar with Github prefer to use a forum, in case they don't want to "learn" Github.

My experience is that sometimes "bugs" are not really "bugs" but misinterpretations of rules or incorrect settings or other OS/Java-related aspects. So these cases can be resolved "quickly" using the forum.

So my vote would be to keep Gihub issues and to use the forum as "add on".

[DanVanAtta]

Closing for now, not sure if we came to much of a resolution, but it does seem clear there is not appetite to start tracking bugs in forums. Much process is still a bit new, it's probably worth while sticking with what we have for a bit longer and re-evaluating down the road.