Mixedrealitytoolkit-unity: vNEXT Task: Beta Checklist

[StephenHodgson]

Overview

Beta release checklist

Requirements

• Project Builds and Runs for all supported platforms and configuration types
  
  
  
  • No Build warnings
  
  
• All Project Unity Tests pass
• Release asset packages should import and install without errors/warnings

Acceptance Criteria

A list of the specific things that must be done for approval / acceptance of the task in checklist form.
This may correspond with the above requirements.

• [x] Fix #2981 Fix service registration failures.
• [x] Merge #2984 All Unit Tests pass
  
  
  
  • [x] Fix #2972 Fix unit tests
  
  
• [x] Fix #2706 Boundary return type invalid
• [x] Merge #2948 Build Window Layout Fixes
• [x] Merge #2988 fix for texture asset loading
  
  
  
  • [x] Fix #2987 Update asset references to not target the Asset folder in the root folder (e.g. Inspectors)
  
  
• [x] Merge #2951 Copy/Paste/Clone Profiles
• [x] Merge #2983 Enable XR Settings on import.
  
  
  
  • [x] Do #2975 Setup XR SDKs if allowed.
  
  
• [x] Merge #2993 Mixed Reality Editor Setting binary drop
  
  
  
  • [x] Fix #2956 by creating MixedRealityEditorSettings.dll
  
  
• [x] Merge #2992 doc cleanup
  
  
  
  • [x] Do #2991 Update docs and clean up empty folders
  
  
• [x] Merge #3001 Input System Profile tidy up
• [x] Merge #2999 Spatial Awareness fixes
• [x] Merge #3008 Fix null ref in controller finder
  
  
  
  • [x] Fix #3007 Null ref in controller finder
  
  
• [x] Fix Solver Demo Scene
  
  
  
  • [x] #2996 In-Between Solver
  
  
  • [x] #2997 Face Headset Solver
  
  
• [x] Merge #3024 CameraCache update
  
  
  
  • [x] Fix #3021 Configuration fails if no main camera tagged in scene.
  
  
• [x] Merge #3003 Mouse Input Fixes
  
  
  
  • [x] Fix #2754 Fix OpenVR Mouse Input
  
  
• [x] Do #2974 Create Getting Started Guide
• [ ] Fix #2960 Drag and Drop Handler
• [x] Create Staging branches
• [x] Update all feature branches with latest changes from mrtk_development
• [x] Test & fix any discovered bugs
• [x] Do #2941 Update version number
• [x] Create unity packages for mrtk_development and each feature branch

[davidkline-ms]

We should get a conference call / chat session to discuss the above list. There are some things that can likely wait until after the first beta release (we plan on updating the beta every 1 - 2 weeks as we make changes).

[StephenHodgson]

These are the items are needed if we want a successful beta release. We also agreed to most/all of these items in the last shiproom meeting.

Please discuss concerns here.

[davidkline-ms]

The last shiproom meeting discussed Spatial Awareness and bug fixes as being critical for beta. 2757 and Spatial Mouse were not discussed at all.

[davidkline-ms]

[StephenHodgson]

Not having mouse support by default in our apps is like shipping a computer without a keyboard.

[davidkline-ms]

Not having mouse support by default in our apps is like shipping a computer without a keyboard.

I don't see that. I have been developing mr/vr apps w/o mouse support for years.

What I was pointing out is that these changes were not brought up in shiproom when we agreed to take the spatial awareness merge and bug fixes for beta. Also, 2757 feels like a rather large item and i do not recall having discussions on the correct intelligence for it.

There will be more beta releases before rtw. We can address these as we go.

[StephenHodgson]

I'm not so sure spatial awareness should be merged in as it is largely untested.

[StephenHodgson]

I don't see that. I have been developing mr/vr apps w/o mouse support for years.

As your customer I am telling you have I have a legitimate business need to have this in beta.
If I fix it before then, I fully expect you to review, test, and merge it in before we release.

[davidkline-ms]

I'm not so sure spatial awareness should be merged in as it is largely untested.

Kurtis and I have been testing this feature branch for as long as it has existed.

[StephenHodgson]

Awesome, now let's get the rest of the community to test it before it makes it into development :)

But I think the priorities are getting bug fixes in first before we add new features that may (or may not) contain bugs.

[davidkline-ms]

Spatial has been on the radar and being actively communicated, developed and reviewed for weeks. I completely disagree that it is not a priority, especially when talking with customers.

[StephenHodgson]

I disagree, it adds more risk to the delivery.

[davidkline-ms]

new features that may (or may not) contain bugs.
Like an input selector?

[StephenHodgson]

We already agreed in the shiproom meeting that features were to be released in additional packages.

[davidkline-ms]

We agreed on package mgr and interactables where architecture was not approved. We all said the spatial merge was needed.

[StephenHodgson]

I don't remember that sorry. Let's stick to the plan and release features separately.

[davidkline-ms]

The plan (as you can see from the notes) was to merge spatial and fix bugs. That is what we will do.

Please discuss concerns here.

Also agreed upon was the ability to request a skype call to resolve these types of issues.

[StephenHodgson]

How about a compromise?

I'll drop the addition of the intelligent controller switching if we ship the spatial in a separate package.

[StephenHodgson]

There's loads of other important issues we need to get done before we can consider these two low priority items. Let's address those first and come back to it.

Priority:

1. Bug fixes
2. Features
3. Packaging

Does this sound reasonable?

[davidkline-ms]

We have had no design discussions or reviews on intelligent controller switching. We have had extensive on sptial.

[davidkline-ms]

It is HIGH priority for HoloLens customers

[StephenHodgson]

It is HIGH priority for HoloLens customers

Agreed, and they'll still be able to get it with the release package.

[StephenHodgson]

Bug fixes first.

[Yoyozilla]

I love the idea of working down a list. However, I have a question of how we decide what goes on the list and assigning priority.
I could give you my prioritized list based on the customers I am talking inside Microsoft, but this list may not necessarily align with the MRTK community asks.
Do we have a process of triaging and prioritization we could all follow together? I think a Skype call would be a good start :)

[Yoyozilla]

Also thanks Steven for writing up this list, I've been meaning to write a remaining work list but haven't gotten around to it.. Thanks for taking the initiative to get this started for us!

[keveleigh]

@Yoyozilla We also have an existing beta list we should make sure is up-to-date. https://github.com/Microsoft/MixedRealityToolkit-Unity/projects/12

[StephenHodgson]

Can we all agree that Bug fixes take a priority before features?

[Yoyozilla]

Depends on the severity of the bug.
If a bug is blocking development, yes we should fix.
If is a nice to have, I think feature > bug given we are still in alpha, and can address bugs/feedbacks in Beta.

I believe the point of alpha and beta release is to get features out to the community, have them try them and give feedback, so we can incorporate these feedback into the official release. In my mind, there is a different quality bar for Beta vs Release. As long as the key functionalites we want to test are there, we should prioritze getting the features out to the hands of our community.

Of course, we should always write good code!

[StephenHodgson]

But isn't that the problem we're having with flights of Win10? Too many features not enough bug fixes?

[SimonDarksideJ]

I'd have to agree that fixes must come before features @DavidKline-ms.

On spatial, the way I understood was that you were finalising the feature branch in the meeting, not merging into dev. That I feel should wait for the next beta drop as it could slow down the initial beta testing /release. As you say, beta 2 should be only a week or so after beta, which would be better than risking the first release.
I know you say its a high priority for HL users, but you yourself said it was incomplete and anything short of parity with the htk will have people questioning why to try Mrtk over htk's full feature. Sure, package it as an addon for now to test, but it seems premature to have it as a full Mrtk feature yet (imho)
Said a few times in shiproom, that we should package these "test" features separately and encourage people to try them.
This also came up in the sharing call, about the difference in a fully fledged component vs a wip feature (Sean's proton networking was used as an example)

As for the mouse discussion. I believe looking at the underlying bug, it was an issue you raised @DavidKline-ms if I read that correctly. @StephenHodgson was just providing the fix.

I think @yoyozilla, you are right and this Warren's further discussion on a call, if we can set one up for early next week, Mon if poss. So we can discuss and finalise the scope for the first beta. Ensuring there's enough time for testing and doc preparation for users to get cracking.

And I always write good code, just not code that works as I expected the first time 😅

(p.s. Writing this in bed as I'm falling asleep, just to try and balance the scales. Or did I just dream this?)