Mixedrealitytoolkit-unity: Play audio on gaze

[paulcanning]

I'm looking to have some my object AudioSource components start playing when they first gazed at (I actually need to figure out something better, because I need to place them in the spatial mesh first, then, later on, have the audio start, but first I need to fix this issue!).

How do I do a raycast from the camera (head position) to the various objects in the scene (that already have colliders available, as they use TapToPlace)

I have tried the following, but the audio starts right away (audio source property "play on awake" is off)

public class PlaySoundOnGaze : MonoBehaviour
{
    protected SpatialMappingManager spatialMappingManager;

    private bool isAudioPlaying = false;

    void Start()
    {
        spatialMappingManager = SpatialMappingManager.Instance;
    }

    void Update()
    {
        if (!isAudioPlaying)
        {
            Vector3 headPosition = Camera.main.transform.position;
            Vector3 gazeDirection = Camera.main.transform.forward;

            RaycastHit hitInfo;
            if (Physics.Raycast(headPosition, gazeDirection, out hitInfo, 30.0f, spatialMappingManager.LayerMask))
            {
                Debug.Log("Playing audio...");
                GameObject child = gameObject.transform.GetChild(0).gameObject;
                child.GetComponent<AudioSource>().Play();
                Debug.Log("is playing? " + child.GetComponent<AudioSource>().isPlaying);
                isAudioPlaying = true;
            }
        }
    }
}

I _think_ the problem is that I'm raycasting into the spatial mesh? The raycast code was lifted out of the TapToPlace script.

How do I make it so the cursor has to be on the object for the raycast to be true?

[killerantz]

Will he input system and gaze manager work? Create a behavior that implements the IFocusable interface and add the functions OnFocusEnter() and OnFocusExit(). Inside the OnFocusEnter() is where the audio should start playing. The gaze manager will handle the raycasting and hit detection, thought it will stop at the parent collider.

[genereddick]

Per @killerantz, OnFocusEnter OnFocusExit is what I would use.

If you wanted to keep your Raycast code, I see you are using the spatialMappingManager.LayerMask. You could create a new layer and use that in its place. Then the Raycast would only hit if it has the correct layer.

[paulcanning]

@genereddick I see your point about layer masks, but could it not work with stuff on the same layer?

Surely there should be a way to figure out if you are hitting different objects in the same layer?

[genereddick]

You can use Physics.RaycastAll (https://docs.unity3d.com/ScriptReference/Physics.RaycastAll.html)

Then loop through all the hits to find the one want. RaycastAll doesn't return them in any order -- so the closest one isn't necessarily the first in the array -- so you have to check by some other criteria (by name, type or whatever).

[jessemcculloch]

I second @killerantz suggestion to use the input manager, and implement the IFocusable interface and use that for your events. This is the recommended way to do all input management at this point (Click, Focus, Manipulation, Navigation, etc)

[jwittner]

@paulcanning I'm closing this as it seems you've got an answer to your question. If not let me know and I'll reopen.