Aframe: Videos and videospheres don't work on mobile

[staus]

I've tried both mp4 and webm format, but at least on my Galaxy S6, it shows up black.
It looks fine on desktop

[gtk2k]

Mobile browser can't autoplay(Require a user action to play).

[staus]

Thanks got it working by having a button at the beginning

[cvan]

Thanks for filing. I think this was reported on the Slack but not filed here. Playing videos on mobile requires a user-initiated event (e.g., click). It's on the "roadmap" (TBD in #299) to design and expose video controls for videos.

[matthewsimo]

@staus do you have a snippet you can share? Playing with this too and my attempts at a button to start have been unsuccessful so far.. thanks! Here is what I'm playing around with - http://codepen.io/matthewsimo/pen/yeaLxP

[cvan]

@gtk2k made a workaround using a component: https://gtk2k.github.io/aframe_videosphere/ (source: https://github.com/gtk2k/gtk2k.github.io/tree/master/aframe_videosphere)

I'll likely be implementing something similar in the interim.

[dustingetz]

How do I start the 360 Video stock aframe demo on mobile? https://aframe.io/examples/showcase/videosphere/

[cvan]

Investigate http://krpano.com/plugins/videoplayer/#iphonepanovideo

[ngokevin]

For iOS, need to set webkit-playsinline or else the video will open in a separate window.

For webkit-playsinline to work, need to pin the website to the homescreen.

[cvan]

If a VR page is pinned and launched from the homescreen (i.e., navigator.standalone), we can play inline videos.

To encourage users to do that, I want to show instructions in a modal/toaster similar to http://forecast.io/'s:

[marshworks]

FYI, if it's helpful: some meta tags are needed to allow for the home screen app:
< meta
name="apple-mobile-web-app-capable"
content="yes"
/>

<!---
    Once we hide the standard Safari chrome, we can also affect
    the way the status bar displays at the very top. By default,
    it's that light gray color, but we can change it to be black
    or translucent.
--->
<meta
        name="apple-mobile-web-app-status-bar-style"
        content="black"
        />
<script>

[cvan]

Interesting: https://github.com/misterburton/ideo-sf-vr-nerd-night#bonus-points

[mobiwu]

@https://gtk2k.github.io/aframe_videosphere/
No picture only voice, nexus 5 chrome for android 44.0.2403.133

[cvan]

@mattdesl brings us https://github.com/jam3/ios-video-test!

[mobiwu]

@cvan
THREE.WebGLRenderer: OES_texture_float_linear extension not supported.
three.js:31371 WebGL: INVALID_VALUE: texImage2D: no video
255192.168.137.1/:1 [.WebGLRenderingContext-0x9fbff200]RENDER WARNING: texture bound to texture unit 0 is not renderable. It maybe non-power-of-2 and have incompatible texture filtering.
192.168.137.1/:1 WebGL: too many errors, no more errors will be reported to the console for this context.

[ngokevin]

@mobiwu Have any more details than that? That might be a result of trying to do cross-origin videos on Safari, which I haven't found a way to support.

[mobiwu]

@ngokevin
A-Frame Version: 0.2.0
index.js:50 three Version: ^0.74.0
index.js:52 webvr-libs Version: ^1.0.4
metaTags.js:29 The key "shrink-to-fit" is not recognized and ignored.
webvr-polyfill.js:2784 Recalculating device params.
webvr-polyfill.js:2821 User agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5 Build/MMB29Q) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.133 Mobile Safari/537.36
webvr-polyfill.js:2822 Pixel width: 1920
webvr-polyfill.js:2823 Pixel height: 1080
webvr-polyfill.js:2850 Rule matched:
webvr-polyfill.js:2851 Object
webvr-polyfill.js:2786 New device parameters:
webvr-polyfill.js:2787 DeviceParams
webvr-polyfill.js:2757 Fetching DPDB...
webvr-polyfill.js:2765 Successfully loaded online DPDB.
webvr-polyfill.js:2784 Recalculating device params.
webvr-polyfill.js:2821 User agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5 Build/MMB29Q) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.133 Mobile Safari/537.36
aframe.js:52159 Pixel width: 1920
aframe.js:52160 Pixel height: 1080
aframe.js:52187 Rule matched:
aframe.js:52188 Object
aframe.js:52123 New device parameters:
aframe.js:52124 DeviceParams
aframe.js:50175 DPDB reported that device params were updated.
aframe.js:32515 THREE.WebGLRenderer 74
aframe.js:36425 THREE.WebGLRenderer: OES_texture_float_linear extension not supported.
aframe.js:38885 WebGL: INVALID_VALUE: texImage2D: no video
255192.168.137.1/:1 [.WebGLRenderingContext-0xaef19300]RENDER WARNING: texture bound to texture unit 0 is not renderable. It maybe non-power-of-2 and have incompatible texture filtering.
192.168.137.1/:1 WebGL: too many errors, no more errors will be reported to the console for this context.
aframe.js:52359 Invalid timestamps detected. Time step between successive gyroscope sensor samples is very small or not monotonic
a-scene.js:495 requestFullscreen() is deprecated on insecure origins, and support will be removed in the future. You should consider switching your application to a secure origin, such as HTTPS. See https://goo.gl/rStTGz for more details.

[ngokevin]

Summary from the docs:

iOS has a lot of restrictions on playing videos in the browser. To play an inline video texture, we must:

- Set the `<meta name="apple-mobile-web-app-capable" content="yes"` metatag.
- Set the `webkit-playsinline` attribute to the video element.
- Pin the webpage to the iOS homescreen.

[villalard]

Sadly none of these examples work on a Nexus 5 (Android 6.0.1) with Chrome for Android 49.0.2623.91. Just a black screen on every example.

[ngokevin]

Android needs user action to trigger video. We have work-in-progress to improve the UX for this.

[ngokevin]

More info https://github.com/mrdoob/three.js/issues/6606

[cvan]

We're spread thin. If people can figure out reliable patches, let us know :+1:

[villalard]

From further testing I have discovered that video textures work as expected on Android using the latest Firefox mobile browser (45.0.1) in non-VR mode but when you switch to VR it crashes the browser after 1 second. Nexus 5 (Android 6.0.1). Video textures in Chrome still show black screens for me.

Great framework by the way. Really like it.

[mobiwu]

@villalard me too.

[cvan]

From further testing I have discovered that video textures work as expected on Android using the latest Firefox mobile browser (45.0.1) in non-VR mode but when you switch to VR it crashes the browser after 1 second. Nexus 5 (Android 6.0.1). Video textures in Chrome still show black screens for me.

Which version of Chrome are you running? And can you provide the video source?

[villalard]

@cvan Chrome 49.0.2623.91
Video source: http://www.wearevr.tech/content/clients/concepts/vr_cinema/index.html

[SonarSystems]

@villalard Does that link work for you on Android devices because it doesn't for me.

[villalard]

@SonarSystems Hi, it works in Firefox for Android but not Chrome for Android. Version numbers are above in the thread.

[SonarSystems]

@villalard Thanks for the very quick response, it worked :D

I will try it in my own implementation tomorrow.

Do you know when Chrome will get fixed for this?

[cvan]

 Do you know when Chrome will get fixed for this?

I would track https://bugs.chromium.org/p/chromium/issues/detail?id=178297 - no time soon it seems

[SonarSystems]

@cvan Thanks for the link.

BTW I am working on a A-Frame tutorial series, should be ready within a week or 2

[SonarSystems]

@cvan @villalard Do you know if the videos have to be encoded or formatted in any particular way for Android, as when I play a video on Firefox on Android, it plays but when I click the VR Goggles button, it switches to VR mode for a second then crashes.

[villalard]

@SonarSystems I am having the same issue as noted above in the thread. Please excuse my simplistic answer but I think it's just that stereoscopic views and video textures is just too much for the browser to take at present.

[SonarSystems]

@villalard I thought that but the wearevr link you shared works on my Android phone in Firefox. Hence why I thought maybe the video is to big or a wrong format.

[villalard]

@SonarSystems I don't think it's an encoding issue because the video textures display in non-vr mode - I think that as soon as the second video instance gets created (stereoscopic) it just maxes out the memory allocation that the browser has set. I will try to bring the video filesize down to see if this helps.

Again, apologies for making general assumptions in a technical forum - my knowledge of what is going on in the background is very minimal.

[SonarSystems]

@villalard Thanks for the response, your feedback is much appreciated. It's weird because the other link works.

[cvan]

@SonarSystems if you can connect your device using USB to use remote debugging in Firefox or Chrome for Android, you get the console logs/warnings. if you have those, paste them here and those will be super helpful.

[SonarSystems]

@cvan Every time I debug using Firefox, when it crashes the console disappears on my computer because of the crash. Can I save the log some how?

[cvan]

@SonarSystems whoa, very odd. I'm not sure. @caseyyee, do you know? I know you're good with mobile debugging.

[villalard]

@villalard Thanks for the response, your feedback is much appreciated. It's weird because the other link works.

@SonarSystems I have re-encoded the videos in my test (http://www.wearevr.tech/content/clients/concepts/vr_cinema/index.html) but the crash still occurs when going into stereoscopic. I have tried all the other examples listed on this link and they all crash Firefox when going into VR mode. Can you tell me which example you are talking about that you say works?

[SonarSystems]

@villalard Well they were all working but now when I go onto it none of them work. Could you send me the original encoded files and I will try them please. You can send them to [email protected]

Cheers

[Chocap]

Hi everybody,
I am also trying to make a VR video working on mobile (iOs and Android) with videosphere, however as I am a very beginner in this field, I don't quite understand, I got that I have to set this metatag and also set the webkit-playsinline, but how do we do those?
I mean, where do I paste the metatag and what do I write to implement this webkit-playsinline?
And I don't understand what "Pin the webpage to the iOS homescreen" means... As you see I do not understand a lot....any help would be soooo welcome!
Thank you very much in advance, I am so lost!

[cvan]

@Chocap Safari for iOS provides an option to pin a web page to the iOS homescreen; see this comment above for an image showing it

There's actually a whole open-source library that adds a modal that makes it more obvious:

http://cubiq.org/add-to-home-screen
https://github.com/cubiq/add-to-homescreen

I considered it, but I'm thinking it'd be overkill to add to A-Frame. Someone could create a third-party A-Frame component (see this curated list of community components).

[mobiwu]

This problem didn't solve before,Not going to study aframe.

[Chocap]

Thank you for your contribution, the video can be played however I am not able to have it displayed in a double screen for VR... I would say it is the webkit-playsinline which does the work?
Does somebody know the code to implement the webkit-playsinline?
Thanks!

[cvan]

Does somebody know the code to implement the webkit-playsinline?

Since #1179, with the latest version of A-Frame (0.2.0 or master), we automatically add this to all videos.

(I've filed #1373 to clarify the docs.)

[jacobpdq]

Here, take this... it should help you on your way:

http://bfred-it.github.io/iphone-inline-video/demo/

[cvan]

Here, take this... it should help you on your way:

http://bfred-it.github.io/iphone-inline-video/demo/

you have to pass the video element or a canvas texture to WebGL. https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/Tutorial/Animating_textures_in_WebGL so that means this iOS trick, which actually just seeks the currentTime, doesn't do anything.

https://github.com/bfred-it/iphone-inline-video (demo) works for only 2D/3D CSS stuff, not WebGL stuff. it's a sweet trick nonetheless.

read https://mobile.twitter.com/feross/status/715826759032287232 for more info.

[ngokevin]

• https://github.com/mayognaise/aframe-video-shader
• https://github.com/mayognaise/aframe-gif-component

[cvan]

@ngokevin this is awesome. great work, @mayognaise - thanks!

I'm glad we now have a (third-party) component for iOS video now. now, we just need a component to enable video autoplay playback (via a user gesture such as a click) for Chrome for Android. since I added support for webkit-playsinline in c6aad59, and we've filed #1008 for adding a help message for "Add to Homescreen" for iOS, IMO we ought to now close this issue.

I could be convinced to add to A-Frame support for autoplay playback for Chrome for Android via some "Play" button if we detect any videos in the scenes on the page. But, iOS inline playback? The hacks (such as this one) work, but the performance leaves a lot to be desired. And I would not want to ship A-Frame with suboptimal workarounds that degrade video performance.

I am almost done with a PR for Firefox for iOS that will enable inline video playback: https://github.com/mozilla/firefox-ios/pull/1651

When it merges and is shipped in the release version of the browser, I'll be so happy :smile:! And we can talk to Apple and Google to try to get them to do the same with their Safari and Chrome browsers for iOS. But, that's the best we can do.

This is not an A-Frame problem. It's a restriction with Safari/Chrome for iOS. (And, although it's less than ideal, having users pin a site/webapp to the homescreen is not the end of the world.)

I understand the desire for out-of-the-box inline video playback for iOS, but A-Frame should not bend over backwards for this.

[mayognaise]

http://bfred-it.github.io/iphone-inline-video/demo/
This is nice! I'll look into it so that it will solve any issue with the current video-shader.
right now it won't show if the video is big or something.
Also hope FF will be the best browser for iOS 👯

[kylehikalea]

Any news on this? Pretty big use killer for Cardboard

[cvan]

Any news on this? Pretty big use killer for Cardboard

You can read my comment above

[kylehikalea]

I could be convinced to add to A-Frame support for autoplay playback for Chrome for Android via some "Play" button if we detect any videos in the scenes on the page

I guess this is what I'm hoping for. Are we under the impression there will never be a way to run videosphere automatically on page load without hacks? Sorry, new to js.

[gnestor]

Has anyone had any luck getting videosphere to work in a iOS WebView? I have:

1. Enabled allowsInlineMediaPlayback in the WebView
2. Added the <meta name="apple-mobile-web-app-capable" content="yes"> tag and webkit-playsinline property to the video element
3. Tried both autoplay=true and autoplay=false

I've also tried to add the videosphere demo on the A-Frame website to the homescreen and open it and I still can't see the video.

[Chocap]

so far, I had not luck on my side :-(
so if I sum up with all of what has been said, the code to implement would be something like that? :

<meta charset="utf-8">
<title>360 Video</title>
<meta name="description" content="360 Video — A-Frame">
<meta name="apple-mobile-web-app-capable" content="yes">
<script src="aframe.min.js"></script>


<a-scene>
  <a-videosphere src="draft3.mp4" controls webkit-playsinline ad-outlet="video" autoplay="true" loop="true" rotation="0 180 0"></a-videosphere>

</a-scene>

let me know!

[gnestor]

The webkit-playsinline property should be applied to the video element.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>360 Video</title>
    <meta name="description" content="360 Video — A-Frame">
    <meta name="apple-mobile-web-app-capable" content="yes">
    <script src="aframe.min.js"></script>
  </head>
  <body>
    <a-scene>
      <a-assets>
        <video id="video" src="test.mp4" loop webkit-playsinline></video>
      </a-assets>
      <a-videosphere src="#video" rotation="0 180 0"></a-videosphere>
    </a-scene>
  </body>
</html>

[Chocap]

Since I am on IOS, I pin it to the home screen and it works!!! (although I have no sound, I can display the image in double for the Google cardboard and the video plays by itself) Thank you!

[cvan]

I've filed the last remaining issue (IMO): #1463 - enabling autoplay of sounds for iOS/Chrome for iOS + Chrome for Android. Other issues have been filed and fixed.

If you have any questions about iOS inline video playback support, please read this comment: https://github.com/aframevr/aframe/issues/316#issuecomment-212608703

If you have any further questions or comments, hop on over to Slack or Stack Overflow.

Thanks, everyone!