Ar.js: Markerless AR could be Possible with AR.js

[ks1505]

Hello Jerome love your work on ar.js, I want to know can ar.js first capture/ take a snap of the live camera feed from user wherever the user clicks and use it as a marker.? if this became possible then ar.js can work like a markerless ar while using a marker.

[janpio]

No. The marker not only gives the code a position, but also the dimensions/size and orientation. Thus simple clicking can never be a replacement for a marker.

[ks1505]

thats why the the first click will capture the image in a specific resolution just like a camera would giving the dimensions/size and orientation and then the object would be placed

[janpio]

AR.js does not support that.

[ks1505]

https://medium.com/chialab-open-source/how-to-handle-click-events-on-ar-js-58fcacb77c4

[janpio]

... which does not help you to solve your initial question at all.
But obviously you don't believe me, so good luck ❤️

[ks1505]

I just think its possible since its a script and after few(/many) tweaks the marker can be initialized as the image captured at the first click (PS-not contradicting that ar.js is marker based)
So the script can detect the captured image as marker and display the object upon it (as it should on a hiro marker)
In this way the object is displayed on the marker (according to the script) but the object is visible without a marker(according to the user)

[ks1505]

(PS- by importing other scripts too (if necessary) and thus not just limiting to ar.js)

[andreyrd]

Interesting idea, but the problem is that markers are not just arbitrary images. Markers need to be in a specific format. They MUST have the thick dark border that you see in the hiro marker. So unless the place your user taps happens to have a perfectly square black border, it will not work :)

[droid001]

@ks1505 You might want to want to take a look at how the image recognition works in more detail. If you have a look at a .patt file and you'll see what the recognition algorithm has to deal with. The pattern is rotated in 4 directions which is what has to be processed. This is a black and white image.
Now if you take a picture and try to use that as a pattern then you're up against finding recognisable features, changing perspective, parallax, lighting reflections and changes. That's when other technologies choices are of better use like SLAM. Apple had a good presentation of this at their WWDC 2018 conference.

[commentatorboy]

I made it without markers.
But the problem is that it very static and not on a plane. No anchor point.
Check it here: https://www.internet-guiden.dk/nomarker

[droid001]

Check it here: https://www.internet-guiden.dk/nomarker

I didn't get your example working. Tried with Chrome on Android and Safari on iOS. What are the specs needed for your demo to work and what's opposed to happen?

[commentatorboy]

Have the phone like the one in the picture. In portrait mode, and pointing straight forward.

Notice that there are no markers. Again the controls are a bit clunky. You have to tilt the phone up or down, and swipe left and right to make it show.

I used the following code:
https://jsfiddle.net/gwo2y4dn/

My specs are the following:
Moto e5. Android 8, Using chrome 73. I did test it on IOS but forgot the specs.

[andreyrd]

But that's not actually tracking anything. It's just creating an aframe VR scene which happens to have a transparent background and overlaying it onto the camera.

[commentatorboy]

That is my point. It is "static" on the camera feed like that. But still markerless.

[nicolocarpignoli]

@ks1505
Hi, as the other guys said, it is not possible, because markers have to created with particular images as input. Markers are .patt files, not images: a screenshot or simply a 'complex' image, with colors and some detail, simply will be processed and will result in a too complex .patt file, thus the camera, using the artoolkit system, will not recognize it.

You linked my article about click events (and thanks for your reading :) ), those are insights on how to click on the marker/3D model and add some interactions to the AR scene. For your question, it will be far more interesting to learn more about markers and how they are built. You can check my other article about markers, it will help you understand the marker thing better. :)

[ks1505]

@commentatorboy https://www.internet-guiden.dk/nomarker is working great ! although not markerless and static but does the job... @nicolocarpignoli I wonder if object rotion, moving object and pinch to zoom in or out could be made to work on this https://jsfiddle.net/gwo2y4dn/ (i.e the source code for https://www.internet-guiden.dk/nomarker).

This link https://jesstelford.github.io/aframe-click-drag-component/basic/ has done dragging of objects and works on the below code but uses aframe 0.3.0 version to drag objects and also works if the camera look controls are enabled (I have tried with look controls enabled for the camera)

[ks1505]

https://jsfiddle.net/ks1505/mz6gp34q/2/ here is the code

[ks1505]

https://jsfiddle.net/ks1505/mz6gp34q/3/ this is with camera look-controls.

[commentatorboy]

I do not think it would be a good idea to use the click and drag component since it works with 0.3.0.
But that is off-topic.
If you are interested in using click and drag, go to aframe's github and make an issue there.
If you believe or @nicolocarpignoli thinks that this question has been answered then please close it.

[hchris0]

@commentatorboy
I implemented a markerless scene in the same way as yours but I am having trouble with the object in Android 8.0 and up when using it in a WebView. The object always stays at the same place regardless of the phone's orientation. It works fine though in the browsers I tested (Firefox and Chrome) and it works fine inside the WebView in Android 7.0. Any ideas why is this happening?

[commentatorboy]

As I stated before. The controls are a bit wonky. You can tilt the phone up and down, but only swipe left and right. Use the code as a reference: https://jsfiddle.net/gwo2y4dn/
Again, this might be out of scope of this topic, and I do not know why that problem is like that.
The problem might lie in the controls that aframe provides.
Make an issue in the aframes github.

[nicolocarpignoli]

The discussion has gone out of topic. Anybody feel free to open an issue for any topic discussed here. The main one is solved: with AR.js is not possible - or at least, for now - to have markerless AR experiences.