Library: iOS use always the front camera

[stevesum]

Hi,

We try to use the zxing-js on iOS 12 with safari, but always show the front camera (independent you choose any of the videoInputDevice's)

[stevesum]

The reason is simple:
navigator.mediaDevices.enumerateDevices() gives back the media devices, but they deviceId is an empty string.

Therefore if you call the decodeFromInputVideoDevice() with any of them it will choose the first camera.

You have to check the device id not only for undefined but empty string also.

[odahcam]

You sure you have the right permissions?

[stevesum]

Yes. It is asked for the permission and I gave it.
And if I call the decodeFromInputVideoDevice(undefined, 'video') then works

[Matth--]

I am having the same issue. Problem started last week. @stevesum have you already been lucky in finding a solution?

[PabloAH]

Same here, I tried replacing the deviceIds that are "" with plain undefined to decodeFromInputVideoDevice hoping it'll use the "facingMode" constraint but still not working (I don't really have an iPhone to debug more)

[YouriHugoBos]

Same issue for me, tried a lot of things to change the facing mode, no luck.. I dont really know if this is an issue with zxing, the https://webrtc.github.io/samples/ examples also have the same problem with the tablet i am using (iOS 12.2).

[samuelting]

Observation

Hi there, I have a testing iPhone and come across the same issue. Maybe I can help in this issue.

The root cause of issue is because iOS and Android handle the official HTML5 API call navigator.mediaDevices.enumerateDevices() differently, which is called by the method getVideoInputDevices of BrowserCodeReader.ts Line 90.

In iOS, the hardware device info will not be disclosed until the user accept camera access request.

After the user accept the access request, I can fetch the device id.

Later when you call decodeFromInputVideoDevice, zxing-js calls official HTML5 API call navigator.mediaDevices.getUserMedia and pass empty string deviceId as its arguments. It will return a available camera, (not necessary the rear camera that you always desires).

Ref: https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia

Android Case

In Android, i can view the deviceId of my camera no matter whether camera permission is granted or not.

A thought on the use case

@PabloAH I have tried settings on my iPhone, by setting deviceId as plain undefined, I can force zxing-js to open camera in "environment" facing mode. If your application does not allow the user to choose which camera to use, it is an option.

Remarks:
Testing Devices Model
Nexus 6 (Android 7.1.1)
iPhone SE (iOS 12.2)

My testing code to fetch deviceId for your reference:

navigator.mediaDevices.enumerateDevices().then(videoInputDevices => {
        videoInputDevices.forEach(device =>
            console.log(device)
        );
    })

[PabloAH]

Thanks @samuelting for the information!

I don't have an iPhone yet to confirm but the sequence makes sense in Android, I just request "undefined" at first, and once the camera is accepted (there is no event in the library but you can use $("#video").on("playing")) I go and get the list of specific cameras to add the drop down. At this point iOS should behave like Android in providing deviceIds.

So this is likely not a problem with the library, but just a correction has to be made on the samples.

[flemx]

Thanks @samuelting , initiating with undefined works for me for using the back camera.

Another workaround which I was using before, was to call the native navigator.mediaDevices.getUserMedia function first so the camera access can be accepted, after which the deviceID can be set using "navigator.mediaDevices.enumerateDevices()".

The "decodeFromInputVideoDevice" can then be initiated with the deviceID, this could be a workaround if you have a requirement for switching between cameras.

           let selectedDeviceId;
           navigator.mediaDevices.getUserMedia({video:true})
           .then(function(stream){
                console.log('Stream1 started with success');
                setDevice();
            })
            .catch(function(){ 
                console.log('Failed to start stream1')
            });

            setDevice();
            function setDevice(){
                navigator.mediaDevices.enumerateDevices()
                .then(function(devices){
                    console.log('Getting device');
                    console.log(devices[1]);
                    selectedDeviceId = devices[1].deviceId
                })
                .catch(function(){
                    console.log('getting device failed');
                  })
               }

[odahcam]

Well, enumerateDevices was precisely chosen because it used to allow us to get the devices info along with their ID without asking permissions, looks like that not anymore. Why? because the component could load all the available devices before bothering the user with permission modals.

The solution I propose is to forget about enumerating devices and properly ask the user for permissions before using any.

Also, I noticed that every time I switch cams in MacBook, the scanner has to ask for permissions again and again and that didn't happened before too. That's another thing I wanna solve.

[odahcam]

As posted here, this may be a workaround:

  ngOnInit(): void {
    this.scanner.camerasFound.subscribe((devices: MediaDeviceInfo[]) => {

      this.availableDevices = devices;

      // selects the devices's back camera by default
      for (const device of devices) {
          if (/back|rear|environment/gi.test(device.label)) {
              this.scanner.changeDevice(device);
              this.currentDevice = device;
              break;
          }
      }
    });
  }

[pepeh]

@odahcam
How to use this fix with this example https://zxing-js.github.io/library/examples/qr-camera/ ?
Or it should be some different library?

[markestocapio]

@odahcam

Thanks for the update. Since I don't need the user to choose between camera source. I manage to force using the back camera. But I still need to check if currentDevice.deviceId = undefined;

For now, this workaround works for us. Thanks

[github-actions[bot]]

Stale issue message

[odahcam]

@pepeh

// on line 121 we have...
      codeReader.getVideoInputDevices()
        .then((videoInputDevices) => {
        // so you just need to filter available devices on the first line and it shall help you understand
        videoInputDevices = videoInputDevices.filter(device => {
            if (/back|rear|environment/gi.test(device.label)) {
                return device
            }
        })

[arthurmmedeiros]

I did something that I thought would work very nice, but after reading this post it seems like it's never going to work.

I created a component to let the user select the camera, then I stored the deviceID on Redux. When I open up the camera, it always shows only the front camera.

The reason I did that is because I'm trying to avoid reendering. I'm getting this warning in my browser console at least three times:

BrowserCodeReader.ts:435 Trying to play video that is already playing.

Even doing what you guys are suggesting I was never able to open the back camera on my Iphone 6. Also, the front camera is not scanning any code on my iphone, only on my lap top.

Anyone facing something similar?

Here is the code I've been working on

const constraints = {
  video: true,
  audio: false,
};

interface IOwnProps{
    handleInputStudentId: () => void;
    title: string;
    description: string;
    deviceId:string;
}

const KioskCameraMode: React.FC<IOwnProps> = ({
  handleInputStudentId, title, description, deviceId,
}: IOwnProps) => {
  const [video, setVideo] = useState(document.querySelector('video'));
  const [scannedResult, setScannedResult] = useState('');
  const [isOpen, setModal] = useState(false);
  const [isCameraSupported, setCameraIsSupported] = useState(true);


  const multiReader = new BrowserMultiFormatReader();

  useEffect(() => {
    console.log('deveria ser component did mount');
    if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
      // Good to go!
      setVideo(document.querySelector('video'));
    } else {
      setCameraIsSupported(false);
      alert('getUserMedia() is not supported by your browser');
    }
  }, []);


  function streamVideo() {
    console.log('stream only once');
    if (video) {
      navigator.mediaDevices.getUserMedia(constraints).then((stream) => {
        video.srcObject = stream;
      });
    }
  }
  useEffect(streamVideo);

  function callbackResult(result: any, err: any) {
    console.log(result);
    // console.log(err);

    if (result !== null) {
      multiReader.reset();
      handleCheckIn();
    }
  }

  function decodeNow() {
    if (deviceId && !isOpen) {
      multiReader.decodeFromInputVideoDeviceContinuously(null, 'video', callbackResult)
        .then((res: any) => {
          console.log('called then');
        })
        .catch((err: any) => console.error(err));
    }
  }

  useEffect(decodeNow, []);



  return (
    <>
      {
        isCameraSupported
          ? (
            <>
              <h1 className="uni-title-h1 uni-blue">{title}</h1>
              <h2 className=" uni-title-h2">{description}</h2>
              <div className="kiosk-camera">
                <div className="camera">
                  <CustomTicket color="#0B5D90" classNameCustom="custom-ticket-camera-box" />
                  <section>
                    <section className="camera-section">
                      <video
                        autoPlay
                        playsInline
                        id="video"
                        width="300"
                        height="200"
                        muted
                        style={{
                          top: '0px',
                          objectFit: 'cover',
                          width: '100%',
                          height: '100%',
                          position: 'absolute',
                          left: '0px',
                        }}
                      />
                    </section>
                  </section>

                </div>

            </>
          )
          : <NotSupported />
    }
    </>

  );
};
export default memo(KioskCameraMode);

[odahcam]

@arthurmmedeiros você percebeu que, nesse trecho de código, em nenhum momento você está passando o deviceId para a biblioteca?

Você deveria tentar algo como:

function decodeNow() {
    if (deviceId && !isOpen) {
        multiReader.decodeFromInputVideoDeviceContinuously(deviceId, 'video', callbackResult)
            .then((res: any) => {
                console.log('called then');
            })
            .catch((err: any) => console.error(err));
    }
}

[arthurmmedeiros]

@odahcam eu retirei o "deviceId" pra testar o que ele faria quando envio "null" no parâmetro. Eu havia criado uma tela antes da câmera pra selecionar o deviceId, guardei na global store e abria a câmera. Isso nunca funcionou. Você tem uma forma de selecionar a câmera que funciona legal?

[odahcam]

Bom, vou te mandar um exemplo pra ti ver uma forma de se fazer.

Tenho esse cara em Angular que faz basicamente isso: https://stackblitz.com/edit/zxing-ngx-scanner
Ele roda em cima dessa biblioteca: https://github.com/zxing-js/ngx-scanner

Se a gente olhar no código-fonte, eu seleciono as câmeras de uma forma completamente customizada:

https://github.com/zxing-js/ngx-scanner/blob/a28b478530e9ed8b94b35ae0aa534b84385de5ce/projects/zxing-scanner/src/lib/zxing-scanner.component.ts#L688-L705

Eu reescrevi isso tudo nessa nova biblioteca porquê na época essa lib aqui não funcionava bem, mas se você ler o código vai ver que é basicamente a mesma coisa que tem aqui hoje, eu copiei de lá pra cá.

Pelo que eu entendi você quer algo tipo isso (exemplo bem simples): https://stackblitz.com/edit/zxing-scanner-react?file=Hello.js

[arthurmmedeiros]

Bom, vou te mandar um exemplo pra ti ver uma forma de se fazer.

Tenho esse cara em Angular que faz basicamente isso: https://stackblitz.com/edit/zxing-ngx-scanner
Ele roda em cima dessa biblioteca: https://github.com/zxing-js/ngx-scanner

Se a gente olhar no código-fonte, eu seleciono as câmeras de uma forma completamente customizada:

https://github.com/zxing-js/ngx-scanner/blob/a28b478530e9ed8b94b35ae0aa534b84385de5ce/projects/zxing-scanner/src/lib/zxing-scanner.component.ts#L688-L705

Eu reescrevi isso tudo nessa nova biblioteca porquê na época essa lib aqui não funcionava bem, mas se você ler o código vai ver que é basicamente a mesma coisa que tem aqui hoje, eu copiei de lá pra cá.

Pelo que eu entendi você quer algo tipo isso (exemplo bem simples): https://stackblitz.com/edit/zxing-scanner-react?file=Hello.js

Era esse exemplo mesmo que eu precisava @odahcam valeu!!!

[odahcam]

Nada, boa sorte. 🖖

[jasoncapriati]

Hello!

mediaDevices.enumerateDevices() returns an array of _connected_ MediaDeviceInfo objects which has properties like deviceId, kind and label.

Safari and iOS Safari do not support this spec [1] yet.

[1] https://developer.mozilla.org/en-US/docs/Web/API/MediaDeviceInfo

[odahcam]

Well, that's new. I have never notice that and had no idea that was happening, maybe because the script had a very weak flow in some points. I hope to have time to improve this, this chunk of code also needs automated tests asap.