Library: QR Decoding Issues

[majestic84]

This library seems to be the best option available from everything I have tried. For some reason though, I had a test user demo this feature in our application and the sample they submitted for decoding does not work, but then I took a screenshot of their sample and (oddly) the screenshot does work. I noticed that the screenshot is a PNG while the original is a JPG, so I thought, ah I will just convert JPGs to PNGs on the fly -- but that didn't help. I should also note that I _am_ able to decode other JPGs, but I can't figure out what it is about this one that doesn't allow it to decode, and why the screenshot of the same exact image does!

Original Image (does't work): https://imgur.com/a/kLUDRCs
Copy of Image (does work): https://imgur.com/a/BYPkEZu

[odahcam]

Sorry but they like the same, is that the intended example?

[majestic84]

Thanks for the quick response :)

That is correct. They are indeed the same QR code, but the first one (jpeg) was taken by my test user's smart phone camera and submitted to the app for decoding, which failed. When I put this image directly through the library (outside of my app) it doesn't work.

The second image (png) is a screenshot of the very same image. You can see it even has the cursor over it, which I would have figured would make it even worse, but instead this second version actually does decode. I am quite puzzled by this!

[odahcam]

Have you tried using try harder or any other hints for testing with the JPEG?

[majestic84]

I do have try harder enabled. Below is the block of code I am using to test this scenario. I can decode other QR jpegs using this code without issue, but not the one from my user's smartphone camera.

const jpeg = require('jpeg-js')

const fs = require('fs')
const jpegData = fs.readFileSync('./img/0.jpg')
const imageData = jpeg.decode(jpegData)

if (!imageData) return false

const { MultiFormatReader, BarcodeFormat, DecodeHintType, RGBLuminanceSource, BinaryBitmap, HybridBinarizer } = require('@zxing/library/esm5')
const formats = [BarcodeFormat.QR_CODE]
const hints = new Map()

hints.set(DecodeHintType.POSSIBLE_FORMATS, formats)
hints.set(DecodeHintType.TRY_HARDER, true)

const reader = new MultiFormatReader()
reader.setHints(hints)

const len = imageData.width * imageData.height
const luminancesUint8Array = new Uint8ClampedArray(len)

for(let i = 0; i < len; i++){
    luminancesUint8Array[i] = ((imageData.data[i*4]+imageData.data[i*4+1]*2+imageData.data[i*4+2]) / 4) & 0xFF
}

const luminanceSource = new RGBLuminanceSource(luminancesUint8Array, imageData.width, imageData.height)
const binaryBitmap = new BinaryBitmap(new HybridBinarizer(luminanceSource))

try {
    const decoded = reader.decode(binaryBitmap)
    if (!decoded || !decoded.text) return false
    console.log(decoded.text)
} catch (err) {
    console.log(err)
}

I can change lines 4 and 5 to be

const pngData = fs.readFileSync('./img/00.png')
const imageData = PNG.sync.read(pngData)

and I can decode the screenshot I took of the nonreadable QR jpg.

In my application the image data actually comes from an HTTP request buffer, but I'm using the example above to rule out something going wrong with the request response.

I should also note that it does make it to the end of the script, where the exception is caught by the try/catch block with the following exception:

NotFoundException: No MultiFormat Readers were able to detect the code.
    at MultiFormatReader.decodeInternal (C:\Node\qrtest\node_modules\@zxing\library\esm5\core\MultiFormatReader.js:178:15)
    at MultiFormatReader.decode (C:\Node\qrtest\node_modules\@zxing\library\esm5\core\MultiFormatReader.js:64:21)
    at Object.<anonymous> (C:\Node\qrtest\harder.js:26:28)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Function.Module.runMain (module.js:694:10)
    at startup (bootstrap_node.js:204:16)

[majestic84]

I just noticed that when I add a console.log(this.hints) to MultiFormatReader.prototype.decodeInternal() it prints "undefined" even though I am setting the hints to the reader. I also noticed that in MultiFormatReader.d.ts the Map constructs are showing "Cannot find name. Do you need to change your target library? Try changing the lib compiler option to es2015 or later." I am using WebStorm.

Adding a jsconfig.json with target: "es6" or lib: ["es6"] fixes the tslint warnings, but this.hints is still undefined.

I can see it does not make it passed var detectorResult = new Detector_1.default(image.getBlackMatrix()).detect(hints); in /lib/esm5/core/qrcode/QRCodeReader.js.

It gets all the way to getVersionForNumber in /qrcode/decoder/Version.js with versionNumber 175 which passes the first conditional if (versionNumber < 1 || versionNumber > 40) { so it throws a IllegalArgumentException_1.default() exception.

[majestic84]

I debugged this issue extensively and found a fix that works for me. Basically, the larger the resolution of the image, the more this library has to work to detect and decode a QR code. Using Jimp, I simply resize the image to a resolution that is easier to digest, and it works great!

I also found that if the image resolution is too small it can cause issues, so I enforced a minimum width and minimum height. Now it is passing all of my tests :)

Complete working example below:

const image = './path/to/image' //note this can also be a buffer if fetching from a request
const imagePromise = new Promise((resolve, reject) => {
        try {
            const Jimp = require("jimp")
            new Jimp(image, async (err, image) => {
                const width = image.bitmap.width
                const height = image.bitmap.height

                if (width > height) {
                        if (width > 400) {
                            await image.resize(400, Jimp.AUTO)
                        }

                        if (image.bitmap.height < 400) {
                            await image.resize(Jimp.AUTO, 400)
                        }
                    }
                    else {
                        if (height > 400) {
                            await image.resize(Jimp.AUTO, 400)
                        }

                        if (image.bitmap.width < 400) {
                            await image.resize(400, Jimp.AUTO)
                        }
                    }

                if (image && image.bitmap) return resolve(image.bitmap)
                else return resolve()

            })
        } catch (e) {
            return resolve()
        }
    })

    const imageData = await imagePromise
    if (!imageData) return false

    const { MultiFormatReader, BarcodeFormat, DecodeHintType, RGBLuminanceSource, BinaryBitmap, HybridBinarizer } = require('@zxing/library/esm5')
    const formats = [BarcodeFormat.QR_CODE]
    const hints = new Map()

    hints.set(DecodeHintType.POSSIBLE_FORMATS, formats)
    hints.set(DecodeHintType.TRY_HARDER, true)

    const reader = new MultiFormatReader()
    reader.setHints(hints)

    const len = imageData.width * imageData.height
    const luminancesUint8Array = new Uint8ClampedArray(len)

    for(let i = 0; i < len; i++){
        luminancesUint8Array[i] = ((imageData.data[i*4]+imageData.data[i*4+1]*2+imageData.data[i*4+2]) / 4) & 0xFF
    }

    const luminanceSource = new RGBLuminanceSource(luminancesUint8Array, imageData.width, imageData.height)
    const binaryBitmap = new BinaryBitmap(new HybridBinarizer(luminanceSource))

    try {
        const decoded = reader.decode(binaryBitmap)
        if (!decoded || !decoded.text) return false
        console.log(decoded.text)
    } catch (err) {
        console.log(err)
    }

[majestic84]

In case anyone else stumbles upon this issue in the future, here is a working example of decoding a QR code from an image that needs to be fetched with an HTTP request:

Requires 'request', 'jimp', and '@zxing/library'

const readQrDataFromURL = async (url) => {
    try {
        if (!url) return false
        const extension = url.substr(-4).toLowerCase() === 'jpeg' ? 'jpg' : url.substr(-3).toLowerCase()
        if (extension !== 'jpg' && extension !== 'png' && extension !== 'gif') return false
        const requestPromise = new Promise((resolve, reject) => {
            try {
                const request = require('request').defaults({ encoding: null })
                request.get(url, (error, response, body) => {
                    if (!error && response.statusCode === 200) {
                        return resolve(body)
                    } else {
                        return resolve()
                    }
                })
            } catch (e) {
                return resolve()
            }
        })

        const imageBuffer = await requestPromise
        if (!imageBuffer) return false

        const imagePromise = new Promise((resolve, reject) => {
            try {
                const Jimp = require("jimp")
                new Jimp(imageBuffer, async (err, image) => {
                    const width = image.bitmap.width
                    const height = image.bitmap.height

                    if (width > height) {
                        if (width > 400) {
                            await image.resize(400, Jimp.AUTO)
                        }

                        if (image.bitmap.height < 400) {
                            await image.resize(Jimp.AUTO, 400)
                        }
                    }
                    else {
                        if (height > 400) {
                            await image.resize(Jimp.AUTO, 400)
                        }

                        if (image.bitmap.width < 400) {
                            await image.resize(400, Jimp.AUTO)
                        }
                    }

                    if (image && image.bitmap) return resolve(image.bitmap)
                    else return resolve()

                })
            } catch (e) {
                return resolve()
            }
        })

        const imageData = await imagePromise
        if (!imageData) return false

        const { MultiFormatReader, BarcodeFormat, DecodeHintType, RGBLuminanceSource, BinaryBitmap, HybridBinarizer } = require('@zxing/library/esm5')
        const formats = [BarcodeFormat.QR_CODE]
        const hints = new Map()

        hints.set(DecodeHintType.POSSIBLE_FORMATS, formats)
        hints.set(DecodeHintType.TRY_HARDER, true)

        const reader = new MultiFormatReader()
        reader.setHints(hints)

        const len = imageData.width * imageData.height
        const luminancesUint8Array = new Uint8ClampedArray(len)

        for(let i = 0; i < len; i++){
            luminancesUint8Array[i] = ((imageData.data[i*4]+imageData.data[i*4+1]*2+imageData.data[i*4+2]) / 4) & 0xFF
        }

        const luminanceSource = new RGBLuminanceSource(luminancesUint8Array, imageData.width, imageData.height)
        const binaryBitmap = new BinaryBitmap(new HybridBinarizer(luminanceSource))
        const decoded = reader.decode(binaryBitmap)
        if (!decoded || !decoded.text) return false

        return decoded.text

    } catch (err) {
        return false
    }
}

[odahcam]

Thanks @majestic84 for your work, it will surely help us all here.