Mqtt.js: Version 4.2.0: ws does not work in the browser
After the latest update, my web page using mqtt and MapBox GL JS library fails to fetch resources and gives this error:
Error: ws does not work in the browser. Browser clients must use the native WebSocket object
The error comes from inside mapbox-gl-js library. Does the new version somehow make global changes to some prototypes etc. which would cause this? The error disappeared after downgrading back to 4.1.0.
ttsirkia
All 24 comments
Both libraries are attached with script tags to the page.
ttsirkia
on 13 Aug 2020
The 4.2.0 does change the internal use of ws to use its internal implementation of streams for web sockets. Can you provide a way to reproduce this issue so it can be investigated and fixed?
Get Outlook for iOShttps://aka.ms/o0ukef
From: Teemu Sirkiä notifications@github.com
Sent: Wednesday, August 12, 2020 10:20:47 PM
To: mqttjs/MQTT.js MQTT.js@noreply.github.com
Cc: Subscribed subscribed@noreply.github.com
Subject: [mqttjs/MQTT.js] Version 4.2.0 doesn't work with mapbox-gl-js (#1141)
After the latest update, my web page using mqtt and MapBox GL JS library fails to fetch resources and gives this error:
Error: ws does not work in the browser. Browser clients must use the native WebSocket object
The error comes from inside mapbox-gl-js library. Does the new version somehow make global changes to some prototypes etc. which would cause this? The error disappeared after downgrading back to 4.1.0.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmqttjs%2FMQTT.js%2Fissues%2F1141&data=02%7C01%7Cyoseph.maguire%40microsoft.com%7C10f5c55e2fbe452cdf9e08d83f48a305%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637328928491529238&sdata=OsM4y2ZyCPj%2FT98qXaIpruqx9dcAyoQPxk7cs8WAAhQ%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAB6VMLHO73YTBVMBHHTYLSTSANZ27ANCNFSM4P5Z5YNA&data=02%7C01%7Cyoseph.maguire%40microsoft.com%7C10f5c55e2fbe452cdf9e08d83f48a305%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637328928491529238&sdata=ZgI3IuZkXRUc%2FQslD7Yqy1Isqn4YXzoD9JlRHTmxotM%3D&reserved=0.
YoDaMa
on 13 Aug 2020
@YoDaMa I'll try to provide a simple example. The problem is with the MapBox authentication token to make it work.
ttsirkia
on 13 Aug 2020
OK, mapbox-gl-js is not related here. It seems that the unpkg version causes this.
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<script src="https://unpkg.com/mqtt/dist/mqtt.min.js"></script>
<script>
var client = mqtt.connect('wss://mqtt.hsl.fi:443');
client.on('connect', function() {});
</script>
</head>
<body>
</body>
</html>
This will cause the error.
ttsirkia
on 13 Aug 2020
Observing the same issue in a browser app built with Webpack, so it does not seem to be related to unpkg. Since the 4.2.0 update, the app breaks with:
Error: ws does not work in the browser. Browser clients must use the native WebSocket object
Downgrading to 4.1.0 fixes it. Is ws used also for when using MQTT in a browser environment?
lucaong
on 13 Aug 2020
@YoDaMa It looks like MQTT now uses ws instead of websocket-stream: https://github.com/mqttjs/MQTT.js/blob/b2c121511c7437b64724e9f1e89ebcd27e3c2cce/lib/connect/ws.js#L3
The problem is that ws only works in Node, not in the browser, differently from websocket-stream. Therefore, 4.2.0 breaks browser builds.
One way to fix this would be to replace ws with the native WebSocket in browser builds, like websocket-stream does.
lucaong
on 13 Aug 2020
So unfortunately the testing infrastructure we have didn’t test for browser functionality like I thought it would during the release process.
The changes made were to remove websocket-stream as a dependency, which in turn uses ws as a dependency, and instead simply use ws for performance and reliability improvements. This change seems to have broken webpack though.
Get Outlook for iOShttps://aka.ms/o0ukef
From: Luca Ongaro notifications@github.com
Sent: Thursday, August 13, 2020 9:08:42 AM
To: mqttjs/MQTT.js MQTT.js@noreply.github.com
Cc: Yoseph Maguire Yoseph.Maguire@microsoft.com; Mention mention@noreply.github.com
Subject: Re: [mqttjs/MQTT.js] Version 4.2.0 doesn't work from unpkg (#1141)
Observing the same issue in a browser app built with Webpack. Since the 4.2.0 update, the app breaks with:
Error: ws does not work in the browser. Browser clients must use the native WebSocket object
Downgrading to 4.1.0 fixes it. Is ws used also for when using MQTT in a browser environment?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmqttjs%2FMQTT.js%2Fissues%2F1141%23issuecomment-673567509&data=02%7C01%7Cyoseph.maguire%40microsoft.com%7Cd4ec276751b44c7431b208d83fa32699%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637329317245154402&sdata=aTE6J92LaKBXGfZM04sjBFB903qfIQ%2FMzlwUnqLxoR8%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAB6VMLHZ6LRXVJZ63KZAN2TSAQFYVANCNFSM4P5Z5YNA&data=02%7C01%7Cyoseph.maguire%40microsoft.com%7Cd4ec276751b44c7431b208d83fa32699%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637329317245154402&sdata=QVjtMnZVN4rSSaJe9iyUy0p7XozT8bLf18YOOwY6EWs%3D&reserved=0.
YoDaMa
on 13 Aug 2020
yeah so like you said we need to implement an isomorphism for the browser compatibility to work with the Websocket in browser builds. I'm working on a PR rn to fix this.
YoDaMa
on 13 Aug 2020
FYI:
Looking at the docs for ws the isomorphic fix seems to be easy:
Note: This module does not work in the browser. The client in the docs is a reference to a back end with the role of a
client in the WebSocket communication. Browser clients must use the native WebSocket object. To make the same
code work seamlessly on Node.js and the browser, you can use one of the many wrappers available on npm,
like isomorphic-ws.
and isomorphic-ws usage seems to be simple as well: https://www.npmjs.com/package/isomorphic-ws
seriousme
on 15 Aug 2020
Yes, the ws module does not work in the browser.
ysfscream
on 18 Aug 2020
Yes there’s a draft PR fixing this.
Get Outlook for iOShttps://aka.ms/o0ukef
From: YuShifan notifications@github.com
Sent: Monday, August 17, 2020 7:50:26 PM
To: mqttjs/MQTT.js MQTT.js@noreply.github.com
Cc: Yoseph Maguire Yoseph.Maguire@microsoft.com; Assign assign@noreply.github.com
Subject: Re: [mqttjs/MQTT.js] Version 4.2.0: ws does not work in the browser (#1141)
Yes, the ws module does not work in the browse.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmqttjs%2FMQTT.js%2Fissues%2F1141%23issuecomment-675217902&data=02%7C01%7Cyoseph.maguire%40microsoft.com%7Ca5976f80b38b474cab9108d843217688%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637333158284608067&sdata=64JptfontSG6TOQygxfRXmkN4AKG3gfr4lKHRori%2BdQ%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAB6VMLFUWF4OX2F67UI37ETSBHT7FANCNFSM4P5Z5YNA&data=02%7C01%7Cyoseph.maguire%40microsoft.com%7Ca5976f80b38b474cab9108d843217688%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637333158284618063&sdata=yozfQUWvPnbal9JFe1WSwhOWN%2BxqhK9UKWIXdIEzHwM%3D&reserved=0.
YoDaMa
on 18 Aug 2020
Hi @YoDaMa
Thanks for you answer to solve this issue.
when the update to solve the bug will be release?
Reggards.
pr0x1
on 18 Aug 2020
Can someone in this thread try using the patch provided in https://github.com/mqttjs/MQTT.js/pull/1145 . I've tested it manually on my machine and it seems to be working but I'd like other folks trying it out. @pr0x1 @ysfscream @lucaong
YoDaMa
on 19 Aug 2020
Thanks, I’ll try it tomorrow and let you know.
lucaong
on 19 Aug 2020
Hi @YoDaMa I was testing but the issue continues.
i'm using the following script
Regards.
pr0x1
on 20 Aug 2020
Hey sorry to clarify you’ll need to manually pull the branch to a folder, run “npm run browser-build” then replace the src line in your code with the full path to the “dist/mqtt.min.js” that was generated
Get Outlook for iOShttps://aka.ms/o0ukef
From: pr0x1 notifications@github.com
Sent: Wednesday, August 19, 2020 5:39:57 PM
To: mqttjs/MQTT.js MQTT.js@noreply.github.com
Cc: Yoseph Maguire Yoseph.Maguire@microsoft.com; Mention mention@noreply.github.com
Subject: Re: [mqttjs/MQTT.js] Version 4.2.0: ws does not work in the browser (#1141)
Hi @YoDaMahttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FYoDaMa&data=02%7C01%7Cyoseph.maguire%40microsoft.com%7C1bf419b61fdb408c3fe208d844a190e0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637334807997085714&sdata=ydJ4XCzncsTd3Og0jGG2kBYIexj7L%2F2%2BS6N7BYg8LcE%3D&reserved=0 I was testing but the issue continues.
i'm using the following script
Reggards.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmqttjs%2FMQTT.js%2Fissues%2F1141%23issuecomment-676830798&data=02%7C01%7Cyoseph.maguire%40microsoft.com%7C1bf419b61fdb408c3fe208d844a190e0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637334807997095705&sdata=6pEgpIccjD5PYfph8H46zpRFvvywZQGIjbPC9lZ1vNk%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAB6VMLA7TWRSGRUGFQTR3Y3SBRWF3ANCNFSM4P5Z5YNA&data=02%7C01%7Cyoseph.maguire%40microsoft.com%7C1bf419b61fdb408c3fe208d844a190e0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637334807997105699&sdata=%2FJv9uSCztvcwDwBi678bLldj9aV%2Bjslm%2BXkBPw8vnwg%3D&reserved=0.
YoDaMa
on 20 Aug 2020
@YoDaMa đź‘Ť Thanks for the fix. I will test it later.
ysfscream
on 20 Aug 2020
really say there is lots of problems with running mqtt in browser. Not only ws, but readable-stream fails to run in browser if node shims are not provided.
https://github.com/nodejs/readable-stream/issues/412 that lead to goal of supporting shim free version in next release https://github.com/nodejs/readable-stream/issues/439 (because webpack 5 will drop node shim support https://webpack.js.org/migrate/5/#test-webpack-5-compatibility, and webpack is most common builder for now)
even if we removed methods that require shiming from mqtt (https://github.com/mqttjs/MQTT.js/pull/1134) we still can't use mqtt.js in browser without hacks because of underlying libs. As a result we need to maintain hacks like that https://github.com/sclausen/ngx-mqtt/pull/158/files
really say I don't see how to improve situation, except providing node shims via webpack in each dependant project. Problem with WS is just one of the symptoms.
nosovk
on 20 Aug 2020
@YoDaMa , I just tried out your patch in #1145 and it works: no error, and MQTT connects successfully while running in the browser. đź‘Ť
lucaong
on 20 Aug 2020
@nosovk I am not sure what you mean, I have been using MQTT.js in the browser for years now, connecting via Websocket, and never experienced any problem. Maybe I miss some context, but I really think that supporting browsers is a very important goal for MQTT.js, as that's one of the primary environments where JS runs.
lucaong
on 20 Aug 2020
yes sorry @nosovk I'm a bit confused at what you're getting at
YoDaMa
on 20 Aug 2020
release 4.2.1 (#1149) fixes this.
YoDaMa
on 24 Aug 2020
Thanks @YoDaMa for fixing this :)
lucaong
on 25 Aug 2020
@YoDaMa @lucaong sorry for late response, I'le provide some explanation.
Current browser version of MQTT.js is build via browserify. Browserify is predecessor of webpack, it's a bundler. What does it mean? When you make browser build of mqtt.js you ship mqtt bundle combined with all deps. For example that one https://github.com/mqttjs/MQTT.js/blob/master/package.json#L76 and other.
It's ok, if you add mqtt js like external module in yours application. That bundle is minified and has no external requirements.
But some time after Browserify there was Webpack/Rollup was invented. They works in a bit different.way. For webpack good library is not the one which provides compiled\minified js file. For webpack\rollup good dependency is js code, which is not minified and surely not bundled with deps. Rollup will try to treeshacke librarry deps with other packages deps, and split to smaller and probably even lazy loaded chunks. And splitting/minifying should be done according to the project config (not library config).
Good example is Angular. Angular requires libraries to provide es, umd, esm etc bundles. Package spec: https://docs.google.com/document/d/1CZC2rcpxffTDfRDs6p1cfbmKNLA6x5O-NtkJglDaBVs/preview
Alternate solution - allow Angular to build all needed versions itself. But it's only possible only if source code of the library is ready to be complied by webpack. In case of mqtt it means that nodejs specific methods shouldn't be invoked. For example process etc. Currently mqtt code could be builded with webpack\rollup, but one of dependant libs - is not.
webpack\rollup couldn't process nodejs/readable-stream.
https://github.com/nodejs/readable-stream/issues/426
https://github.com/nodejs/readable-stream/issues/412
It will be fixed, but currently all libs that depend on readable-stream have to use browserified (non tree shakable) version or enable node shims (polyfils). For webpack versions prior to 5 it wasn't a big problem because of default webpack configuration have those polyfils enabled. But for Rollup or Angular projects polyfills should be explicitly defined. For example check https://github.com/sclausen/ngx-mqtt/commit/b488b801befd047c139da7ea878526382b0dcdd0
It's surely not a good solution
.
Probably after https://github.com/nodejs/readable-stream/issues/439 issues with node intrnals used in browser will disappear, and then we will be able to use mqtt.js in Angular project without hacks.
Using mqtt with webpack now could be a bit tricky (with webpack 5 probably not possible)
nosovk
on 2 Sep 2020
Related issues
markcnunes
·
7Comments
bicccio
·
3Comments
ferm10n
·
5Comments
goliatone
·
4Comments
mcollina
·
8Comments
Most helpful comment
yeah so like you said we need to implement an isomorphism for the browser compatibility to work with the Websocket in browser builds. I'm working on a PR rn to fix this.