Hassio-zigbee2mqtt: Unable to start with custom devices.js

[knutasund]

zigbee2mqtt add-on version (if edge, please report commit hash):

Operating environment (HassOS, Virtual Machine, Device/platform running Hass.io):

Add-on does not start if I try to add custom devices.js.
Download https://github.com/Koenkk/zigbee-herdsman-converters/blob/master/devices.js

This worked fine before upgrade to 1.14.3.

Your entire configuration from the frontend (with sensitive fields redacted):

groups: groups.yaml
homeassistant: true
permit_join: false
mqtt:
  base_topic: zigbee2mqtt
  server: 'mqtt://192.168.0.118'
  user: xxxxx
  password: xxxxxxxxxxxxxxxx
serial:
  port: >-
    /dev/serial/by-id/usb-Texas_Instruments_TI_CC2531_USB_CDC___0X00124B00193673BF-if00
  disable_led: true
advanced:
  pan_id: 6754
  channel: 11
  log_level: info
  network_key:
 - <redacted>
  availability_blacklist: []
ban: []
whitelist: []
queue: {}
socat:
  enabled: false
  master: 'pty,raw,echo=0,link=/dev/ttyZ2M,mode=777'
  slave: 'tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5'
  restartdelay: 1
  initialdelay: 1
  options: '-d -d'
  log: false
zigbee_shepherd_devices: true

Zigbee2MQTT:info 2020-08-30 12:33:29: Logging to console and directory: '/share/zigbee2mqtt/log/2020-08-30.12-33-29' filename: log.txt

internal/modules/cjs/loader.js:968
  throw err;
  ^
Error: Cannot find module './converters/store'
Require stack:
- /zigbee2mqtt-1.14.3/node_modules/zigbee-herdsman-converters/devices.js
- /zigbee2mqtt-1.14.3/node_modules/zigbee-herdsman-converters/index.js
- /zigbee2mqtt-1.14.3/lib/zigbee.js
- /zigbee2mqtt-1.14.3/lib/controller.js
- /zigbee2mqtt-1.14.3/index.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:965:15)
    at Function.Module._load (internal/modules/cjs/loader.js:841:27)
    at Module.require (internal/modules/cjs/loader.js:1025:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/zigbee2mqtt-1.14.3/node_modules/zigbee-herdsman-converters/devices.js:26:21)
    at Module._compile (internal/modules/cjs/loader.js:1137:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1157:10)
    at Module.load (internal/modules/cjs/loader.js:985:32)
    at Function.Module._load (internal/modules/cjs/loader.js:878:14)
    at Module.require (internal/modules/cjs/loader.js:1025:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/zigbee2mqtt-1.14.3/node_modules/zigbee-herdsman-converters/devices.js',
    '/zigbee2mqtt-1.14.3/node_modules/zigbee-herdsman-converters/index.js',
    '/zigbee2mqtt-1.14.3/lib/zigbee.js',
    '/zigbee2mqtt-1.14.3/lib/controller.js',
    '/zigbee2mqtt-1.14.3/index.js'
  ]
}
npm

ERR! code ELIFECYCLE
npm ERR! errno
 1
npm

ERR! [email protected] start: `node index.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-08-30T10_33_30_912Z-debug.log
2020-08-30T12:33:31: PM2 log: App [npm:0] exited with code [1] via signal [SIGINT]
2020-08-30T12:33:31: PM2 log: App [npm:0] starting in -fork mode-
2020-08-30T12:33:31: PM2 log: App [npm:0] online
> [email protected] start /zigbee2mqtt-1.14.3
> node index.js

[Jason-Murray]

Also having this issue. I guess the new devices.js is calling for updated converters files, I've tried placing these in data_path too but they don't get used, I'm guessing the override is only set up to recognise devices.js

[Jason-Murray]

Got around this for now by removing the line near the top of devices.js:
const globalStore = require('./converters/store');

And removing the entire entry for the zigbeeModel: ['3400-D'] (around line 7100) as this is the only place globalStore is used.

[knutasund]

Thanks, same workaround works for me.

[ciotlosm]

Related to https://github.com/danielwelch/hassio-zigbee2mqtt/issues/383 as well. Maybe try to use external_converters as stated here: https://www.zigbee2mqtt.io/information/configuration.html#external-converters-configuration

[ciotlosm]

I think (might be wrong) external_converters implementation broke the devices.js workaround that was implemented in this add-on. See https://github.com/Koenkk/zigbee-herdsman-converters/issues/1343 for details on that implementation, ~but I think the add-on will have to update with support for that~.

@Koenkk would you be able to assist with more details? Maybe it speeds up the implementation / fixes. Seems the implementation done in zigbee-herdsman-converters might cover support for more issues reported here: #383 #373 #194

[lovelylain]

@ciotlosm It's because knutasund's zigbee2mqtt version is 1.14.3, but his devices.js is the latest version from zigbee-herdsman-converters, which has introduced a new file named './converters/store', this file is not exist in 1.14.3.

The new external_converters option is a better solution for custom device. Here is my experience:

module.paths.push(require.main.path+ '/node_modules') //sovle Cannot find module 'zigbee-herdsman-converters'

const zc = require('zigbee-herdsman-converters')
const fz = zc.fromZigbeeConverters
const tz = zc.toZigbeeConverters

Object.assign(zc.findByDevice({modelID:'lumi.switch.b2nacn02'}), { //patch an exist device
  fromZigbee: [fz...],
  toZigbee: [tz...],
})

module.exports = [{ //add new devices
  zigbeeModel: ['lumi.switch.n3acn3'],
  model: 'QBKG26LM',
  vendor: 'Xiaomi',
  description: 'Aqara D1 3 gang smart wall switch (with neutral wire)',
  supports: 'on/off',
  fromZigbee: [fz.on_off],
  toZigbee: [tz.on_off],
  meta: {multiEndpoint: true},
  endpoint: (device) => {
    return {'left': 1, 'center': 2, 'right': 3}
  },
}]

[ciotlosm]

@lovelylain thanks for the details. Would you write some docs to demonstrate external_converters setup with the add-on? I am happy to prepare the PR if you don't have time, the docs can be just a skeleton with the important bits (files, steps).

Later edit:

@knutasund To get the correct devices.js for your version you should:

1. Identify your stable zigbee2mqtt version from the add-on: 1.14.3
2. Navigate to https://github.com/Koenkk/zigbee2mqtt/tags and find tag 1.14.3
3. Click on the commit hash f8066e8 and then browse files button
4. Find package.json and identify zigbee-herdsman-converters version: 12.0.161
5. Navigate to https://github.com/Koenkk/zigbee-herdsman-converters/tags and find tag 12.0.161
6. Click on the commit hash 3a5abc7 and then browse files button
7. Find devices.js file and download it (use raw version)

Let me know if that works. I will close this issue.

@lovelylain I will open a separate issue to address documentation for external_converters

[Koenkk]

@ciotlosm do you still need my help here?

[ciotlosm]

@Koenkk can you let us know (to avoid a lot of digging) if external_converters should cover use cases where people requested ability to change through the add-on more than just devices.js?

People have been requesting to change fromZigbee.js homeassistant.js utils.js.

[Koenkk]

• fromZigbee.js: this is possible, you can the define the converters yourself: https://github.com/Koenkk/zigbee2mqtt.io/blob/master/docs/externalConvertersExample/dummy-converter.js#L15
• homeassistant.js: also possible: https://github.com/Koenkk/zigbee2mqtt.io/blob/master/docs/externalConvertersExample/dummy-converter.js#L66
• utils.js: not possible and not something that should be added I feel. People should use the endpoint names covered here: https://github.com/Koenkk/zigbee2mqtt/blob/master/lib/util/utils.js#L31 in their custom definitions.

[ciotlosm]

Closing this as the original problem was addressed in https://github.com/danielwelch/hassio-zigbee2mqtt/issues/373#issuecomment-695167161. Raised https://github.com/danielwelch/hassio-zigbee2mqtt/issues/392 for updating the docs.