Matterbridge: Bot silently fails and not does bridge messages between IRC and Slack

[jwflory]

Description

Bot successfully starts and claims to join IRC channels, but:

1. Only 1-2 IRC channels are successfully joined
2. Bridging from either Slack or IRC does not work
3. Bot eventually quits from IRC and never comes back (even when running)
4. No logs indicate anything is wrong

To reproduce

Steps to reproduce the behavior:

Run the bot with this config and these variables:

###################################################################
#IRC section
###################################################################

[irc]
[irc.{{ default_irc_network_name }}]
Server="{{ default_irc_network_server }}"
UseTLS=true
SkipTLSVerify=false
UseSASL=false
Nick="{{ default_irc_bot_nick }}"

MessageSplit=true
ColorNicks=true
IgnoreNicks="{{ default_irc_ignore_nicks }}"
RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
ShowJoinPart=true
ShowTopicChange=true


###################################################################
#slack section
###################################################################
[slack]
[slack.{{ default_slack_team_name }}]
Token="{{ default_slack_api_token }}"

IconURL="https://robohash.org/{NICK}.png?size=48x48"
EditSuffix=" (edited)"
PrefixMessagesWithNick=false
IgnoreNicks="{{ default_slack_ignore_nicks }}"

PreserveThreading=true


###################################################################
#General configuration
###################################################################

[general]
RemoteNickFormat="[{PROTOCOL}] <{NICK}> "


###################################################################
#Gateway configuration
###################################################################

[[gateway]]
name="gateway_rit"
enable=true

    [[gateway.inout]]
    account="irc.{{ default_irc_network_name }}"
    channel="{{ matterbridge_config.rit.irc.channel }}"

    [[gateway.inout]]
    account="slack.{{ default_slack_team_name }}"
    channel="{{ matterbridge_config.rit.slack.channel }}"

[[gateway]]
name="gateway_rit_foss"
enable=true

    [[gateway.inout]]
    account="irc.{{ default_irc_network_name }}"
    channel="{{ matterbridge_config.rit.irc.channel }}"

    [[gateway.inout]]
    account="slack.{{ default_slack_team_name }}"
    channel="{{ matterbridge_config.rit.slack.channel }}"

Expected behavior

1. Start bot (/usr/local/bin/matterbridge -conf matterbridge.toml)
2. Bot successfully starts
3. Messages from Slack go to IRC, IRC messages go to Slack

Debug logs

More information is also in FOSSRIT/infrastructure#21.

/usr/local/bin/matterbridge -debug -conf matterbridge.toml 
[0000]  INFO main:         Enabling debug
[0000]  INFO main:         Running version 1.12.1 d2cfd23
[0000]  INFO gateway:      Parsing gateway gateway_ritlug_teleirc
[0000]  INFO gateway:      Parsing gateway gateway_rit
[0000]  INFO gateway:      Parsing gateway gateway_rit_foss
[0000]  INFO gateway:      Parsing gateway gateway_rit_lug_sysadmin
[0000]  INFO gateway:      Parsing gateway gateway_rit_tigeros
[0000]  INFO gateway:      Parsing gateway gateway_ritlug
[0000]  INFO gateway:      Starting bridge: irc.freenode 
[0000]  INFO irc:          Connecting irc.freenode.net:6697
[0000] DEBUG irc:          "CLIENT_INIT :irc.freenode.net:6697"
[0000] DEBUG irc:          ":asimov.freenode.net NOTICE * :*** Looking up your hostname..."
[0000] DEBUG irc:          ":asimov.freenode.net NOTICE * :*** Checking Ident"
[0000] DEBUG irc:          ":asimov.freenode.net NOTICE * :*** No Ident response"
[0000] DEBUG irc:          ":asimov.freenode.net NOTICE * :*** Found your hostname"
[0000] DEBUG irc:          ":asimov.freenode.net CAP * LS :account-notify extended-join identify-msg multi-prefix sasl"
[0000] DEBUG irc:          ":asimov.freenode.net CAP slack-ritlug ACK :account-notify extended-join multi-prefix "
[0000] DEBUG irc:          ":asimov.freenode.net 001 slack-ritlug :Welcome to the freenode Internet Relay Chat Network slack-ritlug"
[0000] DEBUG irc:          "CLIENT_GENERAL_UPDATED"
[0000] DEBUG irc:          Registering callbacks
[0000] DEBUG irc:          "CLIENT_GENERAL_UPDATED"  # this is repeated for many, many lines
[ ... ]
[0002] DEBUG irc:          "CLIENT_CONNECTED :irc.freenode.net:6697"
[0005] DEBUG irc:          ":slack-ritlug MODE slack-ritlug :+Zi"
[0005]  INFO irc:          Connection succeeded
[0005]  INFO bridge:       irc.freenode: joining #ritlug (ID: #ritlugirc.freenode)
[0005]  INFO bridge:       irc.freenode: joining #ritlug-teleirc (ID: #ritlug-teleircirc.freenode)
[0005] DEBUG irc:          handle girc.Event{Source:(*girc.Source)(0xc0007a4840), Tags:girc.Tags(nil), Timestamp:time.Time{wall:0xbef9819888fc1765, ext:5859984620, loc:(*time.Location)(0x18436a0)}, Command:"JOIN", Params:[]string{"#ritlug", "*"}, Trailing:"slack-ritlug", EmptyTrailing:false, Sensitive:false, Echo:false}
[0005] DEBUG irc:          333: Topic set by jwf|matrix [jflory7mat@rit/foss/captain/fedora.jflory7] [2018-06-29 21:12:09 -0400 EDT]
[0029] DEBUG irc:          == Receiving PRIVMSG: jwf slack-ritlug: ?? girc.Event{Source:(*girc.Source)(0xc000731bf0), Tags:girc.Tags(nil), Timestamp:time.Time{wall:0xbef9819e5fb2f0be, ext:29241066566, loc:(*time.Location)(0x18436a0)}, Command:"PRIVMSG", Params:[]string{"#ritlug"}, Trailing:"slack-ritlug: ??", EmptyTrailing:false, Sensitive:false, Echo:false}
[0029] DEBUG irc:          detected UTF-8 confidence 10
[0029] DEBUG irc:          <= Sending message from #ritlug on irc.freenode to gateway

Nothing happens after this message. Even when other activity on either side of the bridge happens, there is silence.

Environment

• OS: Linux, CentOS 7.5
• Matterbridge version: 1.12.1 d2cfd23

[patcon]

Firstly, the good news: this is a great bug report. Thankee, sai!

The alao good news: it's fixed! https://github.com/42wim/matterbridge/issues/639#issuecomment-443755592

The bad news: I am not so good at using snowclone phrasal constructs

Deep apologies for the bother with the breakage.

[jwflory]

@patcon Oh, super! Following the advice in https://github.com/42wim/matterbridge/issues/639#issuecomment-443755592, I downgraded to v1.12.0 for now and the bridge works as expected. It would be great to get a new release cut with this fix if possible. :slightly_smiling_face:

[patcon]

Might be a little while -- we don't tend to release quite that often. @42wim's call. But to clarify, master is always built here:
https://bintray.com/42wim/nightly/Matterbridge/_latestVersion

Despite the link in the other thread perhaps appearing to be 1.12.0, it's in fact the most recent mainline automated build (untagged)

If releases are important to you, I suspect we'd be grateful for help investigating ways to make very frequent releases less of a burden. If you were eager, of course! Again, this is up to @42wim to decide, but maybe things like scripts or researching a semantic release process for Go?

If you would be willing to offer your time, please do open up a new issue! 🎉

[patcon]

Also, would you mind closing this out? :)

[42wim]

@jwflory 1.12.2 is out with the fix :-)

[jwflory]

Yay, thank you @patcon @42wim! Is a new issue for more agile releases still helpful?

[jwflory]

I deployed this in FOSSRIT/infrastructure@54201fc8ea90ee97c3b857aa0852459740df64af and everything is looking good so far. Thanks for the quick resolution to this ticket. :smile:

[patcon]

Ah sweet! Thanks! Would it be worthwhile to cut that out into a unit (playbook?)? I cut my teeth in chef, so not sure if ansible necessarily works best this way :)

But either way, we should at the very least link your role in the related projects section:
https://github.com/42wim/matterbridge#related-projects

If you're game, we'd be psyched to merge a PR for that! 🐱🐱🐱
^^^ cats because why not

[patcon]

(And I'd wait for @42wim on the release script ask, before you spend valuable attention on it :) )

[jwflory]

Ah sweet! Thanks! Would it be worthwhile to cut that out into a unit (playbook?)?

We're using an Ansible Role to do this now. From the docs, they are explained like this:

Roles are ways of automatically loading certain vars_files, tasks, and handlers based on a known file structure. Grouping content by roles also allows easy sharing of roles with other users.

This way, anyone should be able to take the role and use it in their own playbook. You can see how this is used in a playbook here.

But either way, we should at the very least link your role in the related projects section:
https://github.com/42wim/matterbridge#related-projects

If you're game, we'd be psyched to merge a PR for that! catcatcat

Cool! Done in #643. :slightly_smiling_face:

[42wim]

Hey, I think the releases are already agile and semver, that's my goal anyway :)
That's also why you get dev binaries on every commit so people don't need to compile it themselves.
At the moment I've got my own local release scripts that build, push, release and integrate with keybase for PGP signing.

[patcon]

That is rad wim. Sorry, I didn't realize :)

Semantic release is a little diff from semver fwiw.

But if that's not something we'd appreciate from a devops-minded contributor, it might be interesting more generally to riff on ops tasks (in chat channel?) that ppl with those skills could work on. eg, documenting use with pleaserun or things like that. (Pleaserun has great "project principles" section in the readme btw)

[42wim]

Ok misread the sem release and semver. So it's basically the same as the dev builds I'm already doing. Not a fan of making official releases that way.

People have enough options already. stable builds, devbuilds, diy builds.

Regarding to pleaserun, we have a journald service file in contrib. Instead of building, generating, maintening those files I think it would be better to get matterbridge into the different distros. But let's discuss this further on the chat.

[jwflory]

Hey, I think the releases are already agile and semver, that's my goal anyway :)

From my point of view, it was a seamless process and one of the quickest turnarounds I've ever had on an upstream bug being fixed, so no complaints here. :raised_hands:

I think it would be better to get matterbridge into the different distros.

I have a lot of experience in the Fedora community. I'm happy to offer support and guidance for what that process looks like and what to do in the Fedora community.

[patcon]

Oh hey, rockin! Would you be so generous to pop open a general ticket about it, @jwflory? @42wim prob has some thoughts on what he's hoping for, but any references you know would be welcome. (You don't necessarily have to be the one to do the work, but it would be totally welcome if you had time! :)