Mosh: Mosh server crashing on Amazon Linux 2 (AMI)

[scuba-tech]

Hello!
I've just finished installing mosh 1.3.0 (_installation details at end of post_) on my first EC2 instance, specifically running the Amazon Linux 2 AMI on a t2.micro instance.

When running mosh-server, I am greeted with the following error message:

mosh-server: symbol lookup error: mosh-server: undefined symbol: _ZNK6google8protobuf11MessageLite25InitializationErrorStringEv

---

When attempting to connect from the client-side (on a client that I use for mosh every day with other servers), I receive the same error message:

└──╼ $ mosh -ssh="ssh -i ~/.ssh/[redacted].pem" ec2-user@[redacted].amazonaws.com
mosh-server: symbol lookup error: mosh-server: undefined symbol: _ZNK6google8protobuf11MessageLite25InitializationErrorStringEv
Connection to [redacted].amazonaws.com closed.
/usr/local/bin/mosh: Did not find mosh server startup message. (Have you installed mosh on your server?)

---

Here is my version info:

$ cat /etc/*release
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"
Amazon Linux release 2 (Karoo)

Installation commands used:

• sudo amazon-linux-extras install epel
• sudo yum install mosh

I'm unsure of what other info I can help give you in troubleshooting this bug. Anything else I can do to help?

[keithw]

Thanks -- it looks like the EPEL package may need to be updated (at least on this Linux distribution). Can you please raise this with them? @achernya may know more about how to get this fixed.

[keithw]

In the meantime, you can always compile from source (using the instructions at https://mosh.org -- also in the GitHub Wiki).

[scuba-tech]

Thanks -- it looks like the EPEL package may need to be updated (at least on this Linux distribution). Can you please raise this with them? @achernya may know more about how to get this fixed.

Thanks for the help! I'll have to figure out just what/who EPEL is (shameful confession: I ran the installation rote, from a guide, and am learning as I go) and will let them know.

In the meantime, you can always compile from source (using the instructions at https://mosh.org -- also in the GitHub Wiki).

Sounds fun! Will give it a go...

[andersk]

EPEL is built for RHEL and CentOS. According to Amazon, “AL2 does not officially support enabling and installing packages from EPEL repositories, however you should find that the packages are mostly compatible and just work.” This appears to be an example of a package that doesn’t. You’ll probably have to complain to Amazon. Meanwhile, you should be able to build it from source.

[vincent-dm]

FYI: I had to yum install these packages to make Mosh on Amazon Linux 2: protobuf-devel.x86_64 zlib-devel gcc-c++ ncurses-devel.x86_64 openssl-devel.x86_64

In order to run autogen.sh (when building from git, as in the example), I also had to install autoconf and libtool

[eminence]

No recent activity here, so closing for now. Please leave a comment (or open a new issue) if you're still having problems.

[scuba-tech]

Yep, still having some problems, no solution yet from my end, wound up giving up and using SSH like everyone else. :-/

On Aug 13, 2019, at 15:46, Andrew Chin [e-mail] wrote:


No recent activity here, so closing for now. Please leave a comment (or open a new issue) if you're still having problems.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[eminence]

Thanks for the feedback. I just tested this myself and indeed mosh from EPEL doesn't work on these Amazon Linux machines.

But I did confirm that mosh will build and run on these machines, so that's still an option for you.

I guess we'll keep this issue open until Amazon is able to provide a working mosh package.

[scuba-tech]

Thanks! I’m still somewhat new to dev/coding, so building sounds
like a fun (hopefully small) challenge. Didn’t work last time I tried, but I’ve learned a lot since then. I’ll give it a go on my Amazon
Linux t2.micro this weekend, and will report back if I find anything
interesting that could help.

Aside from that and beyond the scope of this issue thread, I just wanted to
say thanks to the MOSH crew for being awesome. I use this just about every
day on my own stuff and it’s truly a game-changer. Not sure if enough
people say thanks but I sure will.

On Thu, Aug 15, 2019 at 18:59 Andrew Chin [e-mail] wrote:

Thanks for the feedback. I just tested this myself and indeed mosh from
EPEL doesn't work on these Amazon Linux machines.

But I did confirm that mosh will build and run on these machines, so
that's still an option for you.

I guess we'll keep this issue open until Amazon is able to provide a
working mosh package.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/mobile-shell/mosh/issues/1026?email_source=notifications&email_token=AIPVM6MYF3QWEAZL2MJJKCDQEX3XJA5CNFSM4G2NSPKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4NMDKA#issuecomment-521847208,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIPVM6MQ3PR7FOFVEANFER3QEX3XJANCNFSM4G2NSPKA
.

[websocket98765]

As an update, I encountered same error as the OP.

Who do we contact at Amazon to voice support for supporting Mosh? Has amazon been contacted?

[ctd]

This is not a bug/issue in mosh, but an issue with binary compatibility between Enterprise Linux 7 and Amazon Linux 2 - in particular for libprotobuf in the protobuf package.

On CentOS 7:

$ nm -D /lib64/libprotobuf.so.8 | grep 'InitializationErrorString' | c++filt
000000000005d500 T google::protobuf::MessageLite::InitializationErrorString() const
00000000000b5de0 T google::protobuf::Message::InitializationErrorString() const

and on Amazon Linux 2:

$ nm -D /lib64/libprotobuf.so.8 | grep 'InitializationErrorString' | c++filt
000000000005fe30 T google::protobuf::MessageLite::InitializationErrorString[abi:cxx11]() const
00000000000b32c0 T google::protobuf::Message::InitializationErrorString[abi:cxx11]() const

Because the build of mosh included in EPEL was built against libraries distributed in EL7, it is linked against the symbol names in that library - hence the "undefined symbol" error.

The simplest workaround for now, as some of you have found, is to rebuild mosh from source on Amazon Linux 2. I've written the rough steps to do this at https://gist.github.com/ctd/9fbd0ed5b7d9be1aa532d11a0701bae1

[yarnball]

I can confirm I'm running into the same issue. Running mosh-server returns:

mosh-server: symbol lookup error: mosh-server: undefined symbol: _ZNK6google8protobuf11MessageLite25InitializationErrorStringEv

Any ideas on a work around?

[yarnball]

Ok, work around is building from source. For a setup on the AWS AMI setup:

$ yum groupinstall 'Development Tools
$ yum install protobuf-devel protobuf-compiler
$ yum install ncurses-devel
$ yum install -y openssl-devel
$ git clone https://github.com/mobile-shell/mosh
$ cd mosh
$ ./autogen.sh
$ ./configure
$ make
# make install

[muyinliu]

Ok, work around is building from source. For a setup on the AWS AMI setup:

$ yum groupinstall 'Development Tools
$ yum install protobuf-devel protobuf-compiler
$ yum install ncurses-devel
$ yum install -y openssl-devel
$ git clone https://github.com/mobile-shell/mosh
$ cd mosh
$ ./autogen.sh
$ ./configure
$ make
# make install

'Development Tools missing a ' here, should be 'Development Tools'

[girotomas]

same comment as above without the "$"

yum groupinstall 'Development Tools'
yum install protobuf-devel protobuf-compiler
yum install ncurses-devel
yum install -y openssl-devel
git clone https://github.com/mobile-shell/mosh
cd mosh
./autogen.sh
./configure
make