Amazon-ssm-agent: Start Session directly fails with websocket: control frame length > 125

[shockie]

When setting up Session Manager, the console seems to hang in the browser (in the cli I only get Starting session with SessionId: # message but hangs afterwards).

I tried to debug the issue and noticed in /var/log/amazon/ssm/amazon-ssm-agent.log (after enabling Debugging level flag) that it was throwing errors about 'control frame lengths'.

Output in log:

INFO Opening websocket connection to: %!(EXTRA string=wss://ssmmessages.eu-central-1.amazonaws.com/v1/data-channel/03e5208a141a9a039?role=publish_subscribe)
INFO Successfully opened websocket connection to: %!(EXTRA string=wss://ssmmessages.eu-central-1.amazonaws.com/v1/data-channel/03e5208a141a9a039?role=publish_subscribe)
DEBUG Successfully reconnected to datachannel 03e5208a141a9a039
WebsocketChannel: Send ping. Message.
DEBUG An error happened when receiving the message. Retried times: 1, MessageType: -1, Error: websocket: control frame length > 125
DEBUG An error happened when receiving the message. Retried times: 2, MessageType: -1, Error: websocket: control frame length > 125
DEBUG An error happened when receiving the message. Retried times: 3, MessageType: -1, Error: websocket: control frame length > 125
DEBUG An error happened when receiving the message. Retried times: 4, MessageType: -1, Error: websocket: control frame length > 125
WARN Reach the retry limit 5 for receive messages. Error: websocket: control frame length > 125
DEBUG Successfully get datachannel token
DEBUG Reconnecting datachannel: 03e5208a141a9a039
INFO Closing websocket channel connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/data-channel/03e5208a141a9a039?role=publish_subscribe
DEBUG Closing websocket connection to:%!(EXTRA string=10.0.0.187:443)
WARN Failed to close websocket: write tcp 10.0.0.30:33960->10.0.0.187:443: write: broken pipe
DEBUG Closing datachannel failed with error: write tcp 10.0.0.30:33960->10.0.0.187:443: write: broken pipe

Versions:
Instance AMI: Amazon Linux 2 AMI 2.0.20190313 x86_64 HVM gp2
SSM Agent: 2.3.372.0
10.0.0.187: is the Endpoint to ssmmessages
10.0.0.30: is the Instance itself.

I've created a specific IAM Role with AmazonEC2RoleforSSM policy added to it.
Tried to run with default VPC and a custom build VPC with Endpoints attached to it (even set the security groups allowance to All traffic so this should't be the problem).

Looks like the issue is in the communication between the SSM agent and AWS System Manager.

[shockie]

After diving into it a bit more, it seems related in having a + sign inside your username. Our usernames are email addresses and I have one in the format [email protected] and a normal user with [email protected], gave the two accounts the same rights, when opening a session via the console, the one with the plus sign fails, the one without succeeds.

(Also the one with the plus sign in it errors on terminating the session "Request not authenticated")

[nitikagoyal87]

Thanks for brining this to our attention. We will investigate this.

[nitikagoyal87]

Issue with '+' in user name has been fixed on service side. Please let us know in case of further queries.