Sensu-go: sensuctl: certain commands returning errors (entity list, event list, cluster health)

[mylesw42]

Expected Behavior

I've built binaries from the latest release (5.16.1) on MacOS 10.14 using Go 1.13.3

I expect to see the list of entities and events that are shown on the Web UI

Direct curl requests to the API endpoints are returning okay for both /entities and /events

Steps to Reproduce (for bugs)

Initializing a single stand-alone backend:
❯ ./sensu-backend start -d . --cache-dir . --agent-port 8079 --log-level debug
❯ ./sensu-backend init --cluster-admin-password admin --cluster-admin-username admin
{"component":"backend.seeds","level":"info","msg":"seeding etcd store with intial data","time":"2020-01-06T10:41:57-07:00"}
❯ ./sensu-agent start --backend-url ws://127.0.0.1:8079 --cache-dir agent-cache --name myles-macbook --subscriptions dev
❯ curl http://localhost:8080/version
{"etcd":{"etcdserver":"3.3.17","etcdcluster":"3.3.0"},"sensu_backend":"5.16.1"}

Once I initialized the admin account on the backend and configured sensuctl I'm able to view the Web UI and see entities/checks/etc...

Current Behavior

❯ ./sensuctl entity list
Error: not found

❯ ./sensuctl event list
Error: not found

❯ ./sensuctl check list
data is returned as expected

❯ ./sensuctl entity info "myles-macbook"
Error: not found

❯ ./sensuctl cluster member-list
=== Etcd Cluster ID: 3b0efc7b379f89be
ID Name Peer URLs Client URLs
8927110dc66458af default http://127.0.0.1:2380 http://localhost:2379

❯ ./sensuctl cluster health
Error: invalid character '<' looking for beginning of value

Context

sensu-backend debug log entries:

{"component":"apid","duration":"0.038ms","level":"info","method":"GET","msg":"request completed","path":"/api/core/v2/namespaces/default/entities","size":33,"status":404,"time":"2020-01-06T14:46:05-07:00"}

{"component":"apid","duration":"0.033ms","level":"info","method":"GET","msg":"request completed","path":"/api/core/v2/namespaces/default/events","size":33,"status":404,"time":"2020-01-06T14:46:40-07:00"}

{"component":"apid","duration":"0.338ms","level":"info","method":"GET","msg":"request completed","path":"/api/core/v2/namespaces/default/checks","size":410,"status":200,"time":"2020-01-06T14:46:35-07:00"}

{"component":"apid","duration":"0.055ms","level":"info","method":"GET","msg":"request completed","path":"/api/core/v2/namespaces/default/entities/myles-macbook","size":33,"status":404,"time":"2020-01-06T14:58:15-07:00"}

Your Environment

• v5.16.1 from release tar.gz
• Compiled binaries following README.md using go 1.13.3
• MacOS 10.14

[echlebek]

This is quite strange. We have QA that covers all of the things that you did. I see that you mentioned on the community slack that you tried this for the docker images too.

Can you give a script that will allow reproducing this error for the docker images?

[mylesw42]

Sure, here's a quick script that reproduces the issue in docker.

#!/bin/bash

docker pull sensu/sensu

docker run \
  -d --name sensu-backend \
  -p 3000:3000 -p 8080:8080 -p 8079:8081 sensu/sensu:latest \
  sensu-backend start --state-dir /var/lib/sensu/sensu-backend --log-level debug

# give time for the process to start
sleep 3

# configure sensuctl
docker exec -t sensu-backend /usr/local/bin/sensuctl configure --url http://localhost:3000 --username admin --password "P@ssw0rd!" -n

# query cluster health, entities and events
echo ''
echo '### status commands ###'
docker exec -t sensu-backend /usr/local/bin/sensuctl cluster health
docker exec -t sensu-backend /usr/local/bin/sensuctl entity list
docker exec -t sensu-backend /usr/local/bin/sensuctl event list

echo ''
echo '### docker logs ###'
docker logs sensu-backend

[echlebek]

Oh gosh, I just realized docker has been broken on my machine since I upgraded to Fedora 31 :sweat_smile:

I'll post back when I have some info.

[echlebek]

I ran the script, and it did indeed reproduce what you saw. However, I noticed pretty soon after that you're connecting sensuctl to port 3000 instead of 8080.

3000 is the port that serves the dashboard, and 8080 is the port that serves the API, of which sensuctl is a client.

There's actually some requests that have the same format for these two services, which explains why some requests worked but others didn't.

Changing the port from 3000 to 8080 for sensuctl configure resolves the problem.