Dashboard: Display all info for a Pod

[digitalfishpond]

Currently, we only show limited information in the details page of any given pod. We should be showing all information

Steps to reproduce

Navigate to the details page of any pod:

Observed result

Expected result

Details should include all info from a pod, such as:

# Please edit the object below. Lines beginning with a '#' will be ignored,`
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubernetes.io/created-by: |
      {"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicationController","namespace":"default","name":"integration-test-valid-rc-saddssdadsasdadsadsasdasdasaddsasdas-sss","uid":"f926c650-1842-11e6-8952-42010af000fb","apiVersion":"v1","resourceVersion":"18735332"}}
    kubernetes.io/limit-ranger: 'LimitRanger plugin set: cpu request for container
      php-redis'
  creationTimestamp: 2016-09-05T13:01:10Z
  generateName: integration-test-valid-rc-saddssdadsasdadsadsasdasdasaddsasdas-sss-
  labels:
    app: guestbook
    tier: frontend2
  name: integration-test-valid-rc-saddssdadsasdadsadsasdasdasaddsapj3nx
  namespace: default
  resourceVersion: "18735345"
  selfLink: /api/v1/namespaces/default/pods/integration-test-valid-rc-saddssdadsasdadsadsasdasdasaddsapj3nx
  uid: d13174b3-7368-11e6-8d2f-42010af000fb
spec:
  containers:
  - image: gcr.io/google_samples/gb-frontend:v4
    imagePullPolicy: IfNotPresent
    name: php-redis
    ports:
    - containerPort: 80
      protocol: TCP
    resources:
      requests:
        cpu: 100m
    terminationMessagePath: /dev/termination-log
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-3kz4k
      readOnly: true
  dnsPolicy: ClusterFirst
  nodeName: gke-bryk-cluster-v1-2-default-pool-b632f6bc-2nvp
  restartPolicy: Always
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  volumes:
  - name: default-token-3kz4k
    secret:
      secretName: default-token-3kz4k
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: 2016-09-05T13:01:19Z
    status: "True"
    type: Ready
  containerStatuses:
  - containerID: docker://ae2c630c8c30825a76662e51478d7732ff822973e246e17827d61beb41ecf407
    image: gcr.io/google_samples/gb-frontend:v4
    imageID: docker://ef7a6d0ef6020b941e0a2824e18dff83ab85f7bd204dd59f23ac2e4c68804eda
    lastState: {}
    name: php-redis
    ready: true
    restartCount: 0
    state:
      running:
        startedAt: 2016-09-05T13:01:18Z
  hostIP: 10.242.0.17
  phase: Running
  podIP: 10.148.0.16
  startTime: 2016-09-05T13:01:17Z`

Comments

This is a huge amount of information, so we need to come up with a concise/pretty/useful way of making it all available.

[bryk]

@romlein @Lukenickerson Can we use it as a testbed for UX design for individual pages? I.e., design UX for this page and then extrapolate it for all other resources. This page will be among the most complex ones.

[digitalfishpond]

I'll get started on a possible page hierarchy that I'll send to @romlein for review/improvement. I'll post here with progress.

[digitalfishpond]

Hierarchy posted [here] - anyone with link can edit. (https://docs.google.com/drawings/d/1vMbsBZlnOeGh3sACaKtrliGgGijJc54Ea4Q0nzfLoqU/edit?usp=sharing).
@romlein @bryk

[bryk]

@digitalfishpond Grat job! What did you use to create this? I'm asking because I'm pretty sure there's some info missing. E.g., containers have env vars.

Also, can you link between objects in the visualization? E.g., link from env var to a config map, because evn var source can be a config map. And link from a pod to a node, because a pod runs on a node :)

[digitalfishpond]

Yes, I'm currently working on improving the info. But for that I need to understand it first, so I'm starting ground up, could take a few hours... The goal is to eventually have a fully linked diagram.

[danielromlein]

Hey just catching up with this now. This looks great @digitalfishpond! I see you're currently editing so I'll let you finish that up before fully trying to wrap my head around the IA.

As a general principle I think we should be cautious about showing everything available simply because we _can_. Has this full list of Pod details been evaluated as necessary to show in the UI?

@bryk I think using this as template for other detail pages is a good idea. Will there be other pages similar enough to this that they'd benefit from some sort of standardization?

[digitalfishpond]

Ok, All the information currently available to me (that I know about) is now included in the diagram. Please let me know if there is more, or if it could be displayed in a more useful way. @bryk was suggesting showing relationships between items. Could I ask you for an example and where I might find the information to add more?

[bryk]

@bryk I think using this as template for other detail pages is a good idea. Will there be other pages similar enough to this that they'd benefit from some sort of standardization?

Controller pages can have similar layout. In fact, they can embed this pod page as all controllers define a pod template. So you could display all you have here, except for the "status" section.

[bryk]

As a general principle I think we should be cautious about showing everything available simply because we can. Has this full list of Pod details been evaluated as necessary to show in the UI?

Yes. But at the end of the day, all info here is needed for some use cases. I'd go with progressive disclosure via expand/collapse or "show more".

[digitalfishpond]

@romlein , what do you think would work best here? I was thinking something like:

• Top level (Pod Detail landing page): essential info (as currently listed) plus three access points for the three top-level objects: Object Meta, Spec, and Status that could be links to seperate pages? or buttons to 'show more' within the same page? Gmail folder-style arrows to open/close hierarchy? hyperlinked cards with essential info that link to page with complete info, or that simply expand to display complete information?
• Depending on the top level solution, the next levels could follow the same pattern, or could show the properties for that object, and any property whose value is a sub-object could just display the word: 'show' as a link that would re-direct to a page for that object, or just open...

Of course whichever solution we choose we must think of an easy way for users to clean up once they've explored a hierarchy path (like a 'close all' or a 'back' button or something).

[bryk]

Also, we may want to display pod spec info for all controller objects (e.g., RCs or deployments).

[danielromlein]

@digitalfishpond sorry for not getting back to you on this sooner – I was out last week.
In your diagram I'm not sure I entirely follow how the external objects like _Volume_ fit with this hierarchy – could you expound on that please?

Also helpful for me would be realistic content to work with. For example, will Pod _State_ be one of those three values, "Waiting", "Running", or "Terminated"? What might the value for _Reason_ look like? Which categories will always have corresponding values? Which might not be shown at all? etc.

I could see here potentially using tertiary nav tabs as we have in the most recent navigation design. Or also perhaps a card appearing from the right side of the screen after being triggered by a user selection.

I'll ping Keith Guerin and see if he's available to help out with this design.

/ cc @bryk

[digitalfishpond]

Yes, of course. So Volume, ConfigMap and Secret are all resources that already exist in Dashboard. In PodSpec, there is a collection (array) called 'Volumes' that references the _Volume_ objects that belong to the Pod. These individual _Volume_ s will already be visible in their own 'Volume' list/detail views.

Still in PodSpec, the 2 objects 'InitContainers' and 'Containers' are both collections (arrays) of objects of type 'Container', and Container objects have a property of type 'EnvVar' (called 'Env') which in turn has a property of type 'EnvVarSource' (called 'ValueFrom'), which in turn has a property of type 'ConfigMapKeySelector' (called 'ConfigMapKeyRef') and another one of type 'SecretKeySelector' (called 'SecretKeyRef') which respectively can refer to a specific ConfigMap and/or a specific Secret (which will already have their own details page in Dashboard).

Do let me know if there is anything I can further clarify, I realise it's not super clear... The file I used to work out what object contains what is this one.

[edouardKaiser]

Regarding displaying more pod information, I think the lastState section might be very useful. In the dashboard we can see that a pod has been restarted (Restarts counter), but there is no way to see why (OOM...). The lastState would provide that information.

[digitalfishpond]

pod>Status>InitContainerStatuses>LastTerminationState
pod>Status>ContainerStatuses>LastTerminationState

..are both of type 'Container State' with json:"lastState,omitempty" (already ncluded in the diagram)

Which leads me to the following question: should we go with the actual property names? WIth the types? With the json names? WIth a human-friendly descriptive name?

[danielromlein]

@digitalfishpond alright, met with some of the GCP UX team at KubeCon and discussed this issue, and I think we've got a good path forward. We decided that since it sounds like use cases for showing a Pod's detailed info are diverse / not well understood, the best approach is to start with a very simple information architecture and iterate on that once it's live and we have some feedback to work with.

Things I need to execute designs:

• Realistic dummy data for wireframing. Sorry, I looked at the types.go file you linked to but I was having a hard time deciphering that. I'd prefer a spreadsheet so we could keep the Drawing neat but still have realistic content to pull from. This I think will help me understand the relationship between things like ValueFrom and ConfigMap (that relationship's a bit unclear to me now).
• Value ranges (e.g. min and max expected number of [Volumes])
• Information around which (if any) attributes will be present in _all_ cases vs. which will be present only some of the time.
• A decision on your question:
• > should we go with the actual property names? With the types? With the json names? With a human-friendly descriptive name?
• And since we have a lot of ungrouped things (RestartPolicy, TerminationGracePeriodSeconds, ActiveDeadlineSeconds, DNSPolicy, NodeSelector, etc) it would be nice to come up with a category for those so we have at least a basic hierarchy in place.

Thanks!

[bryk]

Realistic dummy data for wireframing. Sorry, I looked at the types.go file you linked to but I was having a hard time deciphering that. I'd prefer a spreadsheet so we could keep the Drawing neat but still have realistic content to pull from. This I think will help me understand the relationship between things like ValueFrom and ConfigMap (that relationship's a bit unclear to me now).

We did an object mapping exercise with @rf232 @olekzabl @mijajasinski as explained here: https://www.youtube.com/watch?v=iLwJOh_98uw. We'll process the results and share tomorrow. Hope this will help in this point.

[bryk]

So we're finally getting back. We did the object mapping exercise yesterday with the eng team. It happened to be harder than anticipated. During 4 hours session we listed all objects we have in the UI and analyzed three of them.

Here's the spreadsheet we've created: https://docs.google.com/spreadsheets/d/1KIFKt95Y3nDEhl3li-L7n5zY6jiHGT3GOPB9-SPCgpo/edit#gid=0

Thoughts:

1. Pod has 10 pieces of core info, 8 pieces of metadata, 17 relationships to other objects and 6 actions. And that's when we did not unfold Container object, which is the biggest child of a pod.
2. We didn't know whether "Pod Specification" should be an object or not. In code, and possibly in the UI, this is a thing reused in all types of workloads (RCs, RSs, Pods, etc.). But is not a thing you can look at in isolation.

Is this any helpful?

[danielromlein]

@bryk this is a great start! Might something like a Google Drawing be a more flexible way to drag-and-drop and thus prioritize and establish relationships between objects? Once _that's_ established, I think a spreadsheet would be a fantastic way to collect value ranges and that realistic dummy data I mentioned before.
It'd_ also be helpful for me to see those object types (e.g. _core info_, _metadata_) and their corresponding color displayed in some sort of legend for quick-reference.

Specific questions:

• This seems like a smaller list than @digitalfishpond's original Hierarchy Google Drawing – does this list for a Pod account for all we'd want to show in Dashboard?
• What's the reason for the appearances of _Pod Status_, such as "_Pod Status_ has Pod Conditions"?
• Are these elements you've tackled thus far ordered from most to least important? That's important to capture as well.

@normalnorman thoughts on this progress?

[bryk]

This seems like a smaller list than @digitalfishpond's original Hierarchy Google Drawing – does this list for a Pod account for all we'd want to show in Dashboard?

We lacked the time to unfold all properties of containers. Most of Pod's info is about containers themselves.

What's the reason for the appearances of Pod Status, such as "Pod Status has Pod Conditions"?

The reason is that this is runtime information of a pod. Called status. We made this distinction because all Workloads contain "Pod Spec", but only actual pods contain status. E.g., Deployments have "Pod spec", RCs too, but Pods contain "Pod Spec" + "Pod status". And the question on how to display this arises here.
Should all workloads have "Pod Spec" sections? And then Pods would have additional "Pod status" section? Or Pod should augument "Pod Spec" with live data?

Are these elements you've tackled thus far ordered from most to least important? That's important to capture as well.

Pod is most important here. This is because when we do Pod, then most of the workloads will be pretty similar.

[danielromlein]

Pods contain "Pod Spec" + "Pod status"

I'm still unclear on this point. Is there a section of the docs you could link me to that explains the distinction?

Are these elements you've tackled thus far ordered from most to least important? That's important to capture as well.

Pod is most important here. This is because when we do Pod, then most of the workloads will be pretty similar.

The ordering within each column. Meaning, is Active deadline seconds more important than Termination grace period seconds, etc.?

[danielromlein]

If _core info_, _metadata_, and _relationships_ are the main buckets we're using, what do we want to call these in the UI?

[danielromlein]

Blocking for design:

• That card-sorting spreadsheet prioritized: for each object and category, what are the most important pieces of data? I'd recommend a Google Drawing for this for its draw-and-drop capability, but a table works too.
• Realistic dummy data. A table is probably best for this...?
• Value ranges (e.g. min and max expected number of [Volumes])
• Information around which (if any) attributes will be present in all cases vs. which will be present only some of the time.

@bryk will be putting together a wireframe / information architecture diagram capturing [for a Pod] what all the core pieces of information are: i.e. this list plus _Conditions_, _Resource Graphs_, etc.

Thanks!

cc @rf232

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[floreks]

Right now we display basically all useful information that we can get from a pod.

/close

[k8s-ci-robot]

@floreks: Closing this issue.

In response to this:

Right now we display basically all useful information that we can get from a pod.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.