Azure-docs: Log Analytics queries and Tables are incorrect

[asubmani]

The queries for kube-audit are incorrect:

AzureDiagnostics
| where Category == "kube-audit"
| where log_s contains "nginx"
| project log_s

There is no Category named "kube-audit"
There is no column named log_s
I have enabled kube-audit in log analytics and Storageaccount.

Log Analytics categories

1. kube-apiserver
2. kube-controller-manager
3. kube-scheduler
4. kube-audit
5. cluster-autoscaler

Storage Account Categories

1. kube-audit

I noticed there are 2 new categories "kube-audit-admin" and "guard" would appreciate if the doc updates what is captured in these 2 categories. To be precise what is the different between kube-audit and kube-audit-admin?

AKS 1.16.7, RBAC enabled, MSI cluster.
Container Insights is working (except live stream which broke after AKS moved to MSI)

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 660320d4-fad5-3307-e29f-5f4cfad4b7f7
• Version Independent ID: 85ac2e8b-633f-6e1a-e642-0645ca92e129
• Content: API server authorized IP ranges in Azure Kubernetes Service (AKS) - Azure Kubernetes Service
• Content Source: articles/aks/api-server-authorized-ip-ranges.md
• Service: container-service
• GitHub Login: @mlearned
• Microsoft Alias: mlearned

[himanshusinha-msft]

Thanks for the feedback and bringing this to our notice . At this time we are reviewing the feedback and will update the document as appropriate .

[whoward-msft]

Hi @asubmani,

Can you further elaborate on which doc your referring to? The linked doc to this issue doesn't mention the above queries. Thanks!

[whoward-msft]

Hey @asubmani,

I believe you are referring to this doc. I've assigned the issue to that doc author to be fixed. If you are referring to an example kusto query on another page please feel free to let me know and I'll jump right back in.

[zr-msft]

Thank you @asubmani for the feedback!

The article has been updated to resolve the query issue. We've added an item to our back log for the rest of your feedback and will prioritize accordingly.

[asubmani]

Re-opening this as the updated doc omits "kubernetes auditing" in it's entirety, rather than explain where the kube-audit logs go in Log analytics.

The KubePodInventory table always existed and is certainly not for api-server or audit details.

This looks like a regression or an intentional deprecation of a standard kubernetes logging feature.

Audit-logs are used to find details on Roles,bindings, ClusterRoles,Bindings etc.. They contain information on request Metadata, Request, RequestResponse etc.

kube-apiserver, kube-audit etc.. used to exist as categories within "AzureDiagnostic" in log analytics. If they don't exist any more then either "Azure monitor for Containers" is having issues or it is a regression.

kubepodinventory does NOT contain information about:

• what happened?

• when did it happen?

• who initiated it?
• on what did it happen?
• where was it observed?
• from where was it initiated?
• to where was it going?

Here is a sample query when it used to work before (under AzureDiagnostic)

AzureDiagnostics
| where Category == "kube-audit"
| where parse_json(log_s).verb == "create"
| where parse_json(tostring(parse_json(tostring(parse_json(log_s).requestObject)).roleRef)).name == "cluster-admin"
| where parse_json(tostring(parse_json(log_s).requestObject)).kind == "ClusterRoleBinding"
| extend k8skind = parse_json(tostring(parse_json(log_s).requestObject)).kind
| extend k8sroleref = parse_json(tostring(parse_json(tostring(parse_json(log_s).requestObject)).roleRef)).name
| extend k8suser = parse_json(tostring(parse_json(log_s).user)).username
| extend k8sipaddress = parse_json(tostring(parse_json(log_s).sourceIPs))[0]

If the logs are no longer going to "AzureDiagnostic" table, I would like to know the new destination? They are not going to kubepodinventory now, nor did they ever go to kubepodinventory

[gossion]

Seems that latest document for kube-audit is not right, I have no idea about KubePodInventory either.

@asubmani, your query looks ok, but I guess kube-audit/kube-audit-admin is not enabled (or lost somehow) in your cluster. You can try to disable and re-enable it for a try. If the problem is not solved, please file a support ticket that oncall engineer can take a look.

For kube-audit/kube-audit-admin, yes, we are adding a new category for audit logs, and release note for it should be

• The AKS Kubernetes Audit logs are now split in 2 categories to allow you granularly subscribe and save costs.
  
  
  
  • kube-audit-admin: This category contains only audit events that include write verbs (create,update,delete,patch,post)
  
  
  • kube-audit: This category contains all remaining audit events.
  
  

[miwithro]

@asubmani can you tell me what the Diagnostic Settings are for your cluster? I have the below configured on my cluster and the initial command works.

I am able to query kube-audit results from the Logs.

We are going to clear up the document so that is reflected correctly. Thank you.

[asubmani]

@gossion @miwithro

I deployed a brand new cluster... same issue

NOTES: Log Analytics is in a different subscription. (Kubepodinbentory, kubeevents are fine)

MSFT support asked me to enable managed AAD V2, which I did. (This enables Container Live log stream)

My Log Analytics workspace is not hitting any limits, nor are any "Daily caps" to ingestion set.

Log Analytics usage

[asubmani]

@zr-msft @WinstonHoward-MSFT Please revert the changes to the doc. As per several users, it seems this is peculiar to my environment and the below query is correct.

AzureDiagnostics
| where Category == "kube-audit"

[miwithro]

@asubmani Thanks the changes will have been reverted. It will go out live a couple of days after Ignite. Please stay tuned for the update.