Terraform-provider-azurerm: Configure Data Sources in a Log Analytics Workspace

[josh-barker]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

I'd like to be able to configure Data Sources on a Log Analytics Workspace

New or Affected Resource(s)

• azurerm_log_analytics_workspace

and/or:

• azurerm_log_analytics_workspace_iis_logs
• azurerm_log_analytics_workspace_linux_performance_collection
• azurerm_log_analytics_workspace_linux_performance_object
• azurerm_log_analytics_workspace_linux_syslog
• azurerm_log_analytics_workspace_linux_syslogCollection
• azurerm_log_analytics_workspace_windows_event
• azurerm_log_analytics_workspace_windows_performance_counter

Note: There may be other data sources and properties that can be configured.

Potential Terraform Configuration

Inline example of configuring "datasource" properties in a Log Analytics workspace

resource "azurerm_log_analytics_workspace" "ws" {
  name                = "Demo"
  location            = "US East"
  resource_group_name = "Some-Resource-Group"
  sku                 = "Standard"

  windows_performance_counter { 
    object_name = "LogicalDisk"
    instance_name = "*"
    interval_seconds = 10
    counter_name = "Disk Writes/sec"
  }

  windows_performance_counter { 
    object_name = "LogicalDisk"
    instance_name = "*"
    interval_seconds = 10
    counter_name = "Free Megabytes"
  }

  linux_performance_collection {
    state = "Enabled"
  }

  linux_performance_object {
    performance_counters = ["% Processor Time", "% Privileged Time"]
    object_name = "Processor"
    instance_name = "*"
    interval_seconds = 10
  }
}

Configure "datasource" using a separate resource for a Log Analytics workspace.

resource "azurerm_log_analytics_workspace_iis_logs" "iis_logs" {
  name                = "iis_logs"
  resource_group_name = "Some-Resource-Group"

  state = "OnPremiseEnabled"
}

resource "azurerm_log_analytics_workspace_linux_performance_collection" "lin_perf_collection" {
  name                = "lin_perf_collection"
  resource_group_name = "Some-Resource-Group"

  state = "Enabled"
}

resource "azurerm_log_analytics_workspace_linux_performance_object" "lin_processor_performance" {
  name                = "lin_processor_performance"
  resource_group_name = "Some-Resource-Group"

  performance_counters = ["% Processor Time", "% Privileged Time"]
  object_name = "Processor"
  instance_name = "*"
  interval_seconds = 10
}

resource "azurerm_log_analytics_workspace_linux_syslog_collection" "lin_syslog_collection" {
  name                = "lin_syslog_collection"
  resource_group_name = "Some-Resource-Group"

  state = "Enabled"
}

resource "azurerm_log_analytics_workspace_linux_syslog_collection" "lin_syslog_collection" {
  name                = "lin_syslog_collection"
  resource_group_name = "Some-Resource-Group"

  syslog_name = "kern"
  severities = ["emerg", "alert", "crit", "err", "warning"]
}

resource "azurerm_log_analytics_workspace_windows_event" "win_event" {
  name                = "win_event"
  resource_group_name = "Some-Resource-Group"

  eventLogName = "Application"
  eventTypes = ["Error", "Warning"]
}

resource "azurerm_log_analytics_workspace_windows_performance_counter" "win_disk_writes" {
  name                = "win_disk_writes"
  resource_group_name = "Some-Resource-Group"

  object_name = "LogicalDisk"
  instance_name = "*"
  interval_seconds = 10
  counter_name = "Disk Writes/sec"
}

[laughtonsm]

This would be an awesome feature. Currently looking to do this and surprised to find no support already

[russjury]

I'd also like to see azurerm_log_analytics_workspace_custom_logs and azurerm_log_analytics_workspace_custom_fields be supported in this feature too - I think the mechanism is similar enough to be included. I can help with examples/how you'd do it via the API. An example potential implementation (there's more to it, but skipped for brevity):

resource "azurerm_log_analytics_workspace_custom_logs" "customlog01_CL" {
  name = "customlog01_CL"
  resource_group_name = "Some-Resource-Group"
  workspace_id = "00000000-0000-0000-0000-000000000000"
  description = "Description of custom log"
  extractions = []
  inputs = []
}

[tombuildsstuff]

hi @GCole64

Taking a look at the comments posted above, since they're coming in via email unfortunately we believe that you may be responding to the wrong email? This issue tracker is intended to track/discuss bugs and enhancements in the Terraform Provider for Microsoft Azure - as such we try to limit off-topic conversations.

Thanks!

[diegoitaliait]

Hi,
are there any new features for this new feature?

[RahulWarhekar]

looking forward for this feature

[ssrirama-aka]

My requirement is to use Terraform to link an activity log as data source into a given log analytics workspace. I started looking at issue https://github.com/terraform-providers/terraform-provider-azurerm/issues/4446 and followed it here. What I don't see captured yet in this issue is the need to configure Azure's Activity Log as a data source in a workspace.

Thinking along the lines that @josh-barker outlined, we could define a resource like this:

• azurerm_log_analytics_workspace_activity_log

With properties like this:

resource "azurerm_log_analytics_workspace_activity_log" "my_activity_log" {
  name                  = "some_name"
  subscription_id   = "subscriptionId"
  workspace_id      = "workspaceId"
}

[ssrirama-aka]

@tombuildsstuff Given the votes and utility value of this, could the enhancement be targeted to a 2.x release of the AzureRM provider?

[magodo]

I'll try to implement this, hopefully.

[patrickbsf]

I don't see how this closes out this case. There are still quite a view datasource types to add - it seems like #6321 only handles windows performance counters and windows events.
syslog, custom logs, etc. all still need handled.

[hashibot[bot]]

This has been released in version 2.4.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 2.4.0"
}
# ... other configuration ...

[katbyte]

thanks @patrickbsf, looks like this was automagically closed by that PR. reopening!

[magodo]

I'm wondering whether it make sense to just embed azurerm_log_analytics_workspace_linux_performance_collection and azurerm_log_analytics_workspace_linux_syslog_collection into azurerm_log_analytics_workspace_linux_performance_counter and azurerm_log_analytics_workspace_linux_syslog respectively. As it is rare to setup several "linux performance counter"/"linux syslog" data sources while disable them.

[magodo]

Currently, the progress of this issue is blocked by Azure/azure-rest-api-specs#9072. Once that issue has been addressed, we can move on to implementing the remaining data sources.

[PriyankaRanganath]

Currently, the progress of this issue is blocked by Azure/azure-rest-api-specs#9072. Once that issue has been addressed, we can move on to implementing the remaining data sources.

We have implemented above requested features as separate resources and raised PR for the same.
Regarding the REST API spec issue, We have referred equivalent powershell cmdlets for the log analytics data sources and implemented all properties accordingly.:
1) Linux syslog
2) Linux syslog collection enable and disable
3) Linux performance object
4) Linux performance collection
The PR needs review and approval.

@magodo / @tombuildsstuff, could you kindly review and approve the same?

[defadmindef]

I believe my need falls within this group. I am using terraform with the azurerm_log_analytics_solution to create a "OMSGallery /ServiceDesk". I then need to create a "Workspace Data Sources - ITSM Connections" connector which needs several pieces of data to make a connection to ServiceNow. I have found no way to include the necessary data in terraform and no way to export this Data from Azure. Is this the same as the title says?

[ranjankumarhashedin]

do we have any modules or third party modules to enable data source performance counter for linux type of instances in log analytics??