Atlantis: Azure DevOps - "0 projects are to be planned based on their when_modified config"

[jimmyjamesbaldwin]

Hi, I'm following the instructions to configure atlantis with Azure Devops but am running into some issues; I'm able to get atlantis working with GitLab which adds to the confusion.

Version: runatlantis/atlantis:v0.10.1
Environment vars for azure creds setup and ATLANTIS_REPO_CONFIG_JSON with "allow_custom_workflows":true.

I have a test project structure like so:

Directory structure

|-- atlantis.yaml
|-- testing
|    |-- provider.tf
|    |-- main.tf

atlantis.yaml:

version: 3
projects:
- dir: testing
  workflow: test
workflows:
  test:
    plan:
      steps:
      - run: echo foo
      - plan

testing/main.tf

resource "azurerm_resource_group" "test" {
  name     = "just_a_test_resource"
  location = "West US"
}

testing/provider.tf

provider "azurerm" {
  version = "=1.36.0"
}

if I modify the name of the test resource and create a merge request, atlantis receives the webhook and prints:

2019/11/19 22:34:49+0000 [INFO] middleware.go:38 server: POST /events – from 10.244.0.1:1344
2019/11/19 22:34:49+0000 [DBUG] events_controller.go:131 server: Handling AzureDevops post
2019/11/19 22:34:49+0000 [DBUG] events_controller.go:227 server: Request valid
2019/11/19 22:34:49+0000 [DBUG] events_controller.go:240 server: Handling as pull request event
2019/11/19 22:34:49+0000 [INFO] events_controller.go:511 server: Identified event as type "opened"
2019/11/19 22:34:49+0000 [INFO] events_controller.go:345 server: Executing autoplan
2019/11/19 22:34:49+0000 [INFO] middleware.go:42 server: POST /events – respond HTTP 200
2019/11/19 22:34:49+0000 [DBUG] project_command_builder.go:106 <username>/<repo>/<repo>#47: Got workspace lock
2019/11/19 22:34:49+0000 [DBUG] project_command_builder.go:114 <username>/<repo>/<repo>#47: 2 files were modified in this pull request
2019/11/19 22:34:49+0000 [INFO] working_dir.go:123 <username>/<repo>/<repo>#47: Creating dir "/home/atlantis/.atlantis/repos/<username>/<repo>/<repo>/47/default"
2019/11/19 22:34:50+0000 [DBUG] working_dir.go:189 <username>/<repo>/<repo>#47: Ran: git clone --branch test_merge --depth=1 --single-branch <redacted>@dev.azure.com/<redacted>/<redacted>/_git/<redacted> /home/atlantis/.atlantis/repos/<username>/<repo>/<repo>/47/default. Output: Cloning into '/home/atlantis/.atlantis/repos/<username>/<repo>/<repo>/47/default'...
2019/11/19 22:34:50+0000 [INFO] project_command_builder.go:135 <username>/<repo>/<repo>#47: Successfully parsed atlantis.yaml file
2019/11/19 22:34:50+0000 [DBUG] project_finder.go:84 <username>/<repo>/<repo>#47: Checking if project at dir "testing" workspace "default" was modified
2019/11/19 22:34:50+0000 [INFO] project_command_builder.go:140 <username>/<repo>/<repo>#47: 0 projects are to be planned based on their when_modified config
2019/11/19 22:34:50+0000 [INFO] command_runner.go:121 <username>/<repo>/<repo>#47: Determined there was no project to run plan in

I've tried setting up a new project and have checked the webhook settings, but am at a bit of a loss... I also tried manually setting when_modified in the project definition, but it doesn't appear to make a difference. Can anyone tell me what I'm doing wrong?

screenshots for clarity:

[jimmyjamesbaldwin]

Likely unrelated, but additionally when using the 'Test' service hook button in Azure, the example request sent to Atlantis generates an error in the logs:

2019/11/20 14:34:13+0000 [INFO] middleware.go:38 server: POST /events – from 10.244.0.1:2560
2019/11/20 14:34:13+0000 [DBUG] events_controller.go:131 server: Handling AzureDevops post
JSON unmarshal err: &azuredevops.GitPullRequest{Links:(*map[string]azuredevops.Link)(nil), ArtifactID:(*string)(nil), AutoCompleteSetBy:(*azuredevops.IdentityRef)(nil), ClosedBy:(*azuredevops.IdentityRef)(nil),ClosedDate:(*time.Time)(nil), CodeReviewID:(*int)(nil), Commits:[]*azuredevops.GitCommitRef(nil), CompletionOptions:(*azuredevops.GitPullRequestCompletionOptions)(nil), CompletionQueueTime:(*time.Time)(nil), CreatedBy:(*azuredevops.IdentityRef)(nil), CreationDate:(*time.Time)(nil), Description:(*string)(nil), ForkSource:(*azuredevops.GitRef)(nil), IsDraft:(*bool)(nil), Labels:[]*azuredevops.WebAPITagDefinition(nil), LastMergeCommit:(*azuredevops.GitCommitRef)(nil), LastMergeSourceCommit:(*azuredevops.GitCommitRef)(nil), LastMergeTargetCommit:(*azuredevops.GitCommitRef)(nil), MergeFailureMessage:(*string)(nil), MergeFailureType:(*string)(nil), MergeID:(*string)(nil), MergeOptions:(*azuredevops.GitPullRequestMergeOptions)(nil), MergeStatus:(*string)(nil), PullRequestID:(*int)(nil), Repository:(*azuredevops.GitRepository)(0xc0001ce580), Reviewers:[]*azuredevops.IdentityRefWithVote(nil), RemoteURL:(*string)(nil), SourceRefName:(*string)(nil), Status:(*string)(nil), SupportsIterations:(*bool)(nil), TargetRefName:(*string)(nil), Title:(*string)(nil), URL:(*string)(nil), WorkItemRefs:[]*azuredevops.ResourceRef(nil)}

 &time.ParseError{Layout:"\"2006-01-02T15:04:05Z07:00\"", Value:"\"0001-01-01T00:00:00\"", LayoutElem:"Z07:00", ValueElem:"\"", Message:""}

2019/11/20 14:34:13+0000 [DBUG] events_controller.go:227 server: Request valid
2019/11/20 14:34:13+0000 [EROR] events_controller.go:232 server: Failed parsing webhook: parsing time ""0001-01-01T00:00:00"" as ""2006-01-02T15:04:05Z07:00"": cannot parse """ as "Z07:00" Request-Id=|f1ea1a13dfb0004980c8438c2d127698.6bb7de3024990f46.
2019/11/20 14:34:13+0000 [INFO] middleware.go:42 server: POST /events – respond HTTP 400

[mcdafydd]

Thanks for finding that parsing bug! I'm pretty sure the error above is caused by my go-azuredevops library not handling the time formats sent in the webhooks. The new Microsoft Go azure-devops SDK mentions it.

Just out of curiosity, what ResourceVersion do you have selected in the webhook subscription?

I'll address the time parsing issue in go-azuredevops for now. Maybe we can move to the Microsoft library at some point.

Regarding the other issue, I'll dig into that a bit more. I haven't been relying on auto-planning in our workflow yet so I didn't even notice.

[amasover]

I ran into the time format bug as well. To work around it for now, I've switched to the old UI for Service Hooks, since the new UI doesn't seem to let you save a web hook without testing it first.

[jimmyjamesbaldwin]

Thanks for finding that parsing bug! I'm pretty sure the error above is caused by my go-azuredevops library not handling the time formats sent in the webhooks. The new Microsoft Go azure-devops SDK mentions it.

Just out of curiosity, what ResourceVersion do you have selected in the webhook subscription?

I'll address the time parsing issue in go-azuredevops for now. Maybe we can move to the Microsoft library at some point.

Regarding the other issue, I'll dig into that a bit more. I haven't been relying on auto-planning in our workflow yet so I didn't even notice.

Thanks for looking into it; I'm using Version 1.0 for the Pull Request Created & Pull Request Updated hooks, and Version 2.0 for the Pull Request Commented On hook, as per the docs here: https://www.runatlantis.io/docs/configuring-webhooks.html#azure-devops

[mcdafydd]

@jimmyjamesbaldwin I have a fix that addresses the time bug, at least as far as the test webhooks indicate. I'll submit a PR for that shortly.

It looks like the autoplan wasn't matching based on the when_modified dates because of a missing leading slash in the project dir in the repo atlantis.yaml. One of our repos has a lot of projects and all of them were missing a leading slash. This wasn't working:

- dir: path/to/prod
  workflow: prod

this also wasn't working, but seems like it should have based on the docs:

- dir: ./path/to/prod
  workflow: prod

But this matched successfully and ran autoplan:

- dir: /path/to/prod
  workflow: prod

[jimmyjamesbaldwin]

Adding the slash before the directory indeed solved my problem - many thanks @mcdafydd.

Now that things are working I noticed a minor UI issue when atlantis comments on the PR; it looks like azure devops doesn't support emojis so we see the alt text instead. Doesn't bother me but could make things look prettier:

[t3mi]

Could this please be re-opened and fixed?
As according to the docs there is no indication for the slash in front of the directory path.

[amasover]

@t3mi Perhaps we could open a new issue to track the leading slash thing? I ran into it as well and it confused the heck out of me.

[t3mi]

@amasover it would be better to track it here unless @jimmyjamesbaldwin doesn't want to reopen it for some reason.

[lkysow]

This is documented but I'm going to push a fix that converts absolute dirs because it's unnecessarily confusing!

[lkysow]

Oops! Actually I realize the issue now (disregard my above comment). Azure devops was using absolute paths instead of relative paths so that's why you needed to set an absolute path in the project config. #888 will fix that.