Terraform-provider-cloudflare: waf_rule data source test failure

[jacobbednarz]

Looking at the scheduled builds, we're now seeing two failures related to the WAF rule datasource.

------- Stdout: -------
=== RUN   TestAccCloudflareWAFRules_MatchDescription
=== PAUSE TestAccCloudflareWAFRules_MatchDescription
=== CONT  TestAccCloudflareWAFRules_MatchDescription
--- FAIL: TestAccCloudflareWAFRules_MatchDescription (38.62s)
    testing.go:640: Step 0 error: Check failed: Check 2/2 error: data.cloudflare_waf_rules.qpgfpgkafr: Attribute 'rules.#' expected "20", got "2081"
FAIL

------- Stdout: -------
=== RUN   TestAccCloudflareWAFRules_NoFilter
=== PAUSE TestAccCloudflareWAFRules_NoFilter
=== CONT  TestAccCloudflareWAFRules_NoFilter
--- FAIL: TestAccCloudflareWAFRules_NoFilter (43.18s)
    testing.go:640: Step 0 error: Check failed: Check 2/2 error: data.cloudflare_waf_rules.xhcpgelnxb: Attribute 'rules.#' expected "40", got "2907"
FAIL

Unsure if this is a change in the Cloudflare API response or just that our test case is fragile.

cc @XaF

[XaF]

This is actually (sort-of) both: fragile test case (Test expecting X results) and API client change.
Recently I've fixed a bug on the go client side that was only getting the first page of the results when listing WAF Rules (hence the "20" / "40" that were stable). By fixing this, these tests were thus broken (and probably some others ?). These should probably be replaced by a test checking that we have any result... and consider that the API is doing what it should I guess?
I can check how to make the change to fix those.

[jacobbednarz]

sounds like a plan! i'll have a look at this now and see what i can come up with. thanks for your insight.

[jacobbednarz]

590 should take care of this.

[XaF]

590 seems right!

I would suggest to do the same for waf_packages and waf_groups in the risk that the same might happen if there's packages/groups added/changed!

[jacobbednarz]

👍 i'll add it to my list to take a look at.