Firebase-ios-sdk: AB tests not working when sending activation event right after activating remote config

[domagojb]

[REQUIRED] Step 2: Describe your environment

• Xcode version: 10.3
• Firebase SDK version: 6.6.0
• Firebase Component: Remote Config/AB Testing
• Component version: 4.2.1/3.0.0

[REQUIRED] Step 3: Describe the problem

We have an ongoing issue since last year where activating an experiment (i.e. sending an activation event) exactly after calling [[FIRRemoteConfig remoteConfig] activateFetched] didn't work. That is because activateFetched presumably did some property setting related to AB tests in the background. We figured that out by logging firebase messages to the console.

We found that also to be the case because doing this

[[FIRRemoteConfig remoteConfig] activateFetched];

// This is a workaround for A/B tests not working when the activation event is sent too soon after `activateFetched`.
// Basically we delay further UI initialization and Analytics event dispatch enough that Firebase AB Testing framework has
// time to set-up a conditional user property internally used for marking ab test behaviour with activation event.
dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(0.5 * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{

    // send activation event here
});

made it work. But obviously this is not the way we wan't to use the API.

We were thrilled to see that fetchAndActivateWithCompletionHandler which should have async completion was added. But it seems the issue is still not fixed. AB tests still don't get activated.

I was looking at the newly open sourced code of Remote Config and AB testing and I think I can pin point the problem:

**fetchAndActivateWithCompletionHandler** calls **activateFetched** which in turn calls **activateWithCompletionHandler**

**activateWithCompletionHandler**  calls **[strongSelf updateExperiments]** which does not have a completion handler but if you follow the  updateExperiments method down the rabbit hole you'll find a call like this in **FIRExperimentController**

- (void)updateExperimentsWithServiceOrigin:(NSString *)origin
                                    events:(FIRLifecycleEvents *)events
                                    policy:(ABTExperimentPayload_ExperimentOverflowPolicy)policy
                             lastStartTime:(NSTimeInterval)lastStartTime
                                  payloads:(NSArray<NSData *> *)payloads {
  FIRExperimentController *__weak weakSelf = self;
  dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
    FIRExperimentController *strongSelf = weakSelf;
    [strongSelf updateExperimentsInBackgroundQueueWithServiceOrigin:origin
                                                             events:events
                                                             policy:policy
                                                      lastStartTime:lastStartTime
                                                           payloads:payloads];
  });
}

which as you can see calls something on the background queue and it isn't synched to the original activateWithCompletionHandler. 

Steps to reproduce:

1. Create an AB test that requires an activation even

2. Call fetchAndActivateWithCompletionHandler and in the completion block send the activation event

Relevant Code:

[[FIRRemoteConfig remoteConfig] fetchAndActivateWithCompletionHandler:^(FIRRemoteConfigFetchAndActivateStatus status, NSError * _Nullable error) {

        if (error) {
            return;
        }

        [FIRAnalytics logEventWithName:@"ActivationEvent" parameters:nil];
    }];

[google-oss-bot]

I found a few problems with this issue:

• I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.
• This issue does not seem to follow the issue template. Make sure you provide all the required information.

[dmandar]

Thanks for the detailed report. We will update this thread based on progress on the issue.

[dmandar]

This is tracked in internal bug 141921647.

[domagojb]

I would just like to comment that since 6.9.0 we are getting a really low number of users in the A/B testing dashboard which is how the bug manifested before we implemented our hack. I would assume that our hack broke because of some internal changes?

Either way, this is a big deal for us because now we can't do any A/B testing.

[domagojb]

While debugging I noticed this message I get in the log:

[Firebase/ABTesting][I-ABT000003] Failed to get conditional user properties from Firebase Analytics.

No reason for failure given.

[dmandar]

Chris, we need updateExperimentsWithServiceOrigin to have a completion handler to fix this.

[christibbs]

Reassigning to @dmandar for work on the Remote Config side.

[domagojb]

Not to be a bother, but any update on this? I see that the backend API has been updated.