Flank: Move shards.json to results directory

[adamfilipow92]

Describe the bug

A clear and concise description of what the bug is.

Currently, flank always saves shards file in the current directory. This behavior can cause overriding this file when user run flank multiple times at the same time.

Expected behavior

A clear and concise description of what you expected to happen.

When we run tests we should save shards file to result directory. This behavior can cause overriding this file when user run flank multiple times at the same time.

[bootstraponline]

Makes sense. Thanks for catching this.

[adamfilipow92]

#1475 Save shards.json in the results directory

Change save the location of shard.json file.

Motivation

Currently, Flank always saves shards file in the current directory. This behavior can cause overriding this file when the user runs Flank multiple times at the same time.

Goals

The main goal is avoiding inconsistency between shard.json and results in case when Flank is executed multiple times at the same time.

Design

Currently in dumpShards we using a ANDROID_SHARD_FILE and IOS_SHARD_FILE. This is the reason why Flank saves dump to execution directory.

private fun AndroidMatrixTestShards.saveShards(obfuscateOutput: Boolean) = saveShardChunks(
    shardFilePath =  ANDROID_SHARD_FILE,
    shards = this,
    size = size,
    obfuscatedOutput = obfuscateOutput
)

fun IosArgs.dumpShards(
    shardFilePath: String = IOS_SHARD_FILE,
) 

The dump shard file should be saved to the current execution result directory.

API

Android

private fun AndroidMatrixTestShards.saveShards(obfuscateOutput: Boolean) = saveShardChunks(
    shardFilePath =  Paths.get(path_to_result_dir,ANDROID_SHARD_FILE).toString(),toString(),
    shards = this,
    size = size,
    obfuscatedOutput = obfuscateOutput
)

Flank should also check if the results directory currently exists and create if not.

iOS

Behavior should be similar to android

private fun IosArgs.dumpShardsIfXcTest() = takeIf { isXcTest }?.let {
   createSHards().saveShardChunks(
        shardFilePath =  Paths.get(path_to_result_dir,IOS_SHARD_FILE).
        shards = rawShards,
        size = size,
        obfuscatedOutput = obfuscateDumpShards
    )
    if (disableResultsUpload.not())
        GcStorage.upload(IOS_SHARD_FILE, resultsBucket, resultsDir)
}

Results

Flank was saving shards file to the current directory instead of the current execution result directory. This can cause inconsistency when Flank was used multiple at the same time.

Dependencies

The ticket is not blocked.

Ticket improve quality and reduce data inconsistency.

Testing

Dump shards file should be saved to result in directory instead of execution directory for android and ios.

Android

1. Run flank with android test
2. Wait for results-directory is created
3. Open the results directory and see android_shards.json file
4. Gcs should contain shards file equals to the local file

iOS

1. Run flank with ios test
2. Wait for results-directory is created
3. Open the results directory and see ios_shards.json file
4. Gcs should contain shards file equals to the local file

disable-results-upload

1. Run flank with android or ios test
2. Wait for results-directory is created
3. Open the results directory and see *_shards.json file
4. Flank shouldn't upload shards file to gcs when disable-results-upload set