Google-cloud-ruby: spanner: skipped tests should work with the emulator v0.8.0

[hengfengli]

The emulator team suggests that we can skip the following tests for the new version of emulator:

1) Skipped:
Spanner Batch Client::execute_partition::spanner#test_0002_queries all by default [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/batch_client/execute_partition_test.rb:78]:
Skipped, no message given

  2) Skipped:
Spanner Batch Client::execute_partition::spanner#test_0003_queries all by default with query options [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/batch_client/execute_partition_test.rb:105]:
Skipped, no message given

  3) Skipped:
Spanner Batch Client::execute_partition::spanner#test_0001_reads all by default [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/batch_client/execute_partition_test.rb:48]:
Skipped, no message given

  4) Skipped:
Spanner Client::batch_update::spanner#test_0002_raises InvalidArgumentError when no DML statements are executed in a batch [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/batch_update_test.rb:69]:
Skipped, no message given

  5) Skipped:
Spanner Client::batch_update::spanner#test_0001_executes multiple DML statements in a batch [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/batch_update_test.rb:40]:
Skipped, no message given

  6) Skipped:
Spanner Client::batch_update::spanner#test_0004_runs execute_update and batch_update in the same transaction [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/batch_update_test.rb:115]:
Skipped, no message given

  7) Skipped:
Spanner Client::large_data::spanner#test_0001_writes and reads large random data [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/large_data_test.rb:60]:
Skipped, no message given

  8) Skipped:
Spanner Client::large_data::spanner#test_0002_writes and queries bytes [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/large_data_test.rb:88]:
Skipped, no message given

 10) Skipped:
Spanner Client::snapshot::spanner#test_0001_runs a query [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/snapshot_test.rb:34]:
Skipped, no message given

 11) Skipped:
Spanner Client::snapshot::spanner#test_0004_runs a query with strong option [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/snapshot_test.rb:87]:
Skipped, no message given

 12) Skipped:
Spanner Client::snapshot::spanner#test_0008_runs a query with staleness option [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/snapshot_test.rb:155]:
Skipped, no message given

 13) Skipped:
Spanner Client::snapshot::spanner#test_0014_staleness reads are consistent even when updates happen [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/snapshot_test.rb:278]:
Skipped, no message given

 14) Skipped:
Spanner Client::snapshot::spanner#test_0015_staleness queries are consistent even when updates happen [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/snapshot_test.rb:302]:
Skipped, no message given

 15) Skipped:
Spanner Client::snapshot::spanner#test_0002_runs a query with query options [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/snapshot_test.rb:52]:
Skipped, no message given

 16) Skipped:
Spanner Client::snapshot::spanner#test_0006_runs a query with timestamp option [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/snapshot_test.rb:121]:
Skipped, no message given

 21) Skipped:
Spanner Database Operations::spanner#test_0001_list database operations [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/database_operations_test.rb:23]:
Skipped, no message given

 22) Skipped:
Spanner Client::pdml::spanner#test_0001_executes a simple Partitioned DML statement [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/pdml_test.rb:33]:
Skipped, no message given

 23) Skipped:
Spanner Client::pdml::spanner#test_0002_executes a simple Partitioned DML statement with query options [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/pdml_test.rb:46]:
Skipped, no message given

 25) Skipped:
Spanner Client::single_use::spanner#test_0007_runs a query with bounded_timestamp option [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/single_use_test.rb:118]:
Skipped, no message given

 26) Skipped:
Spanner Client::single_use::spanner#test_0005_runs a query with staleness option [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/single_use_test.rb:90]:
Skipped, no message given

 27) Skipped:
Spanner Client::single_use::spanner#test_0009_runs a query with bounded_staleness option [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/single_use_test.rb:146]:
Skipped, no message given

 28) Skipped:
Spanner Client::single_use::spanner#test_0001_runs a query with strong option [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/single_use_test.rb:34]:
Skipped, no message given

 29) Skipped:
Spanner Client::single_use::spanner#test_0003_runs a query with timestamp option [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/single_use_test.rb:62]:
Skipped, no message given

 30) Skipped:
Spanner Client::transaction::spanner#test_0004_supports tx isolation with read and update [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/transaction_test.rb:131]:
Skipped, no message given

What I have found is that:

No => test fails when unskipping the test.
Yes => test succeed when unskipping the test.

1. No
2. No
3. No
4. No
5. No
6. No
7. Yes
8. Yes
10. No
11. No
12. No
13. No
14. No
15. No
16. No
21. No
22. Yes
23. Yes
25. No
26. No
27. No
28. No
29. No
30. No, tests get stuck forever.

I have unskipped 7,8,22,23 in PR: https://github.com/googleapis/google-cloud-ruby/pull/6822.

Also, 2 and 15 are related to query options and I don't think the emulator can support it right now.

However, the following tests should work with the emulator (because they work fine in other languages):

• batch and partitioned query tests
• the staleness options and large reads

Can we investigate why they do not work? Is it a client lib test issue or an emulator issue?

[skuruppu]

@jiren any updates on this? The emulator v0.8.0 release is already published so it would be good to figure out why these tests that are meant to be passing are failing in Ruby in case customers run into the same issues.

[jiren]

@skuruppu I have debugged a few test cases today, I will complete pending test cases by tomorrow and submit a report with failed reasons.

[hengfengli]

@jiren Is there any progress about this issue?

[jiren]

@jiren Is there any progress about this issue?

@hengfengli I have updated most of the skip test cases to pass against emulator.

[hengfengli]

@jiren Thanks a lot for fixing the skipped tests.

I notice that the following tests are still skipped:

  5) Skipped:
Spanner Client::batch_update::spanner#test_0001_executes multiple DML statements in a batch [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/batch_update_test.rb:40]:
Skipped, no message given

  6) Skipped:
Spanner Client::batch_update::spanner#test_0004_runs execute_update and batch_update in the same transaction [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/client/batch_update_test.rb:115]:
Skipped, no message given

 21) Skipped:
Spanner Database Operations::spanner#test_0001_list database operations [/usr/local/google/home/adirastogi/emulator_080/ruby/google-cloud-ruby/google-cloud-spanner/acceptance/spanner/database_operations_test.rb:23]:
Skipped, no message given

What are the issues? Are they issues from emulator? May I can report them to the emulator team.

[jiren]

@hengfengli Yes issues from emulator.

5) and 6) Result status got nil from emulator for batch dml results.
https://github.com/googleapis/google-cloud-ruby/blob/8e5c1c09fe88c4c9c13915ea93893a858ccd723e/google-cloud-spanner/lib/google/cloud/spanner/service.rb#L332

Poto ref:
https://github.com/googleapis/googleapis/blob/4f4aa3a03e470f1390758b9d89eb1aa88837a5be/google/spanner/v1/spanner.proto#L644

21) list database operations are not implemented on emulator
Error: Google::Cloud::UnimplementedError: 12

[hengfengli]

Thanks. I'll let them know about this.