Grdb.swift: Breaking changes between v4.6.2 and v4.7.0

[micpringle]

What did you do?

Updated from v4.6.2 to v4.7.0

What did you expect to happen?

Everything would work as before since I don't use custom SQLite builds, and therefore the breaking change documented here doesn't apply.

What happened instead?

My app now crashes when attempting to execute the following query...

return try Entity.Motorsport.Event
  .including(all: Entity.Motorsport.Event.fastestLaps
    .including(required: Entity.Motorsport.FastestLap.driver
      .including(required: Entity.Motorsport.Driver.constructor)))
    .including(all: Entity.Motorsport.Event.raceStandings
      .order(Column("position"))
      .including(required: Entity.Motorsport.RaceStanding.driver
      .including(required: Entity.Motorsport.Driver.constructor)))
  .filter(Column("category") == category)
  .order(Column("eventTimestamp"))
  .asRequest(of: Entity.Motorsport.EventInfo.self)
  .fetchAll(db)

...but only when raceStandings isn't empty. The error message I get, although somewhat specific to my app, is as follows...

Fatal error: 'try!' expression unexpectedly raised an error: Swift.DecodingError.keyNotFound(CodingKeys(stringValue: "driverInfo", intValue: nil), Swift.DecodingError.Context(codingPath: [], debugDescription: "No such key: driverInfo", underlyingError: nil)): file /Users/micpringle/Projects/Scores/Pods/GRDB.swift/GRDB/Record/FetchableRecord+Decodable.swift, line 14
2020-03-16 11:20:13.067878+0000 Sp Score[19327:1291018] Fatal error: 'try!' expression unexpectedly raised an error: Swift.DecodingError.keyNotFound(CodingKeys(stringValue: "driverInfo", intValue: nil), Swift.DecodingError.Context(codingPath: [], debugDescription: "No such key: driverInfo", underlyingError: nil)): file /Users/micpringle/Projects/Scores/Pods/GRDB.swift/GRDB/Record/FetchableRecord+Decodable.swift, line 14

As the error mentions, the trigger is line 14 in GRDB.swift/standard/FetchableRecord+Decodable.swift

extension FetchableRecord where Self: Decodable {
  public init(row: Row) {
    // Intended force-try. FetchableRecord is designed for records that
    // reliably decode from rows.
    self = try! RowDecoder().decode(from: row) // <-- THIS LINE
  }
}

Just to reiterate, this query has worked fine in every version from v4.2.1 through to v4.6.2, only breaking in v4.7.0.

I have looked at the diffs and the release notes but nothing jumps out. I also installed each version between v4.2.1 and v4.7.0, and for each version I cleaned the build folder, deleted the app from the sim, and built and run, to determine where the breaking change was introduced.

Environment

GRDB flavor(s): GRDB
GRDB version: 4.7.0
Installation method: (CocoaPods, SPM, manual?) CocoaPods
Xcode version: 11.3.1
Swift version: 5.1.3
Platform(s) running GRDB: (iOS, macOS, watchOS?) iOS
macOS version running Xcode: 10.14.6

[groue]

Hello @micpringle,

Thanks for the report. I suspect this is related to #664, which had me add two commits that attempt at closing a loophole, but I'm not sure. So I don't know if we are witnessing a breaking change, or a bug in the app which has turned visible (I mean that I'm reluctant to call "breaking change" a bug fix, even when it reveals that an app was misusing an API).

I won't be able to do much without a reproducible case, though.

[groue]

return try Entity.Motorsport.Event
  .including(all: Entity.Motorsport.Event.fastestLaps
    .including(required: Entity.Motorsport.FastestLap.driver
      .including(required: Entity.Motorsport.Driver.constructor)))
    .including(all: Entity.Motorsport.Event.raceStandings
      .order(Column("position"))
      .including(required: Entity.Motorsport.RaceStanding.driver
      .including(required: Entity.Motorsport.Driver.constructor)))
  .filter(Column("category") == category)
  .order(Column("eventTimestamp"))
  .asRequest(of: Entity.Motorsport.EventInfo.self)
  .fetchAll(db)

BTW, I'm happy to see such a use of the query interface :-) Debugging those (whether it's a bug in the app or in GRDB) should be made easier!

[micpringle]

I will put together a sample app that reproduces the error, but it won't be today.

[groue]

That's fine. I'll be there when you're back.

[groue]

@micpringle, the error provides a hint, actually:

Swift.DecodingError.keyNotFound(CodingKeys(stringValue: "driverInfo", intValue: nil), ...)

Is there any association in the request whose association key is driverInfo?

return try Entity.Motorsport.Event
  .including(all: Entity.Motorsport.Event.fastestLaps
    .including(required: Entity.Motorsport.FastestLap.driver    // <- "driver", or "driverInfo"?
      .including(required: Entity.Motorsport.Driver.constructor)))
  .including(all: Entity.Motorsport.Event.raceStandings
    .order(Column("position"))
    .including(required: Entity.Motorsport.RaceStanding.driver  // <- "driver", or "driverInfo"?
      .including(required: Entity.Motorsport.Driver.constructor)))
  .filter(Column("category") == category)
  .order(Column("eventTimestamp"))
  .asRequest(of: Entity.Motorsport.EventInfo.self)
  .fetchAll(db)

Let's focus on one driverInfo Swift decodable property that should be fed from the first line commented above:

struct Entity.Motorsport.EventInfo {
  var event: Entity.Motorsport.Event
  var fastestLaps: [Entity.Motorsport.FastestLapInfo]
  var raceStandings: ...
}

struct Entity.Motorsport.FastestLapInfo {
  var lap: Entity.Motorsport.FastestLap
  var driverInfo: Entity.Motorsport.DriverInfo // <- fails
}

struct Entity.Motorsport.DriverInfo {
  var driver: Entity.Motorsport.Driver
  var constructor: Entity.Motorsport.Constructor
}

If I'm not too wrong, Entity.Motorsport.FastestLap.driver has the default association key, derived from the name of the associated database table: "driver":

extension Entity.Motorsport.FastestLap {
    static let driver = belongsTo(Driver.self) // association key "driver"
}

This association can feed a driver decodable property. But not driverInfo.

Before the #664 bugfix, GRDB would actually decode... something. Maybe not what you think. This was definitely a GRDB bug, which used to accept decodable records that don't match the request. A nasty bug, really 😰.

Now such a mismatch is made much more visible 😅💥

If I'm not wrong, a possible fix is the following:

return try Entity.Motorsport.Event
  .including(all: Entity.Motorsport.Event.fastestLaps
    .including(required: Entity.Motorsport.FastestLap.driver
      .forKey("driverInfo")                             // <- name it!
      .including(required: Entity.Motorsport.Driver.constructor))
  .including(all: Entity.Motorsport.Event.raceStandings
    .order(Column("position"))
    .including(required: Entity.Motorsport.RaceStanding.driver
      .forKey("driverInfo")                             // <- name it!
      .including(required: Entity.Motorsport.Driver.constructor))
  .filter(Column("category") == category)
  .order(Column("eventTimestamp"))
  .asRequest(of: Entity.Motorsport.EventInfo.self)
  .fetchAll(db)

I hope this solves your issue!

[groue]

But my favorite option is to keep your initial request, with their default association keys, and totally remove the DriverInfo concept:

struct Entity.Motorsport.EventInfo {
  var event: Entity.Motorsport.Event
  var fastestLaps: [Entity.Motorsport.FastestLapInfo]
  var raceStandings: ...
}

struct Entity.Motorsport.FastestLapInfo {
  var fastestLap: Entity.Motorsport.FastestLap
  var driver: Entity.Motorsport.Driver
  var constructor: Entity.Motorsport.Constructor
}

//    ^ All properties above have names that
//      match the default association keys,
//      that is to say table names.

let fastestLaps = Entity.Motorsport.Event.fastestLaps
  .including(required: Entity.Motorsport.FastestLap.driver
     .including(required: Entity.Motorsport.Driver.constructor))

let raceStandings = Entity.Motorsport.Event.raceStandings
  .including(required: Entity.Motorsport.RaceStanding.driver
     .including(required: Entity.Motorsport.Driver.constructor))

return try Entity.Motorsport.Event
  .including(all: fastestLaps)
  .including(all: raceStandings.order(Column("position")))
  .filter(Column("category") == category)
  .order(Column("eventTimestamp"))
  .asRequest(of: Entity.Motorsport.EventInfo.self)
  .fetchAll(db)

See how nested associations (fastestLap -> driver -> constructor) can be decoded in a flat struct (FastestLapInfo). This avoids defining yet another DriverInfo struct.

This is how I generally deal with chains of nested to-one associations (belongsTo, hasOne, hasOneThrough). Just flatten the whole tree in a single struct.

For more information, see again The Structure of a Joined Request.

[micpringle]

Wow! Thanks @groue. I really do appreciate such detailed responses. As soon as I get an opportunity I'll take a look at your proposals and let you know how I get on.

[groue]

Hi @micpringle, I'm closing this issue due to lack of activity. I hope you and your loved ones are doing well in these particular times.

[micpringle]

Hey @groue – please accept my apologies for the lack of activity on this thread. I'm hoping to get a chance to review your proposals by the end of this week, and then will get back you.

I hope you and your family are safe and well too 👍