Grdb.swift: Subscribing to changes when table is missing yet.

[Vladlex]

What did you do?

I have a simple piece of code to observe database changes:

public func observeAll<T>(type: T.Type) -> Observable<[T]> where T : StoragableProtocol {
        return Observable<[T]>.create({ [storagePool] (observer) in
            let request = T.all()
            let observation = request.observationForAll()
            let dbObserver = observation.start(in: self.storagePool, onError: { (error) in
                // A little pseudocode here for checking
                if error.isTableMissingError {
                    // TODO: Fix with resubscribing or waiting for the table created
                } else {
                    // observer.onError(error)
                }
            }) { (results) in
                observer.onNext(results)
            }
            return Disposables.create {
                storagePool.remove(transactionObserver: dbObserver)
            }
        })
    }

At the TODO-line I would like to write a logic with waiting (async) till table created, but there is no method to subscribe to do it.
I'm not sure, maybe there is an easy way to do subscribe at the moment the table is missing.

What did you expect to happen?

I would like to have a method to observe table creation to start emitting events about changes in the table.

Environment

GRDB flavor(s): GRDB
GRDB version: 4.5
**Installation method: CocoaPods
Xcode version: 11.0
Swift version: 5.0
Platform(s) running GRDB: iOS
macOS version running Xcode: 10.14 (Mojave)

Notes

I know about RxGRDB, for specific reasons I couldn't use it.

[groue]

Hello @Vladlex,

I would like to have a method to observe table creation to start emitting events about changes in the table.

This feature does not exist in GRDB.

It does not exist in SQLite either. The mother of all GRDB observation features, the Data Change Notification Callbacks, tells that:

The update hook is not invoked when internal system tables are modified (i.e. sqlite_master and sqlite_sequence).

Since the only observable effect of a table creation is a modification in the sqlite_master table, which is out of scope of the update hook, GRDB has no way to know that a table was created.

I'm not sure, maybe there is an easy way to do subscribe at the moment the table is missing.

Since those tables are created by your application at some point, your application knows when it can start observing those tables: this is the direction you should look at.

[Vladlex]

Since those tables are created by your application at some point, your application knows when it can start observing those tables: this is the direction you should look at.

Yes, until the target table created from a third-party module =)

Thank you, your answer helps me very much.

[groue]

Yes, until the target table created from a third-party module =)

Oh. Well, if you stay stuck for too long, don't hesitate asking further questions.

[groue]

For example, you could observe all transactions, and check if the list of tables has changed:

// Not tested - hand-written code only
var tableNames: Set<String> = []
let observation = DatabaseRegionObservation(tracking: DatabaseRegion.fullDatabase)
let observer = try observation.start(in: storagePool) { db in
    let newTableNames = try Set(String.fetchCursor(db, sql: """
        SELECT name FROM sqlite_master WHERE type = 'table'
        """))
    let createdTableNames = newTableNames.subtracting(tableNames)
    tableNames = newTableNames
    // Handle createdTableNames by sending it to some Rx subject
}

But even this would not allow you to notice tables created from an external database connection.

And performance-wise, it can not be a good thing to run this request after each and every transaction.

[Vladlex]

For example, you could observe _all transactions_, and check if the list of tables has changed:

// Not tested - hand-written code only
var tableNames: Set<String> = []
let observation = DatabaseRegionObservation(tracking: DatabaseRegion.fullDatabase)
let observer = try observation.start(in: storagePool) { db in
    let newTableNames = try Set(String.fetchCursor(db, sql: """
        SELECT name FROM sqlite_master WHERE type = 'table'
        """))
    let createdTableNames = newTableNames.subtracting(tableNames)
    tableNames = newTableNames
    // Handle createdTableNames by sending it to some Rx subject
}

But even this would not allow you to notice tables created from an external database connection.

And performance-wise, it can not be a good thing to run this request after each and every transaction.

This is a nice workaround, but as long as we waiting for third-party feature request approval we found another possible workaround that can work if GRDB has method that just returns existing database table names.
Could you please tell me, does GRDB has such a method?

[groue]

... if GRDB has method that just returns existing database table names. Could you please tell me, does GRDB has such a method?

It's right in the sample code you have replied to: SELECT name...

Remember that GRDB uses SQLite. Uncommon use cases that do not have a ready-made Swift method can always be achieved at a lower level. SQLite has a great documentation, and GRDB has great support for raw SQL.

[Vladlex]

It's right in the sample code you have replied to: SELECT name...

Oh, I was inattentive, sorry 😔
Thank you very much, that helps a lot!