Pgloader: Introduce a statistics singleton

Created on 3 Apr 2020  ·  4Comments  ·  Source: dimitri/pgloader

_Originally posted by @dimitri in https://github.com/dimitri/pgloader/timeline_

That's good thinking, and I think that's relevant for another PR. The PR introducing the statistics singleton should also take care of the following bits of refactoring of the code:

  • the singleton is named “state” as I initially though I would update the global state of the migration to be able to know about the progress being made while it's being made,
  • when implementing the feature, it quickly enough turned into statistics (count of rows read, imported, errors, timings, etc) rather “state”
  • the API to access and maintain the state objects (stats really) per table etc is rather lacking, so we should clean that up and offer easy access to needed information from threads

At the moment only the monitor thread is creating and updating the data, we certainly want to keep it that way. That allows no-locks (and no dead locks) code, with using a queue to send events and centralise statistics from.

Good First Issue Refactor

Most helpful comment

@phoe @dimitri, I had a look through open issues and PRs and I think I'd like to have a take at this one.
Did any of you already start work on it?

I would not like to duplicate work. If you did and would like to do a handover that would be very welcome.

Thanks!

All 4 comments

Hey @phoe ; thanks for opening the issue to keep track of it. Do you want to take care of this? it's a pretty well isolated piece of pgloader and would allow you to stay in the Common Lisp side of things without having to know much about the database side of things ;-)

Zrzut ekranu z 2020-04-03 23-29-42

Done, I'll poke it in a spare while.

@phoe @dimitri, I had a look through open issues and PRs and I think I'd like to have a take at this one.
Did any of you already start work on it?

I would not like to duplicate work. If you did and would like to do a handover that would be very welcome.

Thanks!

I haven't started any work on it. Please feel free to proceed! Fame and glory awaits.

Was this page helpful?
0 / 5 - 0 ratings