Since Paged.js 0.1.28 (https://github.com/rstudio/pagedown/commit/3168deb12de81d74161754c543ba8720b7abe826), the page numbering in TOC is wrong when we use a TOC depth greater than 1. This happens only with Chrome, not Firefox.
I declared the issue in the Paged.js project: https://gitlab.pagedmedia.org/tools/pagedjs/issues/89
It is fixed in Paged.js 0.1.29 https://gitlab.pagedmedia.org/tools/pagedjs/issues/89#note_502
However, I get a regression with 0.1.29 (see the page number of the lists of figures and tables):

Actually the LOT and LOF are usually not included in the TOC. If you are able to remove these two items from the TOC, that will be great.
I can try to remove LOT and LOF from the TOC.
However, this will lead to a difference with the LaTeX CRC template where LOT and LOF are referenced in the TOC: https://bookdown.org/yihui/bookdown/bookdown.pdf.
The bug will remain for any section of class front-matter.
Oh I misremembered. Sorry. Then let's keep them.
However, I get a regression with 0.1.29 (see the page number of the lists of figures and tables):
I finally found the problem: this is a Chrome redraw issue. This seems to be a well-known bug.
There are many available hacks to solve this problem (try to Google _Chrome redraw issue_). Only this one works for me with Chromium/Ubuntu.
I will upgrade Paged.js to 0.1.31 soon. There is one breaking change (Paged.js CSS variables are now prefixed) but there are many improvements. IMO the most important ones are the fix for page references and the support for the marks property.
Excellent. Thanks!