Currently it is not possible to (properly) size custom radios and checkboxes, as one can do with other form-controls.
One can add .col-form-label-sm or .col-form-label-lg to the custom checkbox/radio markup to scale the label accordingly, but the indicator icon does not scale with the change in label font size.

One could switch the units of the ::before and ::after width/height/top, and the left gutter padding to em units instead of rem units (as these pseudo elements inherit the base font-size from their parent element - the label). This makes the indicators scale nicely when the font-size of the label scales, without needing additional CSS rules for the indicators. All that would be needed would be new classes .custom-control-sm and .custom-control-lg (as an alternative to using .col-form-label-sm and .col-form-label-lg) to control the font-size (and possibly top/bottom padding of the label text)
Example (switching pseudo element units to em):

This trick would also work well with the new custom-switch component.
I'm agree. This is the same point of my old issue https://github.com/twbs/bootstrap/issues/27527 closed by Mark.
Maybe it's time to clarify a way for custom controls sizing, considering also the missing of sizing for new "switch"control.
We can move to em units and make them easily scale. We鈥檒l likely do that in v5 though instead of the v4.3 release to keep things behaving as expected.
This was added in #28450 with the major form rewrite to consolidate native and custom form controls from v4. Closing as fixed!
Most helpful comment
We can move to em units and make them easily scale. We鈥檒l likely do that in v5 though instead of the v4.3 release to keep things behaving as expected.