\dfrac a b
\xrightarrow [2.\, \dfrac c d]
{1.\, \displaystyle\frac c d}
\xrightarrow [4.\, \normalsize \dfrac c d]
{3.\,\Large \displaystyle\frac c d}
{ \displaystyle \frac e f }
yields

even in the display mode.
2.\, \dfrac c d is rendered like \Large applied. (2. and 3. have the same size, and \normalsize in 4. restore that.)
@fennecf0x Thank you for reporting this bug. I see where the problem occurs and I should be able to write up a PR tomorrow that will fix the problem.
Most helpful comment
@fennecf0x Thank you for reporting this bug. I see where the problem occurs and I should be able to write up a PR tomorrow that will fix the problem.