I just noticed that https://github.com/symfony/webpack-encore/pull/642 was actually merged without Encore.configureBabelPresetEnv() being added to the index.js file (no version was released since that merge).
And 0.28.1 got released without this...