Next-i18next: Provide a way to consume locales from outside the filesystem

[d-asensio]

Is your feature request related to a problem? Please describe.

At the company I'm working for, we use next.js for the new front-end we are building, and we are considering to use this package to handle i18n strings management.

I understand that for the vast majority of applications it is fine to load i18n strings from the filesystem and have them stored in json files together with code, I think it makes a lot of sense.

However, to comply with our specific use case, we are not allowed to use this approach, since our translations are stored in a database and we are forced to consume them through an API. And of course we can create a server worker which can generate those jsons from another source and keep them in sync, but I think that it should be easier than this...

Describe the solution you'd like

I don't know exactly which would be the better way to do it, but I think it would be nice to have a sort of "localeLoader" which would be a function responsible to load the strings (from the filesystem or from another place).

So thinking about the API interface, instead of doing this:

const NextI18NextInstance = new NextI18Next({
  defaultLanguage: 'en',
  otherLanguages: ['de'],
  localePath: 'static/locales',
  localeStructure: '{{lng}}/{{ns}}',
  localeSubpaths: false
})

It could be something like this:

const NextI18NextInstance = new NextI18Next({
  defaultLanguage: 'en',
  otherLanguages: ['de'],
  localeLoader: fileSystemLoader({
    path: 'static/locales',
    structure: '{{lng}}/{{ns}}',
    subpaths: false
  })
})

This approach would allow us to write our own "loacaleLoader" which can (for example) request the data to an API, DB, GraphQL or anything else...

[isaachinman]

This question was raised in a previous issue, but I cannot seem to find it now.

My response to this the first time around is basically the same now: why are you dead set on doing this? It's poor design, in my opinion. If your locales live externally and are served by an API, you should add a build step that compiles the resources from your remote to the local filesystem. Then you can trigger a rebuild/deployment on your app whenever translations change (via a webhook, for example).

Otherwise: (1) you're going to be hitting a service over and over when the content presumably has not changed (in most cases), and (2) you have no control over when localisation changes go live, and lose the concept of versioning.

If you _really_ want this functionality, I would be open to any sensible PRs, but this is not something I am going to spend any of my own time working on. In short, fileSystemLoader _already exists_ in that you can just set up a build step on your own.

I am happy to discuss further if you feel it to be necessary.

[d-asensio]

Hi @isaachinman I see your point, and why you could consider this as a poor design. However, depending on the use case and the product you are building, you might be forced to do that.

To provide more context to the discussion, let's figure out that you have to build a front-end at the top of a CMS, and the users of your product are allowed to edit translations from there, and even add new languages, etc.

In this case, I would say, it is not convenient to trigger a new release of the front-end just to update those translations, even if your build process is fast, it will last a couple of minutes till the user can see that the change has been applied effectively.

It's true that with this approach you don't have control over the localization changes, but it can happen that you don't want to. In our case localization is part of the content we serve, not part of the product itself.

Also, you can avoid server stress by caching (which can expire when some translation change), so it is not necessary to hit a service over and over.

I opened the issue to raise my proposal and hear your feedback about it, but as a starting point to discuss if it is worth it or not to implement this feature and agree with you about how to do it.

Of course if we arrive to an agreement and you see this as a valuable change, we will contribute an make a PR to provide this functionality. 😉

[isaachinman]

In this case, I would say, it is not convenient to trigger a new release of the front-end just to update those translations, even if your build process is fast

Your build process would be:

1. Check if git head has changed
2. If so, perform full build
3. If not, refetch translations and restart Node process

This would make the build process just as long as it takes to get content from your API and rollover the server.

Translations are definitely something which should be "deployed", in my opinion. If you require some sort of live view or instantaneous feedback, you'd need to mock that via some sort of live editor (lots of products already exist), and then save/deploy changes once the user has completed their work.

The solution you are suggesting just gives me a general bad feeling - hard to put into words. It feels inherently flakey and prone to failure. Moreover, I'm not sure if i18next even has the concept of fetching translations at runtime.

For what it's worth, you can probably already achieve what you want here by adding some manual work into getInitialProps: call your API, then add the response into the i18n instance by calling addResourceBundle.

[d-asensio]

Okay, thanks for the quick response, I'll take my time to consider how we will solve this and comment here whatever we decide (to help with future similar issues).

[isaachinman]

Okay, no problem. Just let me know. As I said I'm happy to discuss this and review/merge any appropriate PRs, I'm just not sure if there's a whole lot to be done in the next-i18next source. There are probably already ways to achieve this on your own, either via addResourceBundle or setting your backend.loadPath to your remote URI.

[capellini]

Another option (though admittedly less ideal than a pure i18next solution) -- assuming that fetching locale files is strictly a server-side issue, then you can do whatever you'd like in your server code.

For example, there's nothing that precludes someone from doing this:

  server.use(nextI18NextMiddleware(nextI18next))
  server.use((req, res, next) => {
    // do what you would like here when req.url startsWith '/static/locales'
    // call next() if you would like to proceed with the next middleware
  })
  // or
  server.get('/static/locales', (req, res) => {
    // do what you would like here and send your result
    // this can include setting cache headers, reaching out to another service (and,
    //    perhaps caching those files locally), etc
  })
  server.get('*', (req, res) => handle(req, res))  // hand the rest over to Next

Less than ideal, as GET /static/locales/... and a subsequent GET to another service is two round-trips, but that could be mitigated by a custom caching strategy or cache headers. And you have the added flexibility of providing a fallback, should the service you are using be down for some reason.

At any rate, I thought that I would offer it as an option.

[d-asensio]

Thanks @capellini this looks good, I think that we will end up with something similar 😄

[isaachinman]

@capellini That is actually a very simple and elegant solution. I imagine that would solve @d-asensio's case entirely. Nice! We can also refer future users with the same question to this solution.

[isaachinman]

One thing that occurs to me is that i18next-node-fs-backend is still going to complain about the filesystem not being the right shape. Let me know if you run into any issues there. There might also be issues with SSR and the current preload: allLanguages logic.