Next.js: Support code sharing with Next

[ide]

In repositories with multiple projects, sometimes it is appealing to share code between the projects. Next currently requires that all of the site code is under the Next project's root directory. With code sharing, the shared code usually resides outside of the Next project's root. With a few modifications to the Webpack config and Babel plugins, Next can support code outside of its project root.

• [x] I have searched the issues of this repository and believe that this is not a duplicate.

Expected Behavior

In a directory hierarchy that looks like this:

.
├── next
│   └── pages
└── shared
    └── module.js

A file under next/pages should be able to do import module from '../../shared/module' or use a Babel plugin like module-resolver to require code that is outside of the Next project root.

Current Behavior

Next does not run the shared code through the same Webpack loaders as the ones used for code under the Next project's root. Also, the shared code is not compiled and emitted under .next/dist, and require/import paths to that shared code are not correct.

Steps to Reproduce (for bugs)

1. Set up a directory structure like the on in the ASCII art diagram above
2. Add index.js under next/pages with import module from '../../shared
3. Run yarn dev and get errors

Context

In general this is useful for any project that has a website (vs. being only a website) and wants to share code between the Next website and other programs. Specifically I am writing this up since we need it for sharing code across expo.io, snack.expo.io (not yet Next.js), Node servers, and native apps.

This is a list of PRs so far to support code sharing:

• Apply .babelrc's own config when it specifies babelrc: false: #3016

Your Environment

| Tech | Version |
|---------|---------|
| next |4.0.0-beta.2|
| node |8.6.0|
| OS |macOS 10.13|
| browser |*|

[Grokling]

The advice I had from the Zeit team on this topic was to put all the shared code into it's own repository expo-common, and use npm to install this common module in every project that needs it.

[ide]

That works for some people but I find it unproductive to need to publish shared code every time, and npm link and yarn link have significant enough limitations and discrepancies with production builds that they erode trust. We'd be more productive forking Next.js but would rather make Next better so either way I'll put up a few PRs that make sharing code easier.

[ide]

Exploring some more, my current thinking is it is simpler to use Yarn's link: dependencies (which partially simulate publishing, and as Yarn evolves it may support better linking behaviors that are more faithful to production). I have a proof-of-concept of being able to load linked modules without needing to publish shared code every time you edit it nor run Babel in a file watcher. The idea is:

1. Put a link: dependency in your site's package.json. Write some internal script to do this for you; this is out of Next.js's scope anyway.
2. Run yarn, which figures out what your site's node_modules should look like and installs all the dependencies and creates a symlink to your linked dependency. Eventually Yarn may improve how it handles link: dependencies (e.g. https://github.com/yarnpkg/rfcs/blob/master/accepted/0000-yarn-knit.md) so that we can more faithfully match what happens when you install your dependencies for production.
3. Configure Next.js to treat your linked package's source code the same way it treats your other site source code. This means running it through the Babel compiler, for example.
4. Include the compiled code in the Webpack bundle for the client and emit it under .next/dist/node_modules/<package> for SSR.

[brycejacobs]

This is an absolute pain. Is there any progress on this? Or way others can contribute that would be worthwhile? If you run more than one project, you're more than likely to have a repository of shared code.

There are ways around, such as creating a bash script that links against the modules, but it's cumbersome and is sketchy when there is a change to the shared repository. It would be nice if there was a way, such as what @ide is proposing, of specifying a collection of modules that are external to the project to be run through the the SSR compilation.

Can we please get some direction on this?

[znafets]

I think Next should support being able to share common code (components) that do not reside in the next app directory ... its a common case for cross platform repos where you have a shared/components folder on the same level as the /web, /native, /electron code ... why is it so hard to get a next app to import those shared components ?

I struggled with this for days now and are about to drop using next.js for this very reason ... can't believe that I cannot make it work

[timneutkens]

https://github.com/zeit/next.js/pull/3732

[znafets]

seems that it is not merged yet, also it may be not the best strategy to offload the effort of "making this work" to the developers using next.js to create apps ... maybe its just me but we should be able to import components from whatever directory without bending config files ... after all reusability of components is the main reason for using react

[znafets]

I do not find a way to make it work ...

[znafets]

All I want to do is load shared components from a directory not inside the next web-app directory ...

project
|_ next-app
|_shared -- components

even with the provided example next.config.js I seem not to be able to convince next to do that.

````
const webpack = require("webpack")

// Update these to match your package scope name.
const internalNodeModulesRegExp = /@zeit(?!.node_modules)/
const externalNodeModulesRegExp = /node_modules(?!\/@zeit(?!.node_modules))/

module.exports = {
webpack: (config, { dev, isServer, defaultLoaders }) => {
config.resolve.symlinks = false
config.externals = config.externals.map(external => {
if (typeof external !== "function") return external
return (ctx, req, cb) => (internalNodeModulesRegExp.test(req) ? cb() : external(ctx, req, cb))
})
config.module.rules.push({
test: /.+(js|jsx)$/,
loader: defaultLoaders.babel,
include: [internalNodeModulesRegExp]
})
return config
},
webpackDevMiddleware: config => {
const ignored = [config.watchOptions.ignored[0], externalNodeModulesRegExp]
config.watchOptions.ignored = ignored
return config
}
}
```

Is there anyone who can help?

[znafets]

Is there any progress for this matter? I am stuck. A project that needs to run on more than one platform will most likely share UI code and I am not willing to keep the same components in different folders and have to keep them in synch. A directory on root level has components which will be shared in several code projects which are or are not next.js projects.

shared/components
web (next.js)
desktop (electron-next)
native/iOS
native/Android

Choosing next.js was a decision for the "zero-config-paradigm", which sounds now like a joke as I am trying to configure this for over a week without success. But this "problem" seems not to be important enough to actually provide a pointer or actually some help on this matter.

[steida]

@znafets I had a working universal app in one directory in old Este. Now I am trying to figure out how to put it back... Follow related issue.

[znafets]

Well, as good as the initial approach of Next.js may be, there seems to be a need of maturation phase especially in supporting developers who actually want to adopt it for their products.
Having to beg for help is never a good sign for any project open source or not.

So I guess this is not for us ... not yet

[msand]

@znafets Have you seen this? https://github.com/zeit/next.js/issues/706#issuecomment-365975632
Alternatively, if you're ok with moving your folders a bit, you can use this approach: https://github.com/msand/newproject

[znafets]

@msand , thank you for the pointer I'll check them out, ... for now I think we already moved on without Next.js

[cyrilpanicker]

@msand , can you elaborate on how you resolved the issue?

[steida]

That's how I solved it http://github.com/este/este

[stefanosx]

The easiest solution I found is to create symlinks on each project to the shared folder and add the following code on next.config.js.

  webpack: (config, { buildId, dev }) => {
    config.resolve.symlinks = false
    return config
  }

[DullReferenceException]

Does the solution presented here work for this goal? There's a lot in common with our desired use case: https://github.com/zeit/next.js/pull/4406

[romanonthego]

Sharing code with next.js surprisingly comes as the _almost_ impossible task. aliases and babel transpiler gets off the rails and cannot handle shared code. Right now next.js - awesome as it is - cannot be used outside of designed scope (/pages, /static folder at the root, next/babel preset in root .babelrc) without an overwhelming amount of pain...

[steida]

@romanonthego It works in https://github.com/este/este Note two babel configs and package.json.

[romanonthego]

@steida eah, I get it to work by removing .babelrc from web and ignoring it from shared folder (still needed for expo though) and falling back from babel-module-resolver to webpack aliases...

[jmike]

Hey, just wanted to chime in that symlinks, while a great solution, don't work on GAE (Google App Engine) and possibly other vendors as well. Also, not sure what's the case with windows users.

The most robust solution I could find so far has been using Lerna.

[stefanosx]

I also found out that symlinks are not working with Jest.

[martpie]

For people out there with this problem, you can use:

• next-plugin-transpile-modules
• coupled with an npm install ../shared, which will symlink your shared folder to node_modules

Here is an example, the example here uses TypeScript, but it would work the same with plain JS.

cheers!

[scythargon]

@martpie Thank you! It actually worked for me. One thing left is that the redux store does not get passed to such components/modules. Do you by any chance have any ideas related to this? Thank you.

[timneutkens]

Going to track this here: #706 as it's almost the same issue and we'll be tackling them together.