React-rails: Question: Webpacker integration -> using local manifest.json for server-side rendering

[randyv12]

Hello!

I'm using the most recent version of react-rails (2.2.0) and Webpacker (1.2). One of the issues I ran into this week was server-side rendering, not fetching assets locally, but instead fetches from the remote asset host. I'm curious if this is fixed, if it's a bug (first of all), or if it will be-supported in the future or which gem will support it?

Explanation:

If Webpack detects that there is an asset_host set in a config, it will prepend the asset_host url in the manifest.json file in Webpacker (1.2) when it precompiles assets via rake webpacker:compile

webpacker/lib/install/config/webpack/configuration.js

const ifHasCDN = env.ASSET_HOST !== undefined && env.NODE_ENV === 'production'
const devServerUrl = `http://${devServer.host}:${devServer.port}/${paths.entry}/`
const publicUrl = ifHasCDN ? `${env.ASSET_HOST}/${paths.entry}/` : `/${paths.entry}/`
const publicPath = env.NODE_ENV !== 'production' ? devServerUrl : publicUrl

So when we precompile assets, our manifest.json file will contain, https://asset.host.com
and It will look like this

{file.js: 'https://assets.com/assets/file.js}

or like this

{file.js: '//assets.com/assets/file.js}

So when we server-side render using react ujs, using the web packer manifest container. The container will read from the local manifest.json with the asset host in it and recognize the asset_path that starts with 'http'. And when it does that, it will fetch the assets remotely, when server-side rendering. We don't want that, I think, we should just fetch it locally (since we have the files already)

react-rails/lib/react/server_rendering/webpack_manifest_container.rb

      def find_asset(logical_path)
        # raises if not found
        asset_path = Webpacker::Manifest.lookup(logical_path).to_s
        if asset_path.start_with?("http")
          # Get a file from the webpack-dev-server
          dev_server_asset = open(asset_path).read
          # Remove `webpack-dev-server/client/index.js` code which causes ExecJS to 💥
          dev_server_asset.sub!(CLIENT_REQUIRE, '//\0')
          dev_server_asset
        else
          # Read the already-compiled pack:
          full_path = Webpacker::Manifest.lookup_path(logical_path).to_s
          File.read(full_path)
        end
      end

Current Solution

To get around that and solve this issue we will have to create a separate manifest.json file (from Webpack, not Webpacker), which will not include the asset host, so we can locally reference them, without fetching assets remotely.

That, we can do on the Webpacker gem side and add in a separate webpack manifest plugin entry that creates a separate manifest.json for server-side rendering in a separate folder, something like:

webpacker/lib/install/config/webpack/shared.js

  plugins: [
    new webpack.EnvironmentPlugin(JSON.parse(JSON.stringify(env))),
    new ExtractTextPlugin(env.NODE_ENV === 'production' ? '[name]-[hash].css' : '[name].css'),
    new ManifestPlugin({ fileName: paths.manifest, publicPath, writeToFileEmit: true }),
    new ManifestPlugin({
      fileName: '../pre-render/manifest.json',
      publicPath: `${paths.prerender_path}/`,
      writeToFileEmit: true})
  ],

Then, after that, we will need to read that server-side manifest json file in
react-rails/lib/react/server_rendering/webpack_manifest_container.rb
so we can use the local assets that are precompiled, for server-side rendering

something like this:

module React
  module ServerRendering
    class WebpackerManifestContainer
      def find_asset(logical_path)
        path = ::Rails.root.join(File.join(Webpacker::Configuration.output_path, '/pre-render/manifest.json'))
        full_path = ::Rails.root.join(File.join(Webpacker::Configuration.output_path, JSON.parse(File.read(path))[logical_path.to_s]))

        if full_path
          return File.read(full_path)
        end

       // else do original find_asset stuff
    end
  end
end

[justin808]

This discussion might be related, as Webpacker currently wants hosts in the manifest.json: https://github.com/rails/webpacker/issues/464. I'm trying to get this changed so that React on Rails might depend on Webpacker and not Webpacker Lite.

[randyv12]

thanks, we can close this now, i'll just look forward to the changes and keep a close eye on the repo.
i found this earlier today, this is a gist that creates a separate config for client side and server side
https://gist.github.com/jslatts/f9dec699e8fd853590f520f62d5bf21a

[justin808]

In case anybody is curious, I'm working on this right now. Stay tuned.

[joeyparis]

Do we have any updates on this? This is a relatively recent problem for me that didn't affect me for the last couple of months (since ~August) but suddenly I am having the exact problem described in this issue.

[BookOfGreg]

I'm glad to say React-Rails supports this for Webpacker 3! 🙌

$ ASSET_HOST=https://example.com rails c
Loading development environment (Rails 5.1.4)
irb(main):001:0> helper.javascript_pack_tag 'application.js'
=> "<script src=\"//assets.example.com/packs/application-ddc460fb88ce313d87be.js\"></script>"
irb(main):002:0> Webpacker.manifest.lookup('application.js')
=> "/packs/application-ddc460fb88ce313d87be.js"

Thanks for having me check @joeyparis, looks like we fixed it when we added Webpacker 3 support.
https://github.com/BookOfGreg/react-rails-example-app/pull/5

What issue are you having?

[joeyparis]

@BookOfGreg,

When I am using webpacker's webpack-dev-server to watch individual files for compiling I get the following error when calling the react_component function in rails with prerendering set to true: bad URI(is not URI?): http://0.0.0.0:8080http://0.0.0.0:8080/packs/server_rendering.js .

I have traced the problem back to this line in the react-rails source code: https://github.com/reactjs/react-rails/blob/16503ee6243845e3a01a39d20f22cbadbc1f8c15/lib/react/server_rendering/webpacker_manifest_container.rb#L45

The problem is that the asset_path variable already includes the protocol and host_with_port values leading to the weird URI I get in the error above.

I've found that if I manually edit webpacker's public/packs/manifest.json file and remove the leading host from each attribute react-rails works as expected.

From this:

{
  "Digital.svg": "http://0.0.0.0:8080/packs/Digital.svg",
  "DirectMail.svg": "http://0.0.0.0:8080/packs/DirectMail.svg",
  "Email.svg": "http://0.0.0.0:8080/packs/Email.svg",
  "application.css": "http://0.0.0.0:8080/packs/application.css",
  "application.js": "http://0.0.0.0:8080/packs/application.js",
  "loading.gif": "http://0.0.0.0:8080/packs/loading.gif",
  "loading.svg": "http://0.0.0.0:8080/packs/loading.svg",
  "rocket.svg": "http://0.0.0.0:8080/packs/rocket.svg",
  "server_rendering.css": "http://0.0.0.0:8080/packs/server_rendering.css",
  "server_rendering.js": "http://0.0.0.0:8080/packs/server_rendering.js"
}

To this:

{
  "Digital.svg": "/packs/Digital.svg",
  "DirectMail.svg": "/packs/DirectMail.svg",
  "Email.svg": "/packs/Email.svg",
  "application.css": "/packs/application.css",
  "application.js": "/packs/application.js",
  "loading.gif": "/packs/loading.gif",
  "loading.svg": "/packs/loading.svg",
  "rocket.svg": "/packs/rocket.svg",
  "server_rendering.css": "/packs/server_rendering.css",
  "server_rendering.js": "/packs/server_rendering.js"
}

However, this requires manually editing the file each time I restart webpack-dev-server.

---

I am using webpacker 3.0.2 and react-rails 2.4.0.

---

Running your provided example returned the following with webpack-dev-server running:

$ ASSET_HOST=https://example.com rails c
[...]
Loading development environment (Rails 5.1.4)
2.3.1 :001 > helper.javascript_pack_tag 'application.js'
 => "<script src=\"http://0.0.0.0:8080/packs/application.js\"></script>"
2.3.1 :002 > Webpacker.manifest.lookup('application.js')
 => "http://0.0.0.0:8080/packs/application.js"

and the following without webpack-dev-server running:

$ ASSET_HOST=https://example.com rails c
[...]
Loading development environment (Rails 5.1.4)
2.3.1 :001 > helper.javascript_pack_tag 'application.js'
Compiling…
Compiled all packs in /Users/Joey/Sites/LeadJig/public/packs
 => "<script src=\"/packs/application.js\"></script>"
2.3.1 :002 > Webpacker.manifest.lookup('application.js')
 => "/packs/application.js"

[BookOfGreg]

Excellent @joeyparis, I didn't know it's specific to when the dev server is running. Now I can get about fixing this. I'll ping this chat when a fix is inbound 👍

[joeyparis]

Great to hear! Thanks for the quick reply.

Let me know if there's any more information I can provide. I'm happy to help you with debugging and testing in my environment if you need anything.

[BookOfGreg]

I'm pretty low on help in this gem at the moment and I'm currently working on .hydrate() #808 tonight.
I'd very much welcome a PR! My crude solution to this would likely be check asset_path contains // and only prepend #{ds.protocol}://#{ds.host_with_port} if not, then look at the neighbouring code to see if there are any more elegant options.

[joeyparis]

Let me see what I can do. I won't be working on this project again until Monday morning, but that's not to say I can't set aside some time this weekend to create a PR for the gem. I can certainly implement a crude check like that pretty quickly until a more elegant solution can be found. I'll keep you updated!

[joeyparis]

Didn't get a chance to tackle this issue this weekend, but going to get started on it now. Should have a PR here shortly.

[joeyparis]

I've created a crude but quick and easy solution!

# Line 41
def find_asset(logical_path)
  asset_path = Webpacker.manifest.lookup(logical_path).to_s
  if Webpacker.dev_server.running?
    ds = Webpacker.dev_server
    # Remove the protocol and host from the asset path. Sometimes webpacker includes this, sometimes it does not
    asset_path.slice!("#{ds.protocol}://#{ds.host_with_port}")
    dev_server_asset = open("#{ds.protocol}://#{ds.host_with_port}#{asset_path}").read
    dev_server_asset.sub!(CLIENT_REQUIRE, '//\0')
    dev_server_asset
  else
    File.read(file_path(logical_path))
  end
end

My solution was simply to remove the protocol and host/port from loaded asset path. If the protocol and host/port aren't included nothing will change, and if they are included they will be removed so that when they are added in the open they aren't duplicated. I hope you find this solution satisfactory.

I have submitted Pull Request #834

[BookOfGreg]

Fixed by #834 Thanks @joeyparis ! 💙

[randyv12]

Thank you guys!!