Gatsby: [gatsby develop] Properly serve HTML files in the static directory

[simoneb]

Description

This is addressing an issue already reported in #2352 . The issue has reappeared, probably a regression, in even a worse fashion, because files cannot be accessed altogether, as opposed to having to specify the file name as reported in the original issue.

Storing static html files in the static folder causes gatsby to not serve them properly in development mode.

Steps to reproduce

If the file system structure is this:

/static
    /test
        /index.html

Running gatsby develop

Expected result

Browsing to http://localhost:8000/test, http://localhost:8000/test/ or http://localhost:8000/test/index.html should serve the above index.html.

Actual result

The 404 page is served instead.

Environment

System:
    OS: Windows 10
    CPU: (8) x64 Intel(R) Core(TM) i7-4720HQ CPU @ 2.60GHz
  Binaries:
    npm: 6.5.0 - C:\Program Files\nodejs\npm.CMD
  Languages:
    Python: 2.7.10 - /c/Python27/python
  Browsers:
    Edge: 42.17134.1.0
  npmPackages:
    gatsby: ^2.2.3 => 2.2.3
    gatsby-image: ^2.0.34 => 2.0.34
    gatsby-plugin-feed: ^2.0.15 => 2.0.15
    gatsby-plugin-google-analytics: ^2.0.17 => 2.0.17
    gatsby-plugin-manifest: ^2.0.24 => 2.0.24
    gatsby-plugin-offline: ^2.0.25 => 2.0.25
    gatsby-plugin-react-helmet: ^3.0.10 => 3.0.10
    gatsby-plugin-sharp: ^2.0.30 => 2.0.30
    gatsby-plugin-styled-components: ^3.0.7 => 3.0.7
    gatsby-plugin-typography: ^2.2.10 => 2.2.10
    gatsby-remark-copy-linked-files: ^2.0.11 => 2.0.11
    gatsby-remark-images: ^2.0.6 => 2.0.6
    gatsby-remark-prismjs: ^3.2.6 => 3.2.6
    gatsby-remark-responsive-iframe: ^2.1.1 => 2.1.1
    gatsby-remark-smartypants: ^2.0.9 => 2.0.9
    gatsby-source-filesystem: ^2.0.27 => 2.0.27
    gatsby-source-google-sheets: ^1.1.1 => 1.1.1
    gatsby-transformer-remark: ^2.3.7 => 2.3.7
    gatsby-transformer-sharp: ^2.1.17 => 2.1.17

[pieh]

This is difficult to do because of those change - https://github.com/gatsbyjs/gatsby/pull/12243 and https://github.com/gatsbyjs/gatsby/pull/12336 (as we will have lot of potentially stale .html files). Possibly we can create blacklist/whitelist for html files copied from static directory to workaround it.

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

[krischer]

I have a use case where a transformer plugin creates some html pages in the public folder which are later embedded as iframes into other components.

This works fine for production builds but the contents of the iframes are not displayed with the develop server which is a bit of a bummer. A whitelist would work fine for my use case. I'd be happy to contribute this if you point me to the best place where it should go.

[patriziosotgiu]

I'm currently working on a project with a similar use case.
We are using a static index.html that includes custom components and an iframe to serve Gatsby generated content.

[maxie-max]

At https://playchap.io/ we serve games with static html files via iframe and we bound to use Gatsby 2.0.65 due to the aforementioned issue

[yuinchien]

Would be great to have this bug fixed so people can serve their previous projects. Many thanks!

[gatsbot[bot]]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community!

[baerrach]

12336 is killing this for me.

I want those .htm/.html files served in develop.

I've just dropped a reference site in static/ so that I can link to it in Gatsby but because of #12336 everything but the html files are server.

I can kinda work around it by gatsby build; gatsby serve but I'd rather gatsby develop worked

[xBau]

Same issue here. A whitelist would be fine for me.

[diegodorado]

Same issue here. Not serving properly in develop.

[KyleAMathews]

Anyone interested in taking on a PR? It should be possible to track which HTML files are generated by Gatsby and then do serve other ones e.g. from the static folder or other sources.

[hanzhihua1]

Gatsby develop still won't serve static files, but gatsby build/gatsby serve works for now

[lbittner-pdftron]

+1

In the meantime, Ive been serving my static html files with another server and then using Gatsby develop middleware to proxy them:

// package.json
{
    "scripts": {
          "serve-static": "cd static && python -m SimpleHTTPServer 9000"
     }
}

// gatsby-config.js
var proxy = require("http-proxy-middleware")

module.exports = {
  developMiddleware: app => {
    app.use(
      "/some-static-dir",
      proxy({
        target: "http://localhost:9000/" // other server is running on port 9000
      })
    );
  }
}

Then, in an iframe for example, im accessing the file like so:

<iframe src='/some-static-dir/index.html' />

This way my code works in both develop and production.

Would be nice to just have them served by default though :)

[carly-lee]

We also have the same use case as @krischer.
We create .html file in the one of our internal plugin and use that .html in the Gatsby page. It works on gatsby build but shows a 404 page on gatsby develop. The HTML file is located and it works if I open it from finder.

We would like to use Gatsby Preview with Contentful but with this issue, the preview won't work properly. 😞

[jeeyyy]

+1 on the issue. Have a similar need to serve .html files that can be iframe(d) later.

[pieh]

This is difficult issue to solve without reverting perf improvements that were initially implemented in https://github.com/gatsbyjs/gatsby/pull/12243

As mentioned before, we could track html files copied over from static directory and whitelist them, but this wouldn't solve cases like @carly-lee described when those html files are added to public directory in way, that gatsby can't really track:

• using fs module in plugins (directly or indirectly)
• using custom scripts (i.e. cp -R /some-location ./public && gatsby develop)

As a workaround you should be able to use code snippet below (at least until this is properly fixed):

const express = require(`express`)
exports.onCreateDevServer = ({ app }) => {
  app.use(express.static(`public`))
}

Just keep in mind that this might cause issues and potentially serve stale files, that might cause inconsistencies.

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[trumbitta]

I have some demos in my website, and I was able to serve them with Jekyll but now I can't see them in "develop" with Gatsby.

[devpascoe]

ooo need this working for develop when using Auth0 and its callback html page for Silent Auth.

[gatsbot[bot]]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community!

[liamfd]

For those that may encounter this:

Gatbsy seems happy to serve non .html file extensions. My team just needed something html-ish, so we were able to get around this by loading an .xhtml file like:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <title>Example</title>
        <script>alert('hi')</script>
    </head>
    <body>
      <p>
        <b>Hello</b>
      </p>
    </body>
</html>

The browser treats it like an html file, and Gatsby serves it statically. There may be a better way to structure that file, we didn't care about the actual markup much. We just took it from the first thing we found. Seems to work for our particular use case, at least.

[LukyVj]

Hey just an other comment to adds up :)
I'd also like to be able to serve html files from static to iframe them!

Thanks for your work anyway!

EDIT:

I've found a super nice workaround https://github.com/gatsbyjs/gatsby/issues/17761#issuecomment-533816520 from @jonniebigodes, tested and it works nicely!

[simonwidjaja]

Same issue here. A whitelist would be fine for me.

[csath]

I too prefer a whitelist. I need to add my silent token renew html for oidc token refresh.

[stevemarksd]

Hacky fix: Add this code in gatsby-node.js

const express = require('express')
exports.onCreateDevServer=({app})=>{
    app.use(express.static('public'))
}

same here. pdftron can't be used in dev mode https://www.pdftron.com/documentation/web/get-started/npm/

[turistua]

Having similar issue. Store fonts in /static/ folder, which are not available when run 'develop' command. Works fine for build & serve.

Any ideas?

[coreyward]

@turistua As noted above, you can work around this by including the following in your gatsby-node.js file:

const express = require(`express`)

// Enable development support for serving HTML from `./static` folder
exports.onCreateDevServer = ({ app }) => {
  app.use(express.static(`static`))
}

This will only run in develop mode, so it won't interfere with builds.

[turistua]

@coreyward that's true.

If you use themes and want to serve some content from theme folder, the code should be like

const express = require(`express`);
exports.onCreateDevServer = ({ app }) => {
  app.use(
    express.static(path.resolve(process.cwd(), `../gatsby-theme/static`))
  );
};

[joeldbirch]

@coreyward Thanks for the workaround. However, I'm finding that this interferes with the dev environment's ability to accept code changes. As in, if I change some simple UI code I don't see that reflected in the browser until I shut the develop process and run yarn clean && yarn develop.

[coreyward]

@joeldbirch Hmm, I haven't experienced that. I have had a bunch of hot reloading issues where I have to save a file repeatedly before Gatsby picks up the change since the swap to the new file watching setup, though, even when I'm not using the snippet shared above. Is that maybe what you're experiencing?

[Acidic9]

@coreyward I tried to set up the express middleware with the onCreateDevServer function but I am still having the gatsby 404 page being displayed.

[simonjoom]

Yeah personnaly i did a hack in the core of gatsby
In last gatsby v2
in dist/utils/develop-proxy.js
add:

var _path2 = require("./path");

...
  const app = (req, res) => {
    if (process.env.GATSBY_EXPERIMENTAL_ENABLE_ADMIN) {
      const wasAdminRequest = serveAdmin(req, res);

      if (wasAdminRequest) {
        return;
      }
    }  
 //here my hack for a html file to serve properly in develop
    if(req.url=="/hub.html") {
  const directory = input.program.directory; 
  const directoryPath = _path2.withBasePath(directory);
   res.setHeader('Content-Type', 'text/html; charset=UTF-8');
      res.end(_fsExtra.default.readFileSync(directoryPath(`public/hub.html`)));
        return;
    }

If somebody can add this feature to add exception for some path to avoid gatsby to serve the index.html
I guess a whitelist in gatsby-config and make some addition like i proposed upper.

[bwalsh]

surprised this is an issue. need to serve static html so remote services can id site

[joeldbirch]

Quick update on my earlier comment. Using the above workaround, in development I'm serving HTML from an app or theme's static folder instead of the app's public folder to avoid the issue where code changes have no effect.

[coreyward]

@joeldbirch Oh shit, I could have totally been more careful in reviewing what was going on and saved you some time there. I found the same, and I always just use the static folder for this in development as well. Sorry about that!

// Serve files from `static` in development
exports.onCreateDevServer = ({ app }) => {
  app.use(express.static("static"))
}