Express: optimization - static assets check if file exists, but use a static map of fs in memory instead?

[ORESoftware]

I have an optimization idea that might not have been implemented yet.

I was looking at the 'serve-static' repo - if the express.static middleware comes before your API routes, I believe 'serve-static' is always going to try to read the file from the fs, and if the file doesn't exist the middleware will then call next().

That means there is basically always going to be a fs read before your API routes gets hit.

instead, why not create a map of the filesystem in memory, if a server route (path) is not in the map, don't even bother trying to read it in the 'serve-static' middleware, just call next() immediately.

Is this optimization already in place?
I think it would be AWESOME if users could pass an option to express.static that said the FS is static, therefore create a map of it in memory.

hopefully this makes sense.
I have some Express middleware that implemented this idea, although it's pretty simple stuff, you probably get it.

[dougwilson]

Closing because issue opened in appropriate repo already :+1:

[dougwilson]

But yes, your assessment is correct and we recommend either confining your static to a path that doesn't overlay your API or use the static last instead of first. Node.js performs all fs operations on an internal thread pool and it is very easy to get your app bound on the thread pool size if there are a lot of fs operations (even just file stats). A front reverse proxy in front of Node.js to either (a) directly serve the static files or (b) cache them is highly recommended so you don't end up getting bound by the Node.js fs limitations.

[ORESoftware]

ok cool will look to other repo.
it might be cool to warn users if their registered api routes intersect with their static assets dir.
just one logging statement on startup warning users, would be all that's necessary.

right now, we have:

app.use(express.static(path.join(__dirname, '..', 'public')));

and the above middleware is registered before all our API routes, so AFAICT, that means it's hitting the fs for every damn request. damn!

[dougwilson]

Yea, we can consider that. How to determine that without false positives doesn't come to mind, but maybe you have some ideas how to implement such a warning. A pull request would let us look at it and approve!

[ORESoftware]

a very simple warning would be if you do this:

app.use(express.static(path.join(__dirname, '..', 'public')));

users should always be encouraged do this instead (right?):

app.use('/foobar', express.static(path.join(__dirname, '..', 'public')));

of course, users are going to want to be able to turn off the warning, if they actually really want the first scenario.

[ORESoftware]

the problem is app.use probably doesn't have any idea it's registering express.static middleware.

you might have to attach some property to the middleware function, and app.use would read that property. What do you think?

[wesleytodd]

I am not sure we should try to programmatically detect this. Maybe just adding some notes to the documentation to describe (and maybe warn) this behavior would be better?

[ORESoftware]

@wesleytodd that works for me

please weigh in on the original issue too..at the link to serve-static repo instead of here