Hexo: Error: EMFILE: too many open files (ulimit does not work on Windows 10)

[8ctopotamus]

In the Hexo troubleshooting guide they say to try

$ ulimit -n 10000

...but this does not work on Windows.

Environment Info

Node version

v5.6.0

My site _config.yml

# Hexo Configuration
# Hexo Configuration
## Docs: https://hexo.io/docs/configuration.html
## Source: https://github.com/hexojs/hexo/

# Site
title: Cool New Site
subtitle:
description:
author:  8ctopotamus
language:
timezone:

# URL
## If your site is put in a subdirectory, set url as 'http://yoursite.com/child' and root as '/child/'
url: http://yoursite.com
root: /
permalink: :year/:month/:day/:title/
permalink_defaults:

# Directory
source_dir: source
public_dir: public
tag_dir: tags
archive_dir: archives
category_dir: categories
code_dir: downloads/code
i18n_dir: :lang
skip_render:

# Writing
new_post_name: :title.md # File name of new posts
default_layout: post
titlecase: false # Transform title into titlecase
external_link: true # Open external links in new tab
filename_case: 0
render_drafts: false
post_asset_folder: false
relative_link: false
future: true
highlight:
  enable: true
  line_number: true
  auto_detect: false
  tab_replace:

# Category & Tag
default_category: uncategorized
category_map:
tag_map:

# Date / Time format
## Hexo uses Moment.js to parse and display date
## You can customize the date format as defined in
## http://momentjs.com/docs/#/displaying/format/
date_format: YYYY-MM-DD
time_format: HH:mm:ss

# Pagination
## Set per_page to 0 to disable pagination
per_page: 10
pagination_dir: page

# Extensions
## Plugins: https://hexo.io/plugins/
## Themes: https://hexo.io/themes/
theme: trio-theme

# Deployment
## Docs: https://hexo.io/docs/deployment.html
deploy:
  type:

My theme _config.yml

# Hexo Configuration
# Header
menu:
  VISION: /vision
  DESIGN: /design
  EXECUTION: /execution
  REALIZATION: /realization

rss: /atom.xml

# Content
excerpt_link: Read More

# Sidebar
sidebar: right
widgets:
- category
- tag
- tagcloud
- archive
- recent_posts

# display widgets at the bottom of index pages (pagination == 2)
index_widgets:
- category
- tagcloud
- archive

# widget behavior
archive_type: 'monthly'
show_count: false

# Miscellaneous
google_analytics:
favicon: /favicon.png
twitter:
google_plus:
fb_admins:
fb_app_id:

Plugin version(npm ls --depth 0)

[email protected] ~/hexo-vanilla-site
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
└── [email protected]

For BUG

• I am getting the this error almost every time I run "hexo serve". I am on WIndows 10 if that makes a difference.

fs.js:584
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^

Error: EMFILE: too many open files, open 'C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\readdirp\stream-api.js'
    at Error (native)
    at Object.fs.openSync (fs.js:584:18)
    at Object.fs.readFileSync (fs.js:431:33)
    at Object.Module._extensions..js (module.js:421:20)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at readdir (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\readdirp\readdirp.js:55:25)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:355:5)
    at FSWatcher.NodeFsHandler._handleDir (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:406:18)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:455:19)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:460:16)
    at FSReqWrap.oncomplete (fs.js:82:15)

• log with hexo --debug
  This is what happens when I do "hexo serve --debug"

λ hexo serve --debug
17:26:02.458 DEBUG Hexo version: 3.2.2
17:26:02.464 DEBUG Working directory: ~\Documents\GitHub\trio-hexo\
17:26:02.587 DEBUG Config loaded: ~\Documents\GitHub\trio-hexo\_config.yml
17:26:02.605 DEBUG Plugin loaded: hexo-generator-category
17:26:02.609 DEBUG Plugin loaded: hexo-generator-index
17:26:02.614 DEBUG Plugin loaded: hexo-generator-archive
17:26:02.620 DEBUG Plugin loaded: hexo-renderer-ejs
17:26:02.625 DEBUG Plugin loaded: hexo-renderer-stylus
17:26:02.627 DEBUG Plugin loaded: hexo-generator-tag
17:26:02.689 DEBUG Plugin loaded: hexo-server
17:26:02.696 DEBUG Plugin loaded: hexo-renderer-marked
17:26:02.710 DEBUG Loading database.
17:26:02.937 INFO  Start processing
17:26:03.721 DEBUG Processed: Design/index.md
17:26:03.722 DEBUG Processed: Execution/index.md
17:26:03.722 DEBUG Processed: Realization/index.md
17:26:03.723 DEBUG Processed: _posts/hello-world.md
17:26:03.723 DEBUG Processed: _posts/second-post.md
17:26:03.724 DEBUG Processed: vision/index.md
fs.js:584
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^

Error: EMFILE: too many open files, open 'C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\readdirp\stream-api.js'
    at Error (native)
    at Object.fs.openSync (fs.js:584:18)
    at Object.fs.readFileSync (fs.js:431:33)
    at Object.Module._extensions..js (module.js:421:20)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at readdir (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\readdirp\readdirp.js:55:25)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:355:5)
    at FSWatcher.NodeFsHandler._handleDir (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:406:18)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:455:19)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:460:16)
    at FSReqWrap.oncomplete (fs.js:82:15)

[yangg]

Same error, anyone can fix it?

[8ctopotamus]

Hey @yangg, I just keep running hexo s over and over and eventually it starts up again.

Obviously this is not ideal as the issue still persists. Would be nice if someone could suggest a solution!

[yangg]

@8ctopotamus I'm running hexo on Windows 7 (this PC has a bad performance)
hexo g has the same problem.
As the theme use hexo-renderer-scss instead of gulp recently, I just removed the dir theme/even/node_modules, and it works now.

[jcimoch]

I have same problem under windows bash which I described here https://github.com/Microsoft/BashOnWindows/issues/1126 Is there any workaround or fix?

[felixbillon]

Same problem on Windows 10 :(

[8ctopotamus]

Any update on this? I still have to run hexo s repeatedly until it finally starts up...

[pimskie]

Same here. Tried ulimit -u 3200 without result. Server won't start until the error goes away.

I fixed it by removing my own node_modules folder from my themes folder (hexo/themes/[my theme name]/node_modules).
https://github.com/hexojs/hexo/issues/2179

[lucax88x]

updates? this is really annoying..

[ghost]

This happens because hexo is watching ALL files in sub folders. If you're using grunt, gulp in your theme there are thousands of files and hexo doesn't ignore then.

The only workaround is currently to keep your node_modules outside the hexo root and create a gulp task which copy the files to theme folder.

[lucax88x]

Agree with you. This was the first thing I did when I saw this error.

Anyway. I couldn't make my theme work without node_modules because it required some files.

Probably there's an issue with the theme itself (tranquilpeak) but shouldn't hexo avoid to check folders like node_modules, bower_components, etc? maybe making this configurable.

[seaoak]

I seem to find out how to exclude some directories from the targets of watch().
But I have no Windows environment.
(In my Linux (Ubuntu 16.04.1) environment, no error occurs)

Could you please try following patch?

1. If you use Hexo "v3.1.1", modify node_modules/hexo/lib/box/index.js:24

$ diff index.js.org index.js
24c24
<     ignored: /[\/\\]\./
---
>     ignored: /[\/\\](\.[^\/\\]+|node_modules|bower_components)([\/\\]|[\/\\]?$)/
$

1. If you use Hexo "v3.2.2", insert a line into node_modules/hexo/lib/box/index.js:24

$ diff -c index.js.org index.js
*** index.js.org        2016-12-23 17:02:13.315065623 +0900
--- index.js    2016-12-23 17:07:11.864219252 +0900
***************
*** 21,26 ****
--- 21,27 ----
    EventEmitter.call(this);

    this.options = _.assign({
+     ignored: [base + 'node_modules', base + 'bower_components', base + '.git'],
      persistent: true
    }, options);

$

Please check node_modules/hexo/package.json to know the version.

Of course, I will make targets configurable when I make a pull request.

Thank you in advance for your cooperation.

[lucax88x]

Sure!

I have hexo 3.2.0

I applied the .2 patch, but unfortunately, it still not works.

Error: EMFILE: too many open files, open 'C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\readdirp\stream-api.js'
    at Error (native)
    at Object.fs.openSync (fs.js:640:18)
    at Object.fs.readFileSync (fs.js:508:33)
    at Object.Module._extensions..js (module.js:549:20)
    at Module.load (module.js:458:32)
    at tryModuleLoad (module.js:417:12)
    at Function.Module._load (module.js:409:3)
    at Module.require (module.js:468:17)
    at require (internal/module.js:20:19)
    at readdir (C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\readdirp\readdirp.js:56:25)
    at FSWatcher.<anonymous> (C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\chokidar\lib\nodefs-handler.js:355:5)
    at FSWatcher.NodeFsHandler._handleDir (C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\chokidar\lib\nodefs-handler.js:406:18)
    at FSWatcher.<anonymous> (C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\chokidar\lib\nodefs-handler.js:455:19)
    at FSWatcher.<anonymous> (C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\chokidar\lib\nodefs-handler.js:460:16)
    at FSReqWrap.oncomplete (fs.js:123:15)

[seaoak]

Could you taste my new patch?

The patch which I wrote in above comment is not enough.
It makes Hexo turning-off only watch() for node_modules.
Hexo continuously calls readdir() for all subdirectories in node_modules.

I make new patch to turn-off readdir()/watch() for specified directories.
Under the patch, Hexo will ignore node_modules directory and .git directory and all their subdirectories.
Hexo will NEVER call either readdir() nor watch() for these directories.

And also you can configure which directores are ignored.
You can write ignored_path: setting in _config.yml (such as after skip_render: setting) as follows:

ignored_path:
 - node_modules
 - .git
 - bower_components
 - /[\/\\](\.git[^\/\\]*)[\/\\]?$/i

At this time, you can not use glob expressions,
but you can use regular expressions instead.
I'm just getting ahead in myself.
I will reverse them (suit to skip_render: setting).

My repository is:
https://github.com/seaoak/hexo.git

The branch is:
feature/allow-turning-off-reading-and-watching-for-specified-directories

Please checkout the HEAD of above branch.

p.s. When you execute hexo, you will see many debugging messages.
You can filter them by grep -v 'seaoak '.
I'm sorry.

[seaoak]

In addition, if you are not familiar with git command,
please follow the procedure below:

1. Download index.js as node_modules/hexo/lib/box/index.js
2. Download default_config.js as node_modules/hexo/lib/hexo/default_config.js

These links point my commit 743ced890fe29d5bb6be91ba6efa7add1b289ea0
https://github.com/seaoak/hexo/commit/743ced890fe29d5bb6be91ba6efa7add1b289ea0

For example,

$ cd hexo-blog
$ hexo clean
$ (cd node_modules/hexo/lib/box && cp -p index.js index.js.org)
$ curl https://raw.githubusercontent.com/seaoak/hexo/743ced890fe29d5bb6be91ba6efa7add1b289ea0/lib/box/index.js > node_modules/hexo/lib/box/index.js
$ (cd node_modules/hexo/lib/hexo && cp -p default_config.js default_config.js.org)
$ curl https://raw.githubusercontent.com/seaoak/hexo/743ced890fe29d5bb6be91ba6efa7add1b289ea0/lib/hexo/default_config.js > node_modules/hexo/lib/hexo/default_config.js
$ hexo server --debug

And also you can recover these files:

$ hexo clean
$ (cd node_modules/hexo/lib/box && mv index.js.org index.js)
$ (cd node_modules/hexo/lib/hexo && mv default_config.js.org default_config.js)
$ hexo server --debug

[lucax88x]

I'm ok with git ;)

Anyway, with the 2 files patched, seems to be working, and hexo serves also much much faster this way.

[ghost]

@seaoak I would love to see this solved in official hexo, can you try to make a pull request?

[seaoak]

Thanks for your request!

Before I make a pull request, I should make test cases for the patch.
But I found that existing test script for watch() function was not enough.
So I'm trying to improve it.
https://github.com/hexojs/hexo/blob/master/test/scripts/hexo/hexo.js

I'm sorry for the delay, but I'll make a pull request. :wink:

[NoahDragon]

@seaoak thanks for your active contribution. I have invited you to join Hexo as a collaborator. 😃

[NoahDragon]

@seaoak Sorry for my bad memory, did this issue fix?

[erikman]

Probably solved by #1316, documentation issue still open #2855

[stevenjoezhang]

Closing as resolved: https://github.com/hexojs/hexo/pull/3797