projectile-globally-ignored-directories not working with native indexing.

[skrech]

Expected behavior

Ignore non-top-level dirs listed in projectile-globally-ignored-directories with both methods of indexing (native and alien).

Actual behavior

Fix introduced in #1119 fixes ignoring of non-top-level dirs for alien indexing mode but not for native.

Steps to reproduce the problem

1. Put this in init.el:

   projectile-globally-ignored-directories (append '("*__pycache__/")
   projectile-globally-ignored-directories)

2. Run C-u C-c p-f in a python project with generated __pycache__ dirs.

3. The __pycache__ dirs are there even though they have to be ignored.

Environment & Version information

OSX 10.13 and Windows 10.
Emacs 26.1 and older.
projectile 20180616.252 from Melpa

[bbatsov]

Why *__pychache__? I don't think we're doing any globbing here.

[skrech]

Check #1119

There is pull request that is merged which enables globs.

[bbatsov]

Ah, yeah. I remembered now I wanted to clean this up.

Well, the problem is pretty simple:

(defun projectile-dir-files (directory)
  "List the files in DIRECTORY and in its sub-directories.
Files are returned as relative paths to the project root."
  ;; check for a cache hit first if caching is enabled
  (let ((files-list (and projectile-enable-caching
                         (gethash directory projectile-projects-cache)))
        (root (projectile-project-root)))
    ;; cache disabled or cache miss
    (or files-list
        (if (eq projectile-indexing-method 'native)
            (projectile-dir-files-native root directory)
          ;; use external tools to get the project files
          (projectile-adjust-files (projectile-dir-files-external root directory))))))

We're filtering the files that came from the alien branch, but we're not doing this for the native branch.

[ordnungswidrig]

@bbatsov trying to get my hands wet, does this mean fixing to filter the result of projectile-dir-files-native ?

[bbatsov]

@ordnungswidrig I guess it will actually be better to just filter out stuff directly while you're going through the dirs, so we can speed up the recursive native indexing. Seems the actual problem is somewhere here:

(defun projectile-index-directory (directory patterns progress-reporter)
  "Index DIRECTORY taking into account PATTERNS.
The function calls itself recursively until all sub-directories
have been indexed.  The PROGRESS-REPORTER is updated while the
function is executing."
  (apply 'append
         (mapcar
          (lambda (f)
            (unless (or (and patterns (projectile-ignored-rel-p f directory patterns))
                        (member (file-name-nondirectory (directory-file-name f))
                                '("." ".." ".svn" ".cvs")))
              (progress-reporter-update progress-reporter)
              (if (file-directory-p f)
                  (unless (projectile-ignored-directory-p
                           (file-name-as-directory f))
                    (projectile-index-directory f patterns progress-reporter))
                (unless (projectile-ignored-file-p f)
                  (list f)))))
          (directory-files directory t))))

There are already some checks about ignored folders here, but I guess there's some mistake in them.

[ordnungswidrig]

In https://github.com/bbatsov/projectile/blob/master/projectile.el#L343 there's a list of globally ignored directories specified, why is .svn repeated here and .csv is not in the global list. Also why not .git. Should this be unified in favour of using projectile-globally-ignored-directories only?

[bbatsov]

@ordnungswidrig Likely just some old code that has to be cleaned up. I noticed this as well, but didn't have time to investigate it yet. This function is one of the old parts of the codebase and can certainly be improved a lot.

[lanzhiheng]

@bbatsov The projectile-globally-ignored-directories seems not working now. I am working in the rails project with webpack, but because of the node_modules directory it will take me too long time to run project-find-file. So I want to exclude the node_modules, my configuration like below

But the problem is I alway get the node_modules result when I use project-find-file

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contribution and understanding!

[nic-donaldson]

@lanzhiheng are you using the 'alien' indexing method? You could add node_modules to your .gitignore in that case.

[rileyrg]

unignored really should work with hybrid so anything in it overrides the .gitignore.

[fvides]

I've hit the same problem: adding node_modules to projectile-globally-ignored-directories doesn't work. After some digging, found that:

• when projectile-indexing-method is 'hybrid delegates the directory file listing to projectile-dir-files-alien and removes files with the fn projectile-remove-ignored
• when there is no vcs, in turn resorts to projectile-generic-command, which has a default value of "find . -type f -print0", which lists all files in current directory prefixed with "./"
• since projectile-remove-ignored uses string-prefix for checking if a file is inside a globally ignored directory, it doesn't recognize ./node_modules/foo as a file contained in globally ignored node_modules directory -> bummer.

So, as a workaround, prefixing node_modules with an asterisk works just fine:
(add-to-list 'projectile-globally-ignored-directories "*node_modules")

But I guess the more robust solution would be using proper path analysis for determining that a file should be ignored because an ignored parent. If this is the case, I would gladly work on a patch to fix this, but please confirm @bbatsov.

[bbatsov]

@fvides PRs are always welcome! :-)

There's one proposed PR (#1461), but unfortunately it wasn't finished and currently it's not mergeable either.

[nordlow]

For me, projectile-globally-ignored-directories is

• respected on Linux using alien projectile-indexing-method but
• not respected on Windows using native projectile-indexing-method

. Does somebody know where in the sources this error might originate?