Youcompleteme: Autocomplete does not work for Python if source directory is Root directory(/)

[f-squirrel]

Issue Prelude

Please complete these steps and check these boxes (by putting an x inside
the brackets) _before_ filing your issue:

• [x] I have read and understood YCM's CONTRIBUTING document.
• [x] I have read and understood YCM's CODE_OF_CONDUCT document.
• [x] I have read and understood YCM's README, especially the
  Frequently Asked Questions section.
• [x] I have searched YCM's issue tracker to find issues similar to the one I'm
  about to report and couldn't find an answer to my problem. (Example Google
  search.)
• [x] If filing a bug report, I have included the output of vim --version.
• [x] If filing a bug report, I have included the output of :YcmDebugInfo.
• [ ] If filing a bug report, I have attached the contents of the logfiles using
  the :YcmToggleLogs command.
• [x] If filing a bug report, I have included which OS (including specific OS
  version) I am using.
• [ ] If filing a bug report, I have included a minimal test case that reproduces
  my issue, using vim -Nu /path/to/YCM/vimrc_ycm_minimal, including what I
  expected to happen and what actually happened.
• [ ] If filing a installation failure report, I have included the entire output
  of install.py (or cmake/make/ninja) including its invocation
• [x] I understand this is an open-source project staffed by volunteers and
  that any help I receive is a selfless, heartfelt _gift_ of their free time. I
  know I am not entitled to anything and will be polite and courteous.
• [x] I understand my issue may be closed if it becomes obvious I didn't
  actually perform all of these steps.

Thank you for adhering to this process! It ensures your issue is resolved
quickly and that neither your nor our time is needlessly wasted.

Issue Details

I run nvim in Ubuntu 18.04 running in a docker container.
The source directory is mounted in the Root directory(/).
Python autocomplete does not work for User types(classes and functions) but works only for Python builtin modules(os, random etc).

How-to-reproduce:

1. Create source tree as following in /python_test

.
|-- main.py
`-- util
    |-- __init__.py
    `-- user_class.py

1. user_class.py:

 class class_name:
     """description"""
    def foo(self):
        pass

In main.py type:

import util

if __name__ == "__main__":
    util.

After util. autocomplete is to be shown.

Note 1: If the source directory is moved to the user home dir the autocomplete works as expected.
Note 2: With the minimal configuration autocomplete works as expected.

YCM extra config

# This file is NOT licensed under the GPLv3, which is the license for the rest
# of YouCompleteMe.
#
# Here's the license text for this file:
#
# This is free and unencumbered software released into the public domain.
#
# Anyone is free to copy, modify, publish, use, compile, sell, or
# distribute this software, either in source code form or as a compiled
# binary, for any purpose, commercial or non-commercial, and by any
# means.
#
# In jurisdictions that recognize copyright laws, the author or authors
# of this software dedicate any and all copyright interest in the
# software to the public domain. We make this dedication for the benefit
# of the public at large and to the detriment of our heirs and
# successors. We intend this dedication to be an overt act of
# relinquishment in perpetuity of all present and future rights to this
# software under copyright law.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
# IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
#
# For more information, please refer to <http://unlicense.org/>

import os.path as p
import subprocess

DIR_OF_THIS_SCRIPT = p.abspath( p.dirname( __file__ ) )
DIR_OF_THIRD_PARTY = p.join( DIR_OF_THIS_SCRIPT, 'third_party' )


def GetStandardLibraryIndexInSysPath( sys_path ):
  for index, path in enumerate( sys_path ):
    if p.isfile( p.join( path, 'os.py' ) ):
      return index
  raise RuntimeError( 'Could not find standard library path in Python path.' )

# Set path to the right interpreter to allow autocompletion
# Default is python 2.7( python )
def Settings( **kwargs ):
    return {
         'interpreter_path': 'python3'
}


def PythonSysPath( **kwargs ):
  sys_path = kwargs[ 'sys_path' ]

  dependencies = [ p.join( DIR_OF_THIS_SCRIPT, 'python' ),
                   p.join( DIR_OF_THIRD_PARTY, 'requests-futures' ),
                   p.join( DIR_OF_THIRD_PARTY, 'ycmd' ),
                   p.join( DIR_OF_THIRD_PARTY, 'requests_deps', 'idna' ),
                   p.join( DIR_OF_THIRD_PARTY, 'requests_deps', 'chardet' ),
                   p.join( DIR_OF_THIRD_PARTY,
                           'requests_deps',
                           'urllib3',
                           'src' ),
                   p.join( DIR_OF_THIRD_PARTY, 'requests_deps', 'certifi' ),
                   p.join( DIR_OF_THIRD_PARTY, 'requests_deps', 'requests' ) ]

  # The concurrent.futures module is part of the standard library on Python 3.
  interpreter_path = kwargs[ 'interpreter_path' ]
  major_version = int( subprocess.check_output( [
    interpreter_path, '-c', 'import sys; print( sys.version_info[ 0 ] )' ]
  ).rstrip().decode( 'utf8' ) )
  if major_version == 2:
    dependencies.append( p.join( DIR_OF_THIRD_PARTY, 'pythonfutures' ) )

  sys_path[ 0:0 ] = dependencies
  sys_path.insert( GetStandardLibraryIndexInSysPath( sys_path ) + 1,
                   p.join( DIR_OF_THIRD_PARTY, 'python-future', 'src' ) )

  return sys_path

Diagnostic data

Output of vim --version

nvim --version
NVIM v0.4.3
Build type: RelWithDebInfo
LuaJIT 2.0.5
Compilation: /usr/bin/gcc-5 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -O2 -g -DMIN_LOG_LEVEL=3 -Og -g -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wmissing-prototypes -Wvla -fstack-protector-strong -fdiagnostics-color=auto -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -I/home/travis/build/neovim/bot-ci/build/neovim/build/config -I/home/travis/build/neovim/bot-ci/build/neovim/src -I/home/travis/build/neovim/bot-ci/build/neovim/.deps/usr/include -I/usr/include -I/home/travis/build/neovim/bot-ci/build/neovim/build/src/nvim/auto -I/home/travis/build/neovim/bot-ci/build/neovim/build/include
Compiled by travis@travis-job-9a287301-34f2-4b86-92e3-462594f51f36

Features: +acl +iconv +tui
See ":help feature-compile"

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "
/home/travis/build/neovim/bot-ci/build/neovim/build/nvim.AppDir/usr/share/nvim
"

Output of YcmDebugInfo

Printing YouCompleteMe debug information...
-- Completion API: 1
-- Client logfile: /tmp/ycm_dovcd9_p.log
-- Server Python interpreter: /usr/bin/python3
-- Server Python version: 3.6.9
-- Server has Clang support compiled in: False
-- Clang version: None
-- Extra configuration file found and loaded
-- Extra configuration path: /root/.ycm_extra_conf.py
-- Python completer debug information:
--   Python interpreter: /usr/bin/python3
--   Python root: /root
--   Python path: ['/root/python', '/root/third_party/requests-futures', '/root/third_party/ycmd', '/root/third_party/requests_deps/idna', '/root/third_party/requests_deps/chardet', '/root/third_party/requests_de
ps/urllib3/src', '/root/third_party/requests_deps/certifi', '/root/third_party/requests_deps/requests', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/root/third_party/python-future/src', '/usr/lib/python3.6/li
b-dynload', '/root/.local/lib/python3.6/site-packages', '/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages']
--   Python version: 3.6.9
--   Jedi version: 0.17.0
--   Parso version: 0.7.0
-- Server running at: http://127.0.0.1:58621
-- Server process ID: 3016
-- Server logfiles:
--   /tmp/ycmd_58621_stdout_mjn49gm0.log
--   /tmp/ycmd_58621_stderr_vahvk3rj.log

## Output of `YcmDiags`

No errors detecetd

## Output of `git rev-parse HEAD` in YouCompleteMe installation directory

8bc02ee4877bfe07ed92cc4d471c353f1ab84ef6
```

Contents of YCM, ycmd and completion engine logfiles

Reproduce the issue with vim -Nu /path/to/YCM/vimrc_ycm_minimal, which
enabled debug logging and other useful diagnostics. Include a link to a
gist containing all of the log files listed by :YcmToggleLogs.

OS version, distribution, etc.

Linux 7209e519cdf1 5.3.0-53-generic #47~18.04.1-Ubuntu SMP Thu May 7 13:10:50 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Output of build/install commands

Include link to a gist containing the invocation and entire output of
install.py if reporting an installation issue.

[puremourning]

Please can you provide minimal steps to reproduce using the minimal configuration?

Your extra conf looks suspiciously like ours, which is almost certainly not what you want.

It doesn't look like you included the :YcmDebugInfo when your cursor was in a python file, please can we have that?

[f-squirrel]

@puremourning , thank you for the feedback.
I've updated the ticket with the requested info.
Regarding the extra config, it works as is with the only addition:

def Settings( **kwargs ):
    return {
         'interpreter_path': 'python3'
}

Since with the minimal config it works, do you want me to upload my .vimrc?

[puremourning]

Regarding the extra config, it works as is with the only addition:

Why do you have all the other stuff then ? That stuff is clearly copy/pasted from ycmd's extra conf file and almost certainly is _wrong_ for you. I suggest removing it and just using the Settings function you pasted above.

Note 2: With the minimal configuration autocomplete works as expected.

OK so that suggests the problem is in your configuration, or a conflict with another plugin.

Try adding more of your ycm config to the minimal vimrc and see if that reproduces.

Basically, can you bisect your config to work out what's causing the issue, because if it doesn't repro with the minimal vimrc, then we won't be able to repro it.

[f-squirrel]

@puremourning ,
I've changed the extra config to the following:

def Settings( **kwargs ):
    return {
         'interpreter_path': 'python3'
}

However, the root cause was the following line in .vimrc:

let g:ycm_global_ycm_extra_conf = '~/.ycm_extra_conf.py'

Once I comment it out, the autocomplete works normally.

[puremourning]

bonza

[image357]

@puremourning: Actually this issue is not resolved!

I have a similar problem. Adding and using let g:ycm_global_ycm_extra_conf will set the python root to the path where to global ycm_extra_conf resides. This is not how it should work because ycm will not find any packages local to the files that one is working on (i.e. your own development packages in your project root).

Examples:
With g:ycm_global_ycm_extra_conf = '~/.vim/ycm_extra_conf.py' YcmDebugInfo shows:

-- Python root: /home/koepke/.vim

Without it shows:

-- Python root: /home/koepke/src/tensorflow/base/some_project_name

i.e. my project root and is thus able to find my local packages.

If you put a local .ycm_extra_conf.py into your project, the issue is also resolved. But this defeats the purpose of g:ycm_global_ycm_extra_conf.

[bstaletic]

This is fixed in ycmd master. Go into YCM/third_party/ycmd and pull the latest changes.

[image357]

@bstaletic: confirmed. Opened PR #3707 to update the submodule head.

[f-squirrel]

@bstaletic , thank you, it works like a charm!