Youcompleteme: No non-local Rust completion

[sbmueller]

Issue Prelude

Please complete these steps and check these boxes (by putting an x inside
the brackets) _before_ filing your issue:

• [x] I have read and understood YCM's CONTRIBUTING document.
• [x] I have read and understood YCM's CODE_OF_CONDUCT document.
• [x] I have read and understood YCM's README, especially the
  Frequently Asked Questions section.
• [x] I have searched YCM's issue tracker to find issues similar to the one I'm
  about to report and couldn't find an answer to my problem. (Example Google
  search.)
• [x] If filing a bug report, I have included the output of vim --version.
• [x] If filing a bug report, I have included the output of :YcmDebugInfo.
• [x] If filing a bug report, I have attached the contents of the logfiles using
  the :YcmToggleLogs command.
• [x] If filing a bug report, I have included which OS (including specific OS
  version) I am using.
• [x] If filing a bug report, I have included a minimal test case that reproduces
  my issue, including what I expected to happen and what actually happened.
• [x] If filing a installation failure report, I have included the entire output
  of install.py (or cmake/make/ninja) including its invocation
• [x] I understand this is an open-source project staffed by volunteers and
  that any help I receive is a selfless, heartfelt _gift_ of their free time. I
  know I am not entitled to anything and will be polite and courteous.
• [x] I understand my issue may be closed if it becomes obvious I didn't
  actually perform all of these steps.

Thank you for adhering to this process! It ensures your issue is resolved
quickly and that neither your nor our time is needlessly wasted.

Issue Details

Provide a clear description of the problem, including the following key
questions:

• What did you do?

Installed YCM with --rust-completer. Opened a .rs file and started to code, for instance YouCompleteMe/third_party/ycmd/ycmd/tests/rust/testdata/common/src/main.rs.

• What did you expect to happen?

When typing let a = Vec::n I expect to see a completion suggestion like new f pub const fn new() -> Vec<T>.

• What actually happened?

Not all possible suggestions are shown.

Turns out only local completion works, for instance builder.bui:

"Global" completion, for instance from standard library does not work.

Diagnostic data

Output of vim --version

VIM - Vi IMproved 8.1 (2018 May 18, compiled Jun  5 2019 17:57:58)
macOS version
Included patches: 1-1368
Compiled by MacPorts
Huge version without GUI.  Features included (+) or not (-):
+acl               +cmdline_info      -farsi             +libcall           -mouse_sysmouse    +python3           +termguicolors     +vreplace
+arabic            +comments          +file_in_path      +linebreak         +mouse_urxvt       +quickfix          +terminal          +wildignore
+autocmd           +conceal           +find_in_path      +lispindent        +mouse_xterm       +reltime           +terminfo          +wildmenu
+autochdir         +cryptv            +float             +listcmds          +multi_byte        +rightleft         +termresponse      +windows
-autoservername    -cscope            +folding           +localmap          +multi_lang        -ruby              +textobjects       +writebackup
-balloon_eval      +cursorbind        -footer            -lua               -mzscheme          +scrollbind        +textprop          -X11
+balloon_eval_term +cursorshape       +fork()            +menu              +netbeans_intg     +signs             +timers            -xfontset
-browse            +dialog_con        +gettext           +mksession         +num64             +smartindent       +title             -xim
++builtin_terms    +diff              -hangul_input      +modify_fname      +packages          +startuptime       -toolbar           -xpm
+byte_offset       +digraphs          +iconv             +mouse             +path_extra        +statusline        +user_commands     -xsmp
+channel           -dnd               +insert_expand     -mouseshape        -perl              -sun_workshop      +vartabs           -xterm_clipboard
+cindent           -ebcdic            +job               +mouse_dec         +persistent_undo   +syntax            +vertsplit         -xterm_save
-clientserver      +emacs_tags        +jumplist          -mouse_gpm         +postscript        +tag_binary        +virtualedit
+clipboard         +eval              +keymap            -mouse_jsbterm     +printer           -tag_old_static    +visual
+cmdline_compl     +ex_extra          +lambda            +mouse_netterm     +profile           -tag_any_white     +visualextra
+cmdline_hist      +extra_search      +langmap           +mouse_sgr         -python            -tcl               +viminfo
   system vimrc file: "/opt/local/etc/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/opt/local/share/vim"
Compilation: /usr/bin/clang -c -I. -Iproto -DHAVE_CONFIG_H   -I/opt/local/include -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platf
orm/Developer/SDKs/MacOSX10.14.sdk -DMACOS_X -DMACOS_X_DARWIN  -pipe -Os -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Devel
oper/SDKs/MacOSX10.14.sdk -arch x86_64 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: /usr/bin/clang   -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.plat
form/Developer/SDKs/MacOSX10.14.sdk -arch x86_64 -o vim        -lm  -lncurses -liconv -lintl -framework AppKit     -L/opt/local/Library/Frameworks/Python.fr
amework/Versions/3.7/lib/python3.7/config-3.7m-darwin -lpython3.7m -framework CoreFoundation

Output of YcmDebugInfo

Printing YouCompleteMe debug information...
-- Client logfile: /var/folders/zl/jgzhpvtx79jg0d_br665jj2r0000gn/T/ycm_kfeebm_4.log
-- Server Python interpreter: /opt/local/bin/python
-- Server Python version: 3.7.3
-- Server has Clang support compiled in: True
-- Clang version: clang version 8.0.0 (tags/RELEASE_800/final)
-- Extra configuration file found and loaded
-- Extra configuration path: /Users/senpo/.vim/plugged/YouCompleteMe/third_party/ycmd/.ycm_extra_conf.py
-- Rust completer debug information:
--   Rust Language Server running
--   Rust Language Server process ID: 13216
--   Rust Language Server executable: /Users/senpo/.vim/plugged/YouCompleteMe/third_party/ycmd/third_party/rls/bin/rls
--   Rust Language Server logfiles:
--     /var/folders/zl/jgzhpvtx79jg0d_br665jj2r0000gn/T/rust_language_server_stderrw8if9i6r.log
--   Rust Language Server Server State: Initialized
--   Rust Language Server Project Directory: /Users/senpo/.vim/plugged/YouCompleteMe/third_party/ycmd/ycmd/tests/rust/testdata/common
--   Rust Language Server Settings: {}
--   Rust Language Server Project State: Indexing done, building done
--   Rust Language Server Version: 1.36.0 (9692ca8 2019-05-18)
-- Server running at: http://127.0.0.1:51750
-- Server process ID: 13211
-- Server logfiles:
--   /var/folders/zl/jgzhpvtx79jg0d_br665jj2r0000gn/T/ycmd_51750_stdout_6kaci2ov.log
--   /var/folders/zl/jgzhpvtx79jg0d_br665jj2r0000gn/T/ycmd_51750_stderr_03umrcb_.log

Contents of YCM, ycmd and completion engine logfiles

https://www.dropbox.com/s/239s6sn2h7g4veu/rust_language_stderr.log?dl=0
https://www.dropbox.com/s/yyjp8tyzwsucdq2/ycmd_stderr.log?dl=0

OS version, distribution, etc.

macOS 10.14.6

Output of build/install commands

https://www.dropbox.com/s/d7ov0o2cffjgvgq/installation.log?dl=0

I already hat a length discussion on Gitter with @bstaletic . He made sure I'm on the newest YCM commit and confirmed he gets the same installation output. Besides the OS difference (macOS - Linux) we seem to have identical setups, however completion works for him and not for me. Any help would be appreciated!

[bstaletic]

To rehash my conclusion after our discussion, it looks like RLS doesn't find the standard library, which we do install along side RLS. I wouldn't be surprised if this is a symptom of macOS, as we have had that with C already.

@puremourning Could you try to repro the issue on your macOS?

[tanhuacheng]

I have the same problem on "Ubuntu 16.04.1"

[tanhuacheng]

I solved this problem by install rls to $HOME/.rustup where is the place i am install rust toolchain. it's also installed to third_party/ycmd/third_party/rls/ by ycm, but it seems rls not use them.

rustup component add rls rust-analysis rust-src

[bstaletic]

@puremourning Can you give this a test? I can't reproduce the error on linux.

[vasilakisfil]

I have the same problem on linux, what @tanhuacheng said helped me.

[puremourning]

@puremourning Can you give this a test?

Sure. I get the same behaviour as the OP.

However, all the tests pass.

(system3) BenMBP:ycmd ben$ ./run_tests.py --skip-build ycmd/tests/rust
...
----------------------------------------------------------------------
Ran 52 tests in 6.487s

OK

In the racer days you used to have to have the standard library sources installed and configured. Is that still the case ?

[bstaletic]

However, all the tests pass.

Because our completion tests cover only crate-local completion, no dependencies, including the standard library.

In the racer days you used to have to have the standard library sources installed and configured. Is that still the case ?

Right now we are installing rust-src, along with rls and rust-analysis into third_party. That's where we install the entire toolchain. It's supposed to magically work. And it does magically work for me.

[puremourning]

If it's of help: here's the rls log

[puremourning]

Actually, I think that's truncated. the log is 12,381 lines.

Anyone else surprised the log file for the worlds most trivial operation in the worlds most trivial file is 12381 lines ?!

[bstaletic]

Yeah, RLS logs are insane. I compared the rls log with @sbmueller in our gitter room. My log was longer than his and contained things like Vec from the standard library, but I couldn't figure out where the difference lies.

The difference was a couple hundred lines extra in my log.

[bstaletic]

@puremourning Try searching your log for third_party and even files from the standard library that we installed there.

[puremourning]

well it has this stuff:

[2019-11-09T15:05:31Z TRACE rls_analysis::raw] Considering analysis files at /Users/ben/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/rls/lib/rustlib/x86_64-apple-darwin/analysis
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 124874195 }), name: "libunicode_width-40bc6477848e2242.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125012740 }), name: "librustc_std_workspace_alloc-df043cce83b6dd34.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125063255 }), name: "libtest-e6dd79af2b4837e4.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125074294 }), name: "libbacktrace_sys-0f0fd6cb2d0ffa02.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125096018 }), name: "libcompiler_builtins-967a7c840affa3a2.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125330072 }), name: "libpanic_abort-fbfbc94b9e906545.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125392918 }), name: "libpanic_unwind-b8538eedc7e0aa47.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125503187 }), name: "libgetopts-3ef2c6cfd08ec619.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125504903 }), name: "libunwind-0f97cfed0ef82f67.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125507896 }), name: "librustc_asan-c55565a4d1001279.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125662214 }), name: "libterm-5465c10bfd8f7eb9.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125705829 }), name: "librustc_std_workspace_std-559733ba22bce6da.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125840702 }), name: "librustc_std_workspace_core-aededee92e955f4b.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 125849298 }), name: "librustc_demangle-03c3af5ce910211b.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 126086680 }), name: "libbacktrace-c7f6a23a8e1e3f0d.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 126467655 }), name: "liblibc-9cd6fde8086e33c8.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 126964569 }), name: "libstd-a104564b5cd85c3c.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 128484584 }), name: "libhashbrown-17cc4f6d2f17ab1d.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 130485219 }), name: "libproc_macro-d6e25d02026e6a1e.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 151211749 }), name: "libcore-093e765cd9bdca1f.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 154954434 }), name: "liballoc-ac5b59d7ac33292f.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 155188046 }), name: "libprofiler_builtins-9daebda0ed41b83f.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 155255466 }), name: "librustc_tsan-3adcd27409d789cf.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] Considering Listing { kind: File(SystemTime { tv_sec: 1573311800, tv_nsec: 155310726 }), name: "libcfg_if-6308b422d34f5adf.json" }
[2019-11-09T15:05:31Z INFO  rls_analysis::raw] reading 0 crates from /Users/ben/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/rls/lib/rustlib/x86_64-apple-darwin/analysis in 0.000309100s

And refs to the vec source:

[2019-11-09T15:03:48Z TRACE rls_analysis::lowering] record def: Id(21474840201)/Id { krate: 0, index: 3721 } (5): Def { kind: Mod, span: Span { file: "/Users/ben/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/rls/lib/rustlib/src/rust/src/liballoc/vec.rs", range: Range { row_start: Row(0, PhantomData), row_end: Row(0, PhantomData), col_start: Column(0, PhantomData), col_end: Column(0, PhantomData) } }, name: "vec", qualname: "alloc::vec", distro_crate: true, parent: None, value: "src/liballoc/vec.rs", docs: " A contiguous growable array type with heap-allocated contents, written\n `Vec<T>`." }

[bstaletic]

After filtering the RLS log with :v/\<Vec\>/d, this is my output: https://gist.github.com/bstaletic/1dd345c5df5ed3f6118d20cb51c84b61

[bstaletic]

I've updated the gist with the log filtered to only contain references to ycmd/third_party/rls/lib/rustlib/src/rust/src/liballoc/vec.rs.

[bstaletic]

For the record, I managed to repro the issue. I forgot that I had rust-src installed globally. After rustup component remove rust-src the issue was trivial to reproduce.

[sbmueller]

Installing rust with rustup into ~/.rustup resolved the problem for me. However, as mentioned, this is not the desired solution.

[bstaletic]

I've opened rust-lang/rls#1601. Let's see what they have to say.

[bstaletic]

RLS should know about "sysroot". Currently, YCM doesn't set the sysroot to YCM/third_party/ycmd/third_party/rls/. In theory, doing this should fix this problem. However, here's the catch:

1. RLS has a LSP config { 'rust': { 'sysroot': <path> } } and a $SYSROOT environment variable.
2. The linked sys_root_path() is used to discover rust-src.
3. So far, so good, but for some reason that didn't solve the problem.

Help debugging this thing would be appreciated.

 

EDIT:

https://github.com/rust-lang/rls/blob/da014485dfad208be2b97fb920a49d568274f0d9/rls-analysis/src/loader.rs#L127-L129

Looking at that, we shouldn't need to set sysroot, as rustc --print sysroot will be used.
We should be able to just set $RUSTC.
Which we do! https://github.com/ycm-core/ycmd/blob/master/ycmd/completers/rust/rust_completer.py#L95

[juchiast]

I also fixed this issue by installing rust-src globally. It is weird that, goto-definition works without global rust-src, but completing does not.

[bstaletic]

https://rust-lang.github.io/rustup-components-history/

We're planning a move to rust-analyzer. It should be an easy switch and according to my testing, rust-analyzer doesn't suffer this bug.

[bstaletic]

Ycmd now uses rust-analyzer. For those in a hurry, pull in the latest ycmd master.

[ohgree]

Using latest ycmd master, rust-analyzer works fine with cargo based projects.
However, on non-cargo based projects, it fails to fetch any semantic info, both local and non-local.
Is this likely to be fixed when ycmd update makes it to ycm master?

[bstaletic]

Non-cargo based projects are a known limitation of rust-analyzer and there's only basic support for it. I doubt there's much that will change between now and the update of the ycmd submodule. However, there is one thing that will be fixed, though I have no idea if that can help or not. Currently ycmd doesn't play well with the way rust-analyzer expects initializationOptions which should be fixed with https://github.com/rust-analyzer/rust-analyzer/pull/5467 and https://github.com/ycm-core/ycmd/commit/c4fb35b087ec5f831410f9aa9cd1be40d919d81a

[kjeremy]

@bstaletic the fix is now in rust-analyzer. Don't hesitate to file more issues if you see other weirdness.

[bstaletic]

@kjeremy I've seen that. Once again, thanks for the quick action. I did not submit the above ycmd commit for review (it's in my fork), because I'm waiting for rustup to update the nightlies. The relevant branch in my fork works, but it really needs a good review, but that can wait until tomorrow.

[puremourning]

YCM now uses rust-analyser for rust, so this should work. If not we can re-open and/or raise a new issue.