vim.error: Vim:E315: ml_get: invalid lnum: 18

[Yggdroot]

Describe the bug
I encounter an issue https://github.com/Yggdroot/LeaderF/issues/725

To Reproduce
Detailed steps to reproduce the behavior:

suppose there is a file https://github.com/open-mmlab/mmdetection/blob/master/tools/train.py under current directory(~).
suppose there is another file a.txt at the same directory.

a.txt

docs/tutorials/data_pipeline.md
resources/data_pipeline.png
mmdet/datasets/pipelines/compose.py
mmdet/datasets/pipelines/loading.py
mmdet/datasets/pipelines/__init__.py
mmdet/datasets/pipelines/formating.py
mmdet/datasets/pipelines/transforms.py
mmdet/datasets/pipelines/instaboost.py
mmdet/datasets/pipelines/auto_augment.py
mmdet/datasets/pipelines/test_time_aug.py
mmdet/datasets/samplers/__init__.py
configs/_base_/datasets/cityscapes_instance.py
mmdet/datasets/deepfashion.py
configs/_base_/datasets/deepfashion.py
configs/_base_/datasets/cityscapes_detection.py
mmdet/datasets/samplers/distributed_sampler.py
configs/guided_anchoring/ga_retinanet_r101_caffe_fpn_mstrain_2x.py

1. cd && git clone https://github.com/tmhedberg/SimpylFold.git
2.

vim train.py -Nu <(cat <<'EOF'
    filetype plugin on
    set rtp-=~/.vim
    set rtp-=~/.vim/after
    set rtp^=~/SimpylFold
EOF
)

1. execute the following commands in cmdline-mode.

:sp a.txt
:py3 import vim
:py3 b = vim.current.buffer
:py3 aaa = b[:]
:hide
:py3 b[:] = aaa

1. the error occurs

PS. The main job SimpylFold plugin does is

setlocal foldexpr=SimpylFold#FoldExpr(v:lnum)
setlocal foldmethod=expr

Expected behavior
no error

Screenshots

Environment (please complete the following information):

• Vim version 8.2.1891
• OS: [e.g. Ubuntu 18.04, Windows 10 1809, macOS 10.14]
• Terminal: [e.g. GNOME Terminal, mintty, iTerm2, tmux, GNU screen] (Use GUI if you use the GUI.)

[lacygoill]

I reduced the issue to this (no third-party plugin or files required):

vim -Nu NONE -S <(cat <<'EOF'
    vim9
    set fdm=expr fde=Fde(v:lnum)
    b:regex = '^'
    fu Fde(lnum)
        let ld = [{}]
        let lines = bufnr('%')->getbufline(1, '$')
        let was_import = 0
        for lnum in range(1, len(lines))
            let line = lines[lnum]
            call add(ld, {'a': b:regex})
            let ld[lnum].foldexpr = was_import ? 1 : '>1'
            let was_import = 1
        endfor
        return ld[a:lnum].foldexpr
    endfu
    setline(1, repeat([''], 15) + repeat(['from'], 3))
    repeat(['x'], 17)->writefile('/tmp/a.txt')
    sp /tmp/a.txt
    py3 import vim
    py3 b = vim.current.buffer
    py3 aaa = b[:]
    hide
    py3 b[:] = aaa
EOF
)

Which gives this stack trace:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
vim.error: Vim:E315: ml_get: invalid lnum: 18

Backtrace.

---

Interestingly enough, no internal error is raised when the folding function is refactored in Vim9 script:

vim -Nu NONE -S <(cat <<'EOF'
    vim9
    b:regex = '^'
    set fdm=expr fde=Fde(v:lnum) debug=throw
    def Fde(lnum: number): any
        var ld = [{}]
        var lines = bufnr('%')->getbufline(1, '$')
        var was_import = 0
        for _lnum in [1, 18]
            var line = lines[_lnum]
            add(ld, {'a': b:regex})
            ld[_lnum].foldexpr = was_import ? 1 : '>1'
            was_import = 1
        endfor
        return ld[lnum].foldexpr
    enddef
    setline(1, repeat([''], 15) + repeat(['from'], 3))
    repeat(['x'], 17)->writefile('/tmp/a.txt')
    sp /tmp/a.txt
    py3 import vim
    py3 b = vim.current.buffer
    py3 aaa = b[:]
    hide
    py3 b[:] = aaa
EOF
)

Instead, it correctly throws regular errors (visible if 'debug' is set to throw):

E684: list index out of range: 18
...
E121: Undefined variable: b:regex
...

[brammool]

I reduced the issue to this (no third-party plugin or files required):

vim -Nu NONE -S <(cat <<'EOF'
    vim9
    set fdm=expr fde=Fde(v:lnum)
    b:regex = '^'
    fu Fde(lnum)
        let ld = [{}]
        let lines = bufnr('%')->getbufline(1, '$')
        let was_import = 0
        for lnum in range(1, len(lines))
            let line = lines[lnum]
            call add(ld, {'a': b:regex})
            let ld[lnum].foldexpr = was_import ? 1 : '>1'
            let was_import = 1
        endfor
        return ld[a:lnum].foldexpr
    endfu
    setline(1, repeat([''], 15) + repeat(['from'], 3))
    repeat(['x'], 17)->writefile('/tmp/a.txt')
    sp /tmp/a.txt
    py3 import vim
    py3 b = vim.current.buffer
    py3 aaa = b[:]
    hide
    py3 b[:] = aaa
EOF
)

Which gives this stack trace:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
vim.error: Vim:E315: ml_get: invalid lnum: 18

Thanks for figuring this out. I can use it for a test.

It happens because Python changes a hidden buffer using the current
window, which has side effects. I'll make a simple solution that fixes
this case, but there might be other problems.

--
hundred-and-one symptoms of being an internet addict:

1. It's Saturday afternoon in the middle of May and you
   are on computer.

/// Bram Moolenaar -- [email protected] -- http://www.Moolenaar.net \\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\
\\ an exciting new programming language -- http://www.Zimbu.org ///
\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[roachsinai]

@brammool thanks a lot!