elpy-goto-definition does not find definition in some cases

[dmalyuta]

Issue

elpy-goto-definition is able to go to system functions as well as to certain project-defined custom functions, however some project-defined functions cannot be found (No definition found is returned).

Steps to reproduce

1. Use the following setup for elpy in your ~/.emacs.d/init.el:

(use-package elpy
    :ensure t
    :config
    (elpy-enable))

My full init.el can be found here.

1. sudo apt-get install python-pip
2. sudo pip install jedi flake8 importmagic autopep8
3. cd ~ && git clone https://github.com/ethz-asl/kalibr
4. Open Emacs, open file ~/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py.
5. Place cursor at line 49 inside the substring addDesignVariables and do M-x elpy-goto-definition
6. This returns No definition found in the minibuffer, yet this function is defined in the IccSensors.py file, which is in the same directory and is imported in the __init__.py which is also in the same directory.

Versions

• Emacs: 24.5.1
• OS: Ubuntu 14.04
• Python: 2.7.6

Expected behaviour

elpy-goto-definition should be able to find and go to the definition of the addDesignVariables function (which is one of the several project-defined functions for which I get No definition found - you can find other examples in the IccCalibrator.py file by trying out M-x elpy-goto-definition on them).

[jorgenschaefer]

Hello, and thanks for the report! I'm afraid that Python is a rather dynamic language, which makes finding definitions a heuristic process rather than something clean and simple. The backends Elpy uses (Jedi or Rope) try their best, but they can not always find definitions. You can try and report this problem with Jedi – this particular case should be doable for that library.

I'm sorry I can not be of more help. Your bug report is really excellent, I would love to get more like these! :-)

[dmalyuta]

Thanks for replying, I was also thinking that this may be a Jedi problem - so I made an issue over there (davidhalter/jedi#880). Out of interest, what is then your workflow for Python in this context, if it is a large codebase and you cannot autocomplete definitions of certain methods? Do you use etags or something?

[jorgenschaefer]

I don't think I have a particular "workflow" there, unless you count "type the name" as a workflow. For dynamic languages like Python, JavaScript, Ruby etc. I see completions as a nice bonus, not something to expect and take for granted.

[dmalyuta]

I see - what are then the "primary" features that you expect when coding in Python? I'm asking because I'm only starting out with this language and you have a lot of experience with it (enough to have written a great Emacs package for it!) - so I'd like to know what professional developers consider as relevant features.

[jorgenschaefer]

I suspect you come from a Java+IDE background or similar, and Python is your first dynamic language?

Important features tend to rely on: Syntax highlighting and coding style notifications (flake8), primarily. Being able to run tests quickly and easily (C-c C-t and M-x recompile). Being able to quickly grep/ag through the whole project. Completions and "go to definition" are nice, but as I said, for me it's more that I'm happy when they work and just deal with it when they don't.

[dmalyuta]

I come from C++ with Eclipse, more recently C++ with Emacs, background - so yes, I've got a mindset of expecting auto-completion to work flawlessly. Have you had any good/bad experiences using Emacs' etags for Python?

But I see what you mean - thanks for your answers! Hopefully the issue I posted on the Jedi github page will at some point be resolved.

[jorgenschaefer]

Have you had any good/bad experiences using Emacs' etags for Python?

I have no good experiences using any *tags with any language, really. :-)

[dmalyuta]

Ok, thank you very much for your answers.

[analyticd]

This is closed but I'll just add that I use a technique in such cases of
enlisting the help of ag, but limited to my virtual env (for the punchline go
to the gist link at the end):

(require 'ag)
  (after 'my-ag-config  ; my ag config is at the bottom of this post; after is just a macro for eval-after-load
    (defun my-python-jump-to-definition (string directory)
      "Search using ag in a given DIRECTORY for a given literal search STRING,
with STRING defaulting to the symbol under point."
      (interactive (list (ag/read-from-minibuffer "Search string")
                        (read-directory-name
                          "Directory: "
                          (concat python-shell-virtualenv-root
                                  "/"
                                  "/lib/python3.6/site-packages/"))))
      (ag/search string directory)))

and then bind that via

(bind-key "M-C-." 'my-python-jump-to-definition elpy-mode-map)

When invoked, I typically alter the search string to, say, def mean(, instead of
just mean, then I limit what library I want to search in, say pandas, by typing
in the library where my own analysis or understanding of Python libs says it is
likely to exist (again, because Python is dynamic, you have to do a bit more
reasoning to trace things out), then hit enter. I've A/B'd this technique
against using PyCharm and I find that on average, this is actually better when
elpy goto definition fails (which I have bound to M-.).

Another individual does it like this (and I use this too):

(defun goto-def-or-rgrep ()
      "Go to definition of thing at point or do an rgrep in
      project if that fails"
      (interactive)
      (condition-case nil (elpy-goto-definition)
        (error (elpy-rgrep-symbol (thing-at-point 'symbol)))))
  (bind-key "M-." 'goto-def-or-rgrep elpy-mode-map) ; bind-key comes with use-package

But that only will search in the project directory (as determined by elpy - see
source of elpy-rgrep-symbol). You can customize elpy to more broadly define your
project directories by defining a new function that would return also system
libraries you need to jump to def on. However, I find that my
my-python-jump-to-definition works great as the ag emacs package displays things
nicely.

My ag config (referenced above):

(use-package ag
  :ensure t
  ;; :commands (ag ag-mode ag-files ag-regexp)
  :bind ("C-c u s" . ag)
  :init
  (setq ag-highlight-search t)
  (setq ag-reuse-buffers t)
  (defun my-setup-ag ()
    "Function called to set my ag stuff up."
    (toggle-truncate-lines t)
    (switch-to-buffer-other-window "*ag search*"))
  (add-hook 'ag-mode-hook 'my-setup-ag)
  (after 'evil
    (evil-set-initial-state 'ag-mode 'normal))
  :config
  (unbind-key "k" ag-mode-map)   ;; unbind kill window
  (unbind-key "h" ag-mode-map)
  (after 'evil
    (evil-define-key 'normal ag-mode-map (kbd "k") 'nil)
    ;; (evil-define-key 'motion ag-mode-map (kbd "k") 'compilation-previous-error)
    ;; (evil-define-key 'motion ag-mode-map (kbd "j") 'compilation-next-error)
    ))

Finally here is the after macro referenced above:

(defmacro after (feature &rest body)
  "After FEATURE is loaded, evaluate BODY."
  (declare (indent defun))
  `(eval-after-load ,feature
    '(progn ,@body)))

And once I jump to a location either via elpy-goto-definition or
my-python-jump-to-definition I then employ idomenu (emacs package) to jump to
classes and methods. idomenu is a lot smarter in python than a lot of people
realize. If multiple classes are defined in one file, say in pandas somewhere to
where you have jumped, then when you invoke idomenu (which I bind to
evil-leader,i for ease of use) it will first list all classes in the file such
that when you select one, you can then subsequently choose from another method
of everything to do with your class choice.

Oh, and to make this all work I put my Anaconda Python root in my path, i.e.,
~/anaconda/bin, and then I use the conda emacs package (not anaconda-mode) to
manage my virtualenv situation which was previously created with Anaconda via
conda env create (see this helpful blog post, not by me,
http://tdhopper.com/blog/2015/Nov/24/my-python-environment-workflow-with-conda/
for more information on this). I find conda necessary because it uses Anaconda's environment.yml
file in your project root to determine which Python process to invoke, elpy
doesn't do this. To make it easier on future readers, here I recapitulate some
of above and include my total python config at the following gist:

https://gist.github.com/analyticd/cfb253cce2ab5c9b97aed7def1e37a04