Reticulate: Minimal Rmd python chunk with use_python() results in matplotlib error

Created on 2 Apr 2018  路  22Comments  路  Source: rstudio/reticulate

If I make a simple Rmd file with an R chunk

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(reticulate)
use_python("/usr/local/bin/python3")

and a python chunk

```{python}
import sys
print(sys.version_info)

knitting gives the following error:

Quitting from lines 15-17 (test.Rmd) 
Error in py_get_attr_impl(x, name, silent) : 
  AttributeError: module 'matplotlib' has no attribute 'pyplot'
Calls: <Anonymous> ... $.python.builtin.object -> py_get_attr -> py_get_attr_impl -> .Call
Execution halted

If I comment out the use_python() call, then the document knits correctly (but I'm stuck with the OS X default 2.7).

If I import via R, this knits as expected:

```{r}
sys <- import("sys")
print(sys$version_info)

I'm using the RStudio daily build 1.2.502 with the following `sessionInfo()`. I'm not sure why a simple Rmd chunk gives a matplotlib error in this context.

R version 3.4.4 (2018-03-15)
Platform: x86_64-apple-darwin17.3.0 (64-bit)
Running under: macOS High Sierra 10.13.3

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] reticulate_1.6 fortunes_1.5-4 devtools_1.13.5

loaded via a namespace (and not attached):
[1] Rcpp_0.12.16 png_0.1-7 digest_0.6.15 withr_2.1.2 rprojroot_1.3-2
[6] jsonlite_1.5 backports_1.1.2 magrittr_1.5 evaluate_0.10.1 stringi_1.1.7
[11] rmarkdown_1.9 tools_3.4.4 stringr_1.3.0 yaml_2.1.18 compiler_3.4.4
[16] memoise_1.1.0 htmltools_0.3.6 knitr_1.20
```

picture kmiddleton
👍1

Most helpful comment

Same with use_condaenv(). Here's my minimal example:

```{r}
library(reticulate)
use_condaenv("min")
knitr::knit_engines$set(python = reticulate::eng_python)
py_config()
```

```{python}
a = 7
print(a)
```

```{r}
py$a
```

However, if I use another version of Python (even another conda env) without matplotlib installed, it works without giving any error. This makes me think it may be an issue with this function, specifically this line because 1. the same error shows in Python if you try to call pyplot straight from matplotlib without importing matplotlib.pyplot directly (see following code block), and 2. this function exits early if the matplotlib module isn't available.

>>> import matplotlib
>>> plt = matplotlib.pyplot
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'matplotlib' has no attribute 'pyplot'
>>> import matplotlib.pyplot as plt
>>> plt
<module 'matplotlib.pyplot' from '<path>/lib/python3.6/site-packages/matplotlib/pyplot.py'>

Versions:

RStudio: 1.2.496
knitr: 1.20
reticulate: 1.6.0.9000
Python: 3.6.4
OS: Ubuntu 16.04

Edit: I was also able to make it work by uninstalling matplotlib from the first conda env using pip uninstall matplotlib. And then it worked fine after reinstalling matplotlib. :man_shrugging:

picture jdayton3 on 2 Apr 2018
👍3

All 22 comments

Same with use_condaenv(). Here's my minimal example:

```{r}
library(reticulate)
use_condaenv("min")
knitr::knit_engines$set(python = reticulate::eng_python)
py_config()
```

```{python}
a = 7
print(a)
```

```{r}
py$a
```

However, if I use another version of Python (even another conda env) without matplotlib installed, it works without giving any error. This makes me think it may be an issue with this function, specifically this line because 1. the same error shows in Python if you try to call pyplot straight from matplotlib without importing matplotlib.pyplot directly (see following code block), and 2. this function exits early if the matplotlib module isn't available.

>>> import matplotlib
>>> plt = matplotlib.pyplot
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'matplotlib' has no attribute 'pyplot'
>>> import matplotlib.pyplot as plt
>>> plt
<module 'matplotlib.pyplot' from '<path>/lib/python3.6/site-packages/matplotlib/pyplot.py'>

Versions:

RStudio: 1.2.496
knitr: 1.20
reticulate: 1.6.0.9000
Python: 3.6.4
OS: Ubuntu 16.04

Edit: I was also able to make it work by uninstalling matplotlib from the first conda env using pip uninstall matplotlib. And then it worked fine after reinstalling matplotlib. :man_shrugging:

jdayton3 picture jdayton3 on 2 Apr 2018
👍3

@kevinushey Could you take a look at this one?

jjallaire picture jjallaire on 2 Apr 2018

We check for the availability of matplotlib here:

https://github.com/rstudio/reticulate/blob/0abad27db42241efd12f2ad0bcb4b8ea723c74ab/R/knitr-engine.R#L206-L207

And AFAIK all versions of matplotlib should have pyplot and friends (unless there are some 'minimal' installations of matplotlib that come without pyplot or something similar?)

Do you think we need to guard against such installations of matplotlib?

kevinushey picture kevinushey on 2 Apr 2018

I'm not sure if this helps, but I have a full, working matplotlib. e.g., from the command line:

Python 3.6.5 (default, Mar 30 2018, 06:41:53) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
>>> import matplotlib.pyplot as plt

That code works in repl_python() as well.

kmiddleton picture kmiddleton on 2 Apr 2018

@kmiddleton can you share the output of running the following (both in a plain R script and within an R Markdown document)?

library(reticulate)
use_python("/usr/local/bin/python3", required = TRUE)
py_config()

matplotlib <- import("matplotlib")
matplotlib$pyplot
kevinushey picture kevinushey on 2 Apr 2018

@kevinushey Knitting that code gives the same error

Error in py_get_attr_impl(x, name, silent) : 
  AttributeError: module 'matplotlib' has no attribute 'pyplot'
Calls: <Anonymous> ... $.python.builtin.object -> py_get_attr -> py_get_attr_impl -> .Call

Running the three lines to py_config() interactively produces:

> py_config()
python:         /usr/local/bin/python3
libpython:      /usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/config-3.6m-darwin/libpython3.6.dylib
pythonhome:     /usr/local/opt/python/Frameworks/Python.framework/Versions/3.6:/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6
version:        3.6.5 (default, Mar 30 2018, 06:41:53)  [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)]
numpy:          /usr/local/lib/python3.6/site-packages/numpy
numpy_version:  1.14.2

python versions found: 
 /usr/local/bin/python3
 /usr/bin/python
 /usr/local/bin/python

The two /usr/local/bin versions are from homebrew. /usr/bin/python is built-in OS X. which python points to the homebrew version, as does which python3.

Hope this is helpful.

As an update, running the same code from the command line does work:

> matplotlib <- import("matplotlib")
> matplotlib$pyplot
Module(matplotlib.pyplot)
kmiddleton picture kmiddleton on 2 Apr 2018

I was able to reproduce this locally -- I'll dive in and see if I can figure out what's going on.

kevinushey picture kevinushey on 2 Apr 2018

Sorry, nevermind -- my error just occurred because I didn't have matplotlib installed for that particular Python installation.

When I render this document, using rmarkdown::render(<...>):

`````

title: "Untitled"

output: md_document

```{r}
library(reticulate)
use_python("/usr/local/bin/python3", required = TRUE)

display python config

print(py_config())

display module search path

sys <- import("sys")
sys$path

```{r}
py_module_available("matplotlib")

`````

I get the output:

`````
library(reticulate)
use_python("/usr/local/bin/python3", required = TRUE)

# display python config
print(py_config())

## python:         /usr/local/bin/python3
## libpython:      /usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/config-3.6m-darwin/libpython3.6.dylib
## pythonhome:     /usr/local/opt/python/Frameworks/Python.framework/Versions/3.6:/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6
## version:        3.6.5 (default, Mar 30 2018, 06:41:53)  [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)]
## numpy:          /usr/local/lib/python3.6/site-packages/numpy
## numpy_version:  1.13.1
## 
## NOTE: Python version was forced by use_python function

# display module search path
sys <- import("sys")
sys$path

## [1] "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/bin"               
## [2] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python36.zip"           
## [3] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6"              
## [4] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload"  
## [5] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages"
## [6] "/Users/kevin/Library/R/3.4/library/reticulate/python"

py_module_available("matplotlib")

## [1] TRUE

`````

Can you try rendering that and let me know what you see? Also, make sure you're doing the render in a clean R session (if you're executing any other Python / reticulate code you might be forcing reticulate to bind to a version of Python too early)

kevinushey picture kevinushey on 2 Apr 2018

That code in a clean R session with rmarkdown::render("test.Rmd")

library(reticulate)
use_python("/usr/local/bin/python3", required = TRUE)

# display python config
print(py_config())

## python:         /usr/local/bin/python3
## libpython:      /usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/config-3.6m-darwin/libpython3.6.dylib
## pythonhome:     /usr/local/opt/python/Frameworks/Python.framework/Versions/3.6:/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6
## version:        3.6.5 (default, Mar 30 2018, 06:41:53)  [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)]
## numpy:          /usr/local/lib/python3.6/site-packages/numpy
## numpy_version:  1.14.2
## 
## NOTE: Python version was forced by use_python function

# display module search path
sys <- import("sys")
sys$path

## [1] "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/bin"               
## [2] "/usr/local/Cellar/opencv3/3.1.0_3/lib/python3.5/site-packages"                             
## [3] "/Users/kmm/Desktop"                                                                        
## [4] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python36.zip"           
## [5] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6"              
## [6] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload"  
## [7] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages"
## [8] "/usr/local/Cellar/numpy/1.14.2/libexec/nose/lib/python3.6/site-packages"                   
## [9] "/Users/kmm/R_library/reticulate/python"

py_module_available("matplotlib")

## [1] TRUE

That also works with the "knit" button in RStudio and with output html_document.

kmiddleton picture kmiddleton on 3 Apr 2018

If that works, then sorry, I'm out of ideas beyond requesting you just attempt uninstalling and reinstalling matplotlib.

kevinushey picture kevinushey on 3 Apr 2018

FWIW this path looks odd:

## [2] "/usr/local/Cellar/opencv3/3.1.0_3/lib/python3.5/site-packages"

That seems to be a Python 3.5 path but you're using Python 3.6.

kevinushey picture kevinushey on 3 Apr 2018

That path was a bit of cruft leftover from opencv. I cleaned that up and cleared out the entire python site-packages directory to start over. I reinstalled matplotlib and its dependencies.

Still, with any chunk starting with {python}, I get the same error. Things seem fine with {r} chunks.

kmiddleton picture kmiddleton on 3 Apr 2018
👍1

I'm experiencing this same issue using Rstudio Daily Build Version 1.2.510 on macOS

__Here is my Session Info__:

R version 3.4.4 (2018-03-15)
Platform: x86_64-apple-darwin17.4.0 (64-bit)
Running under: macOS High Sierra 10.13.4

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /usr/local/Cellar/openblas/0.2.20_1/lib/libopenblas_haswellp-r0.2.20.dylib

And here is the output from the above mentioned .md:

library(reticulate)
use_python("/usr/local/bin/python3", required = TRUE)

# display python config
print(py_config())

## python:         /usr/local/bin/python3
## libpython:      /usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/config-3.6m-darwin/libpython3.6.dylib
## pythonhome:     /usr/local/opt/python/Frameworks/Python.framework/Versions/3.6:/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6
## version:        3.6.4 (default, Mar  9 2018, 23:15:03)  [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)]
## numpy:          /usr/local/lib/python3.6/site-packages/numpy
## numpy_version:  1.13.1
## 
## NOTE: Python version was forced by use_python function

# display module search path
sys <- import("sys")
sys$path

## [1] "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/bin"             
## [2] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python36.zip"           
## [3] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6"              
## [4] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload"  
## [5] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages"
## [6] "/usr/local/lib/R/3.4/site-library/reticulate/python"

py_module_available("matplotlib")

## [1] TRUE

Here is the test .md I'm trying to knit:

title: "Untitled"
output: md_document
---

    ```{r}
    library(tidyverse)
    library(reticulate)
    use_python("/usr/local/bin/python3")
    ```

    ```{python}
    print("Hello World")
    ```

and here is the error I'm getting

Quitting from lines 13-14 (Untitled.Rmd) 

Error in py_get_attr_impl(x, name, silent) : 
  AttributeError: module 'matplotlib' has no attribute 'pyplot'
Calls: <Anonymous> ... $.python.builtin.object -> py_get_attr -> py_get_attr_impl -> .Call
Execution halted

EDIT: Uninstalling matplotlib using pip3 uninstall matplotlib let's me now fully knit the above .RMD with no issues.

dylanjm picture dylanjm on 4 Apr 2018
👍2

This StackOverflow post seems related:

https://stackoverflow.com/questions/14812342/matplotlib-has-no-attribute-pyplot

I still don't quite understand why this could fail:

matplotlib <- import("matplotlib", convert = FALSE)
matplotlib$pyplot

but the development version of reticulate attempts to work around this by explicitly importing the sub-module. You can test with:

devtools::install_github("rstudio/reticulate")
kevinushey picture kevinushey on 4 Apr 2018
👍2

The development version fixes the issue for me. Thank you very much for working on this issue.

    library(reticulate)
    use_python("/usr/local/bin/python3", required = TRUE)

    # display python config
    print(py_config())

    ## python:         /usr/local/bin/python3
    ## libpython:      /usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/config-3.6m-darwin/libpython3.6.dylib
    ## pythonhome:     /usr/local/opt/python/Frameworks/Python.framework/Versions/3.6:/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6
    ## version:        3.6.5 (default, Mar 30 2018, 06:41:53)  [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)]
    ## numpy:          /usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy
    ## numpy_version:  1.14.2
    ## 
    ## NOTE: Python version was forced by use_python function

    # display module search path
    sys <- import("sys")
    sys$path

    ## [1] "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/bin"               
    ## [2] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python36.zip"           
    ## [3] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6"              
    ## [4] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload"  
    ## [5] "/Users/kmm/Library/Python/3.6/lib/python/site-packages"                                    
    ## [6] "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages"
    ## [7] "/Users/kmm/R_library/reticulate/python"

    py_module_available("matplotlib")

    ## [1] TRUE

    print(1/7)

    ## 0.14285714285714285
kmiddleton picture kmiddleton on 4 Apr 2018
🎉2

Similar problems here. I would like to use conda and use_condaenv. The reticulate-dev version (9002) has not fixed the pyplot-error for me, when I install matplotlib via conda install matplotlib.
Reinstalling matplotlib and avoiding conda makes it possible to create plots (awesome!). I need to learn about and clean up sys.path, I suppose.

henningsway picture henningsway on 10 Apr 2018

I had the same issue. I used this answer from StackOverflow and it fixed the problem for me.

pagepiccinini picture pagepiccinini on 17 Apr 2018
👍1

I'm having a similar problem, which I described in this StackOverflow post. I was able to mostly fix it by going to the development version of reticulate, but I'm still running into some strange issues with Qt5Agg and XCB. I tried the solution in the post that @pagepiccinini linked -- it looks like it should be relevant, but didn't help.

ccjolley picture ccjolley on 17 Apr 2018

I was eventually able to get things working by including a call to use_python('/usr/bin/python3') in my R Markdown document. I'd originally been using use_python('home/craig/anaconda3/bin/python'). My intuition here is that anaconda is setting up matplotlib's backend in some way that reticulate doesn't like.

ccjolley picture ccjolley on 20 Apr 2018

The fix @pagepiccinini suggested worked for me. You can also try (on Mac):

mpl = reticulate::import("matplotlib")
mpl$use('TkAgg')

If you'd like to try this out before switching your backend or for a simple workaround in your script.

muschellij2 picture muschellij2 on 31 Aug 2018
👍2

After installing apt-get -y install python-tk as root, the follow error disappeared for me: Error in py_get_attr_impl(x, name, silent) : AttributeError: 'module' object has no attribute 'pyplot'

fdrennan picture fdrennan on 24 Jan 2019
😄1

issue still exists for python chunk

etheleon picture etheleon on 8 Jun 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

lbocken picture lbocken  路  3Comments
Seito-N picture Seito-N  路  4Comments
cb4github picture cb4github  路  5Comments
VyshaliEnukonda picture VyshaliEnukonda  路  8Comments
berkorbay picture berkorbay  路  4Comments