Manim: Idea: Connection to matplotlib over BytesIO

[kolibril13]

Note: This is only an idea, and should not have high priority.
Related to #363

It would be really cool to have manim smoothly interact with matplotlib.
Therefore one idea:
Do you think it is possible, to save figures created in matplotlib to a byte stream, which is then read by ImageMobject?
Then there is no reading and writing to the disk in between.
One ide based on the idea of https://mellowd.dev/python/using-io-bytesio/, I tried the following:

from manim import *
import matplotlib.pyplot as plt
import io


class ConnectingMatplotlib(Scene):
    def construct(self):
        t = np.arange(0.0, 2.0, 0.01)
        s = 1 + np.sin(2 * np.pi * t)
        fig, ax = plt.subplots()
        ax.plot(t, s)
        ax.set(xlabel='time (s)', ylabel='voltage (mV)',
               title='About as simple as it gets')
        ax.grid()

        b = io.BytesIO()
        plt.savefig(b, format='png')
        img = ImageMobject(b.read())  ## error: IndexError: tuple index out of range
        self.add(img)

But reading the byte stream is not possible just like that.
Do you think this is something, that can be easily solved?

[behackl]

For me, running the code from https://mellowd.dev/python/using-io-bytesio/ always leads to an empty image. I think that before accessing the byte stream, something like b.seek(0) has to be called.

Anyhow, matplotlib also allows to get the rendered figure in a format suitable for ImageMobject by calling buffer_rgba() of the corresponding canvas:

import matplotlib
import matplotlib.pyplot as plt
import numpy as np

from manim import Scene, ImageMobject

class ConnectingMatplotlib(Scene):
    def construct(self):        
        t = np.arange(0.0, 2.0, 0.01)
        s = 1 + np.sin(2 * np.pi * t)

        fig, ax = plt.subplots()
        ax.plot(t, s)

        ax.set(xlabel='time (s)', ylabel='voltage (mV)',
               title='About as simple as it gets')
        ax.grid()

        fig.canvas.draw()
        img = ImageMobject(fig.canvas.buffer_rgba())

        self.add(img)
        self.wait()

[kolibril13]

@behackl : Wow,thank you so much! I'm speechless and happy!
With saving and loading from disk (my method):
--- 35.34524869918823 seconds ---
With your method:
--- 6.816277742385864 seconds ---
And here is a script:

from manim import *
import matplotlib.pyplot as plt


def my_function(amplitude, x):
    return amplitude * np.sin(x)


def get_image_plot(amplitude, x):
    fig, ax = plt.subplots()
    ax.plot(x, my_function(amplitude, x))
    plt.ylim(-1, 1)
    ax.set(xlabel='distance (x)', ylabel='amplitude',
           title='My animated plot')
    fig.canvas.draw()
    img = ImageMobject(fig.canvas.buffer_rgba()).scale(4.5)
    plt.close(fig)
    return img


class ConnectingMatplotlib(Scene):
    def construct(self):
        x_values = np.linspace(0, 30, 400)
        amp1 = 0.5
        amp2 = 1
        tr_amplitude = ValueTracker(amp1)
        image = get_image_plot(amp1, x_values)
        self.add(image)

        def update_image(mob):
            new_mob = get_image_plot(tr_amplitude.get_value(), x_values)
            mob.become(new_mob)

        image.add_updater(update_image)

        self.play(tr_amplitude.set_value, amp2, run_time=3)

import time
from pathlib import Path

if __name__ == "__main__":
    script = f"{Path(__file__).resolve()}"
    start_time = time.time()

    os.system( f"manim  -l --custom_folders  --disable_caching  -p -c 'BLACK" + script)
    print("--- %s seconds ---" % (time.time() - start_time))

[leotrs]

_oh snap_ this is awesome! Perhaps in the future we may have to define a MatplotlibScene if this feature becomes popular.

[kolibril13]

I am also super excited about what this matplotlib connection could become!!
I mean, with this feature, you could take more or less every existing figure, and easily configure them with rate functions, set the timings, and use Valuetrackers for certain parameters. You can also set the figures with functions like next_to, and change with e.g. fade in and fade out, which is a huge advantage about the way matplotlib figures are animated at the moment.
As matplotlib is a very big library, maybe we could even make an own package out of it?
Something likemaniplot? :D

[leotrs]

manimplotlib