Pyo3: Class creation is slow

[ethanhs]

My environment is rustc 1.40, python 3.8, Windows 10.

💥 Reproducing

Make a pyclass and benchmark it!

I realize this could be a quirk of my current setup, but I was doing some benchmarking, and realized that it can take up to 440ns to create a Python class object. This is fine in isolation, but when performance matters, and you are making a lot of these objects (or want to make them as fast as possible) it matters. I created a simple Cython test file to compare, and it was more than 2x faster in creating the class (~200ns).

Here is the sample Cython code:

# cython: language_level=3
cdef class Example:
    def __cinit__(self):
        self.a = 4
        self.b = "Test"
    cdef public long a
    cdef public str b

You can compile an extension from it with cythonize -b test.pyx.

The Rust code is the obvious:

use pyo3::prelude::*;

#[pyclass]
pub struct Example {
    #[pyo3(get, set)]
    pub a: i64,
    #[pyo3(get, set)]
    pub b: String,
}

#[pymethods]
impl Example {
    #[new]
    fn new(obj: &PyRawObject) {
        obj.init({
            Example {
                a: 4,
                b: String::from("Hello"),
            }
        });
    }
}

#[pymodule]
fn example(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<Example>()?;

    Ok(())
}

I realize this is probably not the biggest priority, but it seems odd that class creation would be so much slower.

[ehiggs]

Could you post your rust build commands and commands for how you benchmarked this?

Thanks

[ethanhs]

Sure! I just used cargo build --release (I'd have used target-cpu=native but I'm getting a heap corruption ICE if I do that). Then I write a simple benchmark using pytest-benchmark like the following:

import example
def test_class_creation_rust(benchmark):
    benchmark(example.Example)

This calls example.Example() which is the best benchmark I could think of atm.

[kngwyu]

Thank you for investigating that, but...
PyO3 has some fundamental overhead in its core, so I'm not sure we can fix it :thinking:

[ethanhs]

Ah I see, that's too bad :/

I was under the assumption pyo3 was zero cost. Thank you for the information!

[ijl]

Why does pyo3 track allocations and use PyRef rather than just increment and decrement the PyObject's reference count? This decision might predate everyone maintaining it now, but does anyone know?

[kngwyu]

@ijl
I think I had some discussions about this... but now only found this comment
https://github.com/PyO3/pyo3/issues/271#issuecomment-438225851

Now I'm too drunk to do more detailed explanation..., but I'll do it soon and I want to rethink about this problem.
If we can remove ReleasePool, it's really cool.

[ehiggs]

I was able to reproduce this in ipython. In the following, examplec is the cython code.

$ ipython
Python 3.7.5rc1 (default, Oct  8 2019, 16:47:45) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.9.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import examplec                                                                                            

In [2]: %timeit examplec.Example()                                                                                 
68.7 ns ± 0.315 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [3]: import example                                                                                             

In [4]: %timeit example.Example()                                                                                  
224 ns ± 1.6 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

PyO3 has some fundamental overhead in its core, so I'm not sure we can fix it thinking

What sort of overhead is there that cython doesn't seem to require?

[kngwyu]

@ehiggs
Sorry for the delay but I wrote an explanation #679

[davidhewitt]

@ethanhs we've made steps to improve this in 0.10.1 - would be curious if you'd be interested in rerunning the benchmarks performed to see what the improvement is like?

My guess is we're still not as fast as cython but hopefully we're closer, and maybe we'll find the next steps someday soon!

[ethanhs]

@davidhewitt I ended up making a quick repo of my test for reproducibility. https://github.com/ethanhs/clscreate.

The mean times I got on my Ryzen 9 3900x on Windows with nightly 2020-05-12:
Cython: 75.9655 ns
Python: 247.2162 ns
PyO3: 354.3962 ns

Edit: I realized I should probably also try a non-static class and other things which may get optimized in unrealistic ways by C, but not in Rust, but I don't really have the time atm.

[davidhewitt]

Oh man, it's a bit disappointing that class creation is slower than native Python classes even. I mean it's not suprising that they're also well optimised, but still...

Definitely we shall have to look at improvements here :)

[ethanhs]

Yeah, it is rather unfortunate. If I remember correctly, when I initially posted this was also the case that PyO3 was slower than Python, so at least it isn't a regression.

I look forward to improvements to come! :)