Pyo3: `dyld: Symbol not found: _PyExc_SystemError` in tests if there is `#[pymethods]` in project

[deaz]

Tests fail with error when there are methods in #[pymethods] impl for struct:

Compiling pyo3-test v0.1.0 (/Users/ivan/pyo3-test)
    Finished dev [unoptimized + debuginfo] target(s) in 1.10s
     Running target/debug/deps/pyo3_test-34e869060858761f
dyld: Symbol not found: _PyExc_SystemError
  Referenced from: /Users/ivan/pyo3-test/target/debug/deps/pyo3_test-34e869060858761f
  Expected in: flat namespace
 in /Users/ivan/pyo3-test/target/debug/deps/pyo3_test-34e869060858761f
error: process didn't exit successfully: `/Users/ivan/pyo3-test/target/debug/deps/pyo3_test-34e869060858761f` (signal: 6, SIGABRT: process abort signal)

🌍 Environment

• Your operating system and version: macOS 10.14.2 (18C54)
• Your python version: 3.7.2
• How did you install python (e.g. apt or pyenv)? Did you use a virtualenv?: i don't remeber, probably brew install python3. I used virtualenv via pipenv
• Your rust version (rustc --version): 1.34.0-nightly (8ae730a44 2019-02-04)
• Are you using the latest pyo3 version? Have you tried using latest master (replace version = "0.x.y" with git = "https://github.com/PyO3/pyo3")? I use 0.6.0-alpha.2. With latest master error is the same.

💥 Reproducing

Run cargo test for the following code:

use pyo3::prelude::*;

#[pyclass]
struct Class {}

#[pymethods]
impl Class {
    #[new]
    fn __new__(obj: &PyRawObject) -> PyResult<()> {
        obj.init(|| Class {})
    }
}

#[pymodule]
fn pyo3_test(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<Class>()
}

#[cfg(test)]
mod tests {
    #[test]
    fn test() {}
}

Also i created minimal project for reproducing the issue: https://github.com/deaz/pyo3-test

[kngwyu]

Thanks for reporting!
Basically we need

[dependencies.pyo3]
features = []

to test project.
For extension projects(i.e. projects with features = ["extension-module"]), you can use setuptools-rust to run test.
@konstin
Do you know any workaround other than that?

[deaz]

As workaround I added #[cfg(not(test))] to my struct and impl and it is working fine

[konstin]

First of all, thank you for the test repository!

As @kngwyu said, we need to deactivate features for the tests. A proper solution for this requires https://github.com/rust-lang/cargo/issues/2911, which unfortunately is unlikely to be implemented soon.

For now, you can use the following as workaround:

[package]
name = "pyo3-test"
version = "0.1.0"
authors = ["Ivan Vologdin <[email protected]>"]
edition = "2018"

[lib]
name = "pyo3_test"
crate-type = ["cdylib"]

[dependencies.pyo3]
version = "0.6.0-alpha.2"

[features]
extension-module = ["pyo3/extension-module"]
default = ["extension-module"]

Tests can then be run with cargo test --no-default-features, while cargo build still works as expected.

With a bit context this workaround would be very useful in the guide.

[deaz]

Thanks for help!

[boredstiff]

Just ran into this issue - thank you both for posting really great instructions on how to work around this.

[konstin]

BTW this is also in the guide: https://pyo3.rs/v0.7.0-alpha.1/advanced.html#testing

[liquidscorpio]

The work-around mentioned in https://github.com/PyO3/pyo3/issues/340#issuecomment-461514532 does not work if the test is run on a workspace where one of member crates uses PyO3.

[mvaled]

I'm getting this error as well, however I'm not using #[pymethods] yet. My entire Rust code now is just:

use pyo3::prelude::*;

#[pymodule]
fn travertine(_py: Python, _m: &PyModule) -> PyResult<()> {
    Ok(())
}

#[cfg(test)]
mod test {
    use super::*;
    use pyo3::types::{IntoPyDict, PyDateTime};

    #[test]
    fn pydelta_conversion() {
        let gil = Python::acquire_gil();
        let py = gil.python();
        let datetime = py.import("datetime").unwrap();
        let locals = [("datetime", datetime)].into_py_dict(py);
        let now: &PyDateTime = py
            .eval("datetime.datetime.utcnow()", None, Some(&locals))
            .unwrap()
            .downcast()
            .unwrap();
        println!("{:?}", now);
    }
}

That suffices to trigger the error. The entire repository is in https://github.com/mvaled/pyo3bug340

The issue is reproducible with stable version of pyo3.

[davidhewitt]

Yes. TBH this issue is (I believe) really a dupe of #771

[gilescope]

Maybe we're going around this the wrong way, maybe when maturin builds it should add in --features=pyo3/extension-module because it knows it's required. That way the cargo test would just work.

We can already do:

maturin develop --cargo-extra-args="--features pyo3/extension-module"

Why not add that flag in automatically if it's clear that it's not there? Having pyo3/extension-module turned on as a default feature seems like a footgun as people will just run cargo test and expect stuff to work on cli, IDEs etc.

I've posted this in the maturin issues list here:
https://github.com/PyO3/maturin/issues/325