Materialize: Panic while running `jsonb_agg` with `filter`

[andrioni]

What version of Materialize are you using?

Commit d96243df8

How did you install Materialize?

• [x] Built from source

What was the issue?

Materialize panics and exits after running the following query jsonb_agg and filter:

select jsonb_agg(val) filter (where filter)
from (values (1, true), (2, true), (3, false)) as t (val, filter);

I also managed to make a similar query fail with ERROR: internal error: column 2 is not of expected type jsonb?: [null, "2020-01-16T13:59:00.557Z"], but when trying to reproduce it without actual data, I hit the panic instead.

Is the issue reproducible? If so, please provide reproduction instructions.

Run the following query:

select jsonb_agg(val) filter (where filter)
from (values (1, true), (2, true), (3, false)) as t (val, filter);

Please attach any applicable log files.

The stack trace was:

 thread: <unnamed>
message: Expected datum of type ColumnType { nullable: false, scalar_type: Jsonb }, got value List([Float64(OrderedFloat(1.0)), Float64(OrderedFloat(2.0)), Null])
   0: materialized::handle_panic
             at /home/ec2-user/materialize/src/materialized/src/bin/materialized.rs:388
   1: core::ops::function::Fn::call
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libcore/ops/function.rs:72
   2: std::panicking::rust_panic_with_hook
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libstd/panicking.rs:490
   3: rust_begin_unwind
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libstd/panicking.rs:388
   4: std::panicking::begin_panic_fmt
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libstd/panicking.rs:342
   5: expr::relation::RelationExpr::typ
             at /home/ec2-user/materialize/src/expr/src/relation/mod.rs:216
   6: transform::reduction::FoldConstants::action
             at /home/ec2-user/materialize/src/transform/src/reduction.rs:40
   7: expr::relation::RelationExpr::try_visit_mut::{{closure}}
             at /home/ec2-user/materialize/src/expr/src/relation/mod.rs:860
      expr::relation::RelationExpr::try_visit1_mut
             at /home/ec2-user/materialize/src/expr/src/relation/mod.rs:804
      expr::relation::RelationExpr::try_visit_mut
             at /home/ec2-user/materialize/src/expr/src/relation/mod.rs:860
   8: <transform::reduction::FoldConstants as transform::Transform>::transform
             at /home/ec2-user/materialize/src/transform/src/reduction.rs:33
   9: transform::Optimizer::transform
             at /home/ec2-user/materialize/src/transform/src/lib.rs:168
      transform::Optimizer::optimize
             at /home/ec2-user/materialize/src/transform/src/lib.rs:269
  10: coord::coord::Coordinator<C>::sequence_peek
             at /home/ec2-user/materialize/src/coord/src/coord.rs:1305
      coord::coord::Coordinator<C>::sequence_plan
             at /home/ec2-user/materialize/src/coord/src/coord.rs:815
  11: coord::coord::Coordinator<C>::serve_core
             at /home/ec2-user/materialize/src/coord/src/coord.rs:504
  12: coord::coord::Coordinator<C>::serve::{{closure}}
             at /home/ec2-user/materialize/src/coord/src/coord.rs:398
      tokio::runtime::context::enter
             at /home/ec2-user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.22/src/runtime/context.rs:72
  13: tokio::runtime::handle::Handle::enter
             at /home/ec2-user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.22/src/runtime/handle.rs:76
      coord::coord::Coordinator<C>::serve
             at /home/ec2-user/materialize/src/coord/src/coord.rs:398
  14: materialized::serve::{{closure}}
             at /home/ec2-user/materialize/src/materialized/src/lib.rs:294
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libstd/sys_common/backtrace.rs:130
  15: std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libstd/thread/mod.rs:475
      <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libstd/panic.rs:318
      std::panicking::try::do_call
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libstd/panicking.rs:297
      std::panicking::try
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libstd/panicking.rs:274
      std::panic::catch_unwind
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libstd/panic.rs:394
      std::thread::Builder::spawn_unchecked::{{closure}}
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libstd/thread/mod.rs:474
      core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libcore/ops/function.rs:232
  16: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/liballoc/boxed.rs:1076
      <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/liballoc/boxed.rs:1076
      std::sys::unix::thread::Thread::new::thread_start
             at /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2/src/libstd/sys/unix/thread.rs:87
  17: start_thread
  18: thread_start

[benesch]

I'm looking into this now

[benesch]

Fix is out in #3879. Thanks for the report, as always!

[andrioni]

That was quick, thanks!

[andrioni]

Just ran a test with your PR, and it is working fine. Being able to use jsonb_agg here now instead of max for pivoting caused one of my views to go from ~9.6GB to ~1.2GB now and another from ~16.3GB to ~1.5GB, which is fantastic, thanks again!