Neo4j: ExecutionFailure involving OPTIONAL MATCH and COLLECT(DISTINCT)

[cybersam]

See this SO question for a simple test case and a workaround. You have to first create an (f:DOES_EXIST) node to see the issue.

The ExecutionaFailuer also occurs if the RETURN clause is simply:

RETURN COLLECT(DISTINCT n.property)

[synergeticon]

I can confirm the issue.

OPTIONAL MATCH (n:DOES_NOT_MATCH)
OPTIONAL MATCH (f:DoesMatch)
RETURN COLLECT(DISTINCT f.username), COLLECT(DISTINCT n.test))

leads to:

java.lang.NullPointerException
org.neo4j.cypher.internal.compiler.v2_3.pipes.NiceHasherValue.equals(NiceHasher.scala:70)
scala.collection.mutable.FlatHashTable$class.findElemImpl(FlatHashTable.scala:128)
scala.collection.mutable.FlatHashTable$class.containsElem(FlatHashTable.scala:121)
scala.collection.mutable.HashSet.containsElem(HashSet.scala:40)
scala.collection.mutable.HashSet.contains(HashSet.scala:57)
org.neo4j.cypher.internal.compiler.v2_3.pipes.aggregation.DistinctFunction.apply(DistinctFunction.scala:38)
org.neo4j.cypher.internal.compiler.v2_3.pipes.EagerAggregationPipe$$anonfun$internalCreateResults$1$$anonfun$apply$1.apply(EagerAggregationPipe.scala:84)
org.neo4j.cypher.internal.compiler.v2_3.pipes.EagerAggregationPipe$$anonfun$internalCreateResults$1$$anonfun$apply$1.apply(EagerAggregationPipe.scala:84)
scala.collection.immutable.List.foreach(List.scala:381)
org.neo4j.cypher.internal.compiler.v2_3.pipes.EagerAggregationPipe$$anonfun$internalCreateResults$1.apply(EagerAggregationPipe.scala:84)
org.neo4j.cypher.internal.compiler.v2_3.pipes.EagerAggregationPipe$$anonfun$internalCreateResults$1.apply(EagerAggregationPipe.scala:80)
scala.collection.Iterator$class.foreach(Iterator.scala:742)
scala.collection.AbstractIterator.foreach(Iterator.scala:1194)
org.neo4j.cypher.internal.compiler.v2_3.pipes.EagerAggregationPipe.internalCreateResults(EagerAggregationPipe.scala:80)
org.neo4j.cypher.internal.compiler.v2_3.pipes.PipeWithSource.createResults(Pipe.scala:125)
 org.neo4j.cypher.internal.compiler.v2_3.pipes.PipeWithSource.createResults(Pipe.scala:122)
org.neo4j.cypher.internal.compiler.v2_3.executionplan.DefaultExecutionResultBuilderFactory$ExecutionWorkflowBuilder.createResults(DefaultExecutionResultBuilderFactory.scala:93)
org.neo4j.cypher.internal.compiler.v2_3.executionplan.DefaultExecutionResultBuilderFactory$ExecutionWorkflowBuilder.build(DefaultExecutionResultBuilderFactory.scala:63)
org.neo4j.cypher.internal.compiler.v2_3.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1.apply(ExecutionPlanBuilder.scala:223)
org.neo4j.cypher.internal.compiler.v2_3.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1.apply(ExecutionPlanBuilder.scala:207)
org.neo4j.cypher.internal.compiler.v2_3.executionplan.ExecutionPlanBuilder$$anon$2.run(ExecutionPlanBuilder.scala:164)
org.neo4j.cypher.internal.compatibility.CompatibilityFor2_3$ExecutionPlanWrapper$$anonfun$run$1.apply(CompatibilityFor2_3.scala:200)
org.neo4j.cypher.internal.compatibility.CompatibilityFor2_3$ExecutionPlanWrapper$$anonfun$run$1.apply(CompatibilityFor2_3.scala:200)
org.neo4j.cypher.internal.compatibility.exceptionHandlerFor2_3$.runSafely(CompatibilityFor2_3.scala:116)
org.neo4j.cypher.internal.compatibility.CompatibilityFor2_3$ExecutionPlanWrapper.run(CompatibilityFor2_3.scala:199)
 org.neo4j.cypher.internal.PreparedPlanExecution.execute(PreparedPlanExecution.scala:27)
 org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:120)
org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:114)
org.neo4j.cypher.javacompat.internal.ServerExecutionEngine.executeQuery(ServerExecutionEngine.java:62)
org.neo4j.server.rest.transactional.TransactionHandle.executeStatements(TransactionHandle.java:328)
 org.neo4j.server.rest.transactional.TransactionHandle.commit(TransactionHandle.java:147)\r\n\tat org.neo4j.server.rest.web.TransactionalService$2.write(TransactionalService.java:211)\r\n\tat com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:71)\r\n\tat com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:57)\r\n\tat com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)\r\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)\r\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)\r\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)\r\n\tat com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)\r\n\tat com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)\r\n\tat com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)\r\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\r\n\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)\r\n\tat org.neo4j.server.rest.web.CollectUserAgentFilter.doFilter(CollectUserAgentFilter.java:69)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\r\n\tat org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)\r\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\r\n\tat org.eclipse.jetty.server.Server.handle(Server.java:497)\r\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\r\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)\r\n\tat org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\r\n\tat java.lang.Thread.run(Unknown Source)\r\n

[brandonburkett]

We are seeing this with 3.0 M05 too.

[davidegrohmann]

This problem has been resolved by https://github.com/neo4j/neo4j/pull/6814.
The fix will available in the next releases of 2.2, 2.3, and 3.0.