Presto: Getting error - Mismatched Domain types: smallint vs integer

[anirbanch]

Presto throws the following error when the type in the Parquet file schema is different than the type in the metastore. The query has a WHERE clause like the one below, Presto going against Hive metastore

select cc from tt where _field <= 0004_

Issue maybe similar to https://github.com/prestosql/presto/issues/137

com.facebook.presto.spi.PrestoException: Error opening Hive split hdfs:// (offset=0, length=20433447): Mismatched Domain types: smallint vs integer

          at com.facebook.presto.hive.parquet.ParquetPageSourceFactory.createParquetPageSource(ParquetPageSourceFactory.java:216)

          at com.facebook.presto.hive.parquet.ParquetPageSourceFactory.createPageSource(ParquetPageSourceFactory.java:115)

          at com.facebook.presto.hive.HivePageSourceProvider.createHivePageSource(HivePageSourceProvider.java:157)

          at com.facebook.presto.hive.HivePageSourceProvider.createPageSource(HivePageSourceProvider.java:93)

          at com.facebook.presto.spi.connector.classloader.ClassLoaderSafeConnectorPageSourceProvider.createPageSource(ClassLoaderSafeConnectorPageSourceProvider.java:44)

          at com.facebook.presto.split.PageSourceManager.createPageSource(PageSourceManager.java:56)

          at com.facebook.presto.operator.ScanFilterAndProjectOperator.getOutput(ScanFilterAndProjectOperator.java:218)

          at com.facebook.presto.operator.Driver.processInternal(Driver.java:379)

          at com.facebook.presto.operator.Driver.lambda$processFor$8(Driver.java:283)

          at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:675)

          at com.facebook.presto.operator.Driver.processFor(Driver.java:276)

          at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1065)

          at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162)

          at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:483)

          at com.facebook.presto.$gen.Presto_0_215____20190126_062444_1.run(Unknown Source)

          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

          at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.IllegalArgumentException: Mismatched Domain types: smallint vs integer

          at com.facebook.presto.spi.predicate.Domain.checkCompatibility(Domain.java:229)

          at com.facebook.presto.spi.predicate.Domain.intersect(Domain.java:184)

          at com.facebook.presto.spi.predicate.TupleDomain.intersect(TupleDomain.java:197)

          at com.facebook.presto.spi.predicate.TupleDomain.overlaps(TupleDomain.java:300)

          at com.facebook.presto.parquet.predicate.TupleDomainParquetPredicate.matches(TupleDomainParquetPredicate.java:95)

          at com.facebook.presto.parquet.predicate.PredicateUtils.predicateMatches(PredicateUtils.java:90)

          at com.facebook.presto.hive.parquet.ParquetPageSourceFactory.lambda$createParquetPageSource$2(ParquetPageSourceFactory.java:178)

          at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)

          at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)

          at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)

          at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)

          at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)

          at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)

          at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)

          at com.facebook.presto.hive.parquet.ParquetPageSourceFactory.createParquetPageSource(ParquetPageSourceFactory.java:179)

          ... 17 more

[findepi]

@anirbanch thanks for your issue

I see you're using Presto 0.215.
I fixed this issue in Presto 302 (https://prestosql.io/docs/current/release/release-302.html, https://github.com/prestosql/presto/pull/131).

Could you please give it a try?

[anirbanch]

Verified. The issue does not show up in v304