Presto: Release notes for 318

[sopel39]

Dain Sundstrom

• [ ] all checked
• 08-07 10:41:52 2597804d02 Fix distinct/groupby on UUID

• 08-15 16:04:47 b619848895 Add missing doAs around Hive create empty bucket files

  David Phillips

• [ ] all checked

• 08-02 15:21:45 16a1eebdd6 Update gcs version
• 08-06 22:53:25 0c8706b257 Extract caching Hive metastore to separate package
• 08-06 22:55:04 12328e3159 Improve error messages for loading various plugin services
• 08-07 18:13:20 e82c2d5301 Add Iceberg connector
• 08-13 16:24:02 466edbcf12 Remove Google Sheets Connector's config file
• 08-13 18:20:56 181a7c6006 Should ignore all Exception from close
• 08-14 09:32:45 3de21b893d Add limit pushdown to Information Schema Connector
• 08-14 22:38:36 e02a12c78e Add Google Sheets connector documentation
• 08-15 10:51:23 f82073c5da Fix duplicate field IDs when creating Iceberg tables
• 08-15 15:41:18 efc2bc1749 Improve error reporting for Sheets connector
• 08-15 21:46:13 67443c1219 Move product tests defaults to presto-product-tests
• 08-16 10:05:22 d463bb0750 Fix warnings in PrestoS3FileSystem
• 08-16 10:09:14 d1306b5612 Add hive.s3 option hive.s3.signer-class
• 08-16 15:43:46 2718f1a9cb Add last_day_of_month function
• 08-16 16:05:25 ab0382cdd9 Code cleanup in utility classes
• 08-16 16:05:25 fb5c165a9d Remove unnecessary lenient treatment of null value

• 08-16 16:11:34 8c65b0af99 Add support for requester pays bucket in PrestoS3FileSystem

  Grzegorz Kokosiński

• [x] all checked

• 08-04 11:38:03 8e3d74d342 Fix a small typo in presto-benchto-benchmarks/README.md
• 08-05 03:52:49 66632e239d Minor fixes around aggregation execution
• 08-05 06:48:27 43c25ec651 Extract SecurityContext classes
• 08-05 06:48:27 642fda232b Cleanups in TestFileBasedAccessControl
• 08-06 00:30:33 1e4f1557fc Allow connector to override min schedule split batch size
• 08-07 00:40:44 1998d6068b Fix formatting
• 08-07 00:42:15 4ecdfc04a6 bump snappy version to 1.1.7.1
• 08-07 00:42:15 962d3c4f22 bump kafka version to 1.1.1
• 08-09 01:36:30 92c8ca11bd Add run_presto_cli_on_docker.sh and run_tempto_on_docker.sh
• 08-12 02:02:48 087f1b1d31 Wrap extra credentials on top of existing CredentialProvider
• 08-12 02:02:48 32bd78d7eb Add credential provider for JDBC connectors
• 08-12 04:06:06 0d02779b01 Log JdbcClient invocations
• 08-12 04:06:06 8d11ca844e Move LoggingInvocationHandler to presto-plugin-toolkit
• 08-12 04:23:34 453bf6f084 Avoid nested try-with-resources
• 08-12 12:37:43 a911bba92c Adding google sheets plugin
• 08-13 06:24:39 ab127b817f Fix wrong value in gsheets.properties
• 08-16 23:03:55 8599c9b29f Rename totalPlanningTime to planningTime
• 08-16 23:03:55 8a9bcf0a98 Add analysis time to web ui query detail page
• 08-16 23:03:55 9df4d41d03 Collect query semantic analysis query statistic
• 08-16 23:03:55 b367996af8 Remove obvious comments
• 08-16 23:03:55 d258744b3a Remove analysisTime and distributedPlanningTime

• 08-16 23:03:55 fb7f5bd933 Add planning time to QuerySystemTable

  Karol Sobczak

• [ ] all checked

• 08-02 01:04:55 88eb7b085d Upgrading aws-sdk version to current (1.11.602)
• 08-02 01:39:59 2e9b62dc1f Do not unwrap LazyBlocks in PageUtils#recordMaterializedBytes
• 08-02 01:39:59 de826842fc Add Block#isLoaded method
• 08-06 02:39:18 51131ff1da Simplify RecordPageSource
• 08-06 02:39:18 78f9a89dce Make TPCH connector return lazy pages
• 08-06 02:39:18 85a658f941 Do not use lineitem table in semi join work processor queries
• 08-06 02:39:18 e824116cdf Ensure that work processor queries process less input data
• 08-06 02:39:18 e966976422 Test TopN work processor query with filter condition
• 08-07 02:26:23 100197b2a6 Replace deprecated DistributedQueryRunner constructor
• 08-07 02:26:23 ad99ec94dd Adding memory related session properties
• 08-09 15:29:02 4a9476d4f4 Make TestTable name really unique
• 08-09 15:29:28 bc5590c233 Make Kerberos config properties consistent
• 08-13 00:51:56 570d8970cb Remove unused method ValueInputStream#getCheckpointType

• 08-14 15:24:22 94e08b8877 Make PrestoS3InputStream#read comply with read method contract on EOF

  Martin Traverso

• [ ] all checked

• 08-13 14:55:55 3583d8eef2 Remove unused variable
• 08-13 14:55:55 67bb042005 Remove redundant hash computation
• 08-13 14:55:55 b5d5540d6d Add a PlanMatcher for matching identity projections
• 08-13 16:45:41 a7c351e6f4 Report offending array subscript when out of bounds
• 08-13 16:49:08 ce69ffff3b Fetch SearchHits scroll by scroll as needed than all at once in Elasticsearch
• 08-15 10:00:07 93412b3b79 Remove redundant call
• 08-15 10:00:07 96b9a52b2b Disable product tests on HDP 3.1
• 08-15 10:00:07 a99d301a14 Remove redundant exclusion in test
• 08-15 10:00:07 ec18f1d8cb Code cleanup
• 08-15 10:00:07 f8b79c9b96 Remove unused variable in test

• 08-16 15:43:25 1382c350c4 Remove unused field

  Piotr Findeisen

• [x] all checked

• 08-02 01:38:26 16c33b2557 Update docker images to 16
• 08-02 01:38:26 2166560fce Run product tests also on HDP 3.1 on "branch builds"
• 08-02 01:38:26 f90708111f Open more ports in test environment
• 08-02 12:34:48 03c1e9dfea Fix typo in docs
• 08-03 13:31:02 7c6c31610a Add "Asia/Qostanay" time zone
• 08-04 12:25:15 24a40be118 Pass table handles to beginQuery
• 08-05 08:06:50 28bd075e9e Distribute product tests more evenly on Travis

• 08-06 08:32:13 fdf498a19c Add support for Array of timestamps in PostgreSQL

  Raghav Sethi

• [ ] all checked

• 08-13 17:15:04 2165aee26f Implement periodic cpu usage tracking in resource groups
• 08-13 17:15:04 4f7933973c Enable resource updates in MockManagedQueryExecution

[sopel39]

----

* Improve performance of S3 Object listing. (:issue: `1232`)

https://github.com/prestosql/presto/pull/1232

[sopel39]

---

Add `Block#isLoaded` method. (:issue: `1216`)

https://github.com/prestosql/presto/pull/1216

[findepi]

* Add support for ``Asia/Qostanay`` time zone. (:issue:`1221`)

https://github.com/prestosql/presto/pull/1221

[kokosing]

SPI Changes
-------------
* Introduce ``SecurityContext`` group of classes to store security related information that is passed to plugins. (:issue:`171`)

171

[kokosing]

SPI Changes
-------------
* Allow connector to override minimal schedule split batch size. (:issue:`1251`)

#1251

[kokosing]

Kafka Connector Changes
--------------------------
 * Update Kafka client library to ``1.1.1``. (:issue:`1112`)

1112

[sopel39]

-------

* Add session properties for `query_max_memory_per_node` and
  `query_max_total_memory_per_node`. (:issue:`1212`)

https://github.com/prestosql/presto/pull/1212

[dain]

-------

* Fix ``DISTINCT FROM`` for ``UUID`` type. (:issue:`1180`)

1180

[sopel39]

-------

* Rename Kerberos properties to make them consistent with other
  HTTP server properties. (:issue:`1270`)

https://github.com/prestosql/presto/pull/1270

[kokosing]

Jdbc-based connectors changes
---------------------------------
* Add credential provider for JDBC connectors. Thanks to that user can pass RDBMS credentials in separate file, by using ``credential-provider.type=FILE`` and ``connection-credential-file=credentials.properties``. (:issue:`1124`)

1124

[kokosing]

Jdbc-based connectors changes
-----------------------------
- Allow logging all calls to ``JdbcClient``. This can be enabled by turning on ``DEBUG`` logging for ``io.prestosql.plugin.jdbc.JdbcClient``. (:issue:`1274`)

1274)

[kokosing]

General Changes
-----------------
- Introduce Google Sheets Connector. (:issue:`1030`)

#1030

[phd3]

-----------------
- Add periodic cpu usage tracking in resource groups (:issue: `1128`)

1128

[dain]

Hive Changes
------------

* Fix query failure due to missing credentials while writing empty bucket files. (:issue: `1298`)

1298 (fix is in PR #1303)

[electrum]

Hive Changes
---

* Improve performance when reading data from GCS. (:issue:`1200`)

1200

[electrum]

General Changes
---

* Improve performance for queries over ``information_schema`` that have a limit. (:issue:`999`)

999

[electrum]

---

* Add :func:`last_day_of_month` function. (:issue:`1295`)

1295

[electrum]

Hive Changes
---

* Add support for reading data from S3 Requester Pays buckets. This can be enabled
  using the ``hive.s3.requester-pays.enabled`` configuration property. (:issue:`1241`)

1241

[kokosing]

General Changes
------------------
 - Remove ``distributed_planning_time_ms`` column from ``system.runtime.queries``. (:issue:`1084`)
 - Change the semantics of ``analysis_time_ms`` column from ``system.runtime.queries``. Previously it was showing the time spent on query planning, now it  shows time spent on semantic analysis. of query. (:issue:`1084`)
 -  Add ``planning_time_ms `` column to ``system.runtime.queries`` that shows time spent on query planning. (:issue:`1084`)

1084

[martint]

* Improve query performance when ``optimize_hash_generation`` is enabled. (:issue:`1071`)

1071

[raghavsethi]

I would recommend amending https://github.com/prestosql/presto/issues/1238#issuecomment-521727316 to the following to better explain the user impact.

```rst

• Change query CPU tracking for resource groups to update periodically while
  the query is running. Previously, CPU usage would only update at query
  completion. This improves resource management fairness when using
  CPU-limited resource groups. (:issue: 1128)

[kokosing]

General Changes
------------------
- Allow to display details of query that is in the queue or during semantic analysis. Query details are accessible from the web ui or ``system.runtime.queries``. (:issue: `1079`)
- Allow to cancel query that is in the queue or during semantic analysis. Query can be cancelled from the web ui or using builtin procedure ``system.runtime.kill_query``.(:issue: `1079`)

1079

[findepi]

PostgreSQL Connector Changes
---
* Add support for PostgreSQL ``timestamp[]`` type. (:issue:`1023`, :issue:`1262`)

https://github.com/prestosql/presto/pull/1023, https://github.com/prestosql/presto/pull/1262

[kokosing]

Hive Connector Changes
-----
 - Support ``INSERT INTO`` bucketed unpartitioned Hive table. (:issue:`1127`)

1127

[dain]

-------

* Fix ``DISTINCT FROM`` for ``UUID`` type. (:issue:`1291`)

1291