pg_lake
One fix in Snowflake-Labs/pg_lake, maintained by Snowflake Labs.
Back to all open-source fixesAbout pg_lake
pg_lake is a PostgreSQL extension from Snowflake Labs that lets Postgres create, query, and manage Apache Iceberg tables and data-lake files directly. It plugs into the Postgres planner to push work down to data files and uses partition metadata to skip files that can’t contain matching rows.
My work on it
My work on pg_lake is in the extension’s C code, in the query-planning layer that decides which data files a query can safely skip. I found a type-handling bug in partition pruning by reading the code path and confirming it with a standalone simulation, then fixed it so queries on partitioned smallint columns return every matching row.
Correct partition pruning for smallint columns
The problem. For smallint columns partitioned with truncate, each file’s value range was computed as a 32-bit integer but compared using 16-bit operators. Range ends outside the smallint range wrapped around, so the planner skipped files that actually contained matching rows, and queries silently returned fewer results with no error.
The fix. Range ends for smallint columns are now kept within the valid smallint range, and the upper end is expressed as the last value in the range, so comparisons never see a wrapped value.
How I tested it. Added a regression test for values at both ends of the smallint range, and separately checked the new range logic against every possible smallint value.