Presto: Support for "skip.header.line.count" property for HIVE tables
We have text tables with skip.header.line.count property.
damiencarol
All 24 comments
I hit this as well.. Refreshing that it's an issue.
Downchuck
on 27 Jan 2016
This is still an issue...
jakovm
on 30 Mar 2016
skip.header.line.count is available from metastore. Record cursor will have to skip lines according to that property. Some changes in BackgroundHiveSplitLoader and ColumnarTextHiveRecordCursor will be necessary.
maciejgrzybek
on 29 Jul 2016
No update on this issue ?
pletelli
on 16 Mar 2017
I would expect this to work for reading and writing to existing tables. I don't think there is a way to set this property when creating new tables.
dain
on 16 Mar 2017
馃憤 plus one for this issue
d18s
on 21 Mar 2017
If this is still an issue, please reopen with specifics on how to reproduce the problem.
dain
on 21 Mar 2017
This is still an issue. Hive understands the skip.header.line property and skips header while reading. But presto displays the header record on querying the same table.
Example to reproduce the error:
Step 1: create a csv file with 2 columns including header record (having inserted few records),
Step 2: Create table in hive with following property TBLPROPERTIES("skip.header.line.count"="1");
Step3 : query (select *) the table in hive --> Does not show header, furthermore as an added test, using the value(column header) in where clause produces no rows
Step 4: Query (select *) the table in presto --> Includes Header, furthermore as an added test,using the value(column header) in where clause returns the header record.
rupesh1183
on 30 Mar 2017
facing the same issue. running on AWS EMR with
Hive = Hive 2.1.0
Presto = Presto 0.157.1
chamal-sapumohotti
on 23 May 2017
Is this issue solved since I am also encountering the same issue.
raja-sudhan
on 10 Oct 2017
We are experiencing the same issue, @dain is @rupesh1183 's specifics enough to reproduce the problem on your end?
LouisKottmann
on 17 Oct 2017
Facing the same issue. Any progress since Oct 22, 2014 ?
Micka33
on 17 Oct 2017
any other way around this?
minion96
on 26 Oct 2017
Still an issue.
gseva
on 3 Nov 2017
same issue here
dylancis
on 22 Nov 2017
yeah, same issue here...
robmurtagh
on 8 Dec 2017
This also touches the behaviour when skip.footer.line.count is used
kokosing
on 3 Apr 2018
All - A related update : AWS has fixed this issue in 'Athena' which is AWS's version/product based on Presto.
rupeshmalladi
on 3 Apr 2018
@rupeshmalladi any plans for AWS Athena to contribute this back?
findepi
on 3 Apr 2018
@findepi Not that I am aware of!
rupeshmalladi
on 5 Apr 2018
Fixed with #10323
kokosing
on 13 Apr 2018
Presto is still ignoring skip.header.line.count on latest cluster deployment from AWS (5.13) Presto 0.194 with Hadoop 2.8.3 HDFS and Hive 2.3.2
The "closing fix #10323" doesn't apply to this ticket. The linked ticket was closed for a reason that had nothing to do with Presto.
cvandeve
on 21 May 2018
@cvandeve the fix is available for since Presto 0.199 (https://prestodb.io/docs/current/release/release-0.199.html#hive-changes), while latest release is 0.201.
findepi
on 21 May 2018
Thanks @findepi
cvandeve
on 21 May 2018
Related issues
haozhun
路
4Comments
ghost
路
4Comments
aminalaee
路
3Comments
sopel39
路
3Comments
talnicolas
路
3Comments
Most helpful comment
This is still an issue. Hive understands the skip.header.line property and skips header while reading. But presto displays the header record on querying the same table.
Example to reproduce the error:
Step 1: create a csv file with 2 columns including header record (having inserted few records),
Step 2: Create table in hive with following property TBLPROPERTIES("skip.header.line.count"="1");
Step3 : query (select *) the table in hive --> Does not show header, furthermore as an added test, using the value(column header) in where clause produces no rows
Step 4: Query (select *) the table in presto --> Includes Header, furthermore as an added test,using the value(column header) in where clause returns the header record.