Isis3: kaguyatc2isis cannot ingest data provided by Jaxa online archive

[ladoramkershner]

ISIS version(s) affected: x.y.z
I would assume this applies to all version, however, I am currently using isis3.9.1

Description

kaguyatc2isis was written using the label format of some data provided by a JAXA correspondent and stored locally some years ago. At that time, it was the only available Kaguya data. Since this data delivery, JAXA has created an online data archive to provide their data to the public (http://darts.isas.jaxa.jp/planet/pdap/selene/product_search.html). Additionally, it seems JAXA has rearranged the Kaguya TC label structure, causing kaguyatc2isis to fail with some keyword errors on data downloaded from the JAXA website.

How to reproduce

In /work/projects/kaguya/ladoramkershner/isis_ticket/ there are two folders
online/: contains two random images downloaded from the jaxa online data archive
local/: contains those same two images sourced from usgs asc's internal archive
running
kaguyatc2isis from=TC1W2B0_01_01682N300E0078.img to=TC1W2B0_01_01682N300E0078.cub
kaguyatc2isis from=TC1S2B0_01_05186N225E0040.img to=TC1S2B0_01_05186N225E0040.cub
works successfully on the local data, but fails on the online data with the error

**ERROR** Unable to read [DATA_SET_ID] from label file [TC1W2B0_01_01682N300E0078.img] in main.cpp at 36.
**ERROR** PVL Keyword [DATA_SET_ID] does not exist in [Object = Root] in file [TC1W2B0_01_01682N300E0078.img] in PvlContainer.cpp at 84.

Additional context

We are creating a pilot mosaic of the Apollo 15 landing site, our pipeline is currently dependent on the local data. We intend to do more work with this pipeline next year and will likely continue to use the local versions of the data. In principal we want for ISIS3 users to be able to work with the public (JAXA archive) version of the data as well. Ideally, our code could work on both.

I am not sure if this is a product or missions issue. I am selfishly tagging it as a product, because it may affect my product work.

[jessemapel]

I would classify this as an enhancement and not a bug. Also, do we want to maintain compatibility with the older data set?

[ladoramkershner]

I cannot speak for everyone, but I believe backwards compatibility is ideal.

[lwellerastro]

I agree with Lauren's assessment of backwards compatibility, if possible.

@thareUSGS thoughts (hope you don't mind me sharing):

While the old format is probably going to be harder to come by (from them), it might be out there for others. So backward compatibility would be recommended. However, if it is too difficult to maintain that compatibility, then at least a warning stating new versions are available and should be used.

[jessemapel]

Follow up questions. Does anyone know what has changed between the two or do we need to go in and compare SISs and labels.

[lwellerastro]

[DATA_SET_ID] for sure, however there are possibly other keywords. Lisa made the attempt to manually modify the keyword in the .lbl (I think) and then she got a new error. She documented a number of things. I get her notes uploaded if that helps. But we don't precisely know all of the differences.

[jessemapel]

I get her notes uploaded if that helps. But we don't precisely know all of the differences.

A partial list of differences would still be great. Thanks!

[lwellerastro]

Lisa's notes

KTC_compare_labels_12.9.19.docx

[foobarbecue]

Possibly related to https://github.com/USGS-Astrogeology/ISIS3/issues/3698 , a similar issue regarding new JAXA Hyabusa2 labels.

[lwellerastro]

Hi, I just tested the changes and it appears a DATA_SET_ID was left out - "SLN-L-TC-3-W-LEVEL2B0-V1.0".

Notice that there are two images under @ladoramkershner How To Reproduce section,

kaguyatc2isis from=TC1W2B0_01_01682N300E0078.img to=TC1W2B0_01_01682N300E0078.cub
kaguyatc2isis from=TC1S2B0_01_05186N225E0040.img to=TC1S2B0_01_05186N225E0040.cub

Running using the latest changes to kaguyatc2isis, TC1S2B0_01_05186N225E0040.lbl ingests successfully, however TC1S2B0_01_05186N225E0040.lbl fails with the following message:

Message = "Input file [TC1W2B0_01_01682N300E0078.lbl] does not appear to
           be a supported Kaguya Terrain Camera format. DATA_SET_ID is
           [SLN-L-TC-3-W-LEVEL2B0-V1.0]Valid formats include [TC_MAP,
           TCO_MAP, TC1_Level2B, SLN-L-TC-3-S-LEVEL2B0-V1.0,
           SLN-L-TC-5-MORNING-MAP-V4.0]"

I think this is just a simple oversight.

[jessemapel]

I'll work on getting this fixed. I'm working on RC2 with #3827 and will put this out then.

[jessemapel]

Fix kaguyatc2isis

[jessemapel]

Test W image can be ingested and properly spiceinits under 4.1.0_RC2