Isis3: incomplete cisscal

[ascbot]

---

Author Name: Klaus-Michael Aye (Klaus-Michael Aye)

---

I find it highly problematic that the ISIS3 implementation of cisscal is incomplete compared to the original version of the IDL code, as described in the docstring for cisscal, especially as no instructions are provided to perform the missing steps.

This leads to the creation of products that will be named/identified/thought of the user as "calibrated products" while significant differences between the IDL and the ISIS3 versions can occur.
This misleading situation is worsened by the fact that ISIS3 calls the procedure by the same name as the original calibration procedure.

I attach example screenshots, where I compare the imported cube from N1875229393 with the calibrated data product as provided by the PDS Rings node.

Note the large amount of pixels that are marked as "Hrs", the meaning of which is not described in the documentation for ISIS3' cisscal.

For convenience I also attach the cube with the Hrs "holes".

"image_with_holes.png" is a view export from qview showing the cube as converted from the raw image.

"from_raw.cub" is said cube file

"zoomed_qview.png" shows a zoom with qview into the Hrs holes.

"from_calibrated.png" is a screenshot from displaying the calibrated file from here:
https://pds-rings.seti.org/holdings/calibrated/COISS_2xxx/COISS_2113/data/1875228144_1875491141/N1875229393_1_CALIB.IMG, as read into Python with GDAL

[ascbot]

---

Original Redmine Comment
Author Name: Klaus-Michael Aye (Klaus-Michael Aye)
Original Date: 2018-08-17T17:27:33Z

---

After digging into this further, I have discovered that the ciss2isis importer incorrectly maps the value 255 in the original data to "High Saturation", even so the input data is 16 bit.

This is most likely caused by the line number 87 in the ciss2isis.cpp where it seems that the variable dataConversionType is queried uninitialized:

~~~ cpp
if(dataConversionType != "Table") { //Conversion Type is 12Bit or 8LSB, only save off overclocked pixels
validMax = 255;

~~~

As the variable is un-inititalized, it most likely is not equal "Table".
Let me know if you want a new issue for this, because this is unrelated to the problem that cisscal is incomplete compared to the original calibration procedure.

[ascbot]

---

Original Redmine Comment
Author Name: Makayla Shepherd (Makayla Shepherd)
Original Date: 2018-08-17T23:22:58Z

---

Please create a new ticket for the pixel mapping issue.

[michaelaye]

Is still still open, considering that the changelog for cisscal indicates:

Updated cisscal and documentation to better reflect the steps as of idl cisscal program version, 3.8

[jlaura]

@michaelaye Can you rephrase? I am not sure what you mean.

[jessemapel]

cisscal from the Cassini team is currently on version 4.0

ISIS cisscal is up to date with their version 3.9.1 after #3406

Here's the 4.0 change log:

New in version 4.0:

. Improved logic in option file reader and cisscal_cl.pro
. Changed formatting of bitweight files to ease parsing
. Added logic to handle bias/2-hz noise removal for TABLE images of various gain
states
. Improved CISSCAL_Log formatting
. Updates to polarized filter algorithm in make_polar_image.pro (see
make_polar_image_readme.txt).

It looks like we don't replicate the make_polar_image script because it's a separate calibration program. We also do not do the 2-hz noise removal because that's filtering and not calibration.

So the only difference is support for debiasing of some TABLE images.

[jessemapel]

The new code in cisscal is this block of code at line 67 of cassimg__debias.pro:

       if self.convtype eq 'TABLE' and self.gainstate lt 3 then begin
          If DebugFlag gt 0 then begin
             CISSCAL_Log,'  Table-encoded image with gain state < 3;'
             CISSCAL_Log,'    Using BIAS_STRIP_MEAN to remove bias and skipping 2 Hz removal...'
          ENDIF
          biasstrip = 1
       endif else begin
          biasstrip = 0
       endelse

previously it just set the biasstrip flag to 0.

[michaelaye]

I have to push back against the categorization of a standard noise removal of the CISS team as a separate issue.
While it is technically correct, the CISS team has considered this issue to be a standard calibration issue and not doing it on the ISIS side while it's done on the standard IDL calibration just creates confusion of producing different results.

[jessemapel]

@michaelaye That's reasonable to me. Looking at the revision history it looks like it was decided not to include that way back when it was first ported to ISIS

[blandoplanet]

Naively I agree with @michaelaye. It causes confusion to have ISIS cisscal not perform steps that are standard in the CISS team "calibration."

[jessemapel]

ISIS also does not do anti-bloom correction. I'd assume that's in the same category as the 2-hz correction.

[michaelaye]

yes, have never seen it. One of the initial reviewers of the CISS team, Robert West, pointed these differences out, but I guess there were no means to rectify them at the time.
I could work on these issues, but will take me time to get to it...

[jessemapel]

If the original team is pointing out the difference, we should probably port them over!

[jessemapel]

This is really 3 issues, see:

3491

3493

3494