Vscode-extension-for-zowe: Error searching data sets when result set contains migrated data sets

[MikeBauerCA]

When searching for data sets, if the result set contains a migrated data set I receive the following message:

Retrieving response from zowe.List
Error: z/OSMF REST API Error:
Rest API failure with HTTP(S) status 500
rc:       4
reason:   3
details: 
  - DMS2987 DATA SET CATALOGED TO CA DISK PSEUDO-VOLUME ARCIVE                                                              
  - DMS2987 CA DISK HAS ARCHIVED HLQ.MY.DATASET                                                                  
  -    
  - DMS2971 DO YOU WANT TO RESTORE THE DATA SET? (Y/N)
category: 2
message:  ServletDispatcher failed - received TSO Prompt when expecting TsoServletResponse

The same issue arises if I issue zowe files list ds "HLQ.*" -a but I do not receive an error when issuing the same command without the attributes flag.

[MikeBauerCA]

This issue may not be limited to CA Disk. An issue has also been recorded with DFSMShsm. See https://github.com/zowe/data-sets/issues/127 for additional details.

[Colin-Stone]

Testing locally which is without CA Disk I don't get the same response and indeed get all of the datasets returned. The returned information indicates that a dataset is in fact a migrated dataset which would enable the association of a revised object context and in turn control the availability of subsequent commands and display icon.

[Colin-Stone]

The list works nicely and provides information that a dataset is migrated. It's then a case of creating a migrated object, in this view shown with a "ghost-like" appearance. Options on the migrated type will include HRECALL but requires the appropriate additional code in the CLI https://github.com/zowe/zowe-cli/issues/524

[MatthewB19]

This is actually a well-known issue that happens whenever z/OSMF attempts to retrieve the attributes of an archived data set.

The issue was first reported in zowe/zowe-cli/issues/325 to be affecting the List.dataSet method when passing the "attributes" option.

Essentially, whenever the X-IBM-Attributes header is passed to the API call, z/OSMF uses ISPF to retrieve the attributes of any data sets matching the given dslevel. If a data set is archived, the attempt to retrieve those attributes causes a TSO recall prompt to be thrown, and thus the API call to fail.

For zowe/zowe-cli/issues/325, a clever workaround was implemented (zowe/zowe-cli/pull/489) based on the X-IBM-Max-Items header and the data set collation sequence returned by ISPF - so as to avoid attempting to retrieve attributes of unneeded data sets that could be archived and would cause the request to fail.

[MatthewB19]

In the meantime, it looks like IBM inserted a check into z/OSMF in order to determine whether a data set is archived before attempting to retrieve the attributes. I believe that was meant to solve this issue.

The problem is that it only checks for the MIGRAT pseudo-volser before attempting to retrieve the attributes. Historically, MIGRAT has been DFSMShsm's default pseudo-volser for re-cataloging archived (migrated) data sets.

So this only works for data sets that are cataloged using the MIGRAT pseudo-volser (DFSMShsm’s default pseudo-volser), but not for those using ARCIVE (CA Disk’s default pseudo-volser).

That's the reason why @Colin-Stone's test did not fail, whereas @MikeBauerCA's is getting a REST API error due to the prompt being thrown. So, at the core, this is a problem with z/OSMF - and not an issue with the VSCode extension itself.

[MatthewB19]

By the way, CA Disk also supports backing up/archiving data sets using MIGRAT.

There is a dedicated sysparm (RECATMIG) which, if enabled, will change the default pseudo-volser to MIGRAT without breaking any dependencies for the old ARCIVE data sets.

This means that, if a storage admin decides to enable RECATMIG for their installation, from that point forward CA Disk will start backing up/archiving data sets using MIGRAT instead of ARCIVE but any of the data sets that had been previously cataloged under ARCIVE will still be able to be managed and restored without having to manually re-catalog them to MIGRAT.