E3sm: ERROR: timestep_make_parameters_consistent: divisibility error with ne16 F case

[ndkeen]

Using master, I see an error with ne16 cases, such as SMS_D_Ln5.ne16_ne16.FC5AV1C-L

202:  ERROR: timestep_make_parameters_consistent: divisibility error

[brhillman]

I think this is a case of trying to use a resolution that's not actively supported. Setting timesteps and hyperviscosity coefficients has changed, and what probably happened is that nobody bothered to update the settings for ne16 since nobody uses ne16. Since ne16 isn't tested, I'd probably advocate for removing the default configuration for it so nobody makes the mistake of trying to use it. The alternative is to add a test to make sure it continues to work, which I don't think many folks will have much interest in.

[ambrad]

I agree with Ben. Here are some details:

The full error message (adding some newlines to avoid horizontal scroll here) is:

nsplit was computed as  5.0000E-01
based on dtime    1800,
tstep  6.0000E+02,
and dt_max_factor = max(dt_remap_factor, dt_tracer_factor) = max( 2, 6) = 6,
which is outside the divisibility tolerance.
Set tstep, dt_remap_factor, and dt_tracer_factor so that tstep and dt_max_factor*tstep divide dtime.
 ERROR: timestep_make_parameters_consistent: divisibility error

Thus, you can try setting

dt_remap_factor = 3 ! or 1
dt_tracer_factor = 3
hypervis_subcycle_q = 3

in user_nl_eam and see if that gets the case running. If we want to make this a supported case, I suggest we switch the resolution to pg2. I can imagine interest in a well-running ne16pg2 case. (As Ben wrote, I think ne16np4 won't be of interest, though.)

[PeterCaldwell]

Yeah, I think it would be useful to have an ne16 case for convergence tests even if we don't use that config often.

[ambrad]

If we are going to, I suggest we consult with others to get good time stepping values. For example, it would be great if we could run

dtime = 3600 ! instead of 1800
dt_remap_factor = 2 ! default
dt_tracer_factor = 6 ! default
hypervis_subcycle_q = 6 ! default

That would make ne16pg2 almost 8x faster than ne30pg2. I'm guessing we'd want to set cld_macmic_num_steps to twice what it is for dtime 1800 (so I think 12 instead of 6, but I'm not positive), and there might be other parameters to modify.

[brhillman]

In that case, I'd strongly advocate for adding a corresponding test to E3SM integration along with the ne16pg2 config. It can be just a 1-timestep SMS test, but if it's not tested it will continue to break.

[ndkeen]

fyi, both of those suggested user_nl_eam works, but the second one is very slow and writes many of these:

132:  WARNING:CAAR: theta<   100.000000000000       applying limiter
132:  WARNING:CAAR: theta<   100.000000000000       applying limiter
132:  k,vtheta(k):           72   99.9758045458320
132:  k,vtheta(k):           72   99.9758045458320

[mt5555]

by "the second one", do you mean the one that sets dtime?

I think dtime is ignored in user_nl_eam, and instead you need:

./xmlchange ATM_NCPL=24

[mt5555]

"./xmlchange ATM_NCPL=24" looks good - I ran a 5 day simulation and got no warnings with all default dycore settings.

But someone on the atmosphere physics side should decide if the 2 degree model should use a 1h or 30min physics step. If it should stick with 30min, then my recommendation would be "se_tstep=300" and leave everything else as default.

default set here: (confusingly, it's setting dtime, even though dtime cant go in user_nl_eam):
https://github.com/E3SM-Project/E3SM/blob/master/components/eam/bld/namelist_files/namelist_defaults_eam.xml