Drasil: Generate Input/Output Requirements from SystemInformation

[samm82]

As mentioned in the meeting this morning, the requirements for inputs and outputs should be generated automatically from the inputs and outputs contained in the SystemInformation for an example.

[smiths]

Yes, this is a good idea. As we discussed, we essentially have the same information in two places. Eventually it makes sense to me that the inputs and outputs aren't part of the System Information, but part of the requirements. This doesn't really matter though, as long as we have the information in Drasil; we can worry about the best way to say it later. Since the System Information already knows the inputs (for code generation purposes), we shouldn't have to repeat this information again in the requirements.

[samm82]

A potential issue I see with this is when we want to reference the input requirement, we will probably have to use a "dummy" reference, but I'll see if/when that comes up.

[samm82]

@smiths What should be the cutoff for if the list of inputs/outputs gets put in a list vs. a table? I think five quantities is still small enough to work in a list and any more should be in a table, but it's up to you.

[smiths]

@samm82, if it is easier, I'm fine if the lists always go into a table. If you decide to provide both options, then the cut-off between lists and a table should be an easily customizable number. This would be a parameter of variation that the user could determine. If you are looking for a default value for this parameter, yes, I think 5 is a good choice. (You wouldn't likely want to go higher than that.)

[samm82]

Always having a table would be easier (for now, at least) and would have cleaner code so I'll go that route. 😁 We can always add the extra functionality if we desire it later.

[smiths]

Sounds good. Yes, once the information is in Drasil, we can refine and improve the format and presentation, and flexibility.

[samm82]

I'm also thinking we should standardize a bit - should we use "parameters", "quantities", or "values" in the I/O requirements (and in the rest of them as well)? @smiths

[smiths]

@samm82, I don't understand your question. Are you talking about the text of the requirements? If it is the text of the requirement, I think we can say we are reading "input values". They aren't variables; they are values. Parameters to me implies a function interface. I don't really like "quantities" as a standard terminology.

[samm82]

There is great variation of the Requirements section when referring to values:

• "Output-Quantities: Output the quantities from Table:ReqOutputs." (SWHS)

- "Output-Values: Output s (from IM: messageIM) and d_offset (from IM: offsetIM)." (Projectile)

• "Read-and-Store: Read the inputs, shown in Table:ReqInputs, and store the data." (SSP)
• "Input-Initial-Quantities: Input the following quantities described in Table:ReqInputs, which define the tank parameters, material properties and initial conditions." (SWHS)
• "Input-Glass-Props: Input the quantities from Table:ReqInputs, which define the glass dimensions, type of glass, tolerable probability of failure, and the characteristics of the blast." (GlassBR)

I think that the generated requirements should be standardized to "Input-Values" and "Output-Values",
and the other requirements should follow this pattern as well (eg. "Verify-Input-Values") with the text of each one reflecting this. So the first one would be "Output-Values: Output the values from Table:ReqOutputs." @smiths

[smiths]

Yes, @samm82, I agree. This is a nice benefit of generating the requirements. 😄

[samm82]

Are the only outputs from InstanceModels, and if so, is this something that will always be true in the future? @bmaclach @Mornix @smiths The current output table has a column for Source, which would be super nice and easy to automate if _outputs :: [InstanceModel].

There are also some issues with some GlassBR Requirements:
1) >Output-Values-and-Known-Quantities: Output the input quantities from FR: Input-Glass-Props and the known quantities from FR: System-Set-Values-Following-Assumptions.

  I'm pretty sure that only the IMs are output in the code - should this requirement be removed?

2) > Check-Glass-Safety: If is-safePb ∧ is-safeLR (from TM: isSafeProb and TM: isSafeLoad), output the message "For the given input parameters, the glass is considered safe." If the condition is false, then output the message "For the given input parameters, the glass is NOT considered safe."

  Should this be changed into an IM so that the code actually does this? @smiths (This would be done in a separate branch.)

[bmaclach]

I don't think we can say that outputs will always correspond to instance models. I remember with Projectile how @smiths didn't want to include instance models related to checking the offset and returning a string. He wanted that as a requirement instead, but since code generation can't read from requirements yet we made instance models out of them as a temporary solution.

For your question 1., I can confirm that the generated code does not return the inputs like it says in the requirements. I doubt we want to lose that requirement, though -- probably being able to print inputs to the output file is something we want to be able to handle, we just haven't looked into that yet.

[smiths]

Yes, I agree with @bmaclach. We might eventually decide that requirements do not do anything but summarize what IMs we are using, but that definitely wasn't the original idea. The original idea was that the requirements would include "new" information, like what strings to output as messages to the user. I wouldn't want to be too hasty in changing this.

We should definitely keep the requirement to "output" the "inputs." Even though we don't do this currently, this requirement is "real." Some things in Drasil are just part of the case studies and rather arbitrary, but the output the inputs requirements in motivated by real world uses cases. Scientists need provenance data with their simulation results. If people aren't careful then they have their outputs, but they don't remember how they were generated. (I've had this happen to me!) You get a plot that you like, but then you don't know how to reproduce it. By having the inputs available, the user will know what inputs let to their outputs. This isn't the only way to do it, and it certainly isn't foolproof, since the code itself could change, but it is better than not reproducing the inputs.

[samm82]

Is it acceptable if the output table doesn't include where the outputs come from then? @smiths If not, I'm not sure how to automate the table.

[smiths]

@samm82, we need to say where the outputs come from. Otherwise, the reader has to deduce which equations they need to solve for the outputs. We need this traceability information to appear. It doesn't have to necessarily be in the table, if you can include it somewhere else? Your colleagues may also have an idea on how to get the information.

[samm82]

The issue is that the source information isn't contained within the _outputs field of SystemInformation, so there isn't a way to generate this table solely from the field. Any ideas @Mornix @bmaclach?

[smiths]

@samm82, it would be great to automate, and @Mornix and @bmaclach may have some great ideas, but why do we need full automation? The author of the Drasil documents currently provides the traceability information. We could continue to have the author provide this information until such a time as automation is feasible.

[samm82]

It was mentioned in the meeting last week that we'd want to automate the requirements for inputs and outputs, so I just assumed that the whole requirement should be generated, including the tables. Would we want the requirement to be generated and the table manually created?

[smiths]

As always, we want to automate the easy things, but not stress over those that are too difficult. The user still has an important role in Drasil. Can you automate most of the table, but have the traceability information provided by the user?

[samm82]

A possible workaround would be to have the outputs be of type [(QuantityDict, Sentence)], where the Sentence is the source of the information? Then the code generator would only use the QuantityDicts, and the table could be generated from the tuples, which is the way it is built right now. Would that be a bad idea? @smiths

[JacquesCarette]

This seems like a hack. What would be in that Sentence? Whatever it is, it should be attached to the Quantity itself, not tacked on like this.

[samm82]

That's I was thinking. The Sentence would be the reference to the source of the quantity.

Ideally we would use the the actual source (InstanceModel, TheoryModel, etc.) instead of the QuantityDict, but I think defining the list as (HasShortname r, Referable r) => [r] won't type check. I don't think there's a way currently to attach the source to the quantity.

What if we were to use the utility function mkQRTuple (defined below) when defining the outputs field? The _outputs field would still be of type [(QuantityDict, Sentence)], but we wouldn't be manually passing the source in. For example, a field could look like: _output = mkQRTuple iMods ++ mkQRTuple [outputDD]. This still feels a bit hacky, though.

mkQRTuple :: (Quantity i, MayHaveUnit i, HasShortName i, Referable i) => [i] -> [(QuantityDict, Sentence)]
mkQRTuple = map (\c -> (qw c, makeRef2S c))

[JacquesCarette]

It's a lot less hacky. But I think you should investigate the (HasShortName r, Referable r) => [r] solution more thoroughly. You might need

(Quantity i, MayHaveUnit i, HasShortName i, Referable i) => [i]

.

[samm82]

Even with this type signature, I don't think Haskell will let you put two things that are different types directly in the same list, even if they match all the constraints, so I don't think this will work for a new type for the output field.

[JacquesCarette]

Right, Haskell still won't let you do that. But I'd like to see where it breaks.

[samm82]

I'm not exactly sure what you mean by this. I changed the type signature of _outputs in SystemInformation, which meant I had to add constraints to the requirements builder.

Now, it's breaking at the examples, since the assumption right now is that the outputs are all QuantityDicts. It's pretty trivial to fix this now (since the outputs are [DataDefinition] for HGHC and [InstanceModel] for the rest), but in the future, where the outputs would have different types, we would have bigger issues.

[samm82]

The alternative would be to create a new type for outputs, but I'm not sure if we'd want to.

[JacquesCarette]

We already do a lot of 'casting' when building SystemInformation. In a way, we need to find the smallest data type we have that matches the constraint (Quantity i, MayHaveUnit i, HasShortName i, Referable i) and then we can cast to that in the examples themselves. So I'm not really seeing the problem?

[samm82]

The only data types that have these constraints are DataDefinition, GeneralDefinition, InstanceModel, and TheoryModel. I'm not sure which one would be the "smallest" or if we should even be casting between these.

[JacquesCarette]

We really need a proper 'theory' type that these 4 types specialize...

[samm82]

Should I start making one in another branch, or should more thought be given to it first? I'm not sure if you have any specific ideas about what it should have, and I feel that it would overlap with #1373 and #1601.

[JacquesCarette]

You mean a proper 'theory' type? I have lots of ideas for how to design that. It's tricky.

[samm82]

When trying to generate the input requirement for SSP, I realized that the first entry in the input table isn't in the inputs passed to SystemInformation. I think that this is related to #295, and for now, I am just going to leave the SSP requirements generated manually, so that I can hopefully get a PR before the end of the summer. 😊

[samm82]

GamePhysics has three issues:
1) There are two input requirements.
2) The first requirement isn't for inputs (currently, the assumption is that the generated input requirement should always be the first one in the list).
3) No actual quantities are given for the requirement, just descriptions.

These would be pretty arbitrary to resolve (merge the two requirements to one that will go first, and verify that the (currently populated) inputs field contains all the desired inputs), but I would want to wait for approval from @smiths and/or @oluowoj before proceeding.

[samm82]

For NoPCM, there is currently an issue where the outputs are passed in as inputs. This is done because NoPCM "generates code", and doing this is currently the only way to have a 'path' between the inputs and outputs. I'm guessing generating the input requirement would better be done after #1373, so that it will only contain the inputs (ie. be correct).
https://github.com/JacquesCarette/Drasil/blob/5fed6703273da02f4a11862b57456fb0d7bd68d6/code/drasil-example/Drasil/NoPCM/Body.hs#L184-L185

[smiths]

@samm82 for the game physics question (https://github.com/JacquesCarette/Drasil/issues/1697#issuecomment-523194263), I don't think there is anything you can do in the time available. The problem is that the game physics example is not complete. The input should also include the geometry of the rigid bodies, but there is no facility in the current SRS for representing geometry. This work will have to wait until the Game Physics example is properly fixed.

For NoPCM (https://github.com/JacquesCarette/Drasil/issues/1697#issuecomment-523195428), your suggestion seems reasonable to me.