Drasil: Move Output Data Constraints to Properties of Correct Solution

[samm82]

As mentioned below, the Output Data Constraints table should be moved to the Properties of Correct Solution section in all examples.

• [x] GamePhysics
• [x] GlassBR
• [x] NoPCM
• [x] Projectile
• [x] SSP
• [x] SWHS

[smiths]

Good! Thank you @samm82.

[samm82]

I'm assuming that the automatically generated intro to Data Constraints should be split and the relevant info be moved to properties of correct solution?

[samm82]

A potential side effect of this as well might be code generation. I'm not sure if we have a module for checking output constraints, but I'm pretty sure we get the input constraints from this table for input verification in the code. Unless it is enforced that the output constraints will appear in this section, it could potentially lead to issues. I'm not sure if @bmaclach would want to weigh in here.

[bmaclach]

We don't currently generate any modules for checking output constraints. The input constraints come from the _constraints field of the SystemInformation, which should be independent from the input constraints table. So I don't expect any issues with code generation.

[samm82]

Also, I just realized that output verification literally makes sure that the "properties of a correct solution" are true, so it wouldn't matter, as all information captured here should be checked. 😂 Whoops

[smiths]

Yes, @samm82, the introduction text should be split so that the output data constraints are introduced separately, in the properties of a correct solution section. As @bmaclach already said, and I believe you agreed, moving the output constraints table in the SRS will not impact our eventual ability to verify the output properties are satisfied.

[samm82]

@smiths How much of this change should be enforced? Should the Data Constraints section _have_ to take a table of input constraints, and the Properties of Correct Solution _have_ to take a table of output constraints? In order to split the intro, I have to edit docLang, so I might as well do it right the first time. 😂

[oluowoj]

We don't currently generate any modules for checking output constraints. The input constraints come from the _constraints field of the SystemInformation, which should be independent from the input constraints table. So I don't expect any issues with code generation.

@bmaclach does this mean that we have the input constraints in 2 different places( i mean the values)
@smiths so do we mean that the properties of a correct solution only depends on the output constraints? I was just reviewing this since I have an issue log related to this area for gamePhy. Just wanted to clarify.

[samm82]

@oluowoj SWHS is the only example with more properties of a correct solution than just the output constraints, but as @bmaclach mentioned, we don't generate code for output verification, so it isn't used.

[bmaclach]

@oluowoj There should be a single source of input data constraints, which is used to build the table and the SystemInformation. Looking at the examples, it looks like this is the case for all of them except GlassBR. I'll create an issue to fix that for GlassBR -- thanks for pointing that out.

[samm82]

Similar to my comment on a previous commit, I'm thinking this constraint in SSP should be removed as well, correct? @smiths

Also @oluowoj, I'm just going to remove the unnecessary constraints on this branch, so you don't need to worry about it.

[oluowoj]

@oluowoj SWHS is the only example with more properties of a correct solution than just the output constraints, but as @bmaclach mentioned, we don't generate code for output verification, so it isn't used.

@samm82 , gamePhy has other properties in the manual version, that I need to update in Drasil, so just wanted to be sure the design of that section will allow other properties and not just the constraints, unless we decide otherwise.

[samm82]

@oluowoj Ohhh, I see. This will be made more clear when @smiths decides how standardized the Properties of Correct Solution should be, but as of now on my branch, we can still have extra properties, so there shouldn't be an issue.

[smiths]

I'm not sure I understand the different questions that have come up. In general the properties of a correct solution are relations on the outputs. They definitely will need to take forms more complicated than bounding the output values. As an example: a property of a correct solution could be that as x increases, y will always increase. Conservation of energy, mass and momentum are also good candidates. We want to keep this section general. Giving simple bounds will be fine in many cases, but it won't be the full story.

When these are eventually incorporated into the code generator, it will be as test cases. The test cases will verify that the needed properties of the solution hold.

For those interested, properties of a correct solution are related to metamorphic testing.

[samm82]

Right now, docLang assumes that the first table passed to the Data Constraints section is the inputs, and the second (if present) is the outputs. It also assumes that the Output table will be the first item in the Properties of Correct Solution. Obviously, these are undesirable; I just made these assumptions so that I could at least move the table to the correct section and get stable updated. I'm now going back and fixing the backend. I think I narrowed it down to the following questions @smiths:

1) From this point forward, will the Output Data Constraints table no longer be passed to the Data Constraints section at all?
2) Will the Input Data Constraints table always be passed to the Data Constraints section?
3) Will the Output Data Constraints table always be passed to the Properties of Correct Solution section?

[smiths]

@samm82, I'll answer your questions below.

1. From this point forward, will the Output Data Constraints table no longer be passed to the Data Constraints section at all?

Right. The Data Constraints section is about input data constraints. (We might even rename this section in the future.)

1. Will the Input Data Constraints table always be passed to the Data Constraints section?

If there are inputs (which there always should be), then yes, the input data constraints table should be passed to the Data Constraints section. I believe that we have input data constraints for all of the examples? If there is a case where all of the constraints are trivial (-inf to inf), we might decide to not display the table.

If it is going to cause headaches, we can make passing the input data constraints optional for the moment. In the long run, the user should explicitly say something, even if it is to say that they don't have any meaningful constraints.

1. Will the Output Data Constraints table always be passed to the Properties of Correct Solution section?

As above, there should usually be output constraints, but there might not always be meaningful constraints. We can make this table optional for now. When our document compiler is really working well, the user could get a warning when they don't supply constraints?

[samm82]

Follow-up: do you see anything else being added to the Data Constraints section (ie. should it have the capability to take in more information in docLang)? @smiths

[smiths]

The data constraints section seems fairly stable. I don't currently anticipate anything else being added.