Ifcopenshell: Potential development of I FC selectors / shorthand IFC query or filtering notation

[Moult]

This bug is to track to potential usefulness in developing an IFC shorthand notation. I'll copy the original thread below first.

[Moult]

Good morning Thomas!

For the paper, as well to allow people to easily write unit tests, I have come
up with the concept of an IFC selector. Here's a first draft of it - it would be
great to know your thoughts.

This should be able to allow people to ask complex natural statements like:

• Ensure that all load bearing walls on the first floor have a fire rating of
  2HR or 3HR.

# An IFC selector may be used in a Gherkin statement, with flexibility for
# singular and plural English (or any other language, if implemented). For
# example:
#
# Example: Then the {selector} element is foo
# Example: Then all {selector} elements are foo
#
# A concise and readable method of selecting single and multiple elements in the
# IFC tree is required to maximise flexibility. Inspiration is taken from CSS
# selectors, which are able to filter out elements from a DOM tree.
#
# Selectors can be combined with the Gherkin syntax (e.g. its tabular feature)
# to test complex values, callbacks, or even custom logic for spatial
# calculations.
#
# IFC selector syntax is defined as follows:
#
# # Specify a GlobalId
# Syntax: #[0-3]{1}.{21}
# Example: The #2HybKXRQ9BCRhqdw3O94qv element is foo
#
# # Specify an IFC class
# Example: The .IfcWall elements are foo
#
# Specify an attribute that should exist
# Example: The .IfcWall[Tag]
#
# Specify multiple attributes
# Example: The .IfcWall[Tag][ObjectType]
#
# Specify an attribute value
# Example: .IfcWall[PredefinedType="PARTITIONING"] are foo # Equality test
# Example: .IfcWall[OwnerHistory.ChangeAction="MODIFIED"] # Deep attributes
# Example: .IfcWall[Name^="foo"] # Starts with
# Example: .IfcWall[Name$="foo"] # Ends with
# Example: .IfcWall[Name*="foo"] # Contains
# Example: .IfcWall[Name="foo"i] # Case insenstive
# Example: .IfcWall[Name="foo"s] # Case senstitive
# Example: .IfcWall[Name="[Ff]oo"r] # Regex matching
#
# # Specify a pset value
# Example: The .IfcWall[Pset_WallCommon.LoadBearing="TRUE"]
#
# # Specify an AND relationship
# Example: .IfcWall[Name="foo",Tag="bar"]
#
# # Specify an OR relationship
# Example: .IfcWall[Name="foo"|Tag="bar"]
#
# # Specify logical groupings
# Example: .IfcWall[(Name="foo",Tag="bar")|Name="baz"]
#
# # Specify a qto value with numeric comparison operators
# Example: The .IfcWall[Qto_WallBaseQuantities.Height>="2000"]
#
# # Specify a spatial containment relationship, assuming that the GlobalId is a
# # spatial structure element
# Example: The #2HybKXRQ9BCRhqdw3O94qA>.IfcWall elements are foo
#
# # Specify a type product relationship, assuming that the GlobalId is a
# # product type
# Example: The #2HybKXRQ9BCRhqdw3O94qB>.IfcWall elements are foo
#
# Specify multiple (an AND relationship)
# Example: The #2HybKXRQ9BCRhqdw3O94qv,IfcWall elements are foo

--
Dion Moult

[Moult]

Reply from @aothms below:

I in fact tried to do something similar several years ago, but also enable to select non-building element nodes. So you could start from a rooted entity instance and then use / to follow down the path of attributes. I think this is something to consider: do you want to make statements always in the context of one or more building elements or do you want to make statements about entity instances in general? For example IfcWallStandardCase/ObjectPlacement/RelativePlacement/Position/Z

Instead of CSS you could also look at XPATH/XQUERY for inspiration. The class and id matching from CSS matches nicely, but I like the XPATH syntax much better. The dot is very easy to overlook for example and the > operator for children easy to confuse with greater than predicate. I guess you could also consider the IFC class similar to the HTML tag name, so leave out the dot entirely.

I must say though that conceptually this deviates from the ambition to have human readable rule sets.

Instead of
.IfcWall[PredefinedType="PARTITIONING"] are foo
Shouldn't we be able to say?
All walls with PredefinedType equal to PARTITIONING are foo
Or even better
All partitioning walls are foo
In the last case we can start special casing ObjectType/PredefinedType attributes to become more granular object types. But the great thing is that we not only make it human readable, but also more IFC agnostic because most people are not familiar with IFC terminology and attribute names.

[Moult]

Reply from @Moult below:

Great feedback!

On Sat, Jan 04, 2020 at 10:50:51AM +0100, Thomas Krijnen wrote:

I in fact tried to do something similar several years ago, but also
enable to select non-building element nodes. So you could start from a
rooted entity instance and then use / to follow down the path of
attributes. I think this is something to consider: do you want to make
statements always in the context of one or more building elements or do
you want to make statements about entity instances in general? For
example
IfcWallStandardCase/ObjectPlacement/RelativePlacement/Position/Z

The GlobalId selector (starting with #) covers all rooted elements. The class
selector (currently starting with .), can select any entity, not just building
elements. Combined I think these cover the entire spec, as well as attribute
traversal? In your xpath example, the first token refers to an ifc entity,
whereas the subsequent tokens refer to attributes, whereas in the CSS example it
distinguishes attributes from entities.

Instead of CSS you could also look at XPATH/XQUERY for inspiration. The
class and id matching from CSS matches nicely, but I like the XPATH
syntax much better. The dot is very easy to overlook for example and
the > operator for children easy to confuse with greater than
predicate.

I did consider xpath, but I wanted to experiment with CSS because then the same
CSS could apply in a stylesheet that affects an SVG documentation export.
Perhaps tomorrow I'll have a stab at creating an xpath variant and compare which
looks prettier :) Then again, xpath has lots of benefits too, and potentially
could just run on an IFCXML file...

I guess you could also consider the IFC class similar to
the HTML tag name, so leave out the dot entirely.

Absolutely. Probably just remove it :)

I must say though that conceptually this deviates from the ambition to
have human readable rule sets.
Instead of
.IfcWall[PredefinedType="PARTITIONING"] are foo
Shouldn't we be able to say?
All walls with PredefinedType equal to PARTITIONING are foo
Or even better
All partitioning walls are foo
In the last case we can start special casing ObjectType/PredefinedType
attributes to become more granular object types. But the great thing is
that we not only make it human readable, but also more IFC agnostic
because most people are not familiar with IFC terminology and attribute
names.

I _think_, not sure until I try more in practice, that both are required. I
think that the human language ones would be preferred, since they communicate
natural meaning much more strongly. However, I believe the shorthand should
also still exist to allow people to:

1. Quickly prototype a unit test
2. Efficiently notate a "filter" in a query to the BIM model, something like
   "please retrieve all elements that fit this criteria", useful for extracting
   out items of interest, doing partial imports and exports, analysis on portions
   of the model, etc. Currently, there is no shorthand notation for this. Multiple
   utilities could parse this syntax, such as clash detection, IFC diffs, ifc
   imports, ifc exports, ifc model viewers, ifc drawings...

I'm going to cross-post this on the issue tracker so that others can see the
discussion :)

--
Dion Moult

[Moult]

For those interested, you can see a suite of unit tests created for TfNSW here: https://thinkmoult.com/tmp/tfnsw-bimtester.zip - it doesn't use any IFC selectors, but it demonstrates the ease of writing these unit tests in plain English (or other languages, if implemented).

[berndhahnebach]

very much interested here, but I do not really understand how the tools work you posted a link too. What is bimtester.exe? Never heard of this software?

bernd

[Moult]

@berndhahnebach IFC BimTester is from the code here. It is a very thin wrapper around a unit test runner library called Behave. Under the hood, it is simply a unit tests which uses the IfcOpenShell library.

You might be familiar with unit tests written with things like JUnit, PyUnit, PHPUnit, and so on. Those tests are written with code. There is an alternative style of tests, based around behaviours and scenarios (BDD - Behaviour Driven Development) which are written in a parsed English (or any natural language) which are appropriate in different scenarios (integration testing, unit testing, whole stack testing). You might have heard of tools like Cucumber, Behat, or Behave.

It is my suggestion that an MVD is essentially the same as a unit test (i.e. it asks the question "does this data have what I need?"). So instead of relying on mvdXML which has limitations on what it can describe in validating an IFC file, and is verbose and hard to write, I've developed a vocabulary of tests of simple English sentences you can write down to check an IFC file.

Now, our contracts are accompanied with a test suite - if our consultants deliver IFCs that do not pass our test suite, contractually they have not met our requirements. This is much more useful for us, since we have an unambiguous standard test runner, and also more useful to them, since a sentence "we want all IfcMaterial names to follow the format AA-bbbb-NNNNN" is much more meaningful than "are you sure you delivered the 200 entities exactly correctly as per IFC CV 2.0?", which has no bearing on the quality of the data within.

So far, at work, we have written approximately 50,000 tests (Blender has tools which can help generate some) which validate various aspects of an IFC file. Through the experience of writing them and training our consultants to fix their IFC data to meet these requirements, we are coming up with a vocabulary of tests - see here for some of the ones I have considered very useful. Tests are run on a post-receive hook on a Git repo which is our BIM server, and JUnit XML is parsed by Jenkins, and we also provide a HTML report so that we can give reports to consultants.

We also give the BIMTester tool to consultants so that they can write their own tests, run their own tests, check exactly under the hood what we are testing (most firms seem to have one guy who understands Python), so that we don't waste time with a 1 week turnaround only to find the data still doesn't meet our expectations.

That's the background :) While writing tests, running diffs, clashes, and so on, I figured it might be beneficial to develop a form of IFC selector / shorthand, which all these tools could parse to filter a selection of IFC elements. Hence the bug :)

Hope it made sense.

[Moult]

A CSS style selector has now been implemented in IFCCSV. Marking as closed. I plan to also implement an XPath style selector, then over time see which one is better in which scenarios :)