Powershell: Inheritable ScriptBlock attributes

[KirkMunro]

There are a number of attributes that can be used in ScriptBlocks that I believe should be inherited by any ScriptBlocks whose definitions are nested within the same scope. For example, DebuggerHidden, DebuggerStepThrough, DebuggerNonUserCode, and Experimental. Each of these flags currently only applies to the ScriptBlock in which they are defined, which means you must manually apply those same flags to any nested ScriptBlocks. This is inconvenient, and almost always forgotten.

If you have a script or function to which one of these attributes is applied, and within that script or function you do one of the following, currently you must manually add the same attributes to the nested blocks of code:

1. Define and invoke a nested script block, so that you can run some code with a different level of strict mode (as is done in PowerShell error handling code);
2. Define and invoke a nested script block so that you can follow the DRY principle and maximize code reuse;
3. Define and invoke a nested function for the same reason as the previous;

This design does not make sense, and it makes it more difficult to implicitly apply certain attributes under certain conditions.

I would like PowerShell to automatically apply the attributes mentioned (are there others? feedback welcome) to any nested functions or script blocks that are invoked from within a function or script where these attributes are used, so that they only need to be applied once over a block of code in order for that block of code to properly reflect the use of that attribute.

I've already started looking at how to resolve this, but CompiledScriptBlock doesn't currently have a property that identifies a parent CompiledScriptBlock (if there is one). I think that's the right approach (adding logic to the InitializeMetadata method), but I'm a little unsure at this time how to go about adding it. Suggestions on how to go about doing that, as well as thoughts/comments on the ideas presented here, are welcome and appreciated.

[lzybkr]

I'm not sure the engine should automatically add attributes, at least not in a way that is visible via ScriptBlock.Attributes.

How does C# work, e.g. with local functions?

I realize I've broken this ideal already with format script blocks, but I didn't see a better option. Maybe I should have just marked those script blocks as "format" script blocks and the debugger would also skip those.

For the debugging related attributes, I do think it is reasonable to behave as though the parent attribute was specified, just that it shouldn't be visible via the api.

As for the implementation, you can get the parent from the Ast property.

[KirkMunro]

Blowing the dust off of this as it just came on my radar again.

I'm not sure the engine should automatically add attributes, at least not in a way that is visible via ScriptBlock.Attributes.

I agree. I wouldn't want the attributes to show up in the Attributes collection.

How does C# work, e.g. with local functions?

Local functions in C# need to define their own attributes in order for those attributes to be in effect. The key difference there is that you can define those attributes on local functions, but you cannot define those attributes on script blocks that do not have a param block. With that in mind, I'm thinking if a script block is a "simple" script block (i.e. it does not have a param block, and is therefore not meant to function like a command does), it should inherit the debugger attributes that are set on the nearest parent "command" script block (i.e. one that has a param block). Functionality-wise, that would solve this problem nicely, allowing script blocks to be used as usual, but authors would have to set those attributes on any "command" script block (script block with parameters) if they wanted them set, because they are able to do so.

I realize I've broken this ideal already with format script blocks, but I didn't see a better option. Maybe I should have just marked those script blocks as "format" script blocks and the debugger would also skip those.

Are you referring to the script blocks defined in the C# format ps1xml definitions? From what I can tell, those already have DebuggerHidden automatically set to true on them. It only falls short when they internally invoke a nested script block using a call operator so that they can turn down strict mode for some things, but this change would resolve that.

For the debugging related attributes, I do think it is reasonable to behave as though the parent attribute was specified, just that it shouldn't be visible via the api.

As for the implementation, you can get the parent from the Ast property.

Thanks for pointing out just using Ast.Parent -- that will do nicely. A few things I've noticed since looking more closely at this:

1. Ast has Find, but it does not have ReverseFind. I'm essentially doing a ReverseFind, so I wonder if that would be worth adding to Ast. I can keep it as a one-off thing, but ReverseFind (and ReverseFindAll) could be useful when you want to look up the tree for information. After looking a little more at this though, I think it is faster for me to just process the attributes inline without finding each one, so I'm not moving forward with this.

2. With respect to the DebuggerHidden and DebuggerStepThrough properties, I think I should do something like the following:

   • Add an internal _debuggerAttributeInheritanceChecked boolean field to ScriptBlock.
   • When processing a basic script block, if _debuggerAttributeInheritanceChecked is false, check recursively for a parent script block that has a param block. If one is found, update DebuggerHidden and DebuggerStepThrough according to whether or not that script block has a DebuggerHidden or DebuggerStepThrough attribute on it. Regardless of the outcome of the search, set the _debuggerAttributeInheritanceChecked boolean field to true.

(As it turns out, the extra internal field was not necessary)

[KirkMunro]

Also, something that needs to be considered here is compiled script blocks. Based on my tests, a script block is compiled when it is defined. That means if you come along later and tweak the DebuggerHidden attribute on that script block, it will make that script block hidden or visible to the debugger (depending on how you changed it), _but any script blocks that are defined within that script block will maintain the debugger visibility that they had when the outer script block was defined_.

Assuming my understanding of how script block compilation works in PowerShell, that means toggling the debugger visibility attributes should result in a recompilation of the compiled script block that contains those attributes in order to ensure script blocks have the appropriate debugger visibility settings.

Right now I appear to have this working the way it should locally, except for when someone toggles a DebuggerHidden property on a script block.

[vexx32]

Is it worthwhile looking at whether you can simply set a flag when entering a scriptblock with DebuggerHidden set and retain it until you actually exit that scope? I.e., all child scopes should inherit it by simple virtue of how PS handles the structure, rather than having to recursively set it implicitly?

[lzybkr]

Compilation happens in stages. Each stage happens on demand, e.g. if you access a property that requires that stage. Attributes happen to be one of the first things that get "compiled" because the Attributes property happens to be important in various ways.

There is one design difficulty regarding recompilation - caches. You'd want a design that invalidates caches that might be affected, and also a way to ensure new caches can't be introduced that accidentally miss invalidation upon recompilation.

[KirkMunro]

As I experiment with this more and dig into how these work, I think it would make much more sense if DebuggerHidden acted as a bouncer, keeping the debugger out of a block of code and anything that it invokes (i.e. preventing breakpoints from triggering within that call stack until it immediately steps out of the hidden code). That has been suggested as part of #10530. With that design, it doesn't need to be inherited.

DebuggerStepThrough, however, and DebuggerNonUserCode, should carry though to any script blocks defined and invoked within the script block that contains those attributes.

@vexx32: Your suggestion about setting a flag would work well for what I just described for DebuggerHidden. I'm not sure it would work with the others though, but I'm looking into it. I think it would be useful if a given ScriptBlockAst had an additional parameter called ParentScriptBlock that would refer to the ScriptBlockAst in which a nested script block is defined. If I had that, lookup of debugger attributes for assignment to a given script block would be fast and simple.

@lzybkr: Thanks for those details -- the caches (where they are configured, how to invalidate them) is something I haven't figured out yet.