Winforms: Hidden text not passed to LinkClicked

[weltkante]

.NET Core Version:
see below

Have you experienced this same bug with .NET Framework?:

| Version | RichTextBox.Text | LinkClickedEventArgs.LinkText |
|-|-|-|
| .NET 4.6.1 | includes hidden text | includes hidden text |
| .NET 4.7.2 | excludes hidden text | excludes hidden text |
| .NET Core 3.x | excludes hidden text | excludes hidden text |
| .NET Core 5.0 | includes hidden text (#3399) | excludes hidden text |

Problem description:
.NET 5 reverted hidden text behavior to match .NET 4.6.1 but forgot to update the LinkClickedEventArgs.LinkText behavior. The reversion was necessary because the .NET 4.7+ behavior was a regression, all indexing still operated against hidden text, but when queried for text the RTF control would omit it, making it impossible to calculate indices based on the control text.

There is precedence of people using hidden text to pass URLs into the LinkClicked event handler (for example dotnet/winforms#3632, gitextensions/gitextensions#7162 and NLog/NLog.Windows.Forms#42)

As far as I can tell they were broken in Desktop Framework but could revert the RTF control to the old version via a runtime switch to go back to .NET 4.6.1 behavior. In .NET Core this is no longer possible, furthermore in .NET 5 the treatment of hidden text was fixed to be more consistent and match indexing behavior, but in the process the LinkClicked event was missed.

This leads to an inconsistent experience where RichTextBox.Text reports hidden text but LinkClickedEventArgs.LinkText does not. Previous code from .NET 4.6.1 which used hidden text to pass the URL to the LinkClicked handler cannot be reasonably ported to .NET Core.

Expected behavior:
RichTextBox.Text and LinkClickedEventArgs.LinkText should treat hidden text consistently.

Minimal repro:

public class Form1 : Form
{
    private RichTextBox richTextBox1;

    public Form1()
    {
        richTextBox1 = new RichTextBox();
        richTextBox1.Dock = DockStyle.Fill;
        richTextBox1.DetectUrls = false; // required for manual link creation
        richTextBox1.LinkClicked += richTextBox1_LinkClicked;
        ClientSize = new System.Drawing.Size(640, 480);
        Controls.Add(richTextBox1);
        Load += Form1_Load;
    }

    private void Form1_Load(object sender, EventArgs e)
    {
        richTextBox1.Text = "Click here";
        richTextBox1.SelectAll();
        LinkHelper.ChangeSelectionToLink(richTextBox1, richTextBox1.SelectedText, "http://www.google.com");
        richTextBox1.SelectionStart = richTextBox1.TextLength;
    }

    private void richTextBox1_LinkClicked(object sender, LinkClickedEventArgs e)
    {
        MessageBox.Show($"link \"{e.LinkText}\" clicked\nText=\"{richTextBox1.Text}\"");
    }
}

helper code

// Taken from NLog.Windows.Forms PR to demonstrate how 3rd party code passes links to LinkClicked
internal static class LinkHelper
{
    #region Link support
    /// <summary>
    /// Replaces currently selected text in the RTB control with a link
    /// </summary>
    /// <param name="textBox">target control</param>
    /// <param name="text">visible text of the new link</param>
    /// <param name="hyperlink">hidden part of the new link</param>
    /// <remarks>
    /// Based on http://www.codeproject.com/info/cpol10.aspx
    /// </remarks>
    internal static void ChangeSelectionToLink(RichTextBox textBox, string text, string hyperlink)
    {
        int selectionStart = textBox.SelectionStart;

        //using \v tag to hide hyperlink part of the text, and \v0 to end hiding. See http://stackoverflow.com/a/14339531/376066
        //so in the control the link would consist only of "<text>", but in link clicked event we would get "<text>#<hyperlink>"
        textBox.SelectedRtf = @"{\rtf1\ansi " + text + @"\v #" + hyperlink + @"\v0}";

        textBox.Select(selectionStart, text.Length + 1 + hyperlink.Length); //now select both visible and invisible part
        SetSelectionStyle(textBox, CFM_LINK, CFE_LINK);                     //and turn into a link
    }

    /// <summary>
    /// Sets selection style for RichTextBox
    /// https://msdn.microsoft.com/en-us/library/windows/desktop/bb787883(v=vs.85).aspx
    /// </summary>
    /// <param name="textBox">target control</param>
    /// <param name="mask">Specifies the parts of the CHARFORMAT2 structure that contain valid information.</param>
    /// <param name="effect">A set of bit flags that specify character effects.</param>
    /// <remarks>
    /// Based on http://www.codeproject.com/info/cpol10.aspx
    /// </remarks>
    private static void SetSelectionStyle(RichTextBox textBox, UInt32 mask, UInt32 effect)
    {
        CHARFORMAT2_STRUCT cf = new CHARFORMAT2_STRUCT();
        cf.cbSize = (UInt32)Marshal.SizeOf(cf);
        cf.dwMask = mask;
        cf.dwEffects = effect;

        IntPtr wpar = new IntPtr(SCF_SELECTION);
        IntPtr lpar = Marshal.AllocCoTaskMem(Marshal.SizeOf(cf));
        Marshal.StructureToPtr(cf, lpar, false);

        IntPtr res = SendMessage(textBox.Handle, EM_SETCHARFORMAT, wpar, lpar);

        Marshal.FreeCoTaskMem(lpar);
    }

    /// <summary>
    /// CHARFORMAT2 structure, contains information about character formatting in a rich edit control.
    /// </summary>
    /// see https://msdn.microsoft.com/en-us/library/windows/desktop/bb787883(v=vs.85).aspx
    [StructLayout(LayoutKind.Sequential)]
    private struct CHARFORMAT2_STRUCT
    {
        public UInt32 cbSize;
        public UInt32 dwMask;
        public UInt32 dwEffects;
        public Int32 yHeight;
        public Int32 yOffset;
        public Int32 crTextColor;
        public byte bCharSet;
        public byte bPitchAndFamily;
        [MarshalAs(UnmanagedType.ByValArray, SizeConst = 32)]
        public char[] szFaceName;
        public UInt16 wWeight;
        public UInt16 sSpacing;
        public int crBackColor; // Color.ToArgb() -> int
        public int lcid;
        public int dwReserved;
        public Int16 sStyle;
        public Int16 wKerning;
        public byte bUnderlineType;
        public byte bAnimation;
        public byte bRevAuthor;
        public byte bReserved1;
    }

    private const int WM_USER = 0x0400;
    private const int EM_SETCHARFORMAT = WM_USER + 68;  //EM_SETCHARFORMAT message - Sets character formatting in a rich edit control. https://msdn.microsoft.com/en-us/library/windows/desktop/bb774230(v=vs.85).aspx
    private const int SCF_SELECTION = 0x0001;       //Applies the formatting to the current selection. https://msdn.microsoft.com/en-us/library/windows/desktop/bb774230(v=vs.85).aspx
    private const UInt32 CFE_LINK = 0x0020;         //link effect https://msdn.microsoft.com/en-us/library/windows/desktop/bb787970(v=vs.85).aspx
    private const UInt32 CFM_LINK = 0x00000020;     //mask for CFE_LINK, see https://msdn.microsoft.com/en-us/library/windows/desktop/bb787883(v=vs.85).aspx

    [DllImport("user32.dll", CharSet = CharSet.Auto)]
    private static extern IntPtr SendMessage(IntPtr hWnd, int msg, IntPtr wParam, IntPtr lParam);
    #endregion
}

[weltkante]

Just a quick heads-up, I'll do a more detailed writeup tomorrow (keeping the issue open for now as a reminder) - I tried to implement this and this is not actionable. The two versions of the native control behave fundamentally differently and you need different implementation strategies than what you used up to .NET 4.6. I'll outline how to port code that used the old strategy, I think for NLog.Windows.Forms we ended up with a workable solution.

[weltkante]

So I experimented a bit and I realized the native control changed its behavior between versions and its no longer possible to mix links and hidden text. When you create a link with CFE_LINK it will not apply that attribute to hidden text within the span given, effectively splitting your link into multiple parts. This also happens if you first create a link and then subsequently hide a span within the link.

Given that its no longer possible to mix CFE_LINK and hidden text means the LinkClicked event will always receive the span of the clicked _part_ of the link - if the hidden text is in the middle (and not at the start or end) then you actually do not receive the full span of the link, but only the subspan which you clicked!

In the old native control there was no friendly name for hyperlinks, it did have some hyperlink support for clicking literal URLs and for making your own hyperlinks via CFE_LINK. Apparently the old implementation allowed to freely mix this with hidden text, leading to people using hidden text for passing the URL along with the friendly name of the link.

The new version of the control added support for actual hyperlink markup in addition to extending the automatic hyperlink generation. You have to turn automatic hyperlink generation off if you intend to continue to use CFE_LINK to generate your own links.

The official hyperlinks are also translated into a hidden text part and a friendly name, and the same rules apply - the hidden text part is not marked as link (when you inspect the attributes of the source text via interop). However the native control understands hyperlinks now and will pass the hidden text span _instead of_ the friendly name span to the LinkClicked handler. Like mentioned in the documentation you can then extend the span to discover the friendly name, however doing that in WinForms requires knowning the span (#4431) and using interop since ITextDocument APIs are not exposed through WinForms.

So when porting RichTextBox code written before .NET 4.7 (or using the legacy control on Desktop Framework) you need to switch from CFE_LINK interop to the official hyperlink markup. This will actually simplify your code. For advanced users who do not want to switch to hyperlink markup you will have to wait for #4431 (probably .NET 6) and can then use interop to extend the span to discover the hidden text, just like the native control does.

I considered implementing this issues requirements (passing hidden text to LinkClicked again) by letting WinForms automatically extend the span it got passed by the native control, but this is not sufficient, since you don't know how the span was generated - the hidden text could be in front of the link or behind the link, and if the link was split you cannot recover the full span at all. Also hidden text could exist outside links.

So I believe its best to not perform any automatic processing of the span in the WinForms code and instead expose the span WinForms receive (#4431) and let the user application decide how to process the span, the application has more knowledge of how the link was defined and for example is able to specifically extend it only forward to include hidden text if it knows the hidden text containing further information is always defined behind the link.

[weltkante]

@RussKie not a bug after all, its an incompatibility between the native control versions. WinForms receives a truncated span (excluding hidden text) - this is a change of behavior in the native control, the old version allowed hidden text to be part of the link. Its not possible to recover the hidden text unless you follow a pattern and always put the hidden text behind the link and never in front of it, but that is up to the application to decide and WinForms should not mandate it.

For porting old code its best to just switch to the official hyperlink markup, that will work in .NET 5 already without any further changes, and it actually simplifies the code to embed the links since you don't have to mess with CFE_LINK interop and can define everything inside RTF markup without any interop at all.

[RussKie]

Thank you @weltkante for being thorough, your investigations and the detailed write up. There's nothing we can really add to it.

The currently endorsed way of generating hyperlinks is via {\field{\fldinst…}{\fldrslt…}} construct. This is what the Office team advised:

...this RTF is not the way to represent friendly-name hyperlinks, which is probably what you want. Such links use the {\field{\fldinst…}{\fldrslt…}} construct as you can see by saving a document with a friendly name link as RTF in Word and looking at the file in Notepad. For example, the hyperlink MSW is represented by \field{\*\fldinst{HYPERLINK "http://msw" }}{\fldrslt{\ul\cf1\cf1\ul MSW}}