Smokedetector: The post Markdown sometimes has info which is stripped from the HTML. Should feedback be based on it, or only the HTML?

[makyen]

The information that SD deals with is the rendered HTML, not the Markdown. SD could get the Markdown in addition to the HTML, but doesn't. Do we provide feedback based on information which exists in the Markdown, but which SD never sees because it's stripped from the rendered HTML? Such feedback could result in difficulty in training, if the feedback is used for ML training.

Example ms post

HTML that SD sees:

<p>[enter link description here][۱]</p>

Markdown from post:

[enter link description here][۱]


  [1]: http://%20%[email protected]

While in this case, the Markdown is not completely damning, I recall seeing at least one other case with a post that SD reported where the Markdown made the decision clear due to having stripped a large amount of text out of the rendered HTML. IIRC, it was text that SE's Markdown converter interpreted as HTML and stripped.

[tripleee]

Another example: https://chat.stackexchange.com/transcript/message/45045471#45045471

[makyen]

That example is:
ms post

The HTML SD sees:

<p>this is static website canonical tag  but bellow link check it that is dynamic script we have 100 pages</p>

<p>" /></p>

The Markdown source (edit link) is:

this is static website canonical tag <link rel="canonical" href="https://aapthitech.com"/> but bellow link check it that is dynamic script we have 100 pages

<link rel="canonical" href="<?php echo 'http://'.$canon_url; ?>" />

[Undo1]

We should feedback based on whether the post on SE should have been blocked at the system level or not. Meta doesn't care about our technical details, they care about whether the stuff we catch is actually spam.

[makyen]

With respect to scanning HTML vs. scanning Markdown: We can easily get both at the same time from the SE API call (assuming we're using /questions/{ids}, etc). Doing so would allow us to write some detections which test the Markdown instead, or in addition to, the HTML. Cutting-over to using Markdown would be a significant effort, but just writing some tests which fill in where we identify issues, shouldn't be that difficult.

The difficulty in getting to that point would be the infrastructure that needs to be in place to enable it. This would include having MS store the Markdown, in addition to the HTML. But, once the infrastructure was in place, adding additional detections for the Markdown would be no more difficult than any other detections.

[tripleee]

https://metasmoke.erwaysoftware.com/post/124155 another example

[makyen]

Another example: https://m.erwaysoftware.com/posts/uid/mathoverflow.net/304996

[makyen]

Another Example: https://metasmoke.erwaysoftware.com/post/127060

[iBug]

Same thoughts as Undo has. The only untimate feedback guideline is whether we want to block the post on SE. Complementary/supplementary information can be submitted as a comment on MS so special cases can be filtered out easier.

[stale[bot]]

This issue has been closed because it has had no recent activity. If this is still important, please add another comment and find someone with write permissions to reopen the issue. Thank you for your contributions.