Smokedetector: Should we continue to require 2 feedbacks on each post?

[ArtOfCode-]

We currently, via review, require 2 feedbacks to be submitted on each post before it's dequeued; as of posting this issue, posts review has just over 1500 items.

Do we want to continue this requirement, or drop it back to 1? Running some stats on the latest data dump shows that if the requirement were only 1 feedback, we would currently have just over 400 items in posts review.

We could also simply drop the requirement for _non-autoflagged_ posts to 1 feedback; this has a similar effect (just under 500 items), but ensures that we still maintain two pairs of eyes on autoflags.

cc: @Charcoal-SE/owners

[tripleee]

How about the reverse? Autoflags already get some additional scrutiny, and should at the same time be safer on average just because of the amassed weight.

[angussidney]

I think we should keep the requirement at 2 feedbacks per post, especially for _non_ autoflagged posts (I'm with tripleee on this one).

For autoflagged posts, the main argument is that we need to maintain two pairs of eyes per post. Personally, I am in favour of maintaining this stance for general safety reasons, but I am open to having my opinion changed on the matter. After all, most posts which are autoflagged are either clear-cut spam (anti-aging cream anyone?) or a total mistake (e.g. a weird post on a religion site which trips many of our offensive filters), so it is less likely that additional feedbacks will make any difference.

On the other hand, many non-autoflagged posts require more scrutiny than a quick visual scan of the post - it takes trained eyes to detect some spam tactics e.g. plagiarism. Additionally, there are many instances where is hard to make a distinction between TP, FP, or NAA, e.g.:

• Is a post is an attempt to promote the user's own blog post, or simply a link-only answer? Is it even their own blog post?
• Is the post gibberish or just NAA?
• This website is on the watchlist, but the post looks fairly benign. Is it a hidden spamming attempt or should the site be taken off the watchlist?
• Some of our more infrequent trolls may be hard to spot if you haven't seen them before; having two users review the post increases the likelyhood that they will be caught.

Now, taking a look at conflicted feedbacks: only 4 of 460 (0.87%) posts that are currently unresolved have been autoflagged, indicating that non-autoflagged posts are more likely to have varied opinions (1). Importantly, a large number of these have only two feedbacks - indicating that if only one feedback was required, it is possible that the second feedback would not have been left, and the post would not have been brought to our attention.

I think a better solution to this problem would be to make the reviewing process easier. Currently, there are many friction points that make it hard to review large swathes of posts at once - e.g. the lack of domain record links and detailed 'why' data (both of which are normally available on posts), meaning that you have to click through to the actual MS report for all but the most clear-cut cases. If these issues are ironed out, I suspect that some users (including myself) would be more inclined to do larger batches of reviews.

Finally, if we can bring the number of pending reviews back down to a manageable number, then it'll be a lot easier to keep it down - currently, any attempt to make a dent in the number of the remaining reviews simply leaves you feeling helpless.

---

(1) Note: it is possible that my dataset is biased as it only contains unresolved posts - I don't have the SQL skills to get the data for all posts, but I suspect that would paint a similar picture.

[fortunate-man]

Could we get some statistics on how many posts in the current review queue have no feedbacks, and how many have one feedback?

[ArtOfCode-]

@fortunate-man in the original issue. About 1000 of them have 1 feedback, lessening just over 400 with 0 feedbacks.

[ArtOfCode-]

@angussidney Can you throw suggestions for review improvements in a metasmoke issue? I try to keep on top of review FRs (hey, markdown previews exist now!), so I should be able to get them done.

[angussidney]

Random thought: what if we adjusted AIM so that it only fully greys out a post once it has recieved 2 feedbacks? To show deletion, we could grey out only the link itself - leaving the rest of the report text as normal, signifying that the report hasn't been fully handled yet.

Part of the problem may be that people don't realise the importance of giving feedback to unhandled deleted posts.

[makyen]

@angussidney It's something that I expect to help, and had already implemented for FIRE. Implementing it in AIM could use similar logic, but it relies on both accurately determining which MS post the specific SD report corresponds to and not conflating the feedback for all MS posts for a specific post. AIM currently does neither of these things.

However, I'm not really sure how much this will help. There are a large number of SD reports for posts which don't get deleted, which is, currently, the only time AIM changes them.

[RobCoding]

I agree with angussidney's comment above. Two humans should be a minimum to review everything no matter the source (automatic or another person).

It's unfortunate that the review queue grows, it's simply something we need to remember to nibble away at. Perhaps if it grows more than 15% in one day a one liner could be automatically posted in chat every 4 hours begging for people to tackle the queue.

Making the page more informative would help. For example PRs (one example) have a sentence that says:

"See the MS search here and the Stack Exchange search here."

Followed by a reply from MetaSmoke such as this:

"signyourdoc.com has been seen in 0 true positives, 0 false positives, and 0 NAAs."

Additional information and easy to click on links make doing complete research on each review easier - it's especially useful when the correct information is just there and it's only a matter of confirming it.

[makyen]

We should change MS to not create duplicate MS posts for unchanged SE posts.

I need to double check, but the last time I surveyed a portion of our data, about 20% of the FP posts on MS were duplicate reports of SE posts that had not been edited since previously reported (i.e. were completely unchanged from when previously reported).¹ I, primarily, got frustrated with this due to repeatedly noticing that I was reviewing exactly the same post multiple times, for multiple different SE posts.

The problem is that posts which are FP will be re-reported by SD to MS anytime the SE post is rescanned outside of the window in which SD remembers them, or when we switch to a new instance, unless something has changed in our detections to make them not detected. As things stand, this will keep happening for those SE posts which are FP every time the post is brought to SD's attention outside the relatively brief time SD remembers the post. Over time, this can result in multiple duplicate reports for a single, completely unchanged SE post.

There's no need for us to spend time re-reviewing a post that was previously reported, but has not changed in any way (i.e. where the body, and title on questions, are identical and there are no additional intervening revisions reported by SE²), particularly when the post has already been judged FP. Having theses additional reports just wastes the time of our reviewers and skews our TP/FP statistics (TP posts tend to get deleted, so it's mostly FP posts that are re-reported). In some cases, it also brings an inappropriate level of scrutiny to those FP posts.

We should change MS such that it does not create a new MS post for any SD report that is for an SE post which is unchanged from the most recent prior report (i.e. where the body, and title on questions, are identical (not just the same revision number or where the SE post has only one reported revision, due to grace period edits) and has no intervening revisions). This determination would likely be made by MS after SD begins to post chat messages about the post. Ideally, SD would be signaled by MS to stop making any additional chat messages about the report and delete (or edit) any chat messages it's already made about that new SD report.

Doing this would significantly reduce the number of MS posts / SD reports which need to be reviewed.

---

1. I did not do a full check. The 20% number is from the results when I was tuning a watchlist entry, so it only reflects a subset of MS reports. There were > 100 FP in the sample and all were recent reports (last couple/few months). While this may not be representative of overall MS FP posts, it's likely a decent sample of FP. It wasn't a reasonable sample of the mix of TP and FP, but when only considering FP as representing FP, it was reasonable.

   I first sorted for posts where the title had not changed, then checked for unchanged bodies. While it's possible I could have missed some where the post body had changed, I believe I checked all of them. There were also ones which had been reported 4, or more, times where the SE post had never been edited.

2. Checking that there are no additional intervening revisions, in addition to checking for an exact match to the body, and title on questions, should allow us to report posts that have been reverted to earlier versions (e.g. vandalism or spam that's been edited and rolledback). Checking for intervening revisions could be done just by checking for more reported revisions than previously existed. However, MS would need to change to only counting actual post revisions, rather than the number of entries in the revisions list reported by SE (this is a separate bug, which results in MS thinking a post is edited when it's not).

[ArtOfCode-]

@makyen Your 2: can you throw me an MS issue for metasmoke not counting revisions correctly? I'm not that familiar with that particular bit of code, but it sounds reasonably low-hanging.

In general, agree. Changing metasmoke to reject duplicates doesn't even require Smokey changes: MS can simply tell Smokey "Yes dear, of course I created that report", but since Smokey uses a UID link, it will redirect to the older report.

[makyen]

@ArtOfCode- MS issue created

[RobCoding]

Makyen said: "Having theses additional reports just wastes the time of our reviewers and skews our TP/FP statistics (TP posts tend to get deleted, so it's mostly FP posts that are re-reported).".

I think I noticed evidence of that, if you search MetaSmoke with all fields blank, the default, it returns all the results:

Results (157647)
True positives (119327)
False positives (31407)
NAA (7862)

If you take TPs divided by FPs, (100/(119327/31407)), that equals 26.320112% - that seems like a lot.

Our Charcoal Website main webpage says: "... the bot is pretty accurate these days - some of the reasons that trip our filters have an accuracy of over 99%."

Usually something becomes a FP because a human has decided that, otherwise why would it be reported if Smokey knew it was false; while TPs are often (but not always) confirmed by us (unless the weight is sufficiently high for Autoflagging).

So with so many FPs that's almost all extra work when the claimed accuracy is high, in some cases over 99%.

Cutting the FPs in half would be an enormous time savings, if they were ~10% of what they are that would be great, but we have some necessary experimental tests (the ones not reported in all rooms, can't find a link to them) that push the value higher.

[ArtOfCode-]

Not sure what you're calculating there, @RobCoding. If you're looking for accuracy, that's TP/total: 119329/157655 = 75.68%.

[makyen]

@RobCoding "some of the reasons that trip our filters have an accuracy of over 99%" doesn't say that all, or even most, of our detection reasons have >99% accuracy. Nor does it say that our overall detection rate is >99% accurate.

This project (SD/MS) has two intersecting goals, which have different rates of FP.

1. Detect all TP posts. To do this requires casting a wider net. This will result in a moderately high level of FP. In some cases, we specifically choose to have a higher rate of FP in order to catch some TP which are not otherwise detected. The places we make the choice to have high FP rates are mostly in reasons with names including "potentially", but that doesn't mean we don't make the trade-off to have somewhat lower TP rates (i.e. somewhat higher FP rates) in other detections.

2. Autoflag only TP (that should have red flags). These TP are a subset of those identified in (1). It also means that all the FP should be filtered out of this sub-bin. If filtering out all the FP means that we also filter out some, or even a lot, of TP, then that's OK. This rate is currently 46741/46532/144/91, or 99.54% TP (%TP here counts 4 TP posts with conflicting feedback as non-TP).

There is also the goal/desire of:

1. Obtain feedback indicating what should be blocked, if SD was implemented as a system-level block. This is somewhat different than which posts should be red-flagged, so there's not a 1:1 correlation between TP and "should be red-flagged". All posts that should be red-flagged should be blocked at the system level, but not all posts that should be blocked at the system level should be red-flagged.

As a result of (1), we have a significant number of FP that have been detected (the total TP & FP numbers you are looking at). But, that's significantly different than having FP that have been autoflagged.

So, while we do desire to reduce the number of FP, we don't desire to reduce the number of FP at the expense of not detecting TP.

Yes, sure, if there's a detection that's getting 1 TP for every 50 FP, then we definitely want to take a look at it and see what we can do to reduce the number of FP, but we don't want to eliminate it, if that's the only thing we have which detects that 1 TP. In such cases, we should be looking to see if there are alternate ways of detecting that TP, without the FP. Obviously, in the real world, there is a point at which the % FP is just too high to keep a particular detection, but that's something we need to judge on a case-by-case basis.

[ArtOfCode-]

The fundamental goals are these:

• SmokeDetector: zero false negatives
• Autoflagging: zero false positives

[RobCoding]

⚓🛥️ - I was simply offering agreement with Makyen's comment. My two last paragraphs were missed, I didn't want to overelaborate and finesse.

[ByteCommander]

Not sure about the "age structure" of posts in the review queue, but could it maybe be an option that posts with a single review which are older than X (maybe two days? a week?) get cleared from the queue automatically? After all, they have one feedback already and it's unlikely that there is still any action required on this post.

Other than that, I strongly agree that the MS review interface needs to be improved so that reviewers have all the relevant detail at a glance, as it has been commented here before.

[ArtOfCode-]

(@ByteCommander see 79523cec0548f1e51a4ff8bfad0e600bd68bde28 - review now uses the full post view, so all the details are there)

[angussidney]

I'm not sure that we should clear out older posts with a single review - after all, they haven't had any additional eyes on them other than the intial review, so their age doesn't really mean anything.

[angussidney]

Although only about 8 or 9 people have responded to this discussion, it seems that most are in favour of keeping the system like it is. Additionally, the number of pending reviews has decreased dramatically over the past few days (thanks to the awesome work of a bunch of people!), and /review has seen a dramatic improvement in functionality, so the main motivation behind this discussion isn't so much of an issue anymore.

As such, I'm going to close this issue as [status-agreed] with the conclusion that we will leave the system as is. If anyone has any further points, feel free to leave further comments and we can consider reopening the discussion.