Spoke: flagged words/profanity should put incoming messages in a review queue

[schuyler1d]

Texter volunteers shouldn't need to see responses with slurs or other offensive language.
Before a texter sees the reply, if a message matches the blockword regex (a configured environment variable), then it should not be shown to the texter and the texter should not get the contact (yet) in the list of replies. Instead, the contact and message should be available to an campaign ADMIN where they can choose three 'fates' for the contact/message:

1. optout the contact (with reason 'profane')
2. leave the contact's message in the profane queue but do nothing
3. allow the message (in which case the message gets passed to the contact as a normal reply)

Furthermore, when a texter sends a message that matches the blockword regex, the server should respond with an error message and the texter should be prompted to 'rephrase their message'. -- i.e. texter messages that match the blockword regex should be blocked from being saved or sent.

• [ ] STOPs (see url for other words) texts should auto-optout (for compliance with US texting laws/best-practices. Note: by default Twilio does this already, but accounts can request turning this feature off, so you can track which contacts requested optouts, for e.g. syncing optouts to other systems. STOP/STARTing should follow the behavior described here: https://support.twilio.com/hc/en-us/articles/223134027-Twilio-support-for-opt-out-keywords-SMS-STOP-filtering-
• [ ] A configurable regex string to find bad words.
• [ ] Ideally the regex string would be transliterated so people looking at the config wouldn't need to see a bunch of bad words. ( maybe something like flipping a=>z and z=>a regex_string.replace(/[a-z]/g, ((x) => (String.fromCharCode('a'.charCodeAt()+'z'.charCodeAt()-x.charCodeAt()))))
• [ ] A command (e.g. npm run profaneconfig ... to convert a regular regex to a transliterated regex.
• [ ] Consider adding a message.send_status of PROFANE -- depending on consequences. Pros: might exclude in a good way from messages shown in-thread. Cons: might
• [ ] STOP requests should get a specific 'reason_code' in the opt-out table (say stop). and profane messages should get profane in the opt_out.reason_code.
• [ ] add profane field marker to allmessages api (https://github.com/MoveOnOrg/Spoke/blob/main/src/server/index.js#L127) so that contacts that got marked 'profane optout' are visible (will need to add data out after joining to unless we change the send_status as mentioned above)

Open questions:

• Should we have a new send_status?

[schuyler1d]

@lperson After discussing with the Text Team we've nailed down the desired feature/workflow, and I've updated the title and description.

[shakalee14]

how would folks feel about admins being able to add/subtract words from the flagged list? (question from erik)

and what would it look like to filter & potentially flag outgoing messages?

[lperson]

is this a priority for erik?

I was thinking about having a configurable list of words, which could be tweaked to suit the needs of different organizations. If a message is flagged as profane, it would be reassigned in such a way that it would show up in message review, where such messages can be either reassigned or permanently set aside.

[yolfer]

Yes please!! As the volunteer text team grows, we've seen an uptick in texters getting impatient with the folks they're texting. Of course it also sucks to receive profanity in the replies, but that's gonna happen so I'm less concerned about that than I am about someone using profanity while representing MoveOn and/or a candidate.

[yolfer]

@yolfer yolfer unassigned @lperson lperson just now

Uhhh... not sure how I did that. I'll try to undo but apologies!!

[lperson]

@schuyler1d please reassign this to me

[lperson]

@schuyler1d bump

[DinahSanders]

Dropping a reminder here of this previous discussion on handling obscenity: https://github.com/MoveOnOrg/Spoke/pull/938

Moderation adds a lot of work in what will be a very busy year. Let's consider the potential use of grawlix (comic book swearing like $%&#!) to take away the sting of incoming foul messages. It might work to spare our texters' mental health AND avoid the need of an extra administrative chore moderating incoming messages.

[williamtheaker]

Let's make sure this doesn't impact organizing in the UK though: 🙃

Scunthorpe problem

[arena]

Personally, as a volunteer texter, I don't care about profanity and appreciate the occasional "F*ck Yes!"

[DinahSanders]

Violent obscene threats have been a problem in the past and we have lost texters due to such harassment.

[ibrand]

@arena when I was texting, I saw profanity used in some pretty different contexts that was occasionally targetting at me. Rather than a "Fck yes!" I might see a "FCK YOU" or usually more specific and aggressive stuff. I think it would have been pretty nice to not have to deal with people yelling at me every so often in my responses. I bet the type of profanity is pretty campaign dependent too.

[schuyler1d]

Also, to be clear, we (moveon) would be unlikely even to include "profanity" (i.e. 'fck') -- it's more slurs and epithets (think'***head' or 'n***', or.... 'b*mer' :-P J/K on the last one)

[DinahSanders]

In preparation for 2020 documentation updates, I've been going through lots of old notes from our MoveOn experiences. Profanity-related items:

• moderation takes away limited leader time; better to disguise detected profanity e.g. with grawlix https://github.com/MoveOnOrg/Spoke/pull/938

• administrator being able to add stopwords/stopphrases is helpful as new abusive terms arise and as administrators may wish to block outgoing URLs (e.g. to stop confused texters from sending links to team support documents to contacts)

• the overlap between obscenity and abuse is very strong (e.g. "f**ing kill you self") but cheerfully positive obscenity also comes in and we don't want to lose those highly enthusiastic folks

• the ability to add emoji to the list would be a big help in blocking symbols we just never want our texters to use (e.g. we had a nearsighted texter on a small screen accidentally sending a raised middle finger instead of a thumbs up)

[arena]

😹👍 LOL:
(e.g. we had a nearsighted texter on a small screen accidentally sending a
raised middle finger instead of a thumbs up)
...
P.S. I'm sorry that I'm laughing, and I 100% agree that I never ever want
to send the middle finger emoji when I'm texting for any org or even on my
personal phone. I'm not sure how that got through the Unicode Working Group
..but I fully support taking it out of Spoke. And on a serious note I feel
terrible for the texter who experienced making that mistake.

On Thu, May 7, 2020 at 1:59 PM Dinah Sanders [e-mail]
wrote:

In preparation for 2020 documentation updates, I've been going through
lots of old notes from our MoveOn experiences. Profanity-related items:

-

moderation takes away limited leader time; better to disguise detected
profanity e.g. with grawlix #938
https://github.com/MoveOnOrg/Spoke/pull/938
-

administrator being able to add stopwords/stopphrases is helpful as
new abusive terms arise and as administrators may wish to block outgoing
URLs (e.g. to stop confused texters from sending links to team support
documents to contacts)
-

the overlap between obscenity and abuse is very strong (e.g. "f**ing
kill you self") but cheerfully positive obscenity also comes in and we
don't want to lose those highly enthusiastic folks
-

the ability to add emoji to the list would be a big help in blocking
symbols we just never want our texters to use (e.g. we had a nearsighted
texter on a small screen accidentally sending a raised middle finger
instead of a thumbs up)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MoveOnOrg/Spoke/issues/359#issuecomment-625494358,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADELPRYOI6MZAM2D6KZ4PDRQMOLZANCNFSM4EBBWERQ
.

[ibrand]

There is a WIP PR that starts this work (#938) and it would be great to that taken on and completed!

[KidPluto]

I created this https://github.com/MoveOnOrg/Spoke/issues/1580, before finding this one.
My idea doesn't depend on an additional queue. Having another queue might slow things down, depending on who was available to review the items in that queue.
That being said this one has a lot more detail. My feature request is very high level.

[KidPluto]

Of course it also sucks to receive profanity in the replies, but that's gonna happen

Well, that is what we are trying to address here.

so I'm less concerned about that than I am about someone using profanity while representing MoveOn and/or a candidate.

I think that that should be a separate feature/effort.

[jeffm2001]

I don't know if this a fully sufficient solution (probably not), but I came across this font that uses ligatures to obscure profanity: https://vole.wtf/scunthorpe-sans/

Could be an extremely easy intermediate fix.

[ibrand]

This is really neat @jeffm2001 -- I don't think it filters out racial/other biggotted slurs which I think are some of the worse offenders here

[DinahSanders]

It is very clever, but I don't think it meets our goals for texter experience.

Imagine that right-handedness were treated as powerfully in society as whiteness and that 'southpaw' was deeply derogatory and indicative of pervasive systemic bias.

A texter, particularly a left-handed texter, who got the reply "F◼️◼️◼️ off you s◼️◼️◼️◼️◼️◼️" would have a much more negative emotional experience than with "$%&#! off you $%&#!"

Let's not give hurtful comments any assistance in delivering their loads of pain.

[Frydafly]

Following up here @schuyler1d & @lperson to double check that I can close this issue since it was merged in v7.0?

[schuyler1d]

Due to work-flow reflection, we ended up implenenting this just with a message handler => auto opt-out rather than the more sophisticated idea of a separate review queue -- if there's still demand for that, we should create a new issue and probably could implement it through developing more/better features in Message Review section