Mastodon: "Trust Propagation" can fix the internet. Let's be at the forefront.

[BrianPansky]

Hi, I'd like to implement a trust propagation system, to allow people to see estimate evaluations of others, and to be able to trust that those estimates are probably correct. This is similar to what the "Couch Surfing" website has or had (it's important for them because if you are going to sleep at a stranger's house, you need to know they are trustworthy!).

Basics of the system:

1. each user can evaluate other people.

2. then, through the network of these evaluations made by many users, each user can receive an estimate for the trustworthiness of people that others have evaluated. EDIT: to be clear about this step, the estimates each user receives are simply what is logically entailed by their own evaluations.

I have the calculation for this working on my own computer (currently in Python).

Benefits

I suspect the following problems could be nearly eliminated:

• harassment
• being fooled by fake accounts
• fake news

other benefits:

• works like "peer review", has associated benefits
• trust is a large factor in persuasion
• crowdsourcing effect creates efficiency boosts, division of labor and less redundant duplication of labor
• incentive for people to be more trustworthy
• perhaps more social connection, community

Proposed metrics:

Simple metric:

• "general trust"

More articulate:

• trustworthy source of info
• physical safety
• emotional safety

Could it be implemented as a 3rd party app using the REST API instead?

Yes, perhaps that too! But 3rd party app makes it sound like most users wouldn't even know it exists, which would stunt the network needed for the system to be useful. I also think this needs to be a basic feature of all social media, and Mastodon seems like the natural place to start!

[mkljczk]

but it can be used for harassment, to fool others and boost fake news trustworthiness too

[BrianPansky]

Only if someone builds up a good reputation (thus inviting larger scrutiny) and then decides to throw it all away publicly. I find that implausible.

[Laurelai]

This is a bad idea. This is such a bad idea that if implemented i promise to demostrate just how bad of an idea it is.

[BrianPansky]

It's not clear that either of you actually understand what the idea is, actually.

Please go on and say what is supposed to go wrong, in detail.

[mkljczk]

it would be trivial to abuse this mechanism without complicated algorithms nobody wants there.

How can we meter the trustworthiness of particular ‘reviews’?

How are the reviews gonna be stored? On reviewed user’s home instances? On remote instances?

[BrianPansky]

How can we meter the trustworthiness of particular ‘reviews’?

There you go. You don't understand the idea at all.

I'll have to try again...

[BrianPansky]

Now the storage problem is more interesting, sorry I don't know much about how data is stored on Mastodon. Presumably passwords are kept secure somehow though?

[mkljczk]

Yes, they are stored on home instances of particular users.

[trwnh]

Trivial example: I have 101 accounts, with each account vouching for my 100 others.

One step further: I build a bot army to give everyone good reviews, including other bots. The total number of recommendations expands by n! where n is the number of fediverse users.

Yet another step further: I and a group of humans genuinely rate ourselves highly. Can you deduce which of us are harassers?

For all examples above: suppose we used negative reviews for people we hate, instead of positive reviews to boost ourselves. We bury vulnerable or marginalized people through silent bad reviews that mass-target their accounts. Heck, suppose we did both.

Hopefully this demonstrates why credit systems are a bad idea. At best, they create feedback loops and encourage mob mentality. At worst, they fall to astroturfing efforts. It can even be argued that they are redundant with much simpler signals, such as the number of reports targeting that account, the number of times an account has been blocked, etc. -- and even those metrics are subject to gaming by bad actors publishing mass reports and chainable blocklists, as happens on Twitter for years.

[BrianPansky]

How can we meter the trustworthiness of particular ‘reviews’?

So, this kind of looks like you think we (mastodon coders?) decide for everyone else what should count as trustworthy and what is not?

That's not part of what I outlined. See my section on "basics".

The answer to this is basically: how would trustworthiness be determined without this kind of system?

All these kinds of systems do (google "trust propagation" or "trust aggregation" for academic research and publishing on these kinds of systems) is model the logical consequences of how our own brains already process trust.

[BrianPansky]

Trivial example: I have 101 accounts, with each account vouching for my 100 others.

Right off the bat: irrelevant. The system does not work by number of votes! If those 100 others are not trusted, nothing they say matters to anyone else.

[mkljczk]

All these kinds of systems do (google "trust propagation" or "trust aggregation" for academic research and publishing on these kinds of systems) is model the logical consequences of how our own brains already process trust.

The way our brains process trust is not that effective ;)

[trwnh]

@BrianPansky how is that irrelevant? Those accounts are not untrusted; in fact, they have 100 people vouching that they are trusted! If you trust any one of those 101 accounts, trust propagation would mean you trust the other 100.

[BrianPansky]

If you trust any one of those 101 accounts, trust propagation would mean you trust the other 100.

Not exactly. It would mean that you trust one set of reports in some proportion. But their reports would be weighed against others in your network. And only then would estimates for the 100 get a final computation.

Even still: people get fooled already without this kind of system. So it isn't clear what's worse about using computer assistance. So I don't see this as any kind of objection.

[Laurelai]

You cant use computers to determine how much a human can be trusted. They will always be gamed by people who cannot be trusted. This happens every time. If billion dollar companies cant figure this out, neither can you.

[BrianPansky]

Companies have figured it out when they care to. I linked to the example of the Couch Surfing website. I've been told AirBnb does something that might be similar.

It's so simple (just mathematically model the way we already process trust) that it's inevitable. Probably within the next few decades, especially once AI assistants become smarter. We could have had "my" simple version years ago, of course.

Systems being gamed by people who can't be trusted is a general fact of life, it can't be pinned on me or on any specific new system (google, television, wikipedia, facebook, twitter). But if we ever have a hope of minimizing it, I think it's with the help of a system like this.

[Laurelai]

No the best method of determining trust is by thinking for yourself and it always will be.

[Cassolotl]

So convinced that the idea is flawless that everyone here speaking out against it must simply be misunderstanding the concept, huh? 😬

[Cassolotl]

If I wanted to mess with this system, I can imagine I might do one of these two things:

1. Offer to buy the accounts of trusted people. (Once this idea catches on, there would be people who would try to make money by living reputable online lives for a few weeks/months/years, however long it takes to get super-trusted, with the intention of selling those accounts.)

2. Con my way into trusted accounts, lock out the owners somehow, and then use them to spam people, or sell them to someone who wants a trusted account for some reason. (We know people would do this, because for example I know that people phish Minecraft accounts and then sell them on.)

[BrianPansky]

no Cassoli, people did seem to misunderstand, but it's clear that at least trwnh understood the mechanics in his last reply.

I don't put much credence in your speculation, for reasons I've outlined already.

If people are afraid of the risks, I guess I'll have to accept that. It's fairly understandable, there is uncertainty and risk with any powerful innovation. It's as old as the human use of fire.

I tried...

[BrianPansky]

No the best method of determining trust is by thinking for yourself and it always will be.

Of course, that's why the system is based on your own inputs! And then all it does is literally tell you what they logically entail. If you get garbage results, it's kind of ultimately your own fault.

But surely you know that stuff like peer review (like in science) has some benefit of saving you time/effort if you do trust others to do the work for you.

[nightpool]

Reputation Economy Dystopia: China's new "Citizen Scores" will rate every person in the country – relevant reading for insights into the way these types of systems can be abused and manipulated by state actors.

[BrianPansky]

Yes, what China has is terrifying. Whether talking about the government they have, or the evaluations systems.

[I've basically given up implementing my proposal here, my guess is users would be freaked out. But I'm an advocate so I gotta advocate]

I just thought of an analogy, that hopefully people won't be able to un-see.

Imagine you're in a society without a significant government at all, it's just always been that way. And someone proposes changing to some democratic model. Then someone says "China demonstrates how those kinds of systems (governments) can be abused". Others worry about fraudulent votes.

[bmvjssdkco]

Honestly speaking, due to the lack of elaborate introduction, most people didn't understand this idea. First, using computer systems to calculate trust has its downsides, but it depends on the exact nature of such a system, and often the benefits outweigh the problems. Second, the problem of using multiple fake accounts to forge reputation is a well-known problem and has many potential solutions. Finally, it's not necessarily a governmental, in some systems, it is only enforced at individual-level as their own indicator of trust. Other people are free to "train" the system based on their own standard. In fact, it was first proposed as a privacy-preserving method solve the problem of trust, as an alternative to ask everyone's name to stand behind their words, there are risks, but it's absolutely NOT the social credit system for mass surveillance. And the anarchic aspect of the Cypherpunk's considerations are actually very relevant here, because there is no trusted third-party in the Fediverse in general and the development of a reputation system should not negatively affect the decentralized aspect of the network.

To understand it better, it must be put into the appropriate context. I'm personally very interested in this type of system. I'm not saying that it's necessarily the best idea, especially, whether this idea is suitable to be incorporated to ActivityPub/Mastodon is a still an open question. But personally, I believe this idea is very valuable but under-researched and rarely implemented. An independent community/project may be needed to strengthen the understating of such systems, and facilitate the building of real-world ones.

In this comment, I'll try my best to explain it...

I'm not a good writer, but... Let me try...

An Introduction to Reputation Systems and My Discussions

The Early Proposal of Reputation-based Systems

Anonymity and Encryption are Essential to Privacy

The idea of building a reputation system originally came from the anarchic Cypherpunk movement. In the late 80s, there had been a rapid development of computer networking, and a breakthrough of public-key cryptography. Many pioneers believed systems based on cryptography must be developed immediately to stop the threat of mass surveillance in the new digital age.

By forwarding mail between themselves through remailers, while still identifying themselves in the (encrypted) message contents, they have even more communications privacy than with simple encryption. - Hal Finney, 1993

Even better, cryptography enables us to build an anonymous network, and allows people to interact completely anonymously in an alternative online community. Normally, we evaluate a post based on its author. But in an anonymous communicate, we can interact online based on people's ideas, not their identities.

The more controversial vision associated with anonymous remailers is expressed in such science fiction stories as "True Names", by Vernor Vinge, or "Ender's Game", by Orson Scott Card. These depict worlds in which computer networks are in widespread use, but in which many people choose to participate through pseudonyms. In this way they can make unpopular arguments or participate in frowned-upon transactions without their activities being linked to their true identities. [...] based on the quality of their ideas, rather than their job, wealth, age, or status. - Hal Finney, 1993

Trolls and Frauds!

Even in the 80s, it was well-known, that even a single troll can post 100 useless spam messages per day to a forum, and start infantile flames and run-amok postings, such as racism and sexism, sometimes in a hit-and-run manner. It was a major annoyance on Usenet. But it was not really perceived as a problem per se, because a kill file can always be used on Usenet.

However, if our community is completely anonymous, the problem will be greater. It was expected that most community in its current shape have extremely a low Signal-to-Noise ratio. It was also expected that commercial services, such as mail or streaming, can be provided and be purchased and used by the users in a privacy-preserving way through this anonymous network - so the same problem of fraud would be even greater during financial transactions.

On the Cypherpunks list, I often take "anonymous" messages less seriously. They're often more bizarre and inflammatory than ordinary posts, perhaps for good reason, and they're certainly harder to take seriously and respond to. This is to be expected. (I should note that some pseudonyms, such as Black Unicorn and Pr0duct Cypher, have established reputable digital personnas and are well worth replying to.) - Tim May, 1994

Reputation Matters

The solution? Pseudonyms with reputation.

The idea here is that the ultimate solution to the low signal-to-noise ratio on the nets is not a matter of forcing people to "stand behind their words". People can stand behind all kinds of idiotic ideas. Rather, there will need to be developed better systems for filtering news and mail, for developing "digital reputations" which can be stamped on one's postings to pass through these smart filters, and even applying these reputations to pseudonyms. In such a system, the fact that someone is posting or mailing pseudonymously is not a problem, since nuisance posters won't be able to get through. - Hal Finney, 1993

But what are "reputations" and why are they so important? Tim May said: it's a vague concept related to degree of believability, of trust. We know it when we see it. It's obvious in ordinary life, but in the cyberspatial context, we can have reputation-based systems, such as escrow, things based on expectations, "reputation capital". Book or music recommendations can be seen as a form of it, too. It's also crucial to commercial activities online, though rating and filtering agents and mechanism. Actually it's very common: how most of us deal with our friends, our enemies, the books we read, the restaurants we frequent, etc.

and so what? The world is filled with disinformation, rumors, lies, half-truths, and somehow things go on [...] Reputations will be of central importance, far more important in dealings than even the credit ratings of today. - Tim May, 1988.

Creation of Web-of-Trust

Confidentiality is the pillar of modern cryptography, but we cannot have it without authentication. After all, if a scammer could misrepresent his/her identity as a legitimate merchant, whether the communication is confidential or not does not matter. Thus, verification is central to cryptography and trust in general.

A common method of establishing verification is through a trusted third-party, it can be individuals or institutions. As Bruce Schneier said in Applied Cryptography, "The concept of an arbitrator is as old as society. There have always been people—rulers, priests, and so on—who have the authority to act fairly. Arbitrators have a certain social role and position in our society; betraying the public trust would jeopardize that. Lawyers who play games with escrow accounts face almost-certain disbarment, for example. This picture of trust doesn't always exist in the real world, but it’s the ideal." In cryptography, it is usually accomplished by a Certificate Authority.

Nevertheless, a trusted authority creates a single-point of failure in multiple ways. The burden of proof will be put on this authority, creating high workload for such an authority. A failure to presence can render the system unusable, and the compromise of the authority can undermine trust in the entire system. Also, most trusted authorities rely on checking one's physical proofs, thus unsuitable for most online communities and it's harmful to privacy.

During the development process of PGP, Phil Zimmermann proposed an alternative way to realize trust by using the trusts between individuals. If Alice trusts Bob, it's reasonable that the people who trust Alice will find Bob more likely to be trusted. We can expand this one-dimensional idea to create a much more useful one - by using a graph to represent all the trusts (and distrusts) in the system, we can obtain a Web-of-Trust (WoT). This system also has a more interesting property - it, in fact, is different than just "trust". Because each person has their own network of trust, the "trust" in the system is tensor, rather than scalar.

Case-Studies

Karma and User-Moderation

Karma is a simple mechanism to filter unpopular posts and/or to select interesting content in an online forum. Each user can upvote or downvote for a post. The posts will be displayed according to the votes they received. A user gains/loses 1 point of karma when a post is upvoted/downvoted by other users. A high karma implied the user has contributed interesting posts, and implied a good reputation.

A variant of this scheme is the user-moderation system found on Slashdot/SoylentNews. In this system, most users don't vote. Only a portion of selected users can assign a positive/negative point to a post, with a tag, such as insightful, funny, misleading, etc.

OpenPGP WoT

In public-key cryptography, a public key can be used to construct an encrypted message, which can only be decrypted by a person with its corresponding private key. Often, an identity for identification and communication, such as a nickname or E-mail address, is attached to the public key as additional information, sometimes called a certificate. They are published online and collected by directories. However, since a key/certificate can contain arbitrary information about its identity, so a Web-of-Trust is used to verify the ownership/validity of keys.

1. Each user is represented by a public key, only the owner has the private key.

2. The owner of a key can create and publish their signed statement to "certify" the validity of other public keys, with three certification levels. It is only used to indicate the beliefs of the validity of keys, it's not a indication of trustworthiness or moral judgement. Certification is automatically attached to the corresponding key when published.

• Personal belief but no verification.
• Casual verification.
• Extensive verification.

1. A certification can be withdrawn retroactively.

2. A user can decide how they personally trust the certification made by another user, with five trust levels. The "trust" is not published, and only used for local calculation. They can be changed at any time by the user.

• Don't trust.
• Marginally trusted.
• Fully trusted.
• Ultimately trusted.

1. A user always "ultimately trust" themselves, so all certification made by the user itself is considered valid.

2. When the user adds a new key, the system automatically evaluates the certifications from other users, based on the certification level and trust level of these certifications.

3. The new key is considered valid by the system, if it meets two conditions:

• One has signed it personally, or
  * it has been signed by one fully trusted key, or
  * it has been signed by three marginally trusted keys; and

• The path of signed keys leading from K back to your own key is five steps or shorter.

#bitcoin-otc

Bitcoin-otc is a peer-to-peer system for people to trade Bitcoin, or using Bitcoin to exchange goods and services. It is operated by an IRC bot at #bitcoin-otc IRC channel.

In the early days of Bitcoin, there were no trusted trading platforms, all tradings were needed to be conducted from person to person at an online community, the #bitcoin IRC channel was often used for this purpose. To facilitate trade, soon, an IRC bookkeeper bot was programmed to help people to track individual trades in the order book. Later, it became a separate community known as #bitcoin-otc. Web-of-Trust functionality is also added to the IRC bot soon after, with a web interface.

1. Each user is identified by a OpenPGP public key.

2. Users can give each other ratings between -10 and +10.

3. The system calculates the cumulative trust received by a user from all ratings.

From a technical aspect, the system is worth studying due to several unique properties.

• It works as a community instead of an automatic trading system. Although the IRC bot has the important responsibilities of keep tracks of all orders and maintain the WoT records, but the trades themselves are still negotiated and executed by the users. Ultimately, the IRC bot is only a bookkeeping aid, not a enforcer of the rule.

• IRC is never intended to perform tasks like this, it shows the open, minimalistic platform of communication can have high extensively and can be used in a way that is never intended by the original designers. The trading system is straightforward to setup by anyone, encouraging decentralization.

• Due to the nature of IRC, it allows a user to engage in commercial transactions while exposing minimum personal information. Especially during the early days of Bitcoin, there was no trusted trading platforms, all tradings were needed to be conducted from person-to-person at an online community, the system #bitcoin-otc fulfills the crucial role of establishing basic trust in an untrusted and hostile environment with many scammers, by a very simple and straightforward system.

Failure Modes

A reputation system can be abused in various ways, or fail to enforce its expected functionality in an unexpected way. Many studies and experiments are still needed.

Sybil Attack

The most common threat of any reputation system is a Sybil Attack. In a basic Sybil Attack, an attacker create a large quantities of identities under the control of the attacker, creating a false picture of a majority. Multiple variants of this attack exists, in some cases, the attacker can also forge identities. Actually, a Sybil Attack is the most common threat we face in various systems, and can render most good reputation schemes useless.

• In the Tor network, a Sybil Attack is used to overwhelm the routing selection algorithm to selects the attacker with a high probability. Similar issues exists in any systems where multiple source of information is rated and compared, and sometimes closely related to 51% attack. Publishing fake files and false information on a peer-to-peer file sharing system (KAD or DHT) can be an example. Historically, it was used by copyright interests groups to interfere filesharing, or by security researchers to attack P2P botnet for public interest.

• There's a big warning in #bitcoin-otc's reputation system's manual: by itself, it does not guarantee the trustworthiness, and may be exploited.

• Evil32 Attack of OpenPGP Web-of-Trust. In this attack, the attackers created false identities of every single user in the largest sub-community of users in Web of Trust. Not only misleading identities are created, even the trusting relations are replicated to confuse users.

• In a karma-based system, a Sybil Attack is known as a voting-ring. The members of such voting-ring always give a positive vote to each other, the voting may be voluntary by members of a subgroup, and not necessarily under the control of a single attacker. Historically, there are multiple examples, of which, votings were used for an organized protest or trolling campaign.

• Decentralization means there are no trusted third-party in the Fediverse, the attackers may be easier to take advantage of it. At bare-minimum, a user can trust the computer programs and its administrators of their own instance. But the trust stops at here. A malicious instance can broadcast misleading information, something we need to take a closer took at it.

Loose Ends

Other issues I can think of...

• "Positive reputation is better than negative reputations, because negative reputations can be discarded by pseudonym holders (like allowing a credit card to be used then abandoned with a debt on it)..." - Tim May, 1994.

• In OpenPGP's Web-of-Trust, a certification is based upon objectively standard - whether a key is real. And in #bitcoin-otc, it still has an objective definition - whether a transaction is completed smoothly. But when it comes to voting, there are several ambiguous aspects. In all online communities, we vote for posts, but almost NEVER vote for a person, for good reasons. But it's still common to downvote posts just for ideological reasons, even if the person engages in a reasonable standard, need some serious considerations here...

• If the reputation is used to regulate the propagation of posts (just for example), what standard should be used? Currently on social media, polarization and filtering bubbles is already a significant issue. It won't be good if voting becomes a tool for idealogical fights or popularity contest, etc...

• Power and decentralization. What are the effects the system will have to the neutrality or decentralization of the platform? Personally, I'm a firm believer of instance-rights, ultimately, all powers belong to an instance. I see the centralized approach to content moderation a failure on most social media platforms, because people will never have consensus beyond basic issues, and often ideology is involved. Decentralization provides a better solution, sometimes win-win: take the recent sex censorship as an example, there are always enough people who want strong censorship, weak censorship or no censorship. each instance can have its own standards to achieve localized consensus, I don't want to see this content, you don't want to see that content, yet we often can coexist peacefully on different instances. But if "internet points" (a common joke for karma and reputation) is introduced, it's hard to say, yeah, to maintain the platform neutrality, we can build a more powerful system, where you can also have reputation for instances calculated by each instance individually, etc, etc, but I suspect the finally outcome is something like OpenPGP's web-of-trust - advanced features are so complicated that nobody uses it.

General Conclusions

• Whether this idea is suitable to be incorporated to ActivityPub/Mastodon in general is a still an open question, I find the idea of hardwiring a very specific reputation system into ActivityPub/Mastodon is questionable. As a neutral social media platform, it should provide *mechanism, not policy. * But I think being an open platform, it's very desirable to conduct experiments based on ActivityPub/Mastodon, as plugins or extensions perhaps.

• At least, initially, the system should provide guidance at user/instance/individual-level, it should not be a technocratic overlord. Personally, for me, it should be, and only should be a guidance.

Imagine you're in a society without a significant government at all, it's just always been that way. And someone proposes changing to some democratic model.

Well.. Many believes it's feature of the Internet. The 1985 science fiction The Ungoverned by Vernor Vinge comes to the mind... But, clearly, I agree that governments derive their just powers from the consent of the governed, and obviously many would prefer democracy and may want a strong enforcement of it. So how powerful the reputation system is can be a matter of per-instance policy. Personally, I'm always suspicious to any form of governance, so on my personal instance, I may want the reputation system is simply highlight/label questionable content. On a small instance, if the user agrees, administrators can use stronger-mode of enforcement, such as banning it.

• There's little innovation of social media since 2000. Upvote and Karma was a major innovation at that time in forums, following/followers based microblogging was the next. And now we've stopped at here. I believe a better reputation system is the next important innovation towards a more usable social media system.

• Overall, reputation system is still very promising and needs lots of "real-world" research. The author of this bug report has the time and energy, I strongly suggest the author to build a FOSS community dedicated to the research and implementation of reputation-based social media, and pull all the people and discussions to the community.

Congratulation, you've made it to the end! To all the readers, I hope my terrible writings have provided you some insights...

[BrianPansky]

Thanks, bmvjssdkco.

I just want to reply to two key points:

• The "Sybil Attack"
• and the implementing of consequences

The "Sybil Attack"

I dismissed the suggestion by trwnh that 100 fake accounts would be some kind of problem (beyond the fact that us humans have always been able to trust the wrong people).

I should give mathematical demonstrations of what would happen in various scenarios.

But, for now, wikipedia notes:

A reputation system's vulnerability to a Sybil attack depends on how cheaply identities can be generated, the degree to which the reputation system accepts inputs from entities that do not have a chain of trust linking them to a trusted entity, and whether the reputation system treats all entities identically.

And here is an article that talks about using each of those points to prevent Sybil attacks.

It should be clear that the system I'm proposing has exactly those second two safety features: requiring a chain of trust and, through evaluations, not treating all accounts identically.

One relevant feature of my calculation I suppose I didn't explicitly mention is that the estimates that a user receives will always contain the evaluations they themselves have made, unaltered. This is another way that not all accounts are treated identically.

Consequences

I haven't proposed any consequences.

But any that get implemented should operate in the same way: each user individually decides whether or not people below a certain "score" are muted, blocked, or whatever, and what that score should be.

Just like individual users currently can manually decide to mute or block others. It should be up to them.

[witcheslive]

A good reputation scheme I find is that you pay attention to what goes on on your instance and also to what is going on in the communities you closely federate with, and you take action against people and accounts who don't respect boundaries or otherwise are acting in bad faith. This is a lot easier than recreating Black Mirror but for real. Well, I hope this helps.

[coreyreichle]

I think a number of people here downvoting the idea don't really understand how a web of trust works.

1000 (Or one million) bots with a great reputation amongst themselves isn't very useful. Other people (Real people) would have to trust those bots too, either via transitive trust relationships (ie, I trust Fred, Fred trusts Alice, therefore, I trust Alice).

Those attacks are muted somewhat by lowering your trust value the further away from you that you get in the web of trust (Ie, each step halves the trust level, or a more geometric or exponential fall off).

So, in order for a bot army to have a good reputation, many people, with good reputations, must also trust each entity of the bot army. And, if that were to happen, the bot army would have to be providing content that most people trust anyways, so it'd be a moot point.

This is basically the same way GPG key signing parties work, with the added "weight" value (GPG is only one type of web of trust, trusting identity of key owner).

[Laurelai]

So, in order for a bot army to have a good reputation, many people, with good reputations, must also trust each entity of the bot army. And, if that were to happen, the bot army would have to be providing content that most people trust anyways, so it'd be a moot point.

I think you underestimate the power of bribery. It would be arbitary for someone to hire people to gain trust within networks and then use them as "influencers" to cosign bad actors

[coreyreichle]

You cannot fix bribery, at all. Except, to not trust people you think would accept bribes, and to be able to update how much you trust a person discovered to accept bribes.

What we can do, however, is fix what we can. A web of trust, with a decay algo can be useful, especially so for propaganda (most people don't trust bots, and if they do, you're probably not going to trust that person for long), internet bullying (ie, who trusts a bully that you also trusts?).

Adding in negative trust ratings would assist with this as well. I wouldn't do a scale, but a thumbs up/thumbs down can serve well.

[Laurelai]

internet bullying (ie, who trusts a bully that you also trusts?).

Lots of people trust bullies. Thats how they get away with bullying online. They frame their bullying as a righteous cause. Ive experienced this and seen it happen to others.

[coreyreichle]

And that's fine if THEY trust bullies. Do YOU trust the people who trust bullies.

That's how a transitive web of trust works.

[Laurelai]

And that's fine if THEY trust bullies. Do YOU trust the people who trust bullies.

Its happened before, me not knowing they trusted the bullies until they attacked me. The cold truth is you shouldnt really trust anyone and having artificial gamable trust systems are not only not helpful, they are harmful.

Like ive seen webs of trust exploited numerous times.

[coreyreichle]

And, if you trust nobody, that's fine too! Your "web of trust" extends to 1: You alone.

It's not really game-able. People have tried to game web of trust systems and failed, because you have to game everyone in order to become a trusted person in a community.

And, if they can do that, then they're not gaming a technical solution, they have become the community. Which the other alternative of "Not having a reputation system" doesn't fix, either.

[coreyreichle]

As far as this solution goes, what I propose for a transitive trust solution: A simple thumb up, or thumbs down on each toot. Thumbs up adds a +1 to a trust score for that account (For the individual account), thumbs down adds a -1. Each user builds their own trust score for individual users, and ActivityPub should be able to transmit the trust scores each user has compiled to each follower.

Based on if you positively trust a user, their supplied score is added at 0.5 multiplier (Half their score). If a negative score, then subtracted at -0.5 multiplier.

If you don't know a user (Never trusted or untrusted them), the graph can be traversed if any of the people you follow have a score. That is treated as what their score is with a .25 multiplier (Or, neg 0.25). If you have to walk one more, then .0825, etc etc.

The allows for a decaying amount of trust (I trust a friend, but less so a friend of a friend, etc).

[Laurelai]

People have tried to game web of trust systems and failed.

And people have gamed webs of trust and succeeded. What is your point? Some successes doesnt prove anything, the failures on the other hand do. Let me tell you what would happen if we instituted a web of trust on mastodon. The majority there (read white, male, likely a developer) would down rate marginalized people, the mentally ill, the poor, etc. Their biases would be formalized into a structure that actually would make life harder on the oppressed.

Meanwhile corporations and state actors would slowly infiltrate your system.

Your precious web cant account for this.

[coreyreichle]

You cannot game a transitive web of trust.

The majority there can trust each other. And the minority can trust members of the minority group member. And maybe some gossamer threads of trust connect each other.

The trust score would be a score of how much YOU trust someone. Not how much the community trusts someone. Your trust score for my account would be different for say, BrianPansky's trust score of me, since we have a different social graph.

There is no centralized trust score for each user, hence the "web of trust".

The alternative is "Do nothing", which doesn't solve the problems you've described, either.

[Laurelai]

The majority there can trust each other. And the minority can trust members of the minority group member. And maybe some gossamer threads of trust connect each other.

Which is the problem i just said and you are now saying is a feature.

You cannot game a transitive web of trust.

Assuming it cannot be gamed will only garuntee it.

[coreyreichle]

Yes, individuals choosing who they trust, or don't, is a feature.

The only way to game a transitive trust system is to game the entire community. At which point, no technological problem can solve that, as it's a social problem.

[BrianPansky]

Another galaxy-brain scenario to imagine: the "invention" of language. Before that, people had to rely on direct experience. And there were no lies about other people (only fake personal behavior, like smiling when you aren't happy). With the invention of language, it became possible to say that someone else was good or bad. Scary, perhaps. But very useful.

Now just go beyond the invention of language.

[Laurelai]

Trust is a social problem yes. You have gotten to the core of the issue. You are trying to solve a social problem with technology.

[coreyreichle]

This isn't trying to determine truth. Web of trust is a way to measure (Objectivley) how much you trust another individual, nothing more, nothing less.

If you are uneducated, and fall victim easily to scams, you trust everyone. But, most people would not trust the person that falls victim to scam after scam after scam.

ie, if I believe Person A, because we drink together, and s/he vouches for Person B, I will trust Person B ~1/2 as much as I trust Person A (Who I drink with). If Person C, that I golf with also trusts person B, then I trust person B about 3/4 the amount I trust A and C.

Of course, more than X steps away, is meaningless. Friend of a friend of a friend is near worthless, as far as trust goes, to me.

[Laurelai]

This isn't trying to determine truth. Web of trust is a way to measure (Objectivley) how much you trust another individual, nothing more, nothing less.

Ok? Why? If its simply measuring the trust you have for someone else then its useless since you already know how much you trust someone or dont.

[BrianPansky]

It's faster and more efficient than asking everyone you know to tell you about everyone they know, and so on. And then doing the math by hand or something. And then being questioned by everyone you know (who is also repeating the same process).

[Laurelai]

So its not "a way to measure (Objectivley) how much you trust another individual, nothing more, nothing less." then. It involves other people and thus can be gamed.

[BrianPansky]

Those two things aren't in contradiction. Who you "objectively" trust can be "gamed" even now, without using this tech.

[Laurelai]

And this will make it easier.

[BrianPansky]

I think it will make weeding out lies easier. Lies rely on ignorance, and many hands (doing the "auditing") make light work, and all that.

Admittedly it's hard to prove which side will benefit more.

[Laurelai]

Lies dont rely on ignorance, they rely on power. People who know lies are lies will promote them if they get something out of it. People will adopt lies as truth even if they know its a lie if powerful people they respect promote the lie. Also repeating a lie enough times gets people to believe it.
Its a common fallacy that we can just stop propaganda by disproving it. That never works. If it did then the propaganda arms of nearly every autocracy and some so called democracies would not have done nearly as well as it did.

Propagandists are adaptive because they are people. They adapt. A web of trust exploiter wouldnt look like the fake news peddlers we see now.

They would look like your friend who has a few ideas for you to consider who you totally dont have to agree with, were friends after all, but we should be able to debate the issue right?

[BrianPansky]

Sure, power is another factor in the spread of misinformation. I think strong-arming would be _harder_ with a massively distributed system, with no clear or fixed hierarchy.

[Laurelai]

This wont have a distributed system with no fixed hierarchy. This will be codified social capital with a hierarchy based on existing social biases.

[BrianPansky]

What makes you think that?

[Laurelai]

What makes you think that?

Im a marginalized person who has lived experience with this sort of thing and technology created for these purposes almost inevitably replicates and reifies existing power structures while using the fact that its a computer doing it as an excuse to not examine their own biases.

Quite frankly the reactions is as predictable as a billiard ball path on a table.

[BrianPansky]

technology created for these purposes almost inevitably replicates and reifies existing power structures

What technology do you mean?

[Laurelai]

You know what i mean. Dont play.

[BrianPansky]

Huh? I mean your description sounds a bit like how machine learning can be noobishly (or maliciously) used, is that the example you had in mind? Any others?

[lukepighetti]

One possibility is a spinoff of "Proof of Authority" which registers notaries/lawyers as platform moderators and makes their identities public. (Usually in return for tokens, its not clear to me how this would work on a platform that does not offer crypto rewards such as Mastodon.)

Perhaps those moderators could vouch for people as trustworthy, who can then vouch for other people, and allow that trust to propagate.

Just throwing out ideas, never really thought about this before.

[coreyreichle]

I think rather than auto-trusting moderators (Even though, to a point every user should), root of trust should start with the user.

But, jm2c.

[jelutz77]

I like the idea of having some sort of score weighted by peer proximity. For example, a user who follows the same people you do should have more weight in the scores you see for someone else. This will not only encompass "truth" in posting, but also how likely you are to come up with the same conclusions as your peers, and help identify subjects of interest in common with your peers.
However, I'm not in favor of blocking people with this score.

[trwnh]

having some sort of score weighted by peer proximity [...] This will not only encompass "truth" in posting, but also how likely you are to come up with the same conclusions as your peers

that leads to groupthink.

but anyway, there shouldn't be a "score" aspect in the first place. arguably the only aspects worth considering are "distance" (#11163) and "trust" (a boolean value, indicating the minimum possible value judgement so as not to colour perceptions)

[jelutz77]

Yes, I suppose it would be a cue for groupthink. However, it should be a much better cue than rumor, which is what would be the cue without such cues. Many of us are looking for better cues that save us some work in verifying sources, but reject cues from those who think much more differently than us, such as Google and Facebook. I believe that this is the crux of the matter.

I also fail to see much difference between our proposals. Yours deals with a Boolean trust factor, mine with a more granular score based on degree of similarity with peers. I find your idea of distance a good one for further weighting along with similarity. I see both our algorithms enabling groupthink, but as I stated before, I believe they’re both better for diversity than rumor or some monolithic algorithm like Google or Facebook.

[trwnh]

you can't save work in verifying sources.

and by boolean, i mean "it is true that user a trusts user b" or "it is false that user a trusts user b", e.g. by having some trusts or trusted collection. or something similar to the "featured users" functionality. the point is that the value judgement should be as small as possible, and doing scoring/weighting is not minimal enough.

for example, simply stating the network distance ("degrees of separation"), based on who follows whom, does not meaningfully lead to groupthink, because the value judgement is separated from the metric. it is only additional information that can help you to come to your own judgement (e.g. "this person is a friend-of-a-friend and x generally has good judgement in making friends" or "this person is some rando, why are they talking to me?"). it could lead to a slight bias toward interconnectedness, but imo not meaningfully moreso than incomplete federation already has.

[jelutz77]

I suppose our algorithms are similar thin, in beginning with collecting Boolean values and interpreting them. In reality, such algorithms would have binary domain and range, assuming a standard listing. Even with a score, either it’s on the first page or it’s not. My algorithm simply assumed one level of trust, though rather than trusting friends, trusting someone who trusts the same people or posts that you do.

[BrianPansky]

you can't save work in verifying sources.

Of course you can. That's the whole point of trusted journalists and scientists, we trust they will do it for us, so that hundreds of thousands of us don't have to all waste our time doing the same process of verification. And this useful system can be improved or degraded.

the only aspects worth considering are "distance" (#11163) and "trust" (a boolean value, indicating the minimum possible value judgement so as not to color perceptions)

That's false. Trust isn't boolean, it's a percentage, and distance only matters in relation to trust.

Using proxy measures is worse, it's literally how you get a game that can be gamed.

Use trust as percentages, then multiplying along a chain ("distance") automatically leads to the decay of trust. I'm not just pulling that out of a hat, that's how the (common) importance of "distance" emerges in the first place.

[Gargron]

You might be interested in http://mastodon.social/@Eunomia, a research project I consult that touches on this topic. I doubt that anything within this discussion will become part of Mastodon core, but a separate project like Eunomia might address some of the things.

[tmbdev]

I trust Brian Greene when he makes statements about string theory, but I have no reason to trust him on other subject areas, or that the people he chooses to follow are trustworthy. How does your score account for that?

[BrianPansky]

I have no reason to trust him on other subject areas [...] How does your score account for that?

What I've outlined would only implicitly "account for" that, by way of the trust percentage. i.e. what percentage of the time is he right when he says something? If he's spending a lot of time talking about subjects he's ignorant of, then you ought to account for that yourself when you rate him.

A more advanced system would have more dimensions. The system would work similarly on all dimensions, but they would be about different things (such as string theory VS other subjects).

or that the people he chooses to follow are trustworthy. How does your score account for that?

I'm guessing you're concerned about the "propagation" aspect?

Again, this is implicitly collapsed into the single trust dimension in the outline of my proposal above. And the more advanced option would be to have another _separate_ dimension where you evaluate _how good that person is at evaluating others_.

etc.

I'm not coming up with anything genius here. This is basically "brute force" copying the correct way we already know how to handle these things in our own heads. The point is to scale that capacity up using technology (while also hopefully making it a bit more user-friendly than the dizzying array of mouse-clicks that would be required for a fully accurate copy, and collapsing a few gazillion dimensions is a useful self-correcting way to do that).

[tmbdev]

I'm not coming up with anything genius here. This is basically "brute force" copying the correct way we already know how to handle these things in our own heads.

Take any person from public life, and when you ask 1000 people about how much they trust them on a particular topic, you will get a wide distribution of responses. It seems pretty clear to me that in the real world, the trust one person has in another person (even a public figure) is often very different from the average trust score. So it seems to me that the propagation of a single, average trust score for every user is very different from the way humans handle trust.

If you want to reuse existing technology for trust, it seems to me that "trust" should be treated more like a recommender system. Just like there are some people who like SciFi and others who hate it, there are some people who will trust politician X and others who won't. If you try to reduce that into a single number, you are going to get into conflicts where people are trying to artificially game the scores to agree with their personal preferences. These are the irreconcilable differences that have been hurting centralized social networking platforms so much.

I see two possible solution. First, one could have a system of trust scores similar to what you propose, but the scores are computed separately for different communities; maybe trust scores are computed separately by each instance, or people can join like-minded "communities of trust" or group or separate "trust metric server". Alternatively, one could go "full recommender system". For a recommender system, a few choices that you make then let the system compute other choices that agree with your preferences. Translated into trust that means that by indicating a few other users you trust, the system can then predict other users you might trust as well.

I think having a "trust metric server" separate from instances would be the best choice; people would send their interactions to a server they trust in a community they trust and the trust metric server would then return recommendations on new users based on the interactions of everybody in the same community. Initially, such a trust metric server could use simple statistical methods, but later, it could use AI-based and natural language based scoring. Dividing up the problem like this would both allow for diversity of viewpoints and future innovation.

[BrianPansky]

I think you have misread or misunderstood my proposal.

So it seems to me that the propagation of a single, average trust score for every user is very different from the way humans handle trust.

yes, that's why the scores are NOT averaged in my proposal.

it seems to me that "trust" should be treated more like a recommender system.

That is what my proposal is like.

Just like there are some people who like SciFi and others who hate it, there are some people who will trust politician X and others who won't.

this is exactly why the system I propose _would_ most likely give such people a different estimate of that same politician, and certainly allows the users to manually input different trust values in for that politician if they choose to.

I see two possible solution. First, one could have a system of trust scores similar to what you propose, but the scores are computed separately for different communities

yes, my proposal is to compute scores separately _for each individual user_, just like each individual's brain already does. (communities will emerge naturally from that, and could even be used to cut down computation time)

Read where I said: "the estimates each user receives are simply what is logically entailed by their own evaluations."

Not averaged. Not the same "view" for every user.

Maybe my phrasing is unclear because it doesn't mention how the evaluations made by _other_ users come into play at all. They are used as reports that are only trusted as strongly as that reporter is trusted by the person asking for the report. So the report is a percentage, and the reporter is trusted by a percentage. So the percentages are multiplied and stuff like that, with upper and lower estimates. Multiple conflicting reports are combined not by averaging, but by assigning a weight to each report equal to how much that reporter is trusted by the person asking for the report.

And I have the nonlinear calculation to do this basically finished. I could show you what results it gives for any user in any example network and set of evaluations.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.