The ActivityPub specification was unclear concerning things like webfinger or things like the .well-known path. So for other implementations, it will be good to see how Mastodon did the implementation and to adopt it the same way to keep the compatibility.
So the two questions are:
I'm one of the core developers of Friendica and specialized in our protocol implementations. I can work best with examples and analyzing and mimicking other systems, rather than reading theoretical documentations - especially when they aren't in my native language. The AP documentation is very theoretical, so I hope that I can fill the understanding gaps with practical data from Mastodon.
How can another server detect if a Mastodon server is able to handle AP?
What is Mastodon doing if the AP implementation on the other system isn't complete?
Mastodon will ignore actors without WebFinger (it performs verification for username@domain ownership), and Mastodon will ignore actors without an inbox.
Mastodon will ignore inbox deliveries that do not have a HTTP signature (unauthenticated). The HTTP signature keyId may be the actor URL or the public key URL. If it's the public key URL, public key and actor must link to each other (owner/publicKey).
For legacy, OStatus purposes the keyId can also be [acct:]username@domain, since OStatus used to return magic key in WebFinger.
@annando I've got working code in hubzilla-addons which you can examine and possibly adapt to Friendica; or at least you can read it. We've got a few minor issues with Mastodon that I'm currently investigating: they're presumably fetching a modified actor url that we didn't supply (rewriting https://example.com/channel/foo to https://example.com/channel), and re-writing some of our object ids using Mastodon semantics (breaking them); but overall it's moving forward quickly.
@zotlabs would be interested in hearing about these bugs if you want to file a ticket for them.
Was reported to me and I'm still investigating so don't have firsthand knowledge and I'm pretty slow reading Ruby. The second issue can be seen here:
2017-09-12T08:17:47Z:LOG_INFO:c21e071fd1976dbc7695fc6a530a396d:Mod_Inbox.php:31:post: inbox_activity: {
"@context":[
"https://www.w3.org/ns/activitystreams",
"https://w3id.org/security/v1",
{
"manuallyApprovesFollowers":"as:manuallyApprovesFollowers",
"sensitive":"as:sensitive",
"hashtag":"as:Hashtag",
"ostatus":"http://ostatus.org#",
"atomUri":"ostatus:atomUri",
"inReplyToAtomUri":"ostatus:inReplyToAtomUri",
"conversation":"ostatus:conversation"
}
],
"id":"https://mastodon.social/users/motesting4#accepts/follows/24130",
"type":"Accept",
"actor":"https://mastodon.social/users/motesting4",
"object":{
"id":"https://hub.somaton.com/channel/mario#follows/24130",
"type":"Follow",
"actor":"https://hub.somaton.com/channel/mario",
"object":"https://mastodon.social/users/motesting4"
},
"signature":{
"type":"RsaSignature2017",
"creator":"https://mastodon.social/users/motesting4#main-key",
"created":"2017-09-12T08:17:47Z",
"signatureValue":"qmhhtwhYdcm+J994JyocbcIViz215WrIWQnn4k9UaQg4eAarK8cTRf0DFV6MHqtC4iyRfS+6LC3AfdJ0yd00jlnAhpp5r81PMTrfYgACYQBu3JhXqIa06LByeasyLB4BOjYRnQeoYtPIiH8HBYCbQSQcqt7yFyXvsoDvMcNbv5FJdrlJDCSfXXtYmgvX2mjkjm9XRcH/t50Ndnybq4rH1ESc1dGMXQSIYnbFz52JXbkgGYGItTkybB7146R61GSQRpRdkpo6w2txKpvKB2NbDob5nNR+WR/xG7Xh/e2Pi6Er6mDe4TL7noh6WZp06WUbDFHUTeShbskAAzJHQ0SquQ=="
}
}
The object id should have been of the form https://hub.somaton.com/follow/nnn but was re-written to a Mastodon form. We need it to be the original id to match against the original request we made.
The first issue I don't have the actual logs, but we send out Follow requests which look like this:
{
"@context":[
"https://www.w3.org/ns/activitystreams",
"https://w3id.org/security/v1",
"https://hz.macgirvin.com/apschema"
],
"id":"https://hz.macgirvin.com/follow/186",
"type":"Follow",
"actor":{
"type":"Person",
"id":"https://hz.macgirvin.com/channel/hubzilla",
"preferredUsername":"hubzilla",
"name":"Hubzilla",
"icon":{
"type":"Image",
"mediaType":"image/jpeg",
"url":"https://hz.macgirvin.com/photo/profile/l/2",
"height":300,
"width":300
},
"url":{
"type":"Link",
"mediaType":"text/html",
"href":"https://hz.macgirvin.com/channel/hubzilla"
},
"inbox":"https://hz.macgirvin.com/inbox/hubzilla",
"outbox":"https://hz.macgirvin.com/outbox/hubzilla",
"endpoints":{
"sharedInbox":"https://hz.macgirvin.com/inbox/[public]"
},
"publicKey":{
"id":"https://hz.macgirvin.com/channel/hubzilla/public_key_pem",
"owner":"https://hz.macgirvin.com/channel/hubzilla",
"publicKeyPem":"-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA18JB76lyP4zzL/y7BCej
eJnfZIWZNtM3MZvI1zEVMWmmwOS+u/yH8oPwyaDk4Y/tnj8GzMPj1lCGVRcd8EJa
NrCMd50HODA5EsJtxpsOzRcILYjOcTtIAG1K4LtKqELi9ICAaFp0fNfa+Jf0eCek
vPusx2/ORhy+o23hFoSMhL86o2gmaiRnmnA3Vz4ZMG92ieJEDMXt9IA1EkIqS4y5
BPZfVPLD1pv8iivj+dtN1XjwplgjUbtxmU0/Ej808nHppscRIqx/XJ0XZU90oNGw
/wYoK2EzJlPbRsAkwNqoFrAYlr5HPpn4BJ2ebFYQgWBUraD7HwS5atsQEaxGfO21
lUP0+lDg9t3CXvudDj0UG1jiEKbVIGA+4aG0GN2DSC5AyRq/GRxqyay5W2vQbAZH
yvxPGrZFO24I65g3pjhpjEsLqZ4ilTLQoLMs0drCIcRm5RxMUo4s/LMg16lT4cEk
1qRtk2X0Sb1AMQQ2uRXiVtWz77QHMONEYkf6OW4SHbwcv5umvlv69NYEGfCcbgq0
AV7U4/BWztUz/SWj4r194CG43I9I8dmaEx9CFA/XMePIAXQUuABfe1QMOR6IxLpq
THG1peZgHQKeGz4aSGrhQkZNNoOVNaZoIfcvopxcHDTZLigseEIaPPha4WFYoKPi
UPbZ5o8gTLc750uzrnb2jwcCAwEAAQ==
-----END PUBLIC KEY-----
"
},
"nomadicLocations":[
{
"id":"https://hz.macgirvin.com/locs/hubzilla",
"type":"nomadicLocation",
"locationAddress":"acct:[email protected]",
"locationPrimary":true,
"locationDeleted":false
}
]
},
"object":{
"type":"Person",
"id":"https://hz.macgirvin.com/channel/doper",
"name":"doper",
"icon":{
"type":"Image",
"mediaType":"image/png",
"url":"https://hz.macgirvin.com/photo/b7f5a3c7d660642ef5ef4b701b89fa3d-4",
"height":300,
"width":300
},
"url":{
"type":"Link",
"mediaType":"text/html",
"href":"https://hz.macgirvin.com/channel/doper"
},
"inbox":"https://hz.macgirvin.com/nullbox",
"outbox":"https://hz.macgirvin.com/nullbox"
},
"magicEnv":{
"id":"https://hz.macgirvin.com/follow/186",
"meData":"eyJAY29udGV4dCI6WyJodHRwczovL3d3dy53My5vcmcvbnMvYWN0aXZpdHlzdHJlYW1zIiwiaHR0cHM6Ly93M2lkLm9yZy9zZWN1cml0eS92MSIsImh0dHBzOi8vaHoubWFjZ2lydmluLmNvbS9hcHNjaGVtYSJdLCJpZCI6Imh0dHBzOi8vaHoubWFjZ2lydmluLmNvbS9mb2xsb3cvMTg2IiwidHlwZSI6IkZvbGxvdyIsImFjdG9yIjp7InR5cGUiOiJQZXJzb24iLCJpZCI6Imh0dHBzOi8vaHoubWFjZ2lydmluLmNvbS9jaGFubmVsL2h1YnppbGxhIiwicHJlZmVycmVkVXNlcm5hbWUiOiJodWJ6aWxsYSIsIm5hbWUiOiJIdWJ6aWxsYSIsImljb24iOnsidHlwZSI6IkltYWdlIiwibWVkaWFUeXBlIjoiaW1hZ2UvanBlZyIsInVybCI6Imh0dHBzOi8vaHoubWFjZ2lydmluLmNvbS9waG90by9wcm9maWxlL2wvMiIsImhlaWdodCI6MzAwLCJ3aWR0aCI6MzAwfSwidXJsIjp7InR5cGUiOiJMaW5rIiwibWVkaWFUeXBlIjoidGV4dC9odG1sIiwiaHJlZiI6Imh0dHBzOi8vaHoubWFjZ2lydmluLmNvbS9jaGFubmVsL2h1YnppbGxhIn0sImluYm94IjoiaHR0cHM6Ly9oei5tYWNnaXJ2aW4uY29tL2luYm94L2h1YnppbGxhIiwib3V0Ym94IjoiaHR0cHM6Ly9oei5tYWNnaXJ2aW4uY29tL291dGJveC9odWJ6aWxsYSIsImVuZHBvaW50cyI6eyJzaGFyZWRJbmJveCI6Imh0dHBzOi8vaHoubWFjZ2lydmluLmNvbS9pbmJveC9bcHVibGljXSJ9LCJwdWJsaWNLZXkiOnsiaWQiOiJodHRwczovL2h6Lm1hY2dpcnZpbi5jb20vY2hhbm5lbC9odWJ6aWxsYS9wdWJsaWNfa2V5X3BlbSIsIm93bmVyIjoiaHR0cHM6Ly9oei5tYWNnaXJ2aW4uY29tL2NoYW5uZWwvaHViemlsbGEiLCJwdWJsaWNLZXlQZW0iOiItLS0tLUJFR0lOIFBVQkxJQyBLRVktLS0tLVxuTUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUExOEpCNzZseVA0enpML3k3QkNlalxuZUpuZlpJV1pOdE0zTVp2STF6RVZNV21td09TK3UveUg4b1B3eWFEazRZL3RuajhHek1QajFsQ0dWUmNkOEVKYVxuTnJDTWQ1MEhPREE1RXNKdHhwc096UmNJTFlqT2NUdElBRzFLNEx0S3FFTGk5SUNBYUZwMGZOZmErSmYwZUNla1xudlB1c3gyL09SaHkrbzIzaEZvU01oTDg2bzJnbWFpUm5tbkEzVno0Wk1HOTJpZUpFRE1YdDlJQTFFa0lxUzR5NVxuQlBaZlZQTEQxcHY4aWl2aitkdE4xWGp3cGxnalVidHhtVTAvRWo4MDhuSHBwc2NSSXF4L1hKMFhaVTkwb05Hd1xuL3dZb0syRXpKbFBiUnNBa3dOcW9GckFZbHI1SFBwbjRCSjJlYkZZUWdXQlVyYUQ3SHdTNWF0c1FFYXhHZk8yMVxubFVQMCtsRGc5dDNDWHZ1ZERqMFVHMWppRUtiVklHQSs0YUcwR04yRFNDNUF5UnEvR1J4cXlheTVXMnZRYkFaSFxueXZ4UEdyWkZPMjRJNjVnM3BqaHBqRXNMcVo0aWxUTFFvTE1zMGRyQ0ljUm01UnhNVW80cy9MTWcxNmxUNGNFa1xuMXFSdGsyWDBTYjFBTVFRMnVSWGlWdFd6NzdRSE1PTkVZa2Y2T1c0U0hid2N2NXVtdmx2NjlOWUVHZkNjYmdxMFxuQVY3VTQvQld6dFV6L1NXajRyMTk0Q0c0M0k5SThkbWFFeDlDRkEvWE1lUElBWFFVdUFCZmUxUU1PUjZJeExwcVxuVEhHMXBlWmdIUUtlR3o0YVNHcmhRa1pOTm9PVk5hWm9JZmN2b3B4Y0hEVFpMaWdzZUVJYVBQaGE0V0ZZb0tQaVxuVVBiWjVvOGdUTGM3NTB1enJuYjJqd2NDQXdFQUFRPT1cbi0tLS0tRU5EIFBVQkxJQyBLRVktLS0tLVxuIn0sIm5vbWFkaWNMb2NhdGlvbnMiOlt7ImlkIjoiaHR0cHM6Ly9oei5tYWNnaXJ2aW4uY29tL2xvY3MvaHViemlsbGEiLCJ0eXBlIjoibm9tYWRpY0xvY2F0aW9uIiwibG9jYXRpb25BZGRyZXNzIjoiYWNjdDpodWJ6aWxsYUBoei5tYWNnaXJ2aW4uY29tIiwibG9jYXRpb25QcmltYXJ5Ijp0cnVlLCJsb2NhdGlvbkRlbGV0ZWQiOmZhbHNlfV19LCJvYmplY3QiOnsidHlwZSI6IlBlcnNvbiIsImlkIjoiaHR0cHM6Ly9oei5tYWNnaXJ2aW4uY29tL2NoYW5uZWwvZG9wZXIiLCJuYW1lIjoiZG9wZXIiLCJpY29uIjp7InR5cGUiOiJJbWFnZSIsIm1lZGlhVHlwZSI6ImltYWdlL3BuZyIsInVybCI6Imh0dHBzOi8vaHoubWFjZ2lydmluLmNvbS9waG90by9iN2Y1YTNjN2Q2NjA2NDJlZjVlZjRiNzAxYjg5ZmEzZC00IiwiaGVpZ2h0IjozMDAsIndpZHRoIjozMDB9LCJ1cmwiOnsidHlwZSI6IkxpbmsiLCJtZWRpYVR5cGUiOiJ0ZXh0L2h0bWwiLCJocmVmIjoiaHR0cHM6Ly9oei5tYWNnaXJ2aW4uY29tL2NoYW5uZWwvZG9wZXIifSwiaW5ib3giOiJodHRwczovL2h6Lm1hY2dpcnZpbi5jb20vbnVsbGJveCIsIm91dGJveCI6Imh0dHBzOi8vaHoubWFjZ2lydmluLmNvbS9udWxsYm94In19",
"meDataType":"application/activity+json",
"meEncoding":"base64url",
"meAlgorithm":"RSA-SHA256",
"meCreator":"https://hz.macgirvin.com/channel/hubzilla/public_key_pem",
"meSignatureValue":"wFOzuUbjrgvDuFuagPUaluswlwrLov4ZQv9XYioJKM35h7ywrva6_lU90XpfebTNg-n6Z7I3GrsRT7BAJNRs0i_TIRzjGgtitLmghtdDwohiV98htLRg4yjVWYpT9ynZM5VX2uppbYoGB6XcjU6Nmak7pM2Zh0wypGzWX43-rGDi3am82MJMynXK2b2KVYcCGHZra_ZKO_0Duj1NjmCt7qIEZA71JZWRIwTPNZHiAxEG2NuaoWCiqlSOXAFpgDpoxRIMCl2uHAFFUfspiPf9WUxrqDeZnTIO5Igk1XQfqTZ-GLp_1W4EXHGTv6PIOzcHqxCdda9pw_iZrVtBV8-Oi5qkS6m_lnuKs_UEcFcpE1W7q0YvA2PhbVXxmrJT6OcBR8oUVrIe-_sHRaEYC8bhOLKipm3Yzl4Qu-j61j225UcEhI8q16yZgk6jleDxDTEWbQPhpYTRpTE505jxPe2c9XtNAMi9bxjAp6YmZ_3uFCzSJaXf4qlRP6dbNeMEVeCtXsy-iGZSX2wvms8cdSjaEIANgqiv0eLe6H41Wezhdo2fSB_GxKhrPtSLZRVakorH4hiwbksqTfwmxuKSBqVfKXIEhFoVBj3zAoWa_-XJcEXBuJfjdWdvP1a6ZOTb-F4dH5kVauYYhRfoa8Z3UJotHt4s34DddWHhzBD-s3-DF-4"
},
"signature":{
"@context":[
"https://www.w3.org/ns/activitystreams",
"https://w3id.org/security/v1"
],
"type":"RsaSignature2017",
"nonce":"d43587d46833a14348d2ccb2597fb4219fb07f63856d804f228784063cdc6a37",
"creator":"https://hz.macgirvin.com/channel/hubzilla/public_key_pem",
"created":"2017-09-13T12:56:41Z",
"signatureValue":"WGNyNA9ffkism4KCFhyYA6vhkkD0yLDhaDKdOXqR/q1nsW16HWFkqOr0qio3XBzPTbko6ettIOLE4yV8cb2ptIoB7qD+cSbQp6hzm5Jfle+3BTDCE89amHNT5gGy797FQh4R7/Opj/k9SE1kRqgHwQMAiCDI7gyiV1bUkCwbPOWrWZKNsgXv6JSZzkYaEZWsAvydqo/O00wbO7/vviuBI9gDJLCIPVtAksETyahCJiFvysXdTK8E/ZUknqJbfh7fx6+kxI2WfkClcZUH/QnRV3Yn9WlYvgxJo4vKnrnpQhAi+xmXL2WDHeKCAqzIrPhqGfP0WcjULiSjtxhh6h44iZjwWYQ+Txgfx893WrQ/iA5nVoSzS9941kQZq+cirHMWddlKzQeHfnXWhFznPLkpX/odIpc5VldFeJ/8cKRZW8SAwJt7vwl9zEqqWHBL/vgrJozpCOHCOKzkM9oV5J3rafe9UAehTd2n/OIbYQ5QJ8tTVqbN6DTNrVQcR8l6xIMAoCNf6QatltLPanGNiDKSmDpMd9oMjGVxiGoCGjrg6fETvtxWyCuhkjfVMYXhUeMqevdSheV8Cw3yu1lJr/uCMKAYBz8LkC4namthsFTJWdxuhtbQzH6rvXr7fGu1VzxxBVAgPcNwmMR5v+pFI5pgO2R3Rj7Qeiim+t+vdcaBUUw="
}
}
The report is that Mastodon is fetching the actor URL (in this case) https://hz.macgirvin.com/channel and not https://hz.macgirvin.com/channel/hubzilla - and there's nothing in this packet which specifies the actor URL that Mastodon is actually fetching. I also don't see it in webfinger, so it appears to be a rewrite of the url on the Mastodon side. I suspect it may be attempting to rewrite mastodon urls with '@' in them (and failing) but I haven't been able to find a smoking gun for any rewrites yet.
The object id should have been of the form https://hub.somaton.com/follow/nnn but was re-written to a Mastodon form. We need it to be the original id to match against the original request we made.
Ah yes, apologies for that. I had to do some hacky stuff to get everything to work with the assumptions/storage schema that Mastodon already had from the start. It's all normalized and it so happens that follows/follow requests don't have a URI stored. So it's impossible to pass one. I assume(d) it would be fine because follow/unfollow are such operations that you can't follow the same person twice differently, so the uniqueness of the activity doesn't matter and the follow/request can be inferred from actor/object combination. How much of a problem is this for your system? If this is to be resolved it would take some unpleasant refactoring.
there's nothing in this packet which specifies the actor URL that Mastodon is actually fetching
Mastodon stores actor URL in database, if it can find the local copy of the object it prefers using that instead of whatever is in the JSON, because in fact you could just use "object: actorURL" instead of a fully embedded structure and it would work the same.
How much of a problem is this for your system?
I can work around this one.
Mastodon stores actor URL in database, if it can find the local copy of the object it prefers using that instead of whatever is in the JSON, because in fact you could just use "object: actorURL" instead of a fully embedded structure and it would work the same.
Understood, but I can't figure out where it got or how it calculated the 'https://example.com/channel' URL it is fetching in the discovery or verification phase. All of our URLs (I believe) are https://example.com/channel/username and the one Mastodon uses returns a 404 since we can't parse the username. We were only able to allow communication by hardwiring /channel to /channel/username to get a follow request to succeed, so Mastodon appears to be using the broken form somewhere in the discovery process. Follow activities silently fail if we don't return an actor record at that wrong URL. Mastodon appears to be generating the broken form internally as we didn't provide that specific URL (without the username) anywhere in the Follow packet and I don't see it in webfinger either.
How much of a problem is this for your system? If this is to be resolved it would take some unpleasant refactoring.
Please refactor. This isn't confirming to the standard at all... Let's not put hacks like this into the main AP implementation from day one.
Apologies to @annando that this turned into an issue hijack. I'll open separate issues.
@zotlabs no problem.
so uh, the original question here is answered, right? It would be cool if these answers could be documented (tootsuite/documentation), then this issue could be closed.
if we're collecting ActivityPub interop details here for documenting, here's another minor point worth including: preferredUsername
seems to be required in Person
objects.
e.g. when i tried to deliver a reply to an ActivityPub inbox on from an attributedTo
without preferredUsername
, the inbox POST returned HTTP 422 with {"error":"Validation failed: Username has already been taken"}
.
when i added preferredUsername
to attributedTo
, and to the corresponding AS2 actors i serve, that error went away and inbox delivery worked. example:
https://snarfed.org/2017-10-01_mastodon-for-tech-folks
https://mastodon.technology/@snarfed/3194674
Closing this out now that we have documented compatibility using the implementation report on https://activitypub.rocks/implementation-report/. We're tracking the immediate followup from that report at #5631, and new implementation/compatibility issues not related to the existing documented limitations should go in their own ticket so we can triage them appropriately, thanks!
Ignore me, I hadn’t set Content-Type: application/activity+json; charset=utf-8
on the response 🤦
Hey @Gargron, I’m testing a very basic ActivityPub route for an actor object and stumbled on this issue while trying unsuccessfully to get my account to show by entering the actor object URL into the search box on my Mastodon instance.
You said earlier: “Mastodon will ignore actors without WebFinger (it performs verification for username@domain ownership), and Mastodon will ignore actors without an inbox.”
I’m not seeing a hit for ./well-known/webfinger on my server.
The basic actor object I’m returning is:
{
"@context": ["https://www.w3.org/ns/activitystreams",
{"@language": "en"}],
"type": "Person",
"id": "https://dhobqcaxam.localtunnel.me/",
"following": "https://dhobqcaxam.localtunnel.me/following",
"followers": "https://dhobqcaxam.localtunnel.me/followers",
"liked": "https://dhobqcaxam.localtunnel.me/liked",
"inbox": "https://dhobqcaxam.localtunnel.me/inbox",
"outbox": "https://dhobqcaxam.localtunnel.me/feed",
"preferredUsername": "Aral",
"name": "aral",
"summary": "Just a guy.",
"icon": {
"type": "Image",
"mediaType": "image/jpeg",
"url": "https://ar.al/images/aral-432.jpg"
}
}
Any thoughts?
Most helpful comment
Please refactor. This isn't confirming to the standard at all... Let's not put hacks like this into the main AP implementation from day one.