Staticman: Email notification upon replies

[TWiStErRob]

Usually commenting solutions allow for users to be notified via email about replies to their comments, or in case of a flat commenting stream when anyone posts a new comment. Is it possible to do this with staticman?

[eduardoboucas]

This is currently in development and will be released soon!

[illus0r]

@eduardoboucas looking forward to it! I've just discovered, I've received a first comment on my blog. This comment is useful, but I'm not able to say “Thank you” to it's author.

[eduardoboucas]

The feature is pretty much ready. I could use with some help with testing. Would any of you be interested in being the guinea pig? 😄

[illus0r]

@eduardoboucas ready!

[eduardoboucas]

Okay, here's how to get notifications working.

Emails are sent using Mailgun. There is an account associated with the public instance of Staticman, but it comes with a limit of 10,000 emails a month. This limit shouldn't be an issue in the immediate future, but users are encouraged to use their own Mailgun accounts instead. They are free (for the first 10k emails), but you do need a custom domain to use them with. I can possibly provide *.staticman.net subdomains.

To enable notifications, you need a notifications block in your site config and you must be using the config file format introduced in v2 (i.e. having a staticman.yml file, instead of using Jekyll's _config.yml). See the sample config file for more info.

If you're using your own Mailgun account, you'll need to add a Mailgun API key and domain to the config, which need to be encrypted. To encrypt those, you can use the following endpoint: https://api.staticman.net/v2/encrypt/{TEXT TO BE ENCRYPTED}.

Next, your entries will need to specify that a user wishes to subscribe to a specific entry. This is an example:

```html

Go!

```

In the from above, you can see the following:

• options[origin] contains the full URL of the entry. This will be included in the email sent to subscribers, allowing them to open the post directly.
• options[parent] is a unique identifier to the entry the user is subscribing to. This could be the title of a post (although it needs to be URL-friendly, so this needs to be a slug) or the _id field (which is now automatically added to every entry) if you want to subscribe to another comment.
• options[subscribe] contains the name of the field corresponding to the email address of the subscriber.

Finally, and to make sure we don't send emails asking users to click on links that have been spoofed, you need to add an allowedOrigins field (see docs) to your site config, where you'll specify which domains are valid origins. If you add only mysite.com, then an entry where options[origin] is http://myothersite.com/post1.html will be rejected.

I'm sure there are still a few rough edges, so if any of you can try implementing this and share your feedback that'd be very useful.

Thanks!

[dancwilliams]

Works great! Thanks for your work on this!

[zongren]

Is this working on public staticman instance?
I tried it a couple of times and failed.
Here is my staticman.yml:

allowedFields: ["name", "email", "url", "message", "origin", "parent", "subscribe"]
requiredFields: ["name", "message"]
format: "json"
generatedFields:
  date:
    type: "date"
    options:
      format: "iso8601"
moderation: true
branch: "master"
path: _data/{options.slug}
filename: comment-{@timestamp}
transforms:
  email: md5
name: "宗仁的博客"
allowedOrigins: ["localhost", "zongren.me"]
notifications: 
  enabled: true
  apiKey: "qPB1uWby7FS6T0wgLqUfjcQkkVtLkcergXOqiEoZyTo5yqMmGU/cuzOD825KOZkvbE7m0mOYo2LKPj82v+BcQDxxcIULev8lwpQ1KZJwjv6Ei3f1HbFyIq5N2Ehmya3PyPGga3IaedFVTPFrue67DQ2W5+tu8xJX1S2PahUEgAA="
  domain: "FkFQQFNFX96xVIOYvgtB8IxeBP60lZa/sUhHv0Y3KWV3EdRjMLED0zZr6nGGC5opytzczKvQtR2Y6YJvKQ2ltOBV1aFuAvsN/HPnOZ4e5JMBI+BjGWWlaqUsKp/mNGq/q9oWDk3FT8tdfw7UBqa8lC99eYa+QMXj/k+gpPx5ki0="

and comment form:

<form class="comment-form" method="POST" action="https://api.staticman.net/v2/entry/<%- theme.staticman %>">
    <input type="hidden" name="options[origin]" value="<%- config.url %><%- config.root %><%- item.path %>">
    <input type="hidden" name="options[parent]" value="<%- item.slug %>">
    <input type="hidden" name="options[redirect]" value="<%- config.url %><%- config.root %><%- item.path %>">
    <input type="hidden" name="options[slug]" value="<%- item.slug %>">
    <span class="comment-form__input-wrapper">
      <input autocomplete="off" spellcheck="false" class="comment-form__input" name="fields[name]" type="text" placeholder="请输入姓名"/>
    </span>
    <span class="comment-form__input-wrapper">
      <input autocomplete="off" spellcheck="false" class="comment-form__input" name="fields[email]" type="email" placeholder="请输入邮箱（可选）"/>
    </span>
    <span class="comment-form__input-wrapper">
      <input autocomplete="off" spellcheck="false" class="comment-form__input" name="fields[url]" type="url" placeholder="请输入你的网站（可选）"/>
    </span>
    <span class="comment-form__input-wrapper">
    <textarea class="comment-form__input comment-form__input--textarea" name="fields[message]" placeholder="请输入评论内容（支持Markdown）"></textarea>
    </span>
    <input type="checkbox" name="options[subscribe]" value="email" id="subscribe" /><label for="subscribe">订阅此文章的评论</label>
    <button class="comment-form__input comment-form__input--button">提交评论</button>
  </form>

And there are no logs on my mailgun dashboard.

[eduardoboucas]

@zongren link to the repo, please?

[zongren]

@eduardoboucas the repository is at gitlab

[zongren]

And I used encrypted "zongren.me" as notifications.domain.

[wes-brooks]

Hi Eduardo!
Any chance you could check the logs of the staticman service to help me understand why my attempts to get reply notifications aren't working? I think I've configured staticman properly and have a working mailgun account, but I must be doing something wrong.

The repo is wrbrooks.github.io. Thanks in advance!
-W

[eduardoboucas]

@wrbrooks I don't see any errors in the logs. Can you link to the page you're using to submit entries?

[justinrummel]

Hello @eduardoboucas I'm having issues as well w/ Staticman triggering emails with Mailgun. I'm able to validate with Mailgun's example curl statement but cannot trigger anything from my primary test page: https://www.justinrummel.com/macworld-2010-pictures/

(API Key removed)

justinrummel@Rummel-MBPr ~> curl -s --user 'api:key-12341234123412341234123412341234' \
                                https://api.mailgun.net/v3/mg.justinrummel.com/messages \
                                -F from='Excited User <[email protected]>' \
                                -F [email protected] \
                                -F subject='Hello' \
                                -F text='Testing some Mailgun awesomness!'
{
  "id": "<[email protected]>",
  "message": "Queued. Thank you."
justinrummel@Rummel-MBPr ~>

[eduardoboucas]

@justinrummel Staticman uses the Mailing Lists functionality from Mailgun, where each thread/post is created as a mailing list. If you go to your Mailgun account and navigate to Mailing Lists, do you see any lists created? If so, do you see the email addresses corresponding to the users that have subscribed to that thread?

[justinrummel]

I see one mailing list that I think was created from my API tests, but not from the site.

[chuckmasterson]

I've been planning how to integrate email notifications onto my blog. I have my comment fields defined somewhat idiosyncratically, so the author's email is authoremail instead of just email. I suppose that probably wouldn't work, since I haven't found where to set the name of the field that Staticman looks for... do you know of a way to make it work, or will I just need to sed all my authoremails into emails?

Also, I saw in your reply to #72 (which brilliantly solved a problem I was having, thanks) that the notifications feature isn't considered fully stable yet. Do you have a sense of how likely it is that an existing staticman notification configuration would continue working as the project continues toward stability?

[eduardoboucas]

@chuckmasterson You can specify the name of the field that contains the email address. It's the value of options[subscribe]. In your case, that would be:

<input type="hidden" name="options[subscribe]" value="authoremail">

Regarding the stability of the email notification system, I have to admit I've been struggling to find enough time to dedicate to it. In any case, I think the existing configuration format is flexible enough to accommodate any changes in the near future, so I don't foresee any breaking changes.

[chuckmasterson]

Completely understand about having time. If I knew the first thing about Node.js I'd try contributing, but alas.

I tried setting up with the advice you gave and got as far as getting some mailing lists created. But nothing triggered sending to these lists. I'll look at it again soon... if you find the time to read this and reply, a couple questions that might help me out are:

• How does Staticman create the "Alias Address" in Mailgun for each mailing list? Mine looks like a random series of hex digits, but it seems like it should correspond to the _id of some post, or at least something that I see in the YAML of one of the comments.
• How does Staticman determine what (if any) Alias Address to send notification emails to? I'm not quite wrapping my head around how this is achieved with just the options[parent] field. Does it look at all the files in the post's comment directory and see if any have an _id that matches options[parent]? If so, how does it work if you give it a slug as option[parent], since you aren't telling it whether it's identifying the page or the parent's _id? And how does it make the leap from there to knowing where to send the email?

[chuckmasterson]

After writing that, I took a crack at reading the code anyhow, and I think I may have understood enough to make a suggestion that would get us partway to a solution, though not all the way. (I would write this as a pull request but my understanding of Node.js is so weak that I would almost certainly break more than I'd fix.)

Suggestion:

Use a different variable besides options.parent to control subscriptions, perhaps a new one named options.subscribeTo.

Reasoning:

Though I haven't found anywhere you gave explicit directions on how to use options.parent, @mmistakes is already using it in threaded commenting, and a few other people have copied his strategy (I'm one). This strategy relies on options.parent being written into the YAML comment file as _parent. In this strategy _parent (options.parent) is the _id of a child comment's parent.

From what I understand of the Staticman code, Staticman expects options.parent to be some identifier of a post that the user is currently subscribing to. This conflicts with the other usage. If a user is writing a top-level comment on a post, then that comment's _parent will be null, and hence even if the user clicks the "subscribe me" checkbox, no mailing list is created because this condition doesn't get satisfied:

if (subscriptions && options.parent && options.subscribe && this.fields[options.subscribe]) {
  subscriptions.set(options.parent, this.fields[options.subscribe]).catch(err => {
    console.log(err.stack || err)
  })
}

If I follow the logic aright, the only subscriptions that would work with Staticman as it stands are where a user makes a child comment and clicks "subscribe me" - in which case they would get subscribed to all replies to their comment's parent. No one can subscribe to replies to their own comment, only to replies to some top-level comment that already exists.

Although my and mmistakes' use of options.parent here is apparently nonstandard, I think there's reason to favor it over what's currently in the code:

• It's already employed in several sites.
• The name parent describes a child comment's parent more intuitively than it describes "what comment the user is currently subscribing to".
• I don't think you'd be breaking any currently functioning sites. It seems like all the Jekyll pioneers here who have tried implementing Mailgun notifications have gotten stuck at about the same stage I have, and the only one I'm aware of who apparently has it working is mmistakes. (I've left a comment on his site to try to see if notification works in the strange way I've predicted, but it's in moderation right now.) The sites of @justinrummel and @zongren currently have no "subscribe" options and @wrbrooks 's does but I can't test it because no comments are going through at all for me (hey wrbrooks, did you know that?).

I'm not sure what format would make sense for the hypothetical options.subscribeTo as I still don't fully understand how Staticman currently interprets options.parent.

I'm still confused on one point: you mentioned in #76 that options fields aren't written to the comment file, but options.parent sure is getting written. I'm happy with that, but it is also a bit unusual; maybe it would make sense for it to go into the documentation eventually that "options fields generally don't get written to the entry, with the exception of options.parent, which is prefixed with an underscore (_parent) to denote the difference."

I'm happy to be a guinea pig on this feature as I'd definitely like to have it, but I'm also going to be on extended travels with little to no coding starting April 1, so I hope we can all get it figured out by then.

And thanks for Staticman! I hope this helped and wasn't too full of misunderstandings of how the code works.

[mmistakes]

@chuckmasterson I just merged in your comment. My understanding is you should get a notification email once a new comment is merged in. I'm going to do that now and make it off the main thread (not a child of your comment to see if your hypothesis is true).

My site takes about 20 minutes to build as it crunches through a bazillion images, so give it that long to see if anything shows up.

[eduardoboucas]

Thanks for the suggestions, @chuckmasterson. Let me try to answer some of the questions.

How does Staticman create the "Alias Address" in Mailgun for each mailing list? Mine looks like a random series of hex digits, but it seems like it should correspond to the _id of some post, or at least something that I see in the YAML of one of the comments.

Each mailing list has a corresponding email address. This email address is a MD5 hash of the GitHub username, repository and entry id concatenated together. This happens here.

How does Staticman determine what (if any) Alias Address to send notification emails to?

Whenever we process an entry, we take its id, generate the hash described above and check if there's any mailing list with a matching address. If there is, we fire notifications to all the addresses in it. Otherwise, there's nothing to do.

From what I understand of the Staticman code, Staticman expects options.parent to be some identifier of a post that the user is currently subscribing to. This conflicts with the other usage.

My thinking was that options[parent] describes the "thread" that you want to subscribe to. If you're writing a top-level comment as you mentioned, this parent would be an identifier of the post you're commenting on (it could be a title, slug, or anything else that uniquely identifies the entry). If you're commenting on an existing comment, then the parent would be the id of the comment.

Theoretically, it could go on how many levels necessary, but as @mmistakes will be able to tell you, it's probably not a good idea to nest further than 2 levels because otherwise the site generation process becomes too complex.

So the general idea is: options[parent] can be whatever — a post title, slug or a Staticman entry id. All it does is create a mailing list with subscribers for that particular parent, which we use whenever we want to process that entry.

Does this help?

[eduardoboucas]

And sorry if this has caused confusion and made you have to guess what Staticman does. My intention was to document this whenever I had the chance to make it fully stable, but I understand that people want to use this as soon as it's available, so from now on I'll make sure to document things as they get deployed to the live API.

[eduardoboucas]

Also:

I'm still confused on one point: you mentioned in #76 that options fields aren't written to the comment file, but options.parent sure is getting written. I'm happy with that, but it is also a bit unusual; maybe it would make sense for it to go into the documentation eventually that "options fields generally don't get written to the entry, with the exception of options.parent, which is prefixed with an underscore (_parent) to denote the difference."

This is very true. This should be in the documentation.

[chuckmasterson]

Ahhh, it's becoming clearer. @eduardoboucas, thanks for all the answers.

@mmistakes, your comment has been processed but I didn't get a notification. I think I was right, and if I'm right about being right, we're both stuck until either Eduardo makes the change I suggested or we change our commenting mechanism so it uses options[parent] how Eduardo intended.

In your comment you said that "my understanding is options[parent] are options[origin] are two different things, the later used to track which thread you’re currently subscribed to." I don't think options[origin] is used for this at all, and in fact I haven't been paying any attention to that field. It appears to me that it's only used for (1) preventing spoofing attacks and (2) displaying a link in the processed email.

Sounds like Eduardo is saying that options[parent] is indeed what's used to keep track of the thread (Mailgun mailing list). In SubscriptionsManager.js this happens:

const compoundId = md5(`${this.parameters.username}-${this.parameters.repository}-${entryId}`)

...which would boil down to, for instance, md5("chuckmasterson-chuckmasterson.github.io-[somePostIdentifier]"). That "somePostIdentifier" looks like it's supplied by options[parent].

If not for the condition check (does options.parent exist?) that I quoted earlier, my top-level comment on MadeMistakes would've resulted in a mailing list created with a name of whatever results from md5'ing "mmistakes-made-mistakes-jekyll-", since the entryId (options[parent], blank for a top-level comment) would have been null. Since there is that check, I'm guessing if you looked in your Mailgun mailing lists you'd find that no list with my address in it has been created (it's a mail.chuckmasterson.com address).

So... if I'm right, which of us should change?

[mmistakes]

That makes sense @chuckmasterson. Because I'm using options[parent] for nested comments it will only fire reply emails on child comments. The main thread sends null for that variable so no email notifications.

I'll probably change the language on my site so it's clear that you're only subscribing to "replies made on your comment" and not to the entire thread. Which to be honest is probably fine. I'm less concerned about email notifications than I am with stopping the barrage of comment spam and resulting Staticman PR's I delete daily :wink:

Also I'm not using my own MailGun account. That's probably why I'm one of the few who have email notifications working. I'm just piggy backing on the public instance of Staticman since I never had much luck configuring it.

[zburgermeiszter]

@mmistakes you might want to have a look at the brand new reCAPTCHA feature to stop the spam.

[eduardoboucas]

If I understand correctly, the only thing missing in @mmistakes's implementation is to send a post identifier on options[parent] for the main thread.

@mmistakes I think I already asked you this, but have you tried a honey pot field? I found it to be quite effective in stoping spam on my own site. If that hasn't worked, feel free to open a separate issue so we can find a solution. I'm also more concerned with that than with email notifications.

[mmistakes]

@zburgermeiszter Gave it a go this morning but trying to POST to dev.staticman.net was giving me errors. I had the same problem when I switched to the v2 API and had to jump through some hoops to remove/add Staticman as a collaborator.

[zburgermeiszter]

@mmistakes
Can you please report this the issue #20, with your staticman.yml please, to see what could be wrong?

[chuckmasterson]

@mmistakes People would be subscribing to "other replies made to the comment you're replying to", if that's alright with you.

@eduardoboucas That would work for Michael except that in his comments Liquid (which I got familiar with while I was adapting it), top-level comments are identified as such by how they're missing a _parent field. I suppose we could check whether _parent equals the post slug instead of whether it's null. That seems a little fragile though.

[mmistakes]

@eduardoboucas - That would probably work. I would just need to modify the logic behind the loops. Which may or may not give me a big headache and I'll just punt on. Hahha.

And Yes I have a honey pot in there. I'll drop some notes in a separate issue.

[eduardoboucas]

@chuckmasterson I see. Not sure what the best solution is in that case. My site is using the post slug as parent, but I'm not using threads — all comments are top-level entries.

Any suggestions?

[chuckmasterson]

In my migration from Blogger I ended up defining a variable at the beginning of the comments block called commentslug, which looks something like the markdown filename (for example, "2015-10-some-title" - for legacy reasons mine is without the day). That's constructed of variables that shouldn't change even if the permalink structure does:

{% capture commentslug %}{{ page.date | date: '%Y-%m-' | append: page.title | slugify }}{% endcapture %}

You might be able to do something more simple and more resistant to breaking in case of changing titles, by using {{ page.path }}, which returns the full GitHub URL of the post's original markdown file - if you can figure out a way to reliably get rid of everything up to and including the last /, since the rest will change depending on branch and whether you're on a localhost. Whoops, never mind, just checked page.path and it actually returns a path relative to the root of your repo, like _posts/2015-10-10-some-title.md. As long as you don't make a habit of changing your markdown filenames, page.path should be a good choice as far as I can tell.

[chuckmasterson]

As for getting notifications to work, I'm closer but not there yet. It seems like it should work: I've created a mailing list by subscribing from a top-level comment, and then I've subscribed again from another top-level comment with a different email address, and the hash was the same and the new address was added.

If the mailing list exists and Staticman is getting its name right, I don't know what would be stopping it from delivering. I sent an email to the list from my regular email and it came through fine. I'll keep pondering what may have happened and try not to ask for too much of your time, @eduardoboucas, but if there's any log you can view that tells what's been happening on your Staticman, that might offer an easier route.

[eduardoboucas]

@chuckmasterson There's nothing I can see in the logs. Can you share a link to the repository you're using, if it's public?

[chuckmasterson]

Yeah, it's under this account at chuckmasterson.github.io.

[eduardoboucas]

@chuckmasterson Do you mind switching your Staticman config to temporarily use the public Mailgun account? All you need to do is disable fromAddress, apiKey and domain from under notifications. This is just so I can more easily test.

[chuckmasterson]

Sorry, I was away from computers for a few days.

Just changed the Staticman config over as you requested. If you want to test commenting on a blog post that's up, you can use this post. I don't have a checkbox for subscribing shown right now since I haven't gotten it working, but I'll work on getting a checkbox to show on just that page if that'd be helpful. Otherwise there's also some HTML commented out in the vicinity of the submit button that you could uncomment in DevTools or Firebug.

[mmistakes]

@chuckmasterson Looking at this again I completely missed what you were saying about how options[parent] works in regards to a mailing list. I'm thinking it might make sense to stop using it as a child comment identifier and come up with a different field to identify a comment's parent.

Then most of the liquid could stay the same, just with a variable rename and a batch replace of _parent on all the existing comment data files. options[parent] could then be assigned with something like {{ page.url | absolute_url }} in the form which should correctly subscribe a user to the entire comment thread.

Though I do like your suggestion of options.subscribeTo since that describes that field better than options.parent. Guess that one is up to @eduardoboucas if he wants to change the name and potentially break some sites :wink:

[chuckmasterson]

Yeah, what you described is what I ended up doing - something like sed -i 's/^_parent/replying_to/' */*.yml.

[nakoo]

I also have same problem. Mailgun only makes mailing list, not sending the email.

[ghost]

Trying again to get the notifications working correctly with a mailgun account. For the mailgun domain entry in the staticman.yml, mailgun provides this:
API Base URL https://api.mailgun.net/v3/www.somedomain.com
Does this need to be encrypted before adding to staticman.yml?
Encrypting with this command:
https://api.staticman.net/v2/encrypt/https://api.mailgun.net/v3/www.somedomain.com
fails with this message:
Cannot GET /v2/encrypt/https://api.mailgun.net/v3/www.somedomain.com
Probably something simple ... but I'm not seeing it.
Alec

[chuckmasterson]

@alecsatin Don't use "API Base URL", use the bit that's just labeled "Domain" in the Mailgun interface. In your case I think it'd be just www.alecsatin.com. I tried encrypting that and it worked.

[ghost]

Thank you. Using https://api.staticman.net/v2/encrypt/www.somedomain.com (with the actual domain) to encrypt works. I've updated the staticman.yml.

I'm still having an issue on the mailgun side. One of the txt entries will not validate no matter what I do at the registrar side. The other txt entry (mx._domainkey.www.somedomain.com) is fine.

[nakoo]

Is there any settings to use public instance of mailgun, not just adding enabled option? I have failed so many times for using this.

[mattseemon]

Hi... trying to get notifications working. Right now, my mailgun account seems to be setup correctly. I followed @mmistakes guide and was able to get everything working fine. At this moment, the only thing not working is the actual notification. While I am testing, I used different email ids to register for the notifications. I can see the mailing list getting updated with the new email id's... however, i am still not receiving the notifications. Is there something I missing. I followed @mmistakes guide to the letter.

[Hund]

Will there be any alternative to Mailgun? Something more anonymous, that doesn't require you to own the domain and doesn't require a phone number?

I have my blog at GitHub pages.

[willymcallister]

@eduardoboucas: Staticman (public) and my own MailGun account are working great at spinningnumbers.org. Thanks so much for creating a great service.

A feature request: When Staticman sets up a new mailing list at MailGun, is it possible for Staticman to initialize the "Name" field to the post slug? Some sort of clear text indication of what the mailing list is for.

[eduardoboucas]

@willymcallister That's great news, thanks! Any chance you could write a little guide describing the process you followed to get it working? I'm sure it would be useful to a lot of people.

As for your feature request, it makes perfect sense. I'll try to look into it soon. If you can, please open a separate issue so it doesn't get lost in this thread.

[binarymist]

Hi @eduardoboucas.

I'm currently unable to receive notification emails.

This is my staticman.yml with the relevant notifications config enabled and the encrypted apiKey and domain.

This is where I post comments from in code.

I'm not receiving any error from the Staticman API and no activity in my mailgun account.

Is my config and POST correct?
Do I need to do something in my mailgun account that I've missed from the docs?

Thanks.

[binarymist]

I think my problem was that I added the mailgun API key without the "key-" prefix. Once I added the correct API key as shown in the mailgun domain information and ran it through https://api.staticman.net/v2/encrypt/ along with my [mailgun-prefix].[my.domain], then submitted a test comment with the options[subscribe] checkbox of the submit form checked, I could then see a mailing list entry turn up in my mailgun account.

At this point notifications were working, but there were still errors in the mailgun Logs tab:

No MX for [mailgun-prefix].[my.domain]

At this point I added the two MX records to my DNS provider that mailgun recommends:
name: mailgun
value: mxa.mailgun.org
name: mailgun
value: mxb.mailgun.org

and tested every scenario I could think of. All working at this stage.

[nakoo]

@binarymist what do you mean [mailgun-prefix] of the domain?

[binarymist]

what do you mean [mailgun-prefix] of the domain?

The prefix that mailgun recommend their customers use. I think it can be anything, or possibly nothing (by the look of some of the other participants in this thread), but needs to be specified as part of the domain in the RSA'd domain string in the staticman.yml, and in your mailgun account.

[binarymist]

I've just used this functionality for a subscription to any new blog post. Great for capturing your audience and keeping them up to date directly to their inbox https://github.com/gcushen/hugo-academic/issues/427

[wes-brooks]

I've found two things, either of which can prevent the sending of reply notifications:

1. If moderation is on, notifications won't be sent. This is due to lib/Staticman.js, lines 498-507. moderation==true, we go into the code that writes a Pull Request but if moderation==false we go into the logic that checks whether anyone has subscribed to hear about responses to the parent comment.
2. Both subscriptions and replies are registered to whatever options[parent] was relevant to the comment-leaver, but notifications only fire when these match. This means that only "sibling" comments can notify each other (comments with the same parent.) I think the expected behavior is to see notifications based on replies to your own comment, but the current code will instead notify when someone replies to the same comment you replied to.

I think the way to address (2) is to change the part of lib/Staticman.js line 490 that is like:

 subscriptions.set(options.parent, this.fields[options.subscribe])

to be something like

 subscriptions.set(options._id, this.fields[options.subscribe])

instead (I don't know when the _id variable is assigned, though.) Then, the commenter can be notified when people reply to them.

Edited to add:

I think another common source of confusion comes from the fact that comments with parent set to nil cannot subscribe to reply notifications. This is because parent is part of the string that gets hashed to create a unique comment notification thread. The options[parent] property is set in HTML (usually by a Jekyll variable) but some of the projects linked in the README don't populate options[parent] for top-level comments. When I caught that, I started populating top-level parents with the {{ post.slug }} but I still had to be careful because the for loops that copy out the comments seem to want options[parent] to resolve to an integer. As far as I can tell, there is no reason that has to be, but you'll often see code like:

{% assign p       = comment._parent %}
{% assign parent  = p | to_integer %}

in the for loop. So far I have been able to replace that second line by

{% assign parent  = p %}

with no ill effect.

[nakoo]

@wrbrooks Thanks. I finally can get the notification mail. What I only changed is moderation: true to false. Mailgun Logs show me the success messages. But still, I find the way using notification without moderation to false.

[wes-brooks]

After some further investigation, I figured out how to get notifications to send when moderation is enabled: you have to add a webhook to your repository, pointed at

https://api.staticman.net/v1/webhook

Note that this is true even if you're using the v2 API for comments.

This fix is kind of obvious in retrospect (notifications only go out when the comment is accepted, but how was staticman going to know that the comment had been accepted?), but the documentation only mentions the webhook API in reference to automagically deleting the staticman_ branches after they are merged.

I think there should be a clarification in the documentation section about reply notifications to say that when moderation is enabled, sending notifications requires that you add the staticman webhook to your website's repository.

[nakoo]

@wrbrooks Perfect. I finally made it. Thanks for goodness.

[binarymist]

The main problem I'm (and my commenters) seeing is the multiple Staticman notifications for every comment: #182

[chmac]

@eduardoboucas Haven't read this issue in detail yet, but I could put some time into this if you like. I've just started thinking that I really want reply notifications and saw this and #35. Will wait for the go ahead, and if you're happy, put some effort into this. I'm offline next week, and only online a few days the following week, so might be 3-6 weeks before I get to it...

[samarulmeu]

I finally managed to get notification working with mailgun (in fact only adding the emails to a mailing list). Most of the trouble and stress was because I was trying to use a domain in the EU region. There is a difference in the API base URL: https://api.eu.mailgun.net/v3/ vs https://api.mailgun.net/v3/.
Is there a way to specify the EU region?

[gabeluci]

I managed to setup notifications per-thread (i.e. people will only be notified if someone replied to a thread they started or replied to, rather than any thread on the post), but only because I'm running my own instance of Staticman and I can change the code.

The key was setting the parent property to the auto-generated id if the parent is empty. I did this in the processEntry function of /lib/Staticman.js:

Staticman.prototype.processEntry = function (fields, options) {
  if (!options.parent) {
    options.parent = this.uid
  }
  ...

That may not be the best place to do it, but it works. Maybe that's something that can be worked into the master code, even as a config option. I'm sure others could benefit.

Then in my site code, I leave options[parent] blank for a top-level comment, and I set it to the _id of the parent if it's a reply. I completely got rid of the replying_to property and instead just check if _parent == _id to determine if it's a top-level comment or a reply.

Feel free to look at my code. It is based on the good work of @mmistakes.

The only thing outstanding that I might fix is the wording of the initial email to the original commenter. The email is valuable when moderation is on since it basically tells the person that their comment was approved. But I'd rather it actually say that than say "Someone replied to a comment you subscribed to".

But that's for another day.

[binarymist]

But no PR submitted @gabeluci?

[gabeluci]

No, no PR. I don't know the code base well enough to say that's the best place in the code to do it, or if it will break how others are using Staticman. @eduardoboucas may decide it's better to make it a configurable option, if anything.

[binarymist]

Understood @gabeluci. Looking at the commits vs issues and PR's, it's starting to look like staticman is no longer being maintained. Is this correct @eduardoboucas? If so, have you considered passing the battern?

[purpleidea]

Looking at the commits vs issues and PR's, it's starting to look like staticman is no longer being maintained.

I'm sure @eduardoboucas is busy with other things as well, but send a patch, do a fork, or even host an alternative mail server if you'd like! At least from my POV it's pretty feature complete. The biggest trouble I have is integration with gohugo.

[binarymist]

Oh odd. What exactly are you struggling with there @purpleidea, I have it integrated (https://binarymist.io/blog/2018/02/24/hugo-with-staticman-commenting-and-subscriptions/)

[purpleidea]

@binarymist

I gave up after getting annoyed with the available functions in hugo, and not finding clear docs on how to write one. Code is here:

https://github.com/purpleidea/purpleidea.com

Patches welcome! :)

[eduardoboucas]

Looking at the commits vs issues and PR's, it's starting to look like staticman is no longer being maintained. Is this correct @eduardoboucas? If so, have you considered passing the battern?

The time I've been able to dedicate to the project hasn't been enough, sure. But what exactly do you mean by passing the batten? The repository is open, anyone can contribute with code. I'm more than happy to give write access to people that wish to become regular maintainers (as I've done in the past).

Most issues have to do with new feature requests, most of which I don't want to commit to precisely because I don't have enough time to implement and maintain them. Most PRs are from people wanting to add their site to the README (which, looking back, wasn't the best of ideas).

Maybe what I need to do is go through the list of issues/PRs, close what needs to be closed, merge what needs to be merged, and then start a GitHub Project to give some visibility on what needs to be addressed and in which priority. That should give users (and contributors) an indication on what is being worked on and what we need help with.

Thanks for the patience.

[gabeluci]

I ended up writing a whole article about how I setup my site to use Staticman. More relevant to this thread is the section about comment threading.

[binarymist]

What I mean @eduardoboucas is that it seems that @gabeluci has an answer for this issue in code, and it'd be great to see some colaboration between you two so that the work could be included in Staticman. Respect to both parties involved.

[binarymist]

@purpleidea That's why I provided the available functions and clear docs for you, if they're less than clear please leave a comment on the blog post and I'll do my best to help.

[gabeluci]

I don't think my way is the only way to do it. I think you could do per-thread notifications on the public API as-is. You would have to:

1. For top-level comments, generate a unique identifier in JavaScript (this might help) and use that for the value of parent. That value would be used for replies too. That solves the issue of subscribing the top-level commenter.
2. Add a second field called something like isTopLevel that is true for top-level comments and false for replies. This would be used solely for displaying the comments, since the first loop only looks at top-level comments.

[chmac]

I'm also a committer and have deploy privileges. I'd be open to putting some time into getting this live. Is anyone ready to submit a PR? I'm ready to review it, loop in @eduardoboucas, and if all's well, deploy.

@samarulmeu It would probably be easier to open a new issue about the switch to the EU region. There's a few things getting mixed together here.

[gabeluci]

@chmac Are you talking about setting parent to the generated id? I could attempt a PR for that, but it depends on if it should be a configurable option. Doesn't hurt to make it an option I guess.

I did submit an unrelated PR for something else related to notifications.

[chmac]

@gabeluci Aha, I see your PR. To be honest, I'm not 100% on top of the status of this issue. I'll try to schedule some time to dig into it in the next few weeks. I also now see there's quite a few open PRs which could be reviewed...

[gabeluci]

Well my PR doesn't really have anything to do with this issue, except being related to notifications.