Fsnotes: [Req] Automatically tag or untag notes based on #tags within the note body

[gmenzel]

Bear has a great feature that it tags or untags notes on the fly as hash tags are added or removed from the note's body. I would love this feature, since it would allow me to search for tags with other applications by a simple text search, but would also allow a nice structuring within FSNotes.

[jeff-h]

I like this idea and actually a while back had a look at what the regex would need to look like. Actually, I just found a chunk of code I wrote as a first pass at this, but it's over a year old now and probably not useful.

The most difficult part is probably that there are a bunch of edge-cases to handle when detecting a #tag, eg the fact a hash is significant in markdown, and also to ignore usage such as https://github.com/glushchenko/fsnotes/issues/348#issue-352748030

Can someone describe in detail the usage of a hash in markdown?

[gingerbeardman]

one or more hash denote a heading?

[jeff-h]

Particularly, what are the implications of spaces (and other whitespace characters) before and after the hash.

[gmenzel]

Markdown doesn't require spaces between the hash and the heading text, so Markdown will take a tag for a heading if the tag starts at the beginning of a line. In practice I don't see this as a problem. In my zettelkasten, which is a collection of markdown files, I put tags in a header line at the top of the file like this:

tags: #tag1 #tag2

I've also seen people use HTML comments to hide tags in Markdown-rendered files:

<!-- #tag1 #tag2 -->

Interestingly, FSNotes's Markdown preview does require a space between the hash and the heading text, and does not render the line

#tag

as a heading. I believe this is incorrect behavior. All the other markdown renderers I tried (Byword, Marked 2, iA Writer) render the above line as an h1 heading.

[jeff-h]

I've dug into this a little and here's the story:

The original Markdown spec is vague and ambiguous in many respects — it doesn't mention the need for (or against) spaces between the # and the heading text at all. (1)

FSNotes implements https://github.com/iwasrobbed/Down which in turn is built upon https://github.com/commonmark/cmark. Cmark is a CommonMark implementation. (2)

CommonMark is not ambiguous, and unambiguously requires a space after the hash. This is a beautiful design decision for modern apps, where the use of Twitter-style hash tags is increasingly common. Have a play with it here.

Thus, I would propose that in FSNotes—

• #something, at the beginning of a line or preceded by a "whitespace character" will mean a tag called something
• # something at the beginning of a line will mean a heading with the text something

(1) https://daringfireball.net/projects/markdown/syntax
(2) <personal opinion & rant>There's a colourful exchange from the era when CommonMark was first put together, between the original author of Markdown (who never really specced it out well at all) and the proponents of an actual carefully put-together spec. The latter group were forced (by politeness more than anything) to rename their work CommonMark because the former flatly refused to cooperate or allow anyone other than his godlike self to touch the Markdown "spec", which is kind of annoying, because everyone knows and uses the term Markdown when in fact in many instances they are (hopefully) using CommonMark.</personal opinion & rant>

[aseygo]

i'm using fsnotes for notes containing source code snippets, specifically for languages (e.g. shell, python, etc). where the hash sign # is used to denote a comment. until now I never used markdown in these notes (and did not mark source code with backticks). so no issue that comments could be interpreted as a titles.

but it would now be a pity if all this gets interpreted as tags.

could we disable markdown and tags for .txt files? only have it for .md files? sorry no better solution comes to my mind...

[jeff-h]

@aseygo Excellent point. It made me realise the obvious — search my own notes for those containing a # character. In just a few minutes I came across all the following problematic uses in my notes:

1. customer Acc #101041
2. ref #603
3. let _ = Timer.scheduledTimer(timeInterval: 5, target: self, selector: #selector(self.jeff), userInfo: nil, repeats: false)
4. #log-bin=mysql-bin (ie a commented-out line in a .cnf file)
5. * #1 ranked

I had loads more. All those would fail the #tag logic I described above.

I checked Bear and if a number follows the hash, it doesn't create a tag. That deals with 1, 2 and 5 above., but Bear fails horribly on 3 and 4 — it creates tags for them.

The only solution I'm coming up with is that perhaps we should only look for tags within the first line of each note (or first and second, if you like a note heading in the first? Configurable in prefs?)

How well would that work in others' use cases?

[gingerbeardman]

So would tagging check both .md and .rtf files?

@jeff-h also please link the "colourful exchange" you mention above.

[gmenzel]

@jeff-h did 3 and 4 occur in code blocks? I would think that text inside of code blocks, block quotes, indented preformatted blocks, and between backticks should not be scanned for tag text. Between this and not allowing tags to begin with a number I imagine this will take you most of the way there. Even if there is some pollution of the tags list I would much rather have this feature than not.

A generally useful feature would be the ability to specify a tags black list as a list of regular expressions; a potential tag will not be created if it matches one of these expressions. The no-number rule could then be implemented by seeding the black list with [0-9].*. If you wanted to ignore hex color codes you could add [0-9a-fA-F]{6}. And so on.

@aseygo it does seem like a good option to be able to specify which filetypes should (not) be scanned for tags.

@gingerbeardman my first impression is that rtf files should not be scanned. If you're using rtf files, you're already dealing with files that rely on hidden metadata for formatting; hidden metadata that indicates tags doesn't clash with the rtf design ethic. Also, the potential for tag pollution is higher for rtfs, since rtf files have no standard ways of specifying code blocks and other areas that should not be scanned for tags. It seems like it could be a lot of work for not much benefit.

[gingerbeardman]

I like your style @gmenzel — great to have you on board

[jeff-h]

@gmenzel none of my examples are inside code blocks. I exclusively use rtf for my notes, which doesn't really exclude me using code blocks of course, so I guess that could be an option to tell FSNotes not to create tags there?

Re: your comments on tags in rtf — as someone who exclusively uses rtf, I would disagree — I'd really like this feature :) In fact I've been using it for years simply as an aid to finding notes using the standard search mechanism. I don't see #tag as an "extension" to Markdown/CommonMark so much as a convention that FSNotes could recognise and act on, much like it does now with the auto-bullet feature, which is file-format agnostic.

[gmenzel]

@jeff-h Interesting. I guess there would be no harm in scanning rtfs for tags, especially if there is also a blacklist, a way to specify which file types are scanned, etc.

The thought of interpreting markdown inside of rtfs makes me queasy somehow — is that what you're suggesting by "using code blocks"? Do you feel that this would be a good idea?

[sven-gh]

This is a great idea!
As someone who used Bear for a while in the past I naturally started typing #tags in FSNotes expecting it would just work because for an end user (or at least for me...) it just feels like the logical/natural thing to do
but now I realize it's far from the default.

Agree with above that rtf should be included as well - it would be a good/easy way to categorize content with images

Also, without this functionality it will be slow and cumbersome (i.e. too many steps involved) to assign a tag on iOS compared to being able to just do it directly as you're typing the note content

[micahredding]

As a first step towards this functionality, simply making #tags clickable (triggering a search) would be a huge step—and seems easy enough to implement without introducing unexpected behaviors elsewhere.

[Xiao6]

1Writer the ios app seems to have great support of # hashtag for markdown. If this feature is implemented, I guess 1 writer can be great companion ios app.

[glushchenko]

Added in FSNotes 4.0.0 beta

[gingerbeardman]

Woah!