Foam: Documentation migration from Obsidian to Foam

[jsjoeio]

I think it would be a good idea to show how you can migrate from Obsidian to Foam.

I'm trying to write a script to help with this.

Here's what I have so far:

# Loop over all markdown files
for file in *.md

  do

    # Remove the file extension
    filename=${file%.*}


    # Create new filename it by replacing spaces with - and lowercasing it
    newFileName="`echo $file | sed s/\ /\-/g | tr 'A-Z' 'a-z'`"

    # Rename file
    mv "$file" "$newFileName"

    # Capitalize the first letter for heading
    heading="$(tr '[:lower:]' '[:upper:]' <<< ${filename:0:1})${filename:1}"

    # Add heading to file
    # echo "# $heading\n" >> $file

    # Add heading to file
    # TODO this part is not working
    sed -i '' "# $heading\n" $newFileName
  done

Note: this is my first time writing a script so any feedback is welcome. I can't get the last part working - adding a heading to the start of the file.

Other Notes

• Should the script also delete the ./obsidian directory?
• Should the script grab the latest foam template? Or will that step be manual and done by the user?
• Should it the script do anything else?

The first version could just be renaming files like I have. I'm open to any and all ideas though.

[jevakallio]

@jsjoeio thanks for starting this work! 😍😍😍

I'm not a Obsidian user, but I can create myself an Obsidian project and take a look at this. Foam File Format bubble documents the implicit assumptions we make about files right now, so assuming Obsidian documents are "just markdown", the things it should do is:

• Rename files to conform to kebab-case.md
• Add # Heading
• Generate link reference definitions
  
  
  
  • These aren't necessary until you want to publish your Foam workspace. Right now they are generated only when saving a file in VS Code, so if you wanted to make all the links work in the published Foam, you'd have to go and save every file :)
  
  
  • Automating this out-of-vscode is a high priority for multiple reasons, and I'll try to get to this work soon
  
  

In an ideal world, foam-workspace-manager would export a CLI that could be run with just

npx foam-cli obsidian migrate ./path

Or something similar.

If we wanted to download the latest template, I wouldn't do it in the migration, instead add a separate command like foam-cli init that could be run first.

So far that work is pretty much non-existent, but if anyone would like to run with it, here's a bit that could help generate the link reference definitions:

• FoamWorkspaceManager can ingest markdown files
• FoamWorkspaceManager#getNoteWithLinks can output the links and backlinks for a document
• There's a bit of code in foam-vscode that can generate the markdown link reference lists

I don't expect you to contribute that to foam-workspace-manager directly, just putting the info out there.

I'll add this item to the roadmap and try to find resources to get this solved ASAP, if anyone doesn't do it before me.

Thanks again for this Joe! Btw, if you're more comfortable writing scripts in JS than bash, you could also write this as a Node script, since that's what we'll have to do for the Foam CLI eventually!

[jsjoeio]

I figured out the heading part! 😄

# Loop over all markdown files
for file in *.md

  do

    # Remove the file extension
    filename=${file%.*}

    # Create new filename it by replacing spaces with - and lowercasing it
    newFileName="`echo $file | sed s/\ /\-/g | tr 'A-Z' 'a-z'`"

    # Rename file
    mv "$file" "$newFileName"

    # Capitalize the first letter for heading
    heading="$(tr '[:lower:]' '[:upper:]' <<< ${filename:0:1})${filename:1}"

    # Add heading + old fild together in temp file then rename to new file
    echo "# $heading\n" | cat - $newFileName > temp && echo "y" | mv temp $newFileName

  done

Generate link reference definitions

Ah! Good call. Totally forgot about this.

Btw, if you're more comfortable writing scripts in JS than bash, you could also write this as a Node script, since that's what we'll have to do for the Foam CLI eventually

Good to know! I feel way more comfortable in JS + the fact that it will eventually be in the CLI is enough to convince me! I'll see if I can take a stab at it sometime (but anyone else is welcome to as well!).

And thanks for all the additional context - that's super helpful too!

[jsjoeio]

These aren't necessary until you want to publish your Foam workspace. Right now they are generated only when saving a file in VS Code, so if you wanted to make all the links work in the published Foam, you'd have to go and save every file :)

I'm not publishing my Foam workspace (mine will be in a private repo) so this isn't as important right now.

I tested out my script and it looks as if things are working! (The graph looks a bit funky, but that is okay)

[jevakallio]

Amazing! Can you show me what you mean by funky graph? If you don't want to share your content, maybe zoom it out til the text is illegible, or obfuscate it with a bit of blur?

[jsjoeio]

Sure!

My guess is the daily notes from Obsidian.

1. My script didn't add a heading to these
2. I don't think the backlinks in these files are showing up, which is causing all the islands in the graph

Also, now that I'm thinking about it, I wonder if it's possible to have a file such as 2020-06-02.md but have the heading in the file be June 2, 2020?

[youleaf]

Here a js version of @jsjoeio script.

It works for me; attachments are imported correctly as long as they are all contained in the same folder.
It doesn't generate Link Reference Definitions, but they are created as soon as the file is opened/saved in foam.

Thank you for the info you provided in this issue, they helped a lot!

foam obsidian migration script:
https://gist.github.com/youleaf/54fb5a1d7b22668543ace2764173992a

[jsjoeio]

Woohoo! Thanks for writing a JS version @youleaf 🚀 This is going to be really helpful when we incorporate this into the CLI (we'll give you credit, and if you want to help with that when the time comes, let us know!)

It doesn't generate Link Reference Definitions, but they are created as soon as the file is opened/saved in foam.

We'll have to figure this out down the road.

Thanks for sharing your script too! Looks great, and the comments make it easy to follow too 😍

[youleaf]

Woohoo! Thanks for writing a JS version @youleaf 🚀 This is going to be really helpful when we incorporate this into the CLI (we'll give you credit, and if you want to help with that when the time comes, let us know!)

I think you can import from any markdown-based dir structure (not only obsidian) reusing most of the code: you just need to adjust regexps and file naming accordingly. I'll be glad to help, count me in!

[HyroVitalyProtago]

I've just discovered Obsidian and Foam. So I've not yet choose one to begin with. Seems like Foam have a better philosophy with open source, but maybe isn't completly ready yet. So I'm thinking about starting with Obsidian and waiting some month before switching to Foam. Did you think it's a good move? Did you think that we will be able to use the two without changing files name, maybe with a compatibility parameter, because the only difference seems to be file naming convention?

[youleaf]

I switched from obsidian to foam months ago and I'm pretty happy now - the foam + vscode combo give you tons of customization possibilities.
I wrote a conversion script from obsidian (link in my comment above) and it worked well - so I think you can switch anytime.
If you are ok with working on alpha stage software and you feel comfortable with foam, my suggestion is to not bother with the conversion. Otherwise, Obsidian is simple and user friendly - a good piece of sw to work with.