Codimd: CodiMD command line interface

[pirate]

UPDATE: this exists now, see codimd-cli

I propose we create an official CLI to interact with the hackmd server. It will allow people to build tons of other integrations, and script common workflows to their liking, hopefully providing an easy solution to a handful of open PRs in the process.

Examples of stuff that could be built on top of this command:

• filesystem sync
• automated markdown to pdf/html rendering
• backups to dropbox, google drive, RAID array, or any cloud
• static site generation
• note sharing button
• webhook triggered actions
• automated new user onboarding flow
• automated permissions system linked to LDAP or unix users & groups

./hackmd [import|export|publish] ...:

#!/usr/bin/env bash

help_str="
Usage:
    $ ./hackmd import test.md
    qhmNmwmxSmK1H2oJmkKBQQ       # returns note id on success

    $ ./hackmd publish qhmNmwmxSmK1H2oJmkKBQQ
    /s/S1ok9no3f                 # returns publish url

    $ ./hackmd export --pdf qhmNmwmxSmK1H2oJmkKBQQ my_note.pdf
    $ ./hackmd export --md qhmNmwmxSmK1H2oJmkKBQQ my_note.md
    $ ./hackmd export --html qhmNmwmxSmK1H2oJmkKBQQ my_note.html
    $ ./hackmd export --slides qhmNmwmxSmK1H2oJmkKBQQ my_slides.zip

    $ env HACKMD_SERVER='https://hackmd.example.com' ./hackmd import test.md
"

HACKMD_SERVER=${HACKMD_SERVER:="http://127.0.0.1:3000"}  # could also read this from config.json

function publish_note() {
    curl "$HACKMD_SERVER/$1/publish" 2>/dev/null | perl -pe 's/Found. Redirecting to \/(.+?)$/\/$1\n/gs'
}

function import_note() {
    curl -q -XPOST -H 'Content-Type: text/markdown' --data-binary "@$1" "$HACKMD_SERVER/new" 2>/dev/null | perl -pe 's/Found. Redirecting to \/(.+?)$/$1\n/gs'
}

function export_note() {
    if [[ $1 == "--pdf" ]]; then
        wget -O "$3" "$HACKMD_SERVER/$2/pdf"
    elif [[ $1 == "--md" ]]; then
        wget -O "$3" "$HACKMD_SERVER/$2/md"
    elif [[ $1 == "--html" ]]; then
        publish_url=$(publish_note "$2")
        wget --recursive --convert-links -O "$3" "$HACKMD_SERVER$publish_url"
    elif [[ $1 == "--slides" ]]; then
        wget --recursive --convert-links -O "$3" "$HACKMD_SERVER/$2/slide"
    else
        echo "Usage: ./hackmd export [--pdf|--md|--html|--slides] <note_id> <output_file>"
    fi
}

if [[ "$1" == "import" ]]; then
    import_note "$2"
elif [[ "$1" == "publish" ]]; then
    publish_note "$2"
elif [[ "$1" == "export" ]]; then
    export_note "$2" "$3" "$4"

### TODO
#
# `hackmd inviteuser <email_to_invite>`
# `hackmd chmod <permissions> <note_id>`
# `hackmd chown <user> <note_id>`
# `hackmd delete <note_id>`
# `hackmd list --all` list all notes on the server by id: title
# `hackmd list <user_email>` list notes for a given user by id: title
# `hackmd search <query>` find a note ids matching a given query
#
###

else
    echo "$help_str"
fi

[pirate]

UPDATE: a cli exists now, see codimd-cli

Some PRs/Issues that may benefit from a command line interface:

• Need tests: super easy to write tests with the bash script! now you can write a full UI integration test in 1 line of bash
• Saving to local filesystem: this is easy with ./hackmd export
• Export to PDF: ./hackmd export --pdf <note_id> <output.pdf> bam.
• Arbitrary file upload: we should support this with ./hackmd import <file.extension>
• Export slides as PDF & https://github.com/hackmdio/hackmd/issues/545: we should add this to ./hackmd export --slides once it's ready
• Show all files in database: ./hackmd list --all
• Import/export file in GitHub repository: wget <file from github> > file.md; ./hackmd import file.md
• API to download HTML: works already ./hackmd export --html <note_id> note.html
• CLI for adding new note: can be documented in the README with hackmd command
• "Takeout" feature: ability to download all user data with ./hackmd export

Permissions:

• Support permission of invitee only: ./hackmd inviteuser --permissions r <user_email> could work by first chmod and chowning the note, then sending an email invite to that note
• Find the notes by the owner: ./hackmd list <user_email> works as a rudimentary API to do this form the command line
• Add user administration: setup a script that creates all the users & their notes from .md files on disk, chown & chmods them to their proper permissions, then invites all the users to join. Would be possible to run this on a timer to do regularly as well.

Sync backends:

• Import/Export with Github: trigger ./hackmd import|export script via timer or webhook upon github file modification
• Import/Export with Evernote: same deal as github/dropbox sync, write a script with import/export
• Support for ownCloud: same deal as github/dropbox sync, write a script with import/export
• Support auto sync with dropbox: write a script that does a full import and export or merge in whatever direction desired
• Support auto sync with Google Drive: write a script that does a full import and export or merge in whatever direction desired

Hopefully a CLI will have the effect of drawing some of the complexity of these feature requests out of the node server and into external scripts, allowing the node server to maintain a small API surface area.

[SISheogorath]

Nice work!

I would also use wget --recursive --convert-links for the publish page download.

For most stuff it would be nice to simply expose a /api/ path and put a documented REST API there. This would make it very stable while all the other stuff we have build right now can slowly move over.

I don't know when I'll find the time to do this, but I really like to see some progress here :+1:

(This is just a quick review, will add some more details later :))

[ccoenen]

while we're at it: this could be a hackmd-cli npm package in its own repo. This way, we might also reduce the number of dependencies over time.

[pirate]

@ccoenen as it stands the only cli dependency is wget, and even that could be removed, is a separate package really needed?

[ccoenen]

Depends on how you look at it.

If this is something solely for the admins: no, it should stay in here.
If this is something a tech-savvy end-user should also be allowed to use: yes, please give him a tiny CLI package without the whole dependency tree of hackmd itself.

On a different matter (but also somewhat depending on the question above), I would like this to be JavaScript instead of bash. For one thing, if it is used by end-users it will be used by a lot of people running windows. Also, I believe nobody will argue that JavaScript is a more expressive language and it is a language very familiar to the HackMD Community, so it would help maintainability.

[pirate]

Sorry, ignore the close and reopen, that was an accident.

I don't know if I have the time to re-write this in JS at the moment, but maybe in a few months if someone else doesn't do it before me. For now the bash version is fulfilling all my needs.

[jakubgs]

This is the one thing that I need to really incorporate hackmd use into my workflow.
I agree with @ccoenen, if this is a CLI tool for pro users(like it probably should be) it needs it's own package for easier deployment.

[SISheogorath]

Mhm, I'm still not sure where we want to put this.

An own repo makes sense, but only when the tool is written in something else than 10 lines of bash.

If someone wants to rewrite this (in go or rust or JS) I'll provide a repository for sure. I know that I don't have the time to do this right now :/ but would be awesome!

[ccoenen]

I think the proposed featureset and terminology is spot on. As mentioned before, bash isn't my first choice, but it's better than "oh wait having this in XYZ would be awesome, one day".

So in my opinion: I opened a repo and added @pirate to the contributors. The repo is named codimd-cli, It should contain the script, a readme and everyone lives happily ever after. I added AGPL to it, I believe GPL would also be fine (as AGPL is targeted at server-run code, which does not especially apply there. Feel free to change that.)

[pirate]

Thanks for creating that repo @ccoenen. I have pushed the script along with some documentation: https://github.com/hackmdio/codimd-cli

Let me know if any of you have suggestions/content improvements for the script by using the issue tracker over on on that repo.

[ccoenen]

Thank you :-) I'll close this ticket here, then. Everyone: feel free to chime in over at the other repo as well :-)