Newsboat: Image previews

[Sermak-GIT]

Certain rss entries, for example those from twitter, often contain links to images. My goal is to display these images within newsboat. There is a way to do this for vifm (see distrotube).

Using newsboats pipe-to operation, I use the following script (called nbimagerender) to parse, download and display the images (so far only those on twitter).

#!/bin/bash
cd /tmp
while read line; do
    url=$(echo $line | grep pic.twitter.com | sed 's;.*pic.twitter.com/;;')
    if ! [[ -z $url ]]; then
    wget "pic.twitter.com/$url"
    imgfile=$(wget $(cat $url | grep "<img data-aria-label-part" | tr '"' '\t' | awk -F'\t' '{print $2}') 2>&1 | grep "saved" | awk '{print $6}')
    ~/.config/vifm/scripts/vifmimg draw 2 34 80 80 ${imgfile:1:-1}
    fi
done

Just like vifm, for this to work, newsboat needs to be started using

#!/usr/bin/env bash
export FIFO_UEBERZUG="/tmp/vifm-ueberzug-${PPID}"

function cleanup {
    rm "$FIFO_UEBERZUG" 2>/dev/null
    pkill -P $$ 2>/dev/null
}

rm "$FIFO_UEBERZUG" 2>/dev/null
mkfifo "$FIFO_UEBERZUG"
trap cleanup EXIT
tail --follow "$FIFO_UEBERZUG" | ueberzug layer --silent --parser bash &

newsboat
cleanup

This works nicely, as can be seen here:

Still, there are two problems:
First, the image persists for the livespan of the newsboat instance, so even once I leave the entry and enter the feed view. In the case of vifm, this is solved by it automatically executing vifmimg clear. Unfortunately, there is no way to execute a shell command from within newsboat.
Second, even if there was such an option, the image render and image clear script/command would need to be invoked manually for every new article that is opened and closed.
There already are commands starting with notify- that execute something once a new article arrives, so would it be a stretch to add hooks that execute a shell command every time an article is opened or closed?

For example, a hypothetical config option like auto-pipe-to [cmd] would execute pipe-to [cmd] every time an article is opened. An option such as run-on-article-close [cmd] would execute a shell command every time an article is closed (or even every time the view is switched/refreshed).

[Minoru]

Wow, I'm genuinely impressed! People ask from time to time if it's possible to display images within Newsboat, and I always tell them no. But as Einstein said, "everybody knows that something can't be done but then somebody turns up and he doesn't know it can't be done and he does it" :)

Have you tried using your script as a pager? This setup has the following useful properties:

• pager is called for each article you read;
• Newsboat suspends its UI before calling the pager, so there is no risk of paining over something important. And it restores the UI as well, so there is no risk of breaking something.

If you specify /bin/cat as your html-renderer, the pager will have access to article's HTML, which might be easier/less brittle to parse than Newsboat's text output.

Right now, hooks do sound like a stretch, but let's not dismiss this idea yet.

[Sermak-GIT]

I got it to work almost flawlessly:

Run newsboat using the script above (set an alias in your shell rc file). You need(!) to save this script under the name nbrun (else you need to adapt the xdotool command). Next, go and install https://github.com/EricChiang/pup. Then, save the following script as nbparser:

#!/bin/bash

PUP='/opt/go/bin/pup' # CHANGE THIS (default install under ~/go/bin/pup)
# see https://github.com/EricChiang/pup

lines=$(tput lines)
cols=$(tput cols)

# numbers determined by experimentation
#img_width=50 # $(($cols * 10 / 14))
#img_height=50 # $(($lines * 10 / 7))
#img_x=2
#img_y=$(( (30 * $lines) / 58 )) # $(( ($lines - $img_height ) * 10 / 8))

img_x=2
if [[ $lines = 58 || $lines = 57 ]]; then # my full height
    img_width=80
    img_height=20 
    img_y=35
elif [[ $lines = 27 ]]; then # my half height
    img_width=40
    img_height=20 
    img_y=15
else
    notify-send "newsboat: unsupported size"
fi

cd /tmp
while read line; do
    cp $1 /tmp/testing
    if [[ "$line" =~ ^Feed: ]]; then
    : # to be used later, maybe?
    elif [[ $line =~ "pic.twitter.com" ]]; then
    url=$(echo $line | $PUP 'a text{}' | sed 's;.*pic.twitter.com/;;')
    if ! [[ -z $url ]]; then
        wget "pic.twitter.com/$url"
        imgfile=$(wget $(cat $url | grep "<img data-aria-label-part" | tr '"' '\t' | awk -F'\t' '{print $2}') 2>&1 | grep "saved" | awk '{print $6}')
        ~/.config/vifm/scripts/vifmimg draw $img_x $img_y $img_width $img_height  ${imgfile:1:-1}
    fi
    elif [[ $line =~ "https://www.youtube.com/watch?v=" ]]; then
    url=$(echo $line | sed 's;^Link: ;;')
    if ! [[ -z $url ]]; then
        imgfile=$((wget $(youtube-dl --get-thumbnail $url)) 2>&1 | grep "saved" | awk '{print $6}')
        ~/.config/vifm/scripts/vifmimg draw $img_x $img_y $img_width $img_height  ${imgfile:1:-1}
    fi
    fi


done < $1 1>/dev/null && xdotool key --window "$(xdotool search --name nbrun | head -1)" h+q &

w3m -dump -T text/html <$1 | less +k && ~/.config/vifm/scripts/vifmimg clear

(Depending on your taste, you may remove the && xdotool key --window "$(xdotool search --name nbrun | head -1)" h+q & part. This part is used to fetch the image in the background and then refresh less (by going to help and back). Without it, you fetch in the foreground, which, depending on your internet, might be very fast).
So far, this works for twitter and youtube.
Change the desired image sizes (I tried to make them a function of the terminal size, but failed) and the location of PUP. Also install überzug, xdotool, wget, w3m (I may have missed something) and get the vifmimg script (included here for convenience). Remember to update the paths above, if applicable

#!/usr/bin/env bash
readonly ID_PREVIEW="preview"

#AUTO_REMOVE="yes"
# By enabling this option the script will remove the preview file after it is drawn
# and by doing so the preview will always be up-to-date with the file.
# This however, requires more CPU and therefore affects the overall performance.

if [ -e "$FIFO_UEBERZUG" ]; then
    if [[ "$1" == "draw" ]]; then
        declare -p -A cmd=([action]=add [identifier]="$ID_PREVIEW"
                           [x]="$2" [y]="$3" [width]="$4" [height]="$5" \
                           [path]="${PWD}/$6") \
            > "$FIFO_UEBERZUG"

    elif [[ "$1" == "videopreview" ]]; then
        echo -e "Loading preview..\nFile: $6"
        [[ ! -d "/tmp${PWD}/$6/" ]] && mkdir -p "/tmp${PWD}/$6/"
        [[ ! -f "/tmp${PWD}/$6.png" ]] && ffmpegthumbnailer -i "${PWD}/$6" -o "/tmp${PWD}/$6.png" -s 0 -q 10
        declare -p -A cmd=([action]=add [identifier]="$ID_PREVIEW"
                           [x]="$2" [y]="$3" [width]="$4" [height]="$5" \
                           [path]="/tmp${PWD}/$6.png") \
            > "$FIFO_UEBERZUG"

    elif [[ "$1" == "gifpreview" ]]; then
        echo -e "Loading preview..\nFile: $6"
        [[ ! -d "/tmp${PWD}/$6/" ]] && mkdir -p "/tmp${PWD}/$6/" && convert -coalesce "${PWD}/$6" "/tmp${PWD}/$6/$6.png"
        for frame in $(ls -1 /tmp${PWD}/$6/$6*.png | sort -V); do
           declare -p -A cmd=([action]=add [identifier]="$ID_PREVIEW"
                           [x]="$2" [y]="$3" [width]="$4" [height]="$5" \
                           [path]="$frame") \
            > "$FIFO_UEBERZUG"
            # Sleep between frames to make the animation smooth.
            sleep .07
        done

    elif [[ "$1" == "pdfpreview" ]]; then
        echo -e "Loading preview..\nFile: $6"
        [[ ! -d "/tmp${PWD}/$6/" ]] && mkdir -p "/tmp${PWD}/$6/"
        [[ ! -f "/tmp${PWD}/$6.png" ]] && pdftoppm -png -singlefile "$6" "/tmp${PWD}/$6"
        declare -p -A cmd=([action]=add [identifier]="$ID_PREVIEW"
                           [x]="$2" [y]="$3" [width]="$4" [height]="$5" \
                           [path]="/tmp${PWD}/$6.png") \
            > "$FIFO_UEBERZUG"

    elif [[ "$1" == "clear" ]]; then
        declare -p -A cmd=([action]=remove [identifier]="$ID_PREVIEW") \
            > "$FIFO_UEBERZUG"
        [[ ! -z $AUTO_REMOVE ]] && [[ -f "/tmp${PWD}/$6.png" ]] && rm -f "/tmp${PWD}/$6.png"
        [[ ! -z $AUTO_REMOVE ]] && [[ -d "/tmp${PWD}/$6/" ]] && rm -rf "/tmp${PWD}/$6/"

    fi
fi

All that's left to do is to edit the newsboat config. Add

pager "~/.scripts/nbparser"
html-renderer "/bin/cat"

(or where ever you saved the parser script)

Caveat: since I am replacing the pager with less, I lose all newsboat functionality. So I can't open urls via ´o´ or ´show-urls´, and can't quit the pager using h (use q, which is terrible for my muscle memory).

The only way to resolve this would be to use a better pager. It would be great if one could use the internal newsboat pager, but since that's not possible, are there any other pager that at least allow for opening urls? Some terminals have such a functionality, but it would be nicer to have it in the pager itself.

If someone reading this decides to improve my scripts, please send me a notification (comment here or email).

[Minoru]

This is really cool! We have a special directory, contrib, where stuff like this lives. The contents of that directory is shipped with Newsboat, and starting with next release it will even be installed to /usr/share/doc/newsboat. Would you like to submit a PR to add your scripts there? If so, just create a directory under contrib, put the scripts there, and use your comment as a base for README. I guess the only change it needs is an explicit mention of your GitHub nickname and your email.

I don't like to keep "non-actionable", open-ended issues open, and contrib is much better suited for getting these scripts out to people and soliciting feedback.

[Sermak-GIT]

I just submitted a pull request. Here is the final result:

[Minoru]

Cool! There is nothing to be done about this issue now, so I'm closing it. Thanks for the contribution!