Gallery-dl: [ASK] How to convert local downloaded ugoira zip into GIF/WebM/MP4/etc?

[indrakaw]

The gallery is downloaded locally. How do I do to make video out of it so I can play it?

[mikf]

Assuming it is still possible to access the ugoira metadata on Pixiv, you can convert your already downloaded ZIP files by invoking gallery-dl with the same Pixiv URLs and some "special" options:

$ gallery-dl -o part=false --ugoira-conv --filter "type == 'ugoira'" https://www.pixiv.net/member.php?id=11

• -o part=false: recognize your already downloaded .zip files as "temporary" files. Otherwise it would re-download them.
• --ugoira-conv: enable an ugoira post-processor with more or less default options. You can also leave this one out and setup your own with custom options in your config file, for example like so
• --filter "type == 'ugoira'": ignore everything that's not an ugoira animation

[indrakaw]

Is it possible if the gallery itself is removed by author or bad ID?

[mikf]

Kind of, but not with gallery-dl. You need to somehow replicate the process in postprocessor/ugoira.py, except you won't have any frame-delay data available; unless you used --write-metadata or similar when downloading those files.

Basically it's

• unzip the zip archives
• write a concat file with the last frame duplicated and guessed duration values
  ffconcat version 1.0 file '000000.jpg' duration 0.06 file '000001.jpg' duration 0.06 file '000002.jpg' duration 0.06 ...
• run ffmpeg with -r 1000/60 -i ffconcat.txt ... output.webm as arguments
  
  
  
  • 1000/60 is the input frame rate, basically the same as the duration values
  
  
  • ... can be any ffmpeg video encoding options, for example -c:v libvpx -crf 4 -b:v 5000k -an
  
  

You could also take a look at https://github.com/mikf/ugoira-conv, but that hasn't seen any updates in years. Or maybe you can find the animation on Danbooru. They store the original ZIP with frame-delay data.

[indrakaw]

I see.

As your example above, is it possible to perform single illustration instead the artist url? Your example is artist URL.

[mikf]

Yes, that will work as well.

And another thing that might be important: --ugoira-conv will delete the source ZIP archives after converting them. So copy them beforehand or manually configure an ugoira post-processor and set keep-files to true if you want to keep them.

[indrakaw]

Thanks.

Question answered.

[indrakaw]

Uh oh. I have a trouble. It deleted the zip. Would you please give me an example of gallery-dl conf for this keep-file?

[Hrxn]

Uh, what? You mean like this?

{
    "extractor":
    {
        "base-directory": "./gallery-dl/",
        "archive": "./gallery-dl/archivefile.db",
        "cookies": null,
        "cookies-update": false,
        "proxy": null,
        "skip": true,
        "sleep": 0,
        "path-restrict": "auto",
        "path-remove": "\\u0000-\\u001f\\u007f",
        "user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0",

        "postprocessors": [
            {
                "name": "ugoira",
                "whitelist": ["pixiv", "danbooru"],
                "ffmpeg-twopass": true,
                "ffmpeg-args": ["-c:v", "libvpx", "-crf", "4", "-b:v", "5000k", "-an"],
                "keep-files": true
            },
            {
                "name": "metadata",
                "whitelist": ["danbooru", "yandere", "sankaku"],
                "mode": "tags"
            }
        ],

        "artstation":
        {
            "external": false
        },
        "danbooru":
        {
            "username": null,
            "password": null,
            "ugoira": true
        },
        "deviantart":
        {
            "refresh-token": null,
            "extra": false,
            "flat": true,
            "folders": false,
            "journals": "html",
            "mature": true,
            "metadata": false,
            "original": true,
            "quality": 100,
            "wait-min": 0
        },
        "exhentai":
        {
            "username": null,
            "password": null,
            "original": true,
            "wait-min": 3.0,
            "wait-max": 6.0
        },
        "flickr":
        {
            "access-token": null,
            "access-token-secret": null,
            "videos": true,
            "size-max": null
        },
        "gelbooru":
        {
            "api": true
        },
        "gfycat":
        {
            "format": "mp4"
        },
        "idolcomplex":
        {
            "username": null,
            "password": null,
            "wait-min": 3.0,
            "wait-max": 6.0
        },
        "imgur":
        {
            "mp4": true
        },
        "instagram":
        {
            "highlights": false
        },
        "kissmanga":
        {
            "captcha": "stop"
        },
        "nijie":
        {
            "username": null,
            "password": null
        },
        "oauth":
        {
            "browser": true
        },
        "pixiv":
        {
            "username": null,
            "password": null,
            "ugoira": true
        },
        "reactor":
        {
            "wait-min": 3.0,
            "wait-max": 6.0
        },
        "readcomiconline":
        {
            "captcha": "stop"
        },
        "recursive":
        {
            "blacklist": ["directlink", "oauth", "recursive", "test"]
        },
        "reddit":
        {
            "refresh-token": null,
            "comments": 500,
            "morecomments": false,
            "date-min": 0,
            "date-max": 253402210800,
            "date-format": "%Y-%m-%dT%H:%M:%S",
            "id-min": "0",
            "id-max": "zik0zj",
            "recursion": 0,
            "user-agent": "Python:gallery-dl:0.8.4 (by /u/mikf1)"
        },
        "sankaku":
        {
            "username": null,
            "password": null,
            "wait-min": 3.0,
            "wait-max": 6.0
        },
        "seiga":
        {
            "username": null,
            "password": null
        },
        "tumblr":
        {
            "avatar": false,
            "external": false,
            "inline": true,
            "posts": "all",
            "reblogs": true
        },
        "twitter":
        {
            "content": false,
            "retweets": true,
            "videos": false
        },
        "wallhaven":
        {
            "api-key": null
        },
        "booru":
        {
            "tags": false
        }
    },

    "downloader":
    {
        "part": true,
        "part-directory": null,

        "http":
        {
            "adjust-extensions": true,
            "mtime": true,
            "rate": null,
            "retries": 4,
            "timeout": 30.0,
            "verify": true
        },

        "ytdl":
        {
            "format": null,
            "forward-cookies": true,
            "mtime": true,
            "outtmpl": null,
            "rate": null,
            "retries": 4,
            "timeout": 30.0,
            "verify": true
        }
    },

    "output":
    {
        "mode": "auto",
        "progress": true,
        "shorten": true,
        "log": "[{name}][{levelname}] {message}",
        "logfile": null,
        "unsupportedfile": null
    },

    "netrc": false
}

[indrakaw]

Should it be like this?

{
    "ugoira":
        {
            "keep-files": true
        }
}

or like this?

{
    "extractor":
    {
        "postprocessors": [
            {
                "keep-files": true
            },
        ],
    }
}

Please correct me.

[Hrxn]

Like this:

{
    "extractor":
    {
        "postprocessors": [
            {
                "name": "ugoira",
                "keep-files": true
            }
        ]
    }
}

You need (at least) the "name": "ugoira", in the postprocessor object, otherwise gallery-dl does not know what this is supposed to be.

Remove the comma behind the object in the "postprocessors" array (because this array contains only one object here, and remove the comma behind the end of the array ], because it's the last entry in the extractor object here.

[indrakaw]

Thanks. That helps me a ton.