Gallery-dl: Tumblr gives "Daily API rate limit exceeded" error even when I didn't try to download anything from it today

[KirbyFan102]

Title says it all. And I didn't even access tumblr today at all, I'm prety sure.

[kattjevfel]

It's shared with everyone else also using gallery-dl, sign in with your own account for the best experience. (Unless you've done that, then this is some other issue)

[KirbyFan102]

"tumblr": { "avatar": false, "external": false, "inline": true, "posts": "all", "reblogs": "same-blog", "access-token": "xxxxxx", "access-token-secret": "xxxxxx", "filename": "{filename}.{extension}"

This is my tumblr configuration. I thought the access-token parts were what I needed to login. Do I need my username and password also?

[biznizz]

https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#extractortumblrapi-key--api-secret

Looks like there's an additional api-key and api-secret thing you can do. Sounds interesting, think I'll give it a whirl myself.

[KirbyFan102]

Well, I'm confuddled now. Am I supposed to remove my access-tokens and replace them with my api key? Or keep both? Where did I even an access token without registering an application? Where did I get the idea that I could put these configurations in the config file when I can't even find "access-token" _or_ "access-token-secret" in the list of configurations? And how will this solve the rate api limit exceeded problem when it seems you have to request its removal to the tumblr staff specifically?

[biznizz]

I believe it's like Deviantart where you use both the Oauth access-token and access-token-secret and the api-key and api-secret.

Where as you can have the direct Oauth credentials written to your cache with cache, I think you have to put the actual values of the API's into the config file.

EDIT: It's weird that the DA Oauth is called refresh-token whereas tumblr's Oauth is called access-token, but as far as I'm aware, they're both the same thing.

EDIT 2: As far as I'm aware, you'll have a greater api limit with registering the app than without; but if you hit the barrier, then requesting the removal (or further raising of the limit) is hopefully an easy step; its probably to stop bots or something, so showing you're a human probably won't be a hassle (maybe, it's tumblr, a site that's basically slowly killing itself afterall).

[KirbyFan102]

Okay. Thanks for the clarifications. Hopefully this will make the problem go away.

[biznizz]

Don't forget to run oauth after registering the app. Since you'll have to give permission for the oauth to go through the app.

I ran it myself, and didn't have any issues with my test tumblr.

[indrakaw]

You might want to consider this: https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#extractortumblrdate-min--date-max

This happened to me. I was backing a Tumblr post and it has 80k+ posts since 2014. By design, gallery-dl is backing up by date decent order, this make it start over from recent post to last backed up (which consumes the API; way to wasted api rates). You have take a note on last downloaded file (usually by ID), get its date as start point of back up (tumblr.date-max). For further explanation, please see this: https://github.com/mikf/gallery-dl/issues/337

It has a disadvantage. Recent posts are ignored. You might want to disabled it once you got all past posts backed up, then re-run.

Addendum:
extractor.tumblr.archive doesn't help much since it doesn't record datetime. I wish for better method.