Mediaelch: IMDB - New Layout

[ticao2]

Scrapers that don't work:

• [x] IMDb
• [ ] OFDb
• [ ] TMDb
• [ ] TMDbConcerts
• [ ] TheTvDb
• [ ] TvTunes
• [ ] AEBN
• [ ] AdultDvDEmpire
• [ ] HotMovies
• [ ] VideoBuster

Details which are wrong / not loaded:
I found today in IMDB a yellow bar in the header, with the following text:

Get a sneak peek of the new version of this page

It looks like the Layout is going to be modified.
And it will probably cause some change to the Scraper. Once again.
Sorry to be the bearer of this bad news. :-)

[bugwelle]

Thanks! I don't get such a bar, yet. Could you add a screenshot? :)

[ticao2]

NEW

OLD

[bugwelle]

Ah, in Chrome I get the same! Thanks. Will look into it ASAP.

[bugwelle]

I can use the new format in MediaElch by copying the cookies accordingly., e.g.

static void addCookies(QNetworkRequest& req)
{
    req.setRawHeader("cookie",
        "<header here>");
}

And thanks! The only thing that still works is the released date (and only the year there).

[bugwelle]

Working on it. See #1092

[ticao2]

Just to make sure.
You are doing a lot of "commits" every day for a long time.
But I didn't see anything on bintray.com.
Will nightly version 2.8.1 ever appear there? :-)

[bugwelle]

Will nightly version 2.8.1 ever appear there? :-)

Not soon. See https://github.com/Komet/MediaElch/issues/1052

We have reached our limits on TravisCI. I'm working on some scripts to make it easier to publish Nightlies from my machine, see https://github.com/Komet/MediaElch/pull/1084

I may create a bugfix release this weekend due to some really annoying issues in v2.8.0, for example #1090

[bugwelle]

This may take a while. Their HTML is the worst. Takes forever to adapt our RegExs

[ticao2]

No panic.
Just curious. Also to see if the translations for pt-BR are correct.

[bugwelle]

Ah, ok. I can deploy a nightly tomorrow. I'll update the translations first then. :-)

[ticao2]

Boy, you haven't stopped making commits!
I was almost giving up. :-)
Hallelujah

---

EDIT
I'll even check if there was no new word to translate.
Edit: No, no new ones. 0 to translate.

[bugwelle]

hmm... I wondered why there weren't any new words to translate. I think some strings weren't extracted. I'll look into this in the next few days. :-)

[txtsd]

This may take a while. Their HTML is the worst. Takes forever to adapt our RegExs

Why don't you parse the HTML with lxml or something similar instead of using RegExs?

[bugwelle]

Wouldn't help much. I would still have to search for text-elements that have a certain pattern. They don't tag their HTML elements with e.g. "director" but use general <div>s or <span>s with classes that don't have any semantic meaning.

I thought about this in the past as well. But it would be more work and may break more easily. IMDb changes the nesting of certain elements more often and I can't check for HTML id attributes because there aren't any (useful ones).

And on top of that I don't know how good Qt's HTML parser is. IMDb had some HTML code in the past that was not compliant to the standard. Browsers are just very forgiving in that regard.

[txtsd]

Fair enough.

[ticao2]

hmm... I wondered why there weren't any new words to translate. I think some strings weren't extracted. I'll look into this in the next few days. :-)

Yesterday and today I saw that only 4 strings were not translated into German.
I think this is good news for you. :-)

[bugwelle]

Yesterday and today I saw that only 4 strings were not translated into German.
I think this is good news for you. :-)

Yes. But I think some strings were not extracted from the source, i.e. we won't be able to translate them even if we wanted. That's what I meant ;-)

[ticao2]

I just installed version 2.8.2 2020-12-20 in Portable mode.
In the Movies section I didn't see any of the translation changes I made to pt-BR in the last 30 or 60 days.

EDIT
And Transifex contains 0 (zero) strings to translate to pt-BR.
Therefore, neither the new strings nor the old strings that I modified came in this version 2.8.2 :-)

[bugwelle]

What? For real? That should not happen. I don't select specific languages. I update everything at once. I'll have a look at this in the next days. But as I'm on Christmas vacation starting tomorrow, I'll only be able to update the translations next year (in a week or two) :)

[bugwelle]

Seems that the IMDb summary page stays the same?

https://www.imdb.com/title/tt2277860/reference

I'll look into using that.

[ticao2]

Yes. I believe that only the view for visitors will change.
The new layout looks more pleasant to me.
But some parts of the new one are still under construction.
So we can expect good and bad surprises ahead.
And I believe that the summary/reference page should be the last to be modified.
If it will be modified.