Tachiyomi: Wrong chapter ordering

[j2ghz]

• [x] As you can see there is one extra chapter Onepunch-Man Punch Ver002 028

https://cloud.githubusercontent.com/assets/1708197/12069149/7922750e-b021-11e5-8005-a8436aa57791.png
https://cloud.githubusercontent.com/assets/1708197/12069150/98cd49ba-b021-11e5-801d-7d1fa1401f47.png

Version r285 31 Dec 2015 7:21pm

[j2ghz]

It seems the problem is you get the chapters from html and then parse the number of each chapter and then sort them. OnePunch Man on kissmanga uses different type of naming:
OnePunch Man 001 - 056
OnePunch Man Ver002 001 - 0086
Which is like two mangas in one? (I think) (OnePunch Man 001 starts the same as OnePunch Man Ver002 001)

Chapter Onepunch-Man Punch Ver002 028 is parsed as chapter number 2028 (confirmed when submitted to MAL) and thus sorted as the last one.
The others are sorted like this:

1. OnePunch Man 001
2. OnePunch Man Ver002 001
3. OnePunch Man 002
4. OnePunch Man Ver002 002

[inorichi]

I fixed the chapter being recognized as 2028 here https://github.com/inorichi/mangafeed/blob/master/app/src/test/java/eu/kanade/mangafeed/ChapterRecognitionTest.java#L117

The chapters have to be deleted in order to recognize the number again. If the "Ver002" is a common keyword in Kissmanga (I don't usually read here) we could add another pattern for it.

I want to add an additional sorting method based on the upload date, but I have to save sorting options first.

[inorichi]

I improved the recognition again. I will keep this issue open for future improvements.

If anyone finds a wrong chapter order, post an example here to try to fix it.

[myuc]

Take a look at the manga "capeta", both on mangafox and batoto.

For mangafox: http://mangafox.me/manga/capeta/
Chapters for each volume are divided into chpt 1-4 and in the app, its sorted in alphabetical order ignoring volume. i.e. chapter 1 for each volume followed by 2 and so on

For batoto: http://bato.to/comic/_/comics/capeta-r628
Although you can see Vol:xx Ch.XX, it is still sorted in ch no only and ignore the Vol:xx in the title.

PS: not sure if sorting by date is a good idea as sometimes earlier chapters could be updated/added later on.

[inorichi]

Currently the app ignores the volume, but I didn't expect that ordering. Sorting by date would be an alternative. There won't be a perfect solution. I'm even considering to add a third date to every chapter on the db to sort the same way than the source.

[icewind1991]

Some chapter number parsing improvements: https://github.com/inorichi/tachiyomi/pull/130

[pitman10]

http://i.imgur.com/08s8XhH.png

Another example of wrong sorting.

[inorichi]

Version?

Edit: Nevermind. It's the same case as Capeta that posted @myuc. The chapters parser won't be able to fix that, and I think it's totally unneeded to add a volume field on the db for a few cases.

You'll have to wait until another sorting method based on dates or sources' order is added.

[j2ghz]

• [x] first chapter is last
  http://bato.to/comic/_/comics/oniichan-dakedo-ai-sae-areba-kankeinai-yo-ne-r1715

[j2ghz]

• [x] Not a wrong order, but after reading chapter 4 I get next chapter not available.
  Ch.5v.2 should be chapter 5 (version 2), not chapter 5 part 22

https://cloud.githubusercontent.com/assets/1708197/14706724/b57a659c-07c0-11e6-93f7-fc74b0f5e8bf.png
https://cloud.githubusercontent.com/assets/1708197/14706725/b5eb4faa-07c0-11e6-9ecc-7c393c70afa7.png

[NoodleMage]

The issues mentioned here should be fixed in my pull. If anyone discovers more chapter recognition. mistakes, please let me know and I will at it to the the test class and try to fix it.

[wopian]

Are manga titles ending with a number fixed in the PR? (Ayame 14)

[wopian]

Are manga titles ending with a number fixed in the PR? (Ayame 14)

[NoodleMage]

Yes.

edit (explanation): the third check removes the manga title (Ayame 14) and checks if the first value is a number. Example: Solanin 028 Vol. 2 -> Solanin028Vol.2 -> 028Vol.2 -R> 028 where -R> is the regex.

[Snake87r]

There is another bug with chapter number recognition, see the manga
tower of god in kissmanga source
there are many chapter recognised as chapter -1

If you have this chapter name it's ok:
Tower of god 010
Tower of god 011 Read Online

But if you have a chapter with two number in it the recognition doesn't work, here an example:
Tower of god 091-092
Tower of God 093 The Untrustworthy Room (003)
Tower of God [Season 002] Ep.191

The biggest problem here is the last example, because the program doesn't know where the first season end, and only with the chapter title you can't find out..

[NoodleMage]

I don't have the problem on my device. Did you add tower of god to your library before or after the update?

[Snake87r]

Thanks for the question (because i resolved my problem thanks to it)..
I'm in the beta version and i'm constantly update the app, now i've tried in another android phone and there are no problem, so i deleted the manga tower of god from my app, cleared database and cache readded the manga and now the chapter recognition work without problem..

Maybe it's better if in the option of every manga there is an option to reinitialize chapter recognition of the selected manga by deleting the database entry of the selected manga..

PS: I added the manga before the update

[NoodleMage]

Glad to hear you fixed it. Thanks for the feedback nonetheless ;)

We did a bit of brainstorming in our dev channel but as of now we don't have a perfect solution. We where thinking of implementing something like you suggested (force chapter recognition in advanced tab was the most preferred). But the problem with this is that a new setting is introduced which is not needed in a later stadium (people who install the app after the chapter recognition rewrite commit).

If there is a lot of animo I can (after approval of Inorichi) add a force recognition update and remove it again in a later version. Not sure though.

For the time being in you want to refresh chapter recognition your way is the best.

[j2ghz]

• [x] Nana to kaoru on readmangatoday http://www.readmanga.today/nana-to-kaoru
  051, 052 and 054 don't fit, but that can't be seen from just chapter name, so no action needed

[Taumer]

For all chapters with name like "Ван Пис 83 - 821 Слушаюсь" parsed volume number

[NoodleMage]

@Taumer. Which manga catalogue is it? And what's the returned chapter value?

Btw. If the manga name isn't Ван Пис 83 or the 83 doesn't have any prefix it won't be possible at this moment to get 821 with regex as far as I know.

[Taumer]

@NoodleMage, at this moment i try to rewrite Readmanga/Mintmanga without override chapters number
(https://github.com/Taumer/tachiyomi/tree/rewrite-russian-source)

About example chapter:
Ван Пис 83 - 821 Слушаюсь (from http://readmanga.me/one__piece)
Ван Пис - manga name
83 - volume number
821 - chapter number
Слушаюсь - chapter name

Returned value - 83

[NoodleMage]

@Taumer _Yeah this chapter layout is a problem.
I can't filter on (-) because ranges. I can't take the highest value because again ranges + chapter with number in title._

_Maybe it's possible to remove volume numbers during the chapter fetch or do you perhaps have another idea?_

Edit: I now see that all formats are the same on this website. I have an idea to fix it. Will keep you posted ;).

[j2ghz]

• [ ] Ch.1: Summoned to Another World Part 1 and Ch.1: Summoned to Another World Part 2 both recognized as Chapter 1

http://bato.to/comic/_/comics/arifureta-shokugyou-de-sekai-saikyou-r18091

[NoodleMage]

Problem with that is:
Tokyo ESP 027: Part 002: Chapter 001 3/22/2013
Tokyo ESP 026: Part 001 - END
http://kissmanga.com/Manga/Tokyo-ESP
If i change it to read part as extra. These will be 26.96 and 27.96 (for example) which is also wrong ;(

[j2ghz]

Then how about giving the option to turn off the feature that when going to next chapter, it looks for chapter with number one higher than current? (Instead go to next chapter as shown in chapter list)

[inorichi]

When one day sources' order is finally implemented the loading strategy will be that one.

[j2ghz]

I don't mean as it's on website, I mean as it's in the app. Now If you read Arifureta Shokugyou de Sekai Saikyou, you go chapter 0, chapter 1 part 2 and chapter 2, leaving chapter 1 part 1 out, even though it's in the right place on the list in the app. I want to turn off the "detection of duplicate chapters" (I don't know how to call it) that skips chapters if there are multiple chapters with number 1

[j2ghz]

https://github.com/inorichi/tachiyomi/blob/1d1e5f1f995b5f9757481da35bf5d3241db8d1fc/app/src/main/java/eu/kanade/tachiyomi/data/database/DatabaseHelper.kt#L134-137
You get a chapter that has chapter number higher than the current one but lower or equal than current + 1

Can you use somthing like http://stackoverflow.com/a/27087651/4363604 ?

[inorichi]

The next/previous row isn't reliable either. That's why I want to sync against sources' order.

[j2ghz]

I meant it as temporary solution, until _When one day_ comes. And it's more reliable than the current method.

[NoodleMage]

@Taumer Fixed it: https://github.com/NoodleMage/tachiyomi/commit/a8fea443f86a5e7f7e973d16ca5a4e70dc127925
if there are more problems with chapter please let me know ;).

Edit: note that this change will probably conflict with https://github.com/inorichi/tachiyomi/tree/rewrite-source.

[inorichi]

@NoodleMage that's the solution Taumer used at first but I didn't like the fact that the recognizer is specific source aware (it makes the sources more dependent on the code), that's why I added the option to override it in the source implementation.

How about calling the default recognizer from the source after removing the title and volume? (creating a copy of the chapter first)

@j2ghz working on syncing order with the source right now.

[NoodleMage]

@inorichi Didn't know this ;). But yeah also possible (and better imo).

@Taumer in that case you can use the regex ([0-9]+)(\s-\s[0-9]+)

val name = chapter.name
        val readManga = Regex("""([0-9]+)(\s-\s[0-9]+)""")
        readManga.find(name)?.let {
            val volume = it.groups[1]?.value!!
            chapter.name = chapter.name.replace(volume, "")
        }

and after that you can call the chapter recognition with this new chapter name and it will hopefully get the correct chapter number o/

[inorichi]

I was thinking about something like this (the manga parameter can be added if needed):

    override fun parseChapterNumber(chapter: Chapter, manga: Manga) {
        // Save original title
        val originalTitle = chapter.name

        // Parse chapter number
        chapter.name = /* apply some regex to remove the title and volume */
        ChapterRecognition.parseChapterNumber(chapter, manga)

        // Recover original title
        chapter.name = originalTitle
    }

[NoodleMage]

@inorichi The title doesn't need to get removed.

But @Taumer if you use my regex at:
chapter.name = /* apply some regex to remove the title and volume */ it should do exactly what you wanted ;)

[Taumer]

@NoodleMage @inorichi thanks a lot for help! :)
I was hoping for a common solution, but it did not.

Your solutions dont work with extra chapters (name like "Ван Пис 100 Экстра Вести по всему Миру" where 100 is volume).
I thought about parse chapter number from url ("one__piece/vol100/11") like that

override fun parseChapterNumber(chapter: Chapter) {
        val beginIndex = chapter.url.lastIndexOf("/") + 1
        val endIndex = chapter.url.indexOf("?mature=1", beginIndex)
        chapter.chapter_number = chapter.url.substring(beginIndex, endIndex).toFloat()
    }

Sorry if something wrong, my English is really bad

[NoodleMage]

You can check if volume is empty in: val volume = it.groups[1]?.value!! And if this is the case apply your algorithm?
And don't worry my English isn't the best either ;)

[Taumer]

_@NoodleMage, yes, without volume nonextra chapters have correct chapter number_

_Edit: Not for all chapters
Блич 52 - 452 Эрозия/Имплозия return 4
change to replaceFirst_

Edit2:

    override fun parseChapterNumber(chapter: Chapter, manga: Manga) {
        if (chapter.name.contains("Экстра")) { // Extra chapters doesnt contain chapter number 
            val readManga = Regex("""(vol[0-9]+/)([0-9]+)""")
            readManga.find(chapter.url)?.let {
                val number = it.groups[2]?.value
                chapter.chapter_number = number!!.toFloat() + 0.5f
            }
        } else {
            val name = chapter.name
            val readManga = Regex("""([0-9]+)(\s-\s[0-9]+)""")
            readManga.find(name)?.let {
                val volume = it.groups[1]?.value!!
                chapter.name = chapter.name.replaceFirst(volume, "")
            }
            ChapterRecognition.parseChapterNumber(chapter, manga)
            chapter.name = name
        }
    }

[inorichi]

As of r724 you can sort like the website (overflow menu in chapters tab). It will also load next/prev chapter without caring about the chapter number, so you can jump from chapter 1 to 10 (if there's no chapters 2 to 9) or read the same chapter again (from another scanlator, for example).

Sorting by chapter number will have the old behavior (ensuring no jumps nor repeated chapters).

Probably sorting by source will be made default after some time in release.

[inorichi]

This issue can be finally closed.