Xowa: Changing Shortcuts, Smooth Scrolling, Mouse Buttons. Also, more Category performance

[Ope30]

Heya, can you help me changing the shortcuts? I'm sure it's that simple, but I didn't find a way changing them. Also tried changing the keybinds in the home.xowa file, but that led to XOWA not opening.

Regarding slow categories, can you please try fixing it for both dumps (09 and 010), it is pure annoyance, don't get me wrong, all the work you've done before is excellent, there's nothing that bothers me, I've found no bugs, not an single thing, other than this, especially for the english wikipedia. Category:Use_dmy_dates_from_July_2016 with 10k pages took me about 30-50 seconds while other categories in the german wikipedia, that consist of 500000 pages, took me about 10 seconds. I'm not kidding. Same goes for searching articles, it takes a while. I get instant articles using the german wikipedia. The english wikipedia has way more articles, that might be the reason? However,

Read-only truly isn't causing this. I tested it again.

I usually redownload wikis all 2-3 months, but if I know you fixed/added something new of course I'm going to redownload, you know. In my opinion there's no point redownloading every month. I want to use it's full potential for a while. But I may change my mind regarding this.

[gnosygnu]

Heya, can you help me changing the shortcuts? I'm sure it's that simple, but I didn't find a way changing them. Also tried changing the keybinds in the home.xowa file, but that led to XOWA not opening.

Damn. It looks like I broke this some time ago. I'll fix this for the next release, but if you want to fix it now, try the following:

• Go to home/wiki/Options/Shortcuts
• Click on "Edit" (Between "Read" and "View HTML")
• Find the following lines (lines 7 - 9):

<:['
{{Options/Header}}
']:>

• Replace it with the following lines:

<:['
<div style='float:right; margin-right:10px; margin-top:5px;'>
<a href="javascript:page_save();" accesskey='s'><img src='~{<>app.fsys.root_dir.to_http_file;<>}bin/any/xowa/file/app.window/save.png' width=16 height=16/>Save</a>
</div><br/>
']:>
);

• Press the "Save page" button at the bottom of the page

To test it, try the following:

• Go to home/wiki/Options/Shortcuts
• Find the Key Binding that says "About XOWA"
• Erase "None" and replace with "F2"
• Press the "Save" icon
• Press "F2". The About XOWA page should open

If it still doesn't do what you want, let me know if you had a specific key binding in mind.

Regarding slow categories, can you please try fixing it for both dumps (09 and 010), it is pure annoyance, don't get me wrong, all the work you've done before is excellent, there's nothing that bothers me, I've found no bugs, not an single thing, other than this, especially for the english wikipedia. Category:Use_dmy_dates_from_July_2016 with 10k pages took me about 30-50 seconds while other categories in the german wikipedia, that consist of 500000 pages, took me about 10 seconds. I'm not kidding.

Yeah, I'm guessing you're using https://de.wikipedia.org/wiki/Category:Mann?

I am going to get a fix for it, but it won't be easy. Will definitely keep you posted

Same goes for searching articles, it takes a while. I get instant articles using the german wikipedia. The english wikipedia has way more articles, that might be the reason? However,

Can you let me know what you're using for search? I tried something like "en.wikipedia.org/wiki/Special:Search/earth?fulltext=y&xowa_page_index=0" and it comes back in a second (on the non-SSD drive)

The search should be very fast. The same engine is used for Android, and I use that every day.

Read-only truly isn't causing this. I tested it again.

Yup, agreed.

I usually redownload wikis all 2-3 months, but if I know you fixed/added something new of course I'm going to redownload, you know. In my opinion there's no point redownloading every month. I want to use it's full potential for a while. But I may change my mind regarding this.

Yeah, worse comes to worse, I'll upload new 2016-09 category databases and you'll just download them.

[Ope30]

Worked for me. Thanks. Thought I would keybind "R" for random pages, there's only one little problem, that whether you search a article or a alphabetical title search/address that contains "R" in it, it automatically gets me to a random page :p. Maybe disabling adjusted shortcuts while searching would help. I will just wait for a new release. :'

Yeah, I'm guessing you're using https://de.wikipedia.org/wiki/Category:Mann?

Exactly.

I am going to get a fix for it, but it won't be easy. Will definitely keep you posted

Thanks. I'm aware it's not that easy. Like always, take your time. ^^

The search should be very fast. The same engine is used for Android, and I use that every day.

My bad. It is differently, took me a while last time. Now I get instant popups.

Yeah, worse comes to worse, I'll upload new 2016-09 category databases and you'll just download them.

If that's needed, sure.

Can you publish the 09 swedish, russian and netherlands wikipedia? If possible all on the same date (commons). Working categories/Navigation or not is completely up to you and your schedule.

By the way, does XOWA support Smooth Scrolling?

[gnosygnu]

Worked for me. Thanks.

No problem. Thanks for the follow-up!

Thought I would keybind "R" for random pages, there's only one little problem, that whether you search a article or a alphabetical title search/address that contains "R" in it, it automatically gets me to a random page :p. Maybe disabling adjusted shortcuts while searching would help. I will just wait for a new release. :'

I'm not sure, but are you literally binding "R"? If so, you should bind something like "Ctrl + R", "Alt + R", "Ctrl + Shift + R", "Alt + Shift + R", or "Ctrl + Alt + R". Doing just plain "R" is not advisable. I should probably remove the option to bind to direct keys, but I just wanted to be comprehensive.

Yeah, I'm guessing you're using https://de.wikipedia.org/wiki/Category:Mann?

Exactly.

Cool.

am going to get a fix for it, but it won't be easy. Will definitely keep you posted

Thanks. I'm aware it's not that easy. Like always, take your time. ^^

Yeah, it won't be this weekend, and probably will be next week.

The search should be very fast. The same engine is used for Android, and I use that every day.

My bad. It is differently, took me a while last time. Now I get instant popups.

Ok. Let me know if the search is slow. If you can send me a URL, I'll have something to investigate. Even a screenshot after the slowness gives me some clues.

Can you publish the 09 swedish, russian and netherlands wikipedia? If possible all on the same date (commons). Working categories/Navigation or not is completely up to you and your schedule.

At this point, I'd rather do 2016-10. I'm going to need a week for the category fix, and by then commons will be ready.

By the way, does XOWA support Smooth Scrolling?

Not natively. I think you would only need it when clicking on the Table of Contents (or reference links)? If so, how often do you do this and how important is the experience?

Hope this helps. Thanks.

[Ope30]

I'm not sure, but are you literally binding "R"? If so, you should bind something like "Ctrl + R", "Alt + R", "Ctrl + Shift + R", "Alt + Shift + R", or "Ctrl + Alt + R". Doing just plain "R" is not advisable. I should probably remove the option to bind to direct keys, but I just wanted to be comprehensive.

Yeah, I'm. But I changed it into F2. Works for me. No problems.

Ok. Let me know if the search is slow. If you can send me a URL, I'll have something to investigate. Even a screenshot after the slowness gives me some clues.

I will let you know.

At this point, I'd rather do 2016-10. I'm going to need a week for the category fix, and by then commons will be ready.

Sure.

Not natively. I think you would only need it when clicking on the Table of Contents (or reference links)? If so, how often do you do this and how important is the experience?

I'm not using the ToC at all. Rarely. Whenever I scroll, it lags a little. Smooth Scrolling would fix that for me, you know. Love it in Firefox. Would love it in XOWA too. :D

I was wondering if Mouse Buttons were supported. I googled and found out that Java only allowed 3 buttons. The Mouse Button 5 would fit for going one page back. The "go one page back" symbol is pretty small and sometimes hard to click for me, since I'm using a 4k monitor. I'm still trying to get this done.

[gnosygnu]

Thanks for follow-up. Just a few direct replies:

I'm not using the ToC at all. Rarely. Whenever I scroll, it lags a little. Smooth Scrolling would fix that for me, you know. Love it in Firefox. Would love it in XOWA too. :D

Yeah, unfortunately, that's a limitation of the underlying browser (XULRunner). It's not in my control, and I can only "fix" it by moving to an entirely different GUI toolkit (Chromium?). In other words, I'd probably have to create an entirely new GUI / Windows app. It's something I have planned, but won't be a near-term fix.

I was wondering if Mouse Buttons were supported. I googled and found out that Java only allowed 3 buttons. The Mouse Button 5 would fit for going one page back. The "go one page back" symbol is pretty small and sometimes hard to click for me, since I'm using a 4k monitor. I'm still trying to get this done.

I've been reluctant to support Mouse Buttons 4 and 5 because Java doesn't do it natively. I can do it with hacks, but they never felt clean.

I'll take a look at it for the next release, but I can't make any promises.

Thanks.

[Ope30]

Alright! Thanks for reading.

I think you should see this, I'm getting it while scrolling: http://prnt.sc/cpe7gr
Not annoying at all, just wondering what this is. I've been getting it lately, the few last days. This disappears when re-scrolling or switching to an other programm. Definitely isn't my monitor.

You can pretty much close the issue now.

[gnosygnu]

I think you should see this, I'm getting it while scrolling: http://prnt.sc/cpe7gr
Not annoying at all, just wondering what this is. I've been getting it lately, the few last days. This disappears when re-scrolling or switching to an other programm. Definitely isn't my monitor.

I've seen something like this happen in the past, but that's only after using XOWA for a few hours / a few hundred pages. It hasn't happened for a while, so I figured it was my old machine / old XUL Runner.

Is this easily reproducible? Or is it random? How long do you have to use XOWA before it occurs?

You can pretty much close the issue now.

Let me keep it open longer until I look at the Mouse Buttons.

Thanks.

[Ope30]

I've seen something like this happen in the past, but that's only after using XOWA for a few hours / a few hundred pages. It hasn't happened for a while, so I figured it was my old machine / old XUL Runner.

Is this easily reproducible? Or is it random? How long do you have to use XOWA before it occurs?

Nope it's not. It's pretty random. This time it took me 2-4 minutes for it to appear. I got it through continuously scrolling up and down.

Let me keep it open longer until I look at the Mouse Buttons.

Alright.

[gnosygnu]

Nope it's not. It's pretty random. This time it took me 2-4 minutes for it to appear. I got it through continuously scrolling up and down.

Yeah, that sounds random. Just so you know, I generally proof a wiki by loading 16 - 64 large pages and scrolling from top to bottom. I've not seen this happen when proofing 2016-09 English, French and German Wikipedia. (granted, I was primarily scrolling down)

I tried now for 2 minutes, but couldn't reproduce.

Out of curiosity, which OS are you on again?

[Ope30]

Yeah, that sounds random. Just so you know, I generally proof a wiki by loading 16 - 64 large pages and scrolling from top to bottom. I've not seen this happen when proofing 2016-09 English, French and German Wikipedia. (granted, I was primarily scrolling down)

Mhm.

I tried now for 2 minutes, but couldn't reproduce.

Me neither. I didn't get it this time. :/

Out of curiosity, which OS are you on again?

I'm on Windows 10 64 bit.

Regarding slow categories, whenever I load the "Man" category in the German Wikipedia, I get faster categories. Not only people, every kind of category.
Loading categories in the English Wikipedia is practically "impossible". I'm somehow trying to find a way to get around this.

Can you switch from page to page in de.wikipedia.org/wiki/Kategorie:Begriffsklärung?

[gnosygnu]

I'm on Windows 10 64 bit.

Ok. Thanks for the data point. I don't have Windows 10 but have tried XOWA on it a few times. I've not seen the issue, but have not done the intensive upwards / downwards scrolling

Regarding slow categories, whenever I load the "Man" category in the German Wikipedia, I get faster categories. Not only people, every kind of category. Loading categories in the English Wikipedia is practically "impossible". I'm somehow trying to find a way to get around this.

That's strange. On my machine, de.w:Category:Mann is always slow. On an SSD drive, it takes about 10 seconds. On a SATA drive, it takes several minutes.

Meanwhile, categories like en.w:Category:Earth are sub-second on both an SSD drive and a SATA drive. However, en.w:Category:2001_albums takes 20 or 30 seconds on a SATA drive. en.w:Category:Unprintworthy_directs takes several minutes.

At any rate, I have a test build that should do better with performance on both SSD / SATA. When you get a chance, try the following:

• Download the 64-bit v3.9.4.3 test build from my google drive account: https://drive.google.com/open?id=0B9cb52zjL2rITnhjNTFMdS1YSlE . The 32-bit is https://drive.google.com/open?id=0B9cb52zjL2rIZHczdldzT0ZkNjA
• Download the updated Category files for 2016-09 de.wikipedia.org: https://drive.google.com/open?id=0B9cb52zjL2rITGpJNUZneFA2bzQ
• Backup the following two files at C:\xowa\wiki\de.wikipedia.org (or wherever you've installed XOWA): de.wikipedia.org-xtn.category.core.xowa and de.wikipedia.org-xtn.category.link-db.001.xowa
• Unzip the Category files and replace the existing versions
• Start XOWA and try de.w:Category:Mann

I'm uploading the Category files for en.wikipedia.org now, and will post the link here when it's done.

Can you switch from page to page in de.wikipedia.org/wiki/Kategorie:Begriffsklärung?

Yeah, that seems to be a bug. Let me look at it tomorrow and get a fix for it.

As always, thanks for all the feedback.

[gnosygnu]

I posted the updated 2016-09 en.wikipedia.org category files at https://drive.google.com/open?id=0B9cb52zjL2rIYnZsUmhlUmhDWWM. They're about 2 GB zipped.

Also, I posted a new v3.9.4.4 built to handle Kategorie:Begriffsklärung . 64-bit is https://drive.google.com/open?id=0B9cb52zjL2rINnRnWTdEc2F1T3c; 32-bit is https://drive.google.com/open?id=0B9cb52zjL2rIM3E1Ymg5cERlRjg.

The problem was that the Next page link had a #. Sadly, this exact page and issue was reported and fixed in the past once before, but broken since. I'll add an automated test for this in the next release

Let me know how it goes. Thanks.

[Ope30]

I can confirm that categories are faster. Same goes for "Begriffserklärung". Thank you very much!
I went through certain articles, and it definitely takes less time now. You did great work!

There are only two small things that I've found (regarding this):
Switching from page to page takes 2 seconds in any category
en.wikipedia.org/wiki/Category:All_stub_articles is averagely broken. The next page has a "#" in it, probably. There may be more out there.

One thing I found:
fr.wikipedia.org/wiki/Catégorie:Article_contenant_un_appel_à_traduction_en_anglais is broken as well. I don't care that much about the fr wikipedia, but since I'm not the only one it could bother others, you know.

You literally fixed everything 2016-09 wikis were missing. 2016-09 is pretty much checked off. I would say let's focus on the 2016-10 wikis. ^^

Other than that, I decided to redownload everything again. The 2016-10 de, en, sv, es, it ,nl, fr and russ wikis, 2016-10 de-en wikivoyage, books, source etc all on the same date (commons) once you uploaded them. Working categories, Navigation, Interaction, fast categories, basically everything I've listened, if possible. This takes alot of time and work, writing, rewriting, deleting, adding, coding, I know that. You should know, I'll support XOWA as much as I can. I use it 20% of the day. I look for possible bugs, mistakes, missing images, missing articles simply because I really want to keep this going, you know.

---

Just so you know, I'm going to keep the 2016-09 wikis. I won't delete them. Therefore I wanted to ask you, how do I change the name of a specific wiki? Let's say, I want to change "en.wikipedia.org" into "en.wikipediaold.org", how exactly do I do that? I tried changing the name of the german wikipedia into "detext.wikipedia.org", but that led to the wiki not opening. Do I have to change the file names of the wikis?

---

If indigent, the current size of Wikipedia is 376 GB.

[gnosygnu]

I changed the title to reflect the recent work with Categories. Ultimately, I should have posted my comments to https://github.com/gnosygnu/xowa/issues/81 instead of here, but for now it is what it is.

I can confirm that categories are faster. Same goes for "Begriffserklärung". Thank you very much!
I went through certain articles, and it definitely takes less time now. You did great work!

Cool! Thanks for the quick feedback! I really appreciate your work in confirming as well!

Switching from page to page takes 2 seconds in any category

Yeah, I still think that a few seconds is going to be unavoidable. Without going into technical details, there's over 100 million rows to search, and the indexes only help so much. On the bright side, it should be the same few seconds regardless of whether the category has 2,000 pages (en.wikipedia.org/wiki/Category:2001_albums) or 1 million (en.wikipedia.org/wiki/Category:Unprintworthy_redirects)

en.wikipedia.org/wiki/Category:All_stub_articles is averagely broken. The next page has a "#" in it, probably. There may be more out there.

Ouch. That's a problem. It looks like Wikipedia changed some Category collation issues back in 2016-05 and it broke XOWA. See: https://phabricator.wikimedia.org/T136150 . Because of the change, XOWA is sorting a title like "1230s_BC" under "0".

I'm still investigating the issue, but it looks like this will need a new set of Category databases. Sorry.

fr.wikipedia.org/wiki/Catégorie:Article_contenant_un_appel_à_traduction_en_anglais is broken as well

I took a look now, and it seemed fine. Specifically, I opened up the page and pressed "(200 suivantes)" 10 times. The pages changed correctly. Are you able to get past this first page?

You literally fixed everything 2016-09 wikis were missing. 2016-09 is pretty much checked off. I would say let's focus on the 2016-10 wikis. ^^

Yup. Agreed.

Other than that, I decided to redownload everything again. The 2016-10 de, en, sv, es, it ,nl, fr and russ wikis, 2016-10 de-en wikivoyage, books, source etc all on the same date (commons) once you uploaded them. Working categories, Navigation, Interaction, fast categories, basically everything I've listened, if possible. This takes alot of time and work, writing, rewriting, deleting, adding, coding, I know that. You should know, I'll support XOWA as much as I can. I use it 20% of the day. I look for possible bugs, mistakes, missing images, missing articles simply because I really want to keep this going, you know.

Of course! If you tell me you use them (and at 20% of the day!) I'll definitely produce them. Let me know your priority. I'm thinking something like: en, de, fr, sv, es, it, nl, and ru. If there aren't major bugs, then I should be able to get it done for October.

BTW: it's amazingly impressive that you're fluent in so many languages. I've never been any good outside of English. :)

Just so you know, I'm going to keep the 2016-09 wikis. I won't delete them. Therefore I wanted to ask you, how do I change the name of a specific wiki? Let's say, I want to change "en.wikipedia.org" into "en.wikipediaold.org", how exactly do I do that? I tried changing the name of the german wikipedia into "detext.wikipedia.org", but that led to the wiki not opening. Do I have to change the file names of the wikis?

It's actually not possible right now. I depend on the domain name being constant for the directory name (/wiki/de.wikipedia.org) and the file names (de.wikipedia.org-core.xowa).

With that said, let me look into putting a hack into this for next week.

If indigent, the current size of Wikipedia is 376 GB.

Not sure I follow. Are you saying 376 GB for all of English, German and French?

[Ope30]

Yeah, I still think that a few seconds is going to be unavoidable. Without going into technical details, there's over 100 million rows to search, and the indexes only help so much. On the bright side, it should be the same few seconds regardless of whether the category has 2,000 pages (en.wikipedia.org/wiki/Category:2001_albums) or 1 million (en.wikipedia.org/wiki/Category:Unprintworthy_redirects)

That's alot. Jesus. However.

Ouch. That's a problem. It looks like Wikipedia changed some Category collation issues back in 2016-05 and it broke XOWA. See: https://phabricator.wikimedia.org/T136150 . Because of the change, XOWA is sorting a title like "1230s_BC" under "0". I'm still investigating the issue, but it looks like this will need a new set of Category databases. Sorry.

Damnit..

I took a look now, and it seemed fine. Specifically, I opened up the page and pressed "(200 suivantes)" 10 times. The pages changed correctly. Are you able to get past this first page?

Nope I'm not. Pressed it 10 times, but it didn't bring me to the next page. But it started to load categories or more articles I think, it used 2.7 GB of RAM. I've gotten an Java error. Here it is:
http://prnt.sc/cqxj5j

Of course! If you tell me you use them (and at 20% of the day!) I'll definitely produce them. Let me know your priority. I'm thinking something like: en, de, fr, sv, es, it, nl, and ru. If there aren't major bugs, then I should be able to get it done for October.

Amazing! Yeah, in that order. ^^

BTW: it's amazingly impressive that you're fluent in so many languages. I've never been any good outside of English. :)

Thanks for your lovely compliment :D. But to be honest, I actually don't speak these languages, if that's what you meant. But that may change in the future. ^^

It's actually not possible right now. I depend on the domain name being constant for the directory name (/wiki/de.wikipedia.org) and the file names (de.wikipedia.org-core.xowa).

With that said, let me look into putting a hack into this for next week.

Alrighty. It would help.

Not sure I follow. Are you saying 376 GB for all of English, German and French?

The folder size of all wikis, English, French, German, CS, EN-DE Wikibooks etc, Wikidata, Commons, Species, Simple are all 376 GB. :-p

I forgot to list Simple Wikipedia, Species Wikipedia, if that is okay.

[gnosygnu]

I'm still investigating the issue, but it looks like this will need a new set of Category databases. Sorry.

Damnit..

Yeah, I'm working thru it now, and will definitely need new databases. The Category sorting system was more complicated than I expected, and the recent change from 2016-05 really breaks my core assumptions.

Nope I'm not. Pressed it 10 times, but it didn't bring me to the next page. But it started to load categories or more articles I think, it used 2.7 GB of RAM. I've gotten an Java error. Here it is:
http://prnt.sc/cqxj5j

Thanks for the screenshot. I think this is a bug that I fixed in the past day. I'll check later, but right now it does work on my side, so hopefully it's not a thing.

BTW: it's amazingly impressive that you're fluent in so many languages. I've never been any good outside of English. :)

Thanks for your lovely compliment :D. But to be honest, I actually don't speak these languages, if that's what you meant. But that may change in the future. ^^

I think just having the wherewithal to work with such a diverse set is impressive enough. :)

With that said, let me look into putting a hack into this for next week.

Alrighty. It would help.

On my list. Will see.

The folder size of all wikis, English, French, German, CS, EN-DE Wikibooks etc, Wikidata, Commons, Species, Simple are all 376 GB. :-p

Ah, should have done the math myself. Thanks!

I forgot to list Simple Wikipedia, Species Wikipedia, if that is okay.

Yup, perfectly okay. ;)

[Ope30]

Yeah, I'm working thru it now, and will definitely need new databases. The Category sorting system was more complicated than I expected, and the recent change from 2016-05 really breaks my core assumptions.

Alright!

Thanks for the screenshot. I think this is a bug that I fixed in the past day. I'll check later, but right now it does work on my side, so hopefully it's not a thing.

I don't think it's a big thing neither. ^

I think just having the wherewithal to work with such a diverse set is impressive enough. :)

^^

Ah, should have done the math myself. Thanks!

I'm sure it is going to reach atleast 450 GB next time, lol.

Yup, perfectly okay. ;)

Nice. :D

[gnosygnu]

@Ope30 Hi again, sorry for the delay, but I didn't want to post again until I made more progress.

I have another version of the Category fix for 2016-09 dewiki and enwiki. Here are the main changes:

• "(Next 200)" always works, even for large categories like "Category:All_stub_articles". This was the most complicated fix as it involved integrating icu4j and adding new columns to the databases.
• Numeric pages are sorted in natural order. For example, "0", "2", "10", not "0", "10", "2". Also, they're grouped under "0-9"
• Category urls now work from history. For example, you can navigate directly to "en.wikipedia.org/wiki/Category:2001_albums?pagefrom=Beautiful%2BView#mw-pages"
• Some pages would be skipped when pressing "Next 200". For example, pressing "Next 200" on "en.wikipedia.org/wiki/Category:2001_albums" will eventually end up with a final page of 116 pages out of 2316. It used to be 115 pages (one page was skipped)

If you want to give it a try, please do the following:

• Download XOWA v3.9.4.5 from here: https://drive.google.com/open?id=0B9cb52zjL2rIMFZXeTAxNUp1QkU. I've only made a Windows 64 version, but will upload any other OS on request
• Download the new de.wikipedia.org.7z file from here: https://drive.google.com/open?id=0B9cb52zjL2rIa1UtcWVZVVoxamc
• Backup de.wikipedia.org-core.xowa, de.wikipedia.org-xtn.category.core.xowa, de.wikipedia.org-xtn.category.link-db.001.xowa
• Repeat with the new en.wikipedia.org.7z file from here: https://drive.google.com/open?id=0B9cb52zjL2rIM1dnNllQWFlfaWs

Also, v3.9.4.5 won't work with the previous category files from v3.9.4.4. Specifically those I listed in https://github.com/gnosygnu/xowa/issues/87#issuecomment-251877824 and https://github.com/gnosygnu/xowa/issues/87#issuecomment-251891833. Once you verify that v3.9.4.5 works and the new category files are good, I'd throw away the old set. The new category files should be the final format.

Don't worry, if you don't get a chance to test them. I've tested them a bit and will do some more in the coming days. If you're curious, here are my basic tests:

• en.wikipedia.org/wiki/Category:2001_albums: Pressed Next 200 until the end and Previous 200 until the beginning
• en.wikipedia.org/wiki/Category:All_stub_articles: Pressed Next 200 about 20 times. It used to stop moving forward after a few pages
• de.wikipedia.org/wiki/Kategorie:Begriffsklärung: Pressed Next 200 about a few times. Confirmed that it didn't fail because of the "#" articles
• de.wikipedia.org/wiki/Kategorie:Mann: Pressed Next 200 several times.

Finally, I discovered the problem with fr.wikipedia.org/wiki/Catégorie:Article_contenant_un_appel_à_traduction_en_anglais. The best way to fix it is to move / delete "fr.wikipedia.org-html-ns.014.xowa". If this file is present, XOWA will load Categories from there. Unfortunately, each Category has all its member pages dumped into one HTML page. So, Catégorie:Article_contenant_un_appel_à_traduction_en_anglais will have all 130,000 articles in one page, instead of just 200 at a time. This causes the "out of memory" error

Hope this helps. Let me know if anything above isn't clear. As always, thanks for your help and testing. You've been absolutely great in spotting issues!

[Ope30]

@Ope30 Hi again, sorry for the delay, but I didn't want to post again until I made more progress.

No problemo!

I have another version of the Category fix for 2016-09 dewiki and enwiki. Here are the main changes:

If you want to give it a try, please do the following:

Yeah, you did very nice work! I did test it a bit using the German Wikipedia, I'm sorry to disappoint you, but I forgot to check the categories at all, Next 200, etc this time. Same goes for the English Wikipedia, I couldn't check the categories, because I had to replace several files. I just wanted to leave the way it is. But still, I'm sure you're right. Just so you know, I wasn't lazy or anything. ^^
I still have the files, so I can try them out anytime I want. :-l

Also, v3.9.4.5 won't work with the previous category files from v3.9.4.4. Specifically those I listed in #87 (comment) and #87 (comment). Once you verify that v3.9.4.5 works and the new category files are good, I'd throw away the old set. The new category files should be the final format.

It should work. I remember opening a category, that consists of 2000 pages. It didn't take long, I'm talking about slow categories, if that's what you meant. ^

Finally, I discovered the problem with fr.wikipedia.org/wiki/Catégorie:Article_contenant_un_appel_à_traduction_en_anglais. The best way to fix it is to move / delete "fr.wikipedia.org-html-ns.014.xowa". If this file is present, XOWA will load Categories from there. Unfortunately, each Category has all its member pages dumped into one HTML page. So, Catégorie:Article_contenant_un_appel_à_traduction_en_anglais will have all 130,000 articles in one page, instead of just 200 at a time. This causes the "out of memory" error

Nice. I'll use your solution!

Hope this helps. Let me know if anything above isn't clear. As always, thanks for your help and testing. You've been absolutely great in spotting issues!

It definitely did! I thank you as well for always fixing all these issues. :D

[gnosygnu]

Yeah, you did very nice work! I did test it a bit using the German Wikipedia

Cool! Thanks for trying out the new version.

I couldn't check the categories, because I had to replace several files. I just wanted to leave the way it is. But still, I'm sure you're right. Just so you know, I wasn't lazy or anything. ^^

Yup, completely up to you. Keep in mind that your category files for v3.9.4.4 won't work well with v3.9.4.5 and up. So, whenever you get a chance, you should try replacing the files. ;)

It should work. I remember opening a category, that consists of 2000 pages. It didn't take long, I'm talking about slow categories, if that's what you meant. ^

Other wikis will still work. But English and German will definitely be off with the old category files. The new ones I uploaded yesterday will work.

---

Also, I looked at the MouseButtons issue, and this looks like an SWT bug. I added a comment to the existing bug report: https://bugs.eclipse.org/bugs/show_bug.cgi?id=147424#c5 . I don't know if they will fix it, but for now, there's not much I can do (besides implementing a JNI hack for Windows: https://github.com/kristian/system-hook)

[Ope30]

Yup, completely up to you. Keep in mind that your category files for v3.9.4.4 won't work well with v3.9.4.5 and up. So, whenever you get a chance, you should try replacing the files. ;)

Yeah, I know. Will do that probably. :p

Other wikis will still work. But English and German will definitely be off with the old category files. The new ones I uploaded yesterday will work.

Right.

Also, I looked at the MouseButtons issue, and this looks like an SWT bug. I added a comment to the existing bug report: https://bugs.eclipse.org/bugs/show_bug.cgi?id=147424#c5 . I don't know if they will fix it, but for now, there's not much I can do (besides implementing a JNI hack for Windows: https://github.com/kristian/system-hook)

Oh wow. Let's hope they're going to fix it. ^^

Out of interest, have you gotten an 2016-10 dump yet? Especially commons, did they dump it yet?

[gnosygnu]

Out of interest, have you gotten an 2016-10 dump yet?

I'm working on 2016-10 enwiki now. I'm trying to get a cleaner dump with Category data. Hopefully ready for Sunday. dewiki and frwiki will follow afterwards.

Especially commons, did they dump it yet?

Yeah, it's dumped. See https://dumps.wikimedia.org/commonswiki/. I forget: was I supposed to upload it? If I remember correctly, it's not really useful without the images (which would require 22+ TB of downloads)

[Ope30]

I'm working on 2016-10 enwiki now. I'm trying to get a cleaner dump with Category data. Hopefully ready for Sunday. dewiki and frwiki will follow afterwards.

Wonderful. Take your time. ^^

Yeah, it's dumped. See https://dumps.wikimedia.org/commonswiki/. I forget: was I supposed to upload it? If I remember correctly, it's not really useful without the images (which would require 22+ TB of downloads)

Oh, I meant commons and all the other wikis together, not commons itself (the wiki), just whether they have the same date you know, missing images. :-d

[gnosygnu]

Oh, I meant commons and all the other wikis together, not commons itself (the wiki), just whether they have the same date you know, missing images. :-d

Ah, didn't realize. Yup, they do. Won't make that mistake again. Thanks!

[gnosygnu]

It's actually not possible right now. I depend on the domain name being constant for the directory name (/wiki/de.wikipedia.org) and the file names (de.wikipedia.org-core.xowa).

With that said, let me look into putting a hack into this for next week.

Just a quick note. I added support to the next release for renamed folders. So you can have a folder called "de.wikipedia.org-201609" and navigate to "de.wikipedia.org-201609/wiki/Main_Page" in XOWA

---

I think I covered all items in this issue. To recap:

• Smooth scrolling: Not possible. XULRunner limitation
• Mouse Buttons 4 / 5: Dependent on SWT. Commented on a bug, and hoping they'll respond: https://bugs.eclipse.org/bugs/show_bug.cgi?id=147424#c5
• Category issues: Generated yet another Category database schema change. Will be part of v3.10.3.1 as well 2016-10 wikis
• Renamed wiki folders: Will be part of v3.10.3.1

Let me know if I missed anything. Otherwise, I'll close this issue out in the next few days. Thanks!

[gnosygnu]

As indicated above, the items in this issue have been covered. I've also uploaded 2016-10 English / German Wikipedia, and will upload more wikis over the coming weeks.

Thanks again for the help!