Xowa: Wikis: 2017-01 progress thread

[gnosygnu]

This is a general thread for XOWA wikis produced for 2017-01.

I'm going to be on vacation for the 3rd and 4th weeks of this month, so this will be a short month. Also, the Wikimedia dump files don't tend to be ready until sometime around the 10th, so there isn't much time for producing.

With that said, here's the queue for 2017-01.

• 2017-02 English Wikipedia released 2017-02-13
• 2017-02 French Wikipedia released 2017-03-03
• 2017-02 German Wikipedia released 2017-02-26
• 2017-01 Simple wikis released 2017-01-10
• 2017-02 Wikidata released 2017-03-02
• 2017-01 Commons
• 2017-01 Russian wikis
• 2017-01 Haitian wikis
• 2017-01 Czech wikis
• 2017-01 Chinese wikis release 2017-01-28

Also, as a general reminder, here's how to get any of these wikis in XOWA or XOWA Android

• Go to Download Central
  
  
  
  • On XOWA: Tools -> Download Central (also home/wiki/Special:XowaDownloadCentral)
  
  
  • ON XOWA Android: Left menu -> XOWA -> Download button
  
  
• Look at the "Available" table
• Click the "blue down arrow" all the way to the right
• You'll see a status message in the bottom right about the recent additions

[Ope30]

I'm going to be on vacation for the 3rd and 4th weeks of this month, so this will be a short month. Also, the Wikimedia dump files don't tend to be ready until sometime around the 10th, so there isn't much time for producing.

Ah alright. If you want we can wait until 2017-02, if you'd like!

I have some requests, can you add the CS wikis, RU (big one), HT wikis, Wikidata and Commons to the list? Also, af.wikibooks, af.wiktionary, sw.wikibooks, sw.wikipedia and sw.wiktionary as well? They're missing the Navigation Page and categories. Plan re/downloading them. :P

[gnosygnu]

Ah alright. If you want we can wait until 2017-02, if you'd like!

Actually, it looks like the Wikimedia dump process changed last month. The files are available now, so I kicked off the build process. With any luck, 2017-01 English Wikipedia may be ready by Sunday

I have some requests, can you add the CS wikis, RU (big one), HT wikis, Wikidata and Commons to the list?

• I'll upload Wikidata and Commons, since that's just a matter of uploading my files (no extra-checking on my side)
• HT (2016-07) and CS (2016-09) are old. They also happen to be relatively small, so I can upload them as well.
• RU (2016-11) is relatively recent. It's also sort of large. Is there any particular reason you want a refresh so quickly? Otherwise, I'd like to leave it for 2017-02 / 2017-03.

Also, af.wikibooks, af.wiktionary, sw.wikibooks, sw.wikipedia and sw.wiktionary as well? They're missing the Navigation Page and categories. Plan re/downloading them. :P

Ah, nice catch on the missing Navigation sidebar. I have to look into how they fell out.

Category pages are there. Here are some random ones I checked in the different wikis. Note that sw.wikibooks doesn't have any as it's a very small wiki

• af.wiktionary.org/wiki/Kategorie:Woorde_in_Frans
• af.wikibooks.org/wiki/Kategorie:Aardrykskunde
• sw.wikipedia.org/wiki/Jamii:Masanduku_ya_uanapwa_ya_familia_za_mamalia
• sw.wiktionary.org/wiki/Jamii:Kiingereza

Let me know if there's anything else. Thanks!

[Ope30]

RU (2016-11) is relatively recent. It's also sort of large. Is there any particular reason you want a refresh so quickly? Otherwise, I'd like to leave it for 2017-02 / 2017-03.

Uhm, the previous RU Wiki had missing images. Just wanted to redownload it so you don't have to make a patch or anything. ^^

Category pages are there. Here are some random ones I checked in the different wikis. Note that sw.wikibooks doesn't have any as it's a very small wiki

Right. sw.wikipedia isn't missing categories, only the Navigation. My bad!
Still, I plan downloading them. Just want them to be functioning. :p

[gnosygnu]

Uhm, the previous RU Wiki had missing images. Just wanted to redownload it so you don't have to make a patch or anything. ^^

Oops. Forgot about that. :flushed:

Yeah, will definitely bump it up to this month then. I updated the list above. Thanks for the catch

Right. sw.wikipedia isn't missing categories, only the Navigation. My bad!

Cool. Also, I think none of the SW or AF wikis are missing categories (except for SW.wikibooks which is missing them because it's such a small wiki). Let me know which Category pages are missing and I'll investigate further

Still, I plan downloading them. Just want them to be functioning. :p

Actually, I think I can fix this in the next XOWA release. It turns out this isn't a download issue. Those 5 wikis don't have an official MediaWiki:Sidebar page. (Almost all other wikis do) In those cases, XOWA should use a default sidebar, not use nothing. I'll add this logic later.

[Ope30]

Oops. Forgot about that. 😳

No thing mate. ^^

Cool. Also, I think none of the SW or AF wikis are missing categories (except for SW.wikibooks which is missing them because it's such a small wiki). Let me know which Category pages are missing and I'll investigate further

Yeah. They're so small. af.wikibooks definitely has categories. You're right about sw.wikibooks. I thought they don't have categories because whenever I got to an Random Page there were no categories.

Actually, I think I can fix this in the next XOWA release. It turns out this isn't a download issue. Those 5 wikis don't have an official MediaWiki:Sidebar page. (Almost all other wikis do) In those cases, XOWA should use a default sidebar, not use nothing. I'll add this logic later.

Alright! I'm excited!

[gnosygnu]

Some bad news. I ran into an issue with 2017-01 English Wikipedia. I don't think I'll be able to fix it in the next few days. As I'll be out on vacation the following two weeks, that probably means no new wikis until 1-29.

I'll post again when I have new info but it may be several days.

[gnosygnu]

I'm still working on 2017-01 English Wikipedia, but I went ahead and generated 2017-01 Simple Wikipedia. Thanks.

[Ope30]

Some bad news. I ran into an issue with 2017-01 English Wikipedia. I don't think I'll be able to fix it in the next few days. As I'll be out on vacation the following two weeks, that probably means no new wikis until 1-29.

I'm still working on 2017-01 English Wikipedia, but I went ahead and generated 2017-01 Simple Wikipedia. Thanks.

Thanks for uploading Simple Wikipedia! I just got into it and it's working. At the first time it wasn't because I had to cut out the previous date files into the newer one, but yeah. There are some mistakes I found, categories are missing, Table of Contents and the Navigation Page. Also, when you click on an image and you go back every image disappears.

By the way, if you want we can wait until 2017-02, or maybe 03. Up to you. To be honest, I would say let us wait until 2017-02 just to be sure. Your choice mate.

Thanks!

[gnosygnu]

Thanks for uploading Simple Wikipedia! I just got into it and it's working. At the first time it wasn't because I had to cut out the previous date files into the newer one, but yeah.

Hmm... This could be an issue with the new wiki, as the format changed (from 4 files to 10+ files). I deleted the old wiki first, so I didn't notice this. I'll try to reproduce later.

There are some mistakes I found, categories are missing, Table of Contents and the Navigation Page. Also, when you click on an image and you go back every image disappears.

I tried this now, and it works on my side. :( Specifically:

• Category exists; for example, simple.wikipedia.org/wiki/Category:Earth
• Table of Contents exists; for example, simple.wikipedia.org/wiki/Earth
• Navigation page exists; you can see it in my screenshot below
• Clicking on an image and coming back still shows images; I did this for the Earth image and had no issues

Can you try the first 2 specifically? For the navigation page, a screenshot will be fine

By the way, if you want we can wait until 2017-02, or maybe 03. Up to you. To be honest, I would say let us wait until 2017-02 just to be sure. Your choice mate.

Yeah, at this point, this issue may be a short one. I may have to postpone till 2017-02 depending on the fix. I'll know more by tomorrow.

Thanks!

[Ope30]

I tried this now, and it works on my side.

Hmm... This could be an issue with the new wiki, as the format changed (from 4 files to 10+ files). I deleted the old wiki first, so I didn't notice this. I'll try to reproduce later.

I'm sorry. I don't know what I done wrong but, it works now, lol. I just redownloaded it. My guess is, it may be because I cut the old files into the new one. So to be clear, I redownloaded, extracted it into a new folder I made and, yeah, that's all I did. Weird. However, everything I mentioned works now.

Yeah, at this point, this issue may be a short one. I may have to postpone till 2017-02 depending on the fix. I'll know more by tomorrow.

Roger that!

[gnosygnu]

So to be clear, I redownloaded, extracted it into a new folder I made and, yeah, that's all I did. Weird. However, everything I mentioned works now.

Cool. Glad to hear it works. As always, thanks for the follow-up.

This is probably a bug with the format change. I'll look at it later, but deleting the old version and updating to the new one would work.

Yeah, at this point, this issue may be a short one. I may have to postpone till 2017-02 depending on the fix. I'll know more by tomorrow.

Roger that!

Well, the build failed this morning. I'm running it again now, but it doesn't look like I'll have anything ready until end of month...

[Ope30]

Well, the build failed this morning. I'm running it again now, but it doesn't look like I'll have anything ready until end of month...

No problem. Let us wait until next month. I'm very patient. :D

[Ope30]

Hey, just a random question, would it be possible in the future if you include the "Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization." at the very bottom such as "Privacy policy", "About Wikipedia" etc? Also talking about "WIKIMEDIA project" and "Powered By MediaWiki" as well. It would make it look more alike, you know.

Here's something, I changed all my wikis into a new name, for example en.wikipedia into English Wikipedia, but when you do this you won't see any images and Wikidata wouldn't work if you wanted to change the language. Is that somehow fixable?

[gnosygnu]

Hey, just a random question, would it be possible in the future if you include the "Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization." at the very bottom such as "Privacy policy", "About Wikipedia" etc? Also talking about "WIKIMEDIA project" and "Powered By MediaWiki" as well. It would make it look more alike, you know.

Sure! I've been meaning to add it, but haven't gotten around to it. Will do so in one of the new few builds.

Here's something, I changed all my wikis into a new name, for example en.wikipedia into English Wikipedia, but when you do this you won't see any images and Wikidata wouldn't work if you wanted to change the language. Is that somehow fixable?

You mean you renamed the folder from "/xowa/wiki/en.wikipedia.org" to "/xowa/wiki/English Wikipedia"? If so, it should be fixable but would require a new release. I'll take a look at it later, but for now, can you keep it as "en.wikipedia.org"? Thanks!

[Ope30]

Sure! I've been meaning to add it, but haven't gotten around to it. Will do so in one of the new few builds.

Thanks! I would love to see it in the next build!

You mean you renamed the folder from "/xowa/wiki/en.wikipedia.org" to "/xowa/wiki/English Wikipedia"? If so, it should be fixable but would require a new release. I'll take a look at it later, but for now, can you keep it as "en.wikipedia.org"? Thanks!

Exactly. In my opinion it looks way better than "en.wikipedia.org". It looks better overall. It would be great if you fix this in the next release.

Thanks for reading!

[Ope30]

Hey again. There's something I would like to see in the next coming builds too, can you include the Navigation "Werkzeuge" and "Drucken/­exportieren" on all the next coming wikis? Only these that have it. I don't know if it's automatically. Also, "Categories" in the German Wikipedia into "Kategorien".

Regarding the "Talk" page, could you add that as well? Of course without it having the function, it just has to be there. You can't even click on it. It just has to be there. It would make it more alike!

[gnosygnu]

Sorry for the delay, but I'm out on vacation for the next week and a half, so my replies will be spotty.

Exactly. In my opinion it looks way better than "en.wikipedia.org". It looks better overall. It would be great if you fix this in the next release.

Cool. Thanks for confirming. Let me look at it next week. XOWA uses the domain name (en.wikipedia.org) by default everywhere, so it may be interesting...

can you include the Navigation "Werkzeuge" and "Drucken/­exportieren" on all the next coming wikis?

I actually removed them intentionally as XOWA doesn't support any of those links. I can add them back under an option, but think it will be misleading (you click on the link, and you go to an error page). I guess you want them there for visual similarity to the online version, but do you really feel it's worth it? They would be functionally meaningless....

Also, "Categories" in the German Wikipedia into "Kategorien".

Hmmm, I checked the German Wikipedia front page now and don't see "Category" anywhere. Where are you seeing "Categories" when it should be "Kategorien"?

Regarding the "Talk" page, could you add that as well? Of course without it having the function, it just has to be there. You can't even click on it. It just has to be there. It would make it more alike!

This is actually available already as an option. Do the following:

• Go to home/wiki/Special:XowaCfg?grp=xowa.html.page. Or, go to Options and choose "HTML - Page"
• Change "Talk namespace" from "Hide" to "Show as blue link"

XOWA also supports Talk pages, but it has to be imported manually through Tools -> Import Offline

Hope this helps. Thanks!

[Ope30]

Sorry for the delay, but I'm out on vacation for the next week and a half, so my replies will be spotty.

It's alright. I already knew that you were on vacation. :-)

Cool. Thanks for confirming. Let me look at it next week. XOWA uses the domain name (en.wikipedia.org) by default everywhere, so it may be interesting...

Oh yeah. It is very important. I would love to be able to change the names without disadvantages.

I actually removed them intentionally as XOWA doesn't support any of those links. I can add them back under an option, but think it will be misleading (you click on the link, and you go to an error page). I guess you want them there for visual similarity to the online version, but do you really feel it's worth it? They would be functionally meaningless....

Right. I would just like it to have it there. I know it'd make no sense at all, but like you said, similarity. I'm sorry too, lol.

Hmmm, I checked the German Wikipedia front page now and don't see "Category" anywhere. Where are you seeing "Categories" when it should be "Kategorien"?

My bad! I had to change it back to "de.wikipedia.org". Changing the name into something else makes a big difference.

This is actually available already as an option

Thank you! Again, I'm aware this literally makes no sense since it's functionless. I honestly just do it to be similar to the original. :D

I have one question and a request:

• Can you add "Not logged in", "Talk", "Contributions" etc, "View Source" and "View History" to my request list? What I'm wondering is, is "View Source" automatically going to be "Quelltext anzeigen" since it's German? Goes this for all the other wikis?

• Is there a way to permanently hide the Calendar when you get to an year article, let's say 1995, in the German Wikipedia?

[gnosygnu]

Hey. I think we're on the same page on all the earlier points. Regarding your question and request

Can you add "Not logged in", "Talk", "Contributions" etc,...

Probably, but again, they wouldn't be functional, and only be there for display purposes. "Contributions" is not possible, logging in doesn't really apply to offline wikis, and even something like going to the Talk page is a bit interesting

.... "View Source" and "View History" to my request list? What I'm wondering is, is "View Source" automatically going to be "Quelltext anzeigen" since it's German? Goes this for all the other wikis?

View History falls in the same category as above: can show, but won't be functional.

View Source is a bit more interesting.

• "View Source" is the same as "Quelltext anzeigen". XOWA should show "Quelltext anzeigen" instead of "View Source" for the German wikis, but I haven't updated the translations for awhile
• "View Source" does the same action on all wikis. It shows the Wikitext for the page
• However, "View Source" only works for Wikitext databases (wikis built with Import Online / Import Offline). For HTML databases (wikis downloaded from Download Central), it only shows a blank page. I'll probably change this in the future to show the HTML, but editing HTML is not easy (it's one of the reasons why Wikitext was invented)

Is there a way to permanently hide the Calendar when you get to an year article, let's say 1995, in the German Wikipedia?

Not really. One could write some Javascript to search for a NavFrame with "Kalenderübersicht", but it would be kludgey.

Just so I understand, why would you want to hide the Calendar? Is it because it takes up too much space?

[Ope30]

Probably, but again, they wouldn't be functional, and only be there for display purposes.

can show, but won't be functional

That's the point! Honestly, I really would love it to be there. Functionless or not, doesn't matter. As long as it's there. I just love the way they designed the wikis you know, and it would look good on XOWA too! Maybe you're going to give them a function in the future. All I want to reach is to have the exact same feeling as if you're browsing using the original. Make it a Option or something like that, if possible. So users can decide whether they would like to see it or not.

However, "View Source" only works for Wikitext databases (wikis built with Import Online / Import Offline). For HTML databases (wikis downloaded from Download Central), it only shows a blank page. I'll probably change this in the future to show the HTML, but editing HTML is not easy (it's one of the reasons why Wikitext was invented)

Yeah. If you want it to have a function it would take some codes. It wouldn't be worth it anyway other than having it for display, you know. :l

Just so I understand, why would you want to hide the Calendar? Is it because it takes up too much space?

Exactly! In my opinion it's pretty annoying. :( Having a Option to turn it off would be pleasant!

Thanks for reading and taking the time!

[gnosygnu]

Just a quick note. I uploaded 2017-01 Chinese wikis as per #123

I worked around my issue with 2017-01 English wiki. However, I'm going to postpone any release until 2017-02 because

• The month is almost over and 2017-02 is just around the corner. I don't want to upload 40 GB now just to do it again in a week.
• I want to do one other thing to try to reduce image sizes.

Once I get the English wiki update done, I'll work on the other wikis.

Thanks.

[Ope30]

Just a quick note. I uploaded 2017-01 Chinese wikis as per #123

Hey mate, thanks for uploading, but I cannot find them in the Download Central. And yes, I clicked on the Blue Arrow. :(

I worked around my issue with 2017-01 English wiki. However, I'm going to postpone any release until 2017-02 because

The month is almost over and 2017-02 is just around the corner. I don't want to upload 40 GB now just to do it again in a week.

I could not agree more. :)

I want to do one other thing to try to reduce image sizes.

Once I get the English wiki update done, I'll work on the other wikis.

Take your time. By the way, what do you exactly mean by trying to reduce the image file sizes?

[gnosygnu]

Hey mate, thanks for uploading, but I cannot find them in the Download Central. And yes, I clicked on the Blue Arrow. :(

Oops. My mistake. Forgot to update the version file. I fixed it now. Thanks!

By the way, what do you exactly mean by trying to reduce the image file sizes?

Fair question. Here's a breakdown.

• The online wikis continually obsolete images. For example, a page can use "A1.png" in 2016-11 and later change it to "A2.png" in 2016-12
• The XOWA 2016-11 update would include "A1.png" and the 2016-12 update would include "A2.png". However, now you have "A1.png" and "A2.png" even though only "A2.png" is being used.
• By design, the XOWA monthly updates only add images. They don't delete images. This is done primarily for simplicity of deployment, but also for speed.
• I've worked around this issue, by
  
  
  
  • For existing users: offering a "delete-images" database every year or so. This database would delete all images that aren't being used. It's less than 1 MB to download. However, this process can take about an hour or so to complete
  
  
  • For new users: uploading a brand new set of images. This would be the full 90 GB of images
  
  
• I'm also planning on offering a "smaller" image database set for English Wikipedia only. This will try to get the set down from 90-100 GB to 50-60 GB by using 50% quality images. I don't think I'll have this ready for a few weeks, but it may be possible.

Hope this is enough info. Thanks!

[Ope30]

Just wanted to say that I tested the wikis. They're working! Thanks for uploading them!

[gnosygnu]

Cool. Thanks for testing!

[Ope30]

Thanks for the reference! I'm sure your solution would work but, it would take me a very long time to rename all the files in the wikis. I got alot of wikis and apparently the wiki folder consists of 3732 files. I'm rather going to wait until you officially fix this! Thanks!

[gnosygnu]

I looked at this today, and I don't think it will work.

I can put in a hack to get (a) images to show and (b) "In other languages" to appear at the bottom . However, the code really assumes that the domain name (en.wikipedia.org) will always match the folder name (in your case "English Wikipedia").

For example, using the same "In other languages" features:

• The user opens "English Wikipedia/wiki/Earth"
• The user scrolls down to "In other languages" and "German"
• The user clicks "Erde" to go to "de.wikipedia.org/wiki/Erde"
• The user scrolls back down to "In other languages" and "English"
• The user clicks on "Earth"
• XOWA will try to take the user to "en.wikipedia.org/wiki/Earth". However, XOWA fails because there is no "en.wikipedia.org/wiki/Earth", just "English Wikipedia/wiki/Earth"

Even if there was a way to bandage this behavior, it would break in other circumstances. For example, what if the user has two English Wikipedias: One at "C:\xowa\wiki\English Wikipedia 2016" and one at "C:\xowa\wiki\English Wikipedia 2015". In the above case, which wiki should "In other languages" and "English" go to?

There are probably other places that will break as well. This assumption ("the domain name matches the folder name") is a core assumption and I would really need to rewrite a lot of the wiki initialization code to even get it to work correctly. Otherwise, I'm just going to end up putting in one ugly patch after another.

Hopefully the above makes sense. Thanks.

[Ope30]

Thank you for your well explanation! Yeah, I get it. Very sad that there is no way around it. Maybe in the future. Who knows. Seems like I have to rename everything again. :(

Any news regarding the 2017-02 Wikipedias?

[gnosygnu]

Thank you for your well explanation! Yeah, I get it. Very sad that there is no way around it. Maybe in the future. Who knows. Seems like I have to rename everything again. :(

Well, I'm going to put in some code in this Sunday's release to handle images and in other languages. However, it'll probably still not be good for your usage.

Out of curiosity, are you renaming it to "English Wikipedia" because ....

• (a) You want to see it as "English Wikipedia" in the file explorer? For example, "C:\xowa\wiki\English Wikipedia"
• (b) You want to see it as "English Wikipedia" in the XOWA url bar? For example, "English Wikipedia/wiki/Earth"
• (c) Both of the above
• (d) Something else

I ask, because the changes should basically allow (b). (b) should also be useful for setting up parallel versions of the same wiki. For example, "en.wikipedia.org/wiki/Earth" and "en.wikipedia.org-2016.12/wiki/Earth"

(a) is a different matter altogether for the reasons detailed above. If it's (a) does it bother you to see "en.wikipedia.org" in the file explorer as opposed to "English Wikipedia"? Or is there something else?

Any news regarding the 2017-02 Wikipedias?

Well, they just kicked off the dumps today. I'll know more in about a week (it takes that long for them to generate all the files)

[Ope30]

Out of curiosity, are you renaming it to "English Wikipedia" because ....

(b) You want to see it as "English Wikipedia" in the XOWA url bar? For example, "English Wikipedia/wiki/Earth"

Yep.

I ask, because the changes should basically allow (b). (b) should also be useful for setting up parallel versions of the same wiki. For example, "en.wikipedia.org/wiki/Earth" and "en.wikipedia.org-2016.12/wiki/Earth"

Yeah. I once thought about having a wiki with images and without images (Wikitext) at the same time.

(a) is a different matter altogether for the reasons detailed above. If it's (a) does it bother you to see "en.wikipedia.org" in the file explorer as opposed to "English Wikipedia"? Or is there something else?

Nah.

"English Wikipedia" instead of "en.wikipedia.org" looks better and more comfortable in my opinion.
"en.wikipedia.org" bothers me a bit you know. It scratches me. :p

[gnosygnu]

"English Wikipedia" instead of "en.wikipedia.org" looks better and more comfortable in my opinion.
"en.wikipedia.org" bothers me a bit you know. It scratches me. :p

Okay. I guess I understand now. I'll think about it some more but as I said above the change would be deeply embedded in code

Thanks!

[Ope30]

Hey.

It seems you released a new build. Firstly, thank you so much for adding the Creative Commons footer! I'm already loving it. I cannot wait for the rest. :D

Regarding the build now, I did not find any flaws that the new build could've brought in.

I have questions which I'm very tense about.

• Html: Show missing left sidebar on small wikis.

What did you mean by that again?

• Wiki: Support renamed folders.

Is it only working for (example): en.wikipedia.org-2014.5? Because it is not working for "English Wikipedia", or did I misunderstood?

[gnosygnu]

It seems you released a new build. Firstly, thank you so much for adding the Creative Commons footer! I'm already loving it. I cannot wait for the rest. :D

Cool. Thanks for giving it a try.

I took a look at the rest, but it's actually not easy. MediaWiki has customizable logic to add / remove items and from the footer depending on the wiki. I'm going to postpone the rest for now, as it really does involve some code, and it really doesn't add any functionality

Html: Show missing left sidebar on small wikis.

Yeah, it was from above. See: https://github.com/gnosygnu/xowa/issues/120#issuecomment-270487721 and the following:

Actually, I think I can fix this in the next XOWA release. It turns out this isn't a download issue. Those 5 wikis don't have an official MediaWiki:Sidebar page. (Almost all other wikis do) In those cases, XOWA should use a default sidebar, not use nothing. I'll add this logic later.

Wiki: Support renamed folders.

Oops. That was a mistake on my side. I uploaded a temp build to my google drive account if you want to give it a try. https://drive.google.com/open?id=0B9cb52zjL2rIZHdGRkNwQm5tZWM

Thanks!

[Ope30]

I took a look at the rest, but it's actually not easy. MediaWiki has customizable logic to add / remove items and from the footer depending on the wiki. I'm going to postpone the rest for now, as it really does involve some code, and it really doesn't add any functionality

Okay.

Yeah, it was from above. See: #120 (comment) and the following:

Ah. Yeah.

Oops. That was a mistake on my side. I uploaded a temp build to my google drive account if you want to give it a try. https://drive.google.com/open?id=0B9cb52zjL2rIZHdGRkNwQm5tZWM

Can confirm that this build works, mostly if you use the Random Article function. If you search for the article "Wikipedia", the images are going to appear, but if you search for "Joker (Comics)" images will not appear for some reason. Tested this in the English Wikipedia.

I also gave the German Wiki a try. So far it doesn't has the same problem as above.

Nice work on fixing renamed folders by the way! I really appreciate all your work you put on your programm.

[gnosygnu]

If you search for the article "Wikipedia", the images are going to appear, but if you search for "Joker (Comics)" images will not appear for some reason. Tested this in the English Wikipedia.

Nice catch! The problem is that https://en.wikipedia.org/wiki/Joker_(comics) has images that reside in en.wikipedia.org, not commons.wikimedia.org. The code needed to look in a repository called "en.wikipedia.org", but was instead looking in "en.wikipeda.org-2016.12" (or whatever you renamed it to)

I fixed this in the new build here: https://drive.google.com/open?id=0B9cb52zjL2rIbV91TmZpQ3E1M1k

Thanks!

[Ope30]

Nice catch! The problem is that https://en.wikipedia.org/wiki/Joker_(comics) has images that reside in en.wikipedia.org, not commons.wikimedia.org. The code needed to look in a repository called "en.wikipedia.org", but was instead looking in "en.wikipeda.org-2016.12" (or whatever you renamed it to)

Yep. I can now confirm that it works! Thanks so much again. It really looks better now than just "en.wikipedia.org" or "de.wikipedia.org".

I've talked about this already and the only fix for it was to rename the folder back to "de.wikipedia.org", but that is not needed anymore. The thing is, "Kategorien" is now "Categories" since I renamed the folder. Would you fix that for me as well? I'd appreciate it.

Oh, one thing, is there a option to hide the "Generated by XOWA" thingy at the bottom?

[gnosygnu]

Yep. I can now confirm that it works! Thanks so much again. It really looks better now than just "en.wikipedia.org" or "de.wikipedia.org".

Cool. As always, thanks for confirming!

The thing is, "Kategorien" is now "Categories" since I renamed the folder. Would you fix that for me as well? I'd appreciate it.

Ok. I found the issue and uploaded a new build now. See: https://drive.google.com/open?id=0B9cb52zjL2rIRmxuVXIyZFZfaFE

Oh, one thing, is there a option to hide the "Generated by XOWA" thingy at the bottom?

I'm planning to make the entire page format configurable in the future, but unfortunately, for now, there is no option. If you want to remove it you can open up "C:\xowa\bin\any\xowa\cfgapp\xowa.gfs" and remove the HTML around "Generated by". Keep in mind this file gets replaced with every release though, so you'll have to reapply it after upgrading.

Hope this helps. Thanks!

[gnosygnu]

I released 2016-02 English Wikipedia last night. It's up on Download Central now.

My main Linux box is having issues, so I'm going to have to take a small break to rebuild it. It may be another week or so before I can get something up.

Sometime this week I'll upload a deletion database. This will compact the size of English Wikipedia Images to about 95 GB. I'll also upload this 95 GB set for new users.

[Ope30]

I released 2016-02 English Wikipedia last night. It's up on Download Central now.

Thank you for uploading! Just wanted to tell you that I will probably download it in 2 days. So you know why so late.

Thanks!

Edit: Done downloading. There is not much to talk about. Everything is working the way it should!
Two things:
Category:Articles_incorporating_a_citation_from_the_1913_Catholic_Encyclopedia_with_Wikisource_reference?pagefrom=Pierre%2BJoseph%2BPelletier#mw-pages
Can you get to the next page? Also, when you go back and forward again and again,, it bugs out a little. Basically you go back and forward until you have no article when you go forward.
Edit 2: It definitely has to be a bug in this category. I believe every other has the same issue.

Apparently, the Main Page is from 2017-01. Probably just a bug, or? I wanted to Sync the page but there's no option to do so on the left. Or how do I Sync again?

Oh, how is it going with the other wikis? I'd like to redownload the German, Russian and Czech Wikis (big)!

[Ope30]

Oh, did you miss my comment? Sorry, I edited it instead of making a new one. :/

[gnosygnu]

Oh, did you miss my comment? Sorry, I edited it instead of making a new one. :/

Yeah, sorry. I gave it a thumbs up, but your subsequent edit didn't send me an email. Thanks for the second notification.

Category:Articles_incorporating_a_citation_from_the_1913_Catholic_Encyclopedia_with_Wikisource_reference?pagefrom=Pierre%2BJoseph%2BPelletier#mw-pages
Can you get to the next page? Also, when you go back and forward again and again,, it bugs out a little. Basically you go back and forward until you have no article when you go forward.
Edit 2: It definitely has to be a bug in this category. I believe every other has the same issue.

Hmmm.... I tried now, and couldn't reproduce it.

• I pressed Next 200 and got Ladislaus Pyrker
• I pressed Next 200 again and got Saint Catherine's Monastery
• I went Previous / Next a few times and the view updated accordingly

Can you check if you have all the category databases? This is what I get for en.wikipedia.org/wiki/Special:XowaDiag?type=fs.check&wiki=en.wikipedia.org

fil|en.wikipedia.org-xtn.category.core.xowa|23015424|20170210 115125.797
fil|en.wikipedia.org-xtn.category.link-db.001.xowa|1665716224|20170210 115125.796
fil|en.wikipedia.org-xtn.category.link-db.002.xowa|1662754816|20170210 115125.797
fil|en.wikipedia.org-xtn.category.link-db.003.xowa|1685577728|20170210 115125.797
fil|en.wikipedia.org-xtn.category.link-db.004.xowa|1688784896|20170210 115125.797
fil|en.wikipedia.org-xtn.category.link-db.005.xowa|1685344256|20170210 115125.796
fil|en.wikipedia.org-xtn.category.link-db.006.xowa|1687609344|20170210 115125.796
fil|en.wikipedia.org-xtn.category.link-db.007.xowa|1687244800|20170210 115125.797
fil|en.wikipedia.org-xtn.category.link-db.008.xowa|1689300992|20170210 115125.797
fil|en.wikipedia.org-xtn.category.link-db.009.xowa|1688117248|20170210 115125.797
fil|en.wikipedia.org-xtn.category.link-db.010.xowa|289656832|20170210 115125.797

Apparently, the Main Page is from 2017-01. Probably just a bug, or?

Yeah, I forgot to mention this. For all intents and purposes, it's just the main page. More complicated explanation follows:

• All the articles are from the 2017-02 dump
• Wikipedia has a function that says what day is today? This function is called from the Main_Page as well as other places
• XOWA can change the answer the answer to the "what day is today?" I manually update this answer every month, but forgot to change it to 2017-02-01

I wanted to Sync the page but there's no option to do so on the left. Or how do I Sync again?
Apparently, the Main Page is from 2017-01. Probably just a bug, or? I wanted to Sync the page but there's no option to do so on the left. Or how do I Sync again?

I changed it a little for the Options redesign. Try the following:

• Go to home/wiki/Special:XowaCfg?grp=xowa.bldr.page_sync
• Check Manual for Enabled. A "Page sync" option will appear on the left

Oh, how is it going with the other wikis? I'd like to redownload the German, Russian and Czech Wikis (big)!

I haven't started yet. :( My main Linux box is having issues, and I need to rebuild it. I should be reuploading next week.

[Ope30]

Hmmm.... I tried now, and couldn't reproduce it.

This is weird. I had the issue because it was renamed to something else (not en.wikipedia.org).
My wiki is called "English Wikipedia", if you change it to something else you get no "Sync Page" and the categorie(s) in the English Wikipedia will not work at all.
I still have my old 2016-12 EN Wiki and it is renamed to something else. It does not have the Sync Function and categories are bugged, for example Category:Primeira Liga players. I cannot get to the next 200. I don't have it in the German Wikipedia. I may be wrong there.
Try renaming it to something else. Do you get the same issues?

The reason why I'm saying this is weird is because, didn't you fix it in the previous release? As I remember, it used to be working fine. Lol.

I haven't started yet. :( My main Linux box is having issues, and I need to rebuild it. I should be reuploading next week.

Ah. No problemo. I can wait.

[gnosygnu]

My wiki is called "English Wikipedia", if you change it to something else you get no "Sync Page" and the categorie(s) in the English Wikipedia will not work at all.

Ah, thanks. I see the issue now.

I'm not sure how I can fix this. Again, a lot of code ultimately depends on the Folder Name (for you, "English Wikipedia") being the same as the domain name ("en.wikipedia.org"). Categories is running afoul of this.

The sync function is the same, as it's going to try to sync with "https://English Wikipedia" instead of "https://en.wikipedia.org"

The reason why I'm saying this is weird is because, didn't you fix it in the previous release? As I remember, it used to be working fine. Lol.

Categories wasn't working b/c of something called UCA collation. I fixed it so that it does work for UCA collation. If you change English Wikipedia back to en.wikipedia.org you'll see it works too. :)

Let me think about this, but I think in the end, I'll just be monkey-patching various issues, and I don't like going down that road.

I haven't started yet. :( My main Linux box is having issues, and I need to rebuild it. I should be reuploading next week.

Ah. No problemo. I can wait.

Cool. Thanks.

[Ope30]

I'm not sure how I can fix this. Again, a lot of code ultimately depends on the Folder Name (for you, "English Wikipedia") being the same as the domain name ("en.wikipedia.org"). Categories is running afoul of this.

Categories wasn't working b/c of something called UCA collation. I fixed it so that it does work for UCA collation. If you change English Wikipedia back to en.wikipedia.org you'll see it works too. :)

Let me think about this, but I think in the end, I'll just be monkey-patching various issues, and I don't like going down that road.

Okay. You're right about en.wikipedia.org. When needed I will just change it back to en.wikipedia.org and wait for a fix (if possible), you know. :d

The sync function is the same, as it's going to try to sync with "https://English Wikipedia" instead of "https://en.wikipedia.org"

Hm. Weird. I'm still getting no "Sync Page" when it's English Wikipedia if that's what you mean?!
However. I'll just change it to en.wikipedia.org. As well as the rest.

Thanks!

[Ope30]

Hey.

Sorry for the late response. I noticed you have uploaded a new build with high DPI which I absolutely love!
You also uploaded a wiki although I didn't request but I still tested it. The wiki is alright.

Regarding the new build, I don't have anything to say about it. So far.
As always, you put great work into your programm my friend. Especially the high DPI fix, I love it. I have a 4k monitor. :)

[gnosygnu]

Hey, sorry, but forgot to reply to the earlier comments:

Okay. You're right about en.wikipedia.org. When needed I will just change it back to en.wikipedia.org and wait for a fix (if possible), you know. :d

Yeah, let me look at it for next week. Had a bunch of other issues the past week

Hm. Weird. I'm still getting no "Sync Page" when it's English Wikipedia if that's what you mean?!
However. I'll just change it to en.wikipedia.org. As well as the rest.

Yeah, it won't be there because it thinks that "https://English Wikipedia" does not exist. Compare that to "https://en.wikipedia.org"

I noticed you have uploaded a new build with high DPI which I absolutely love!

Wow. My apologies many times for this. I didn't know you had a high DPI monitor. The URL bar was always half the height. I thought it was unworkable. I'm surprised you've been suffering through this. :o

You also uploaded a wiki although I didn't request but I still tested it. The wiki is alright.

Cool. It was requested in #130 and the wiki was small enough for me to do on my machine

Regarding the new build, I don't have anything to say about it. So far.

Thanks! Appreciate the confirmation. 👍

---

Also, I finally figured out the Linux issue this week. I'll post more details in either a blog post or a github issue. In short, it was related to XULRunner and recent versions of Java. I had to switch to webkit

I'm going to start updating wikis this week. I already uploaded 2016-02 German Wikipedia today. I'm trying to do French Wikipedia, but hit a different type of snag. Afterwards, I'm hoping to hit a more regular stride again.

[Ope30]

Wow. My apologies many times for this. I didn't know you had a high DPI monitor. The URL bar was always half the height. I thought it was unworkable. I'm surprised you've been suffering through this. :o

No worries

I'm going to start updating wikis this week. I already uploaded 2016-02 German Wikipedia today. I'm trying to do French Wikipedia, but hit a different type of snag. Afterwards, I'm hoping to hit a more regular stride again.

Thanks so much for uploading the German one. For your information, it has no bugs or anything in it. The wiki itself is very fast functioning. :=)

Regarding the French Wikipedia, do whatever you have to do. I only want to redownload the Czech Wikis, Russian Wikis, Wikidata, the big French one and HT!

Yeah, it won't be there because it thinks that "https://English Wikipedia" does not exist. Compare that to "https://en.wikipedia.org"

Oh, and one question, would it be possible for you to write a function that only changes the DISPLAY name of the wikis instead of having to rename the whole folder?

[gnosygnu]

Thanks so much for uploading the German one. For your information, it has no bugs or anything in it. The wiki itself is very fast functioning. :=)

Cool. As always, thanks for testing it.

Regarding the French Wikipedia, do whatever you have to do. I only want to redownload the Czech Wikis, Russian Wikis, Wikidata, the big French one and HT!

Yup. Wanted to do French Wikipedia because the last one was from October. I'll do the others next.

However, I'll start uploading Wikidata tomorrow.

Oh, and one question, would it be possible for you to write a function that only changes the DISPLAY name of the wikis instead of having to rename the whole folder?

That's an interesting idea. Would you want navigation to be the same as well? For example, you're in "English Wikipedia/wiki/Earth" and you enter in "German Wikipedia/wiki/Erde" into the URL bar? I think typing "German Wikipedia" is more work than "de.wikipedia.org"

[Ope30]

However, I'll start uploading Wikidata tomorrow.

I agree. Thanks!

That's an interesting idea. Would you want navigation to be the same as well? For example, you're in "English Wikipedia/wiki/Earth" and you enter in "German Wikipedia/wiki/Erde" into the URL bar? I think typing "German Wikipedia" is more work than "de.wikipedia.org"

Yeah, sure! If it's not too much work for you, why not. I really would love to see this!

How big is your DE/EN Wiki folder?
DE is 57,5 GB (61.814.939.590 Bytes)
EN is 160 GB (172.140.814.794 Bytes)

[gnosygnu]

Yeah, sure! If it's not too much work for you, why not. I really would love to see this!

Let me take a look at it. I think the hard point is adding the options to configure the display

How big is your DE/EN Wiki folder?
DE is 57,5 GB (61.814.939.590 Bytes)
EN is 160 GB (172.140.814.794 Bytes)

• DE is 57.8 GB
• EN is 144 GB . Keep in mind I shrank the image databases to get rid of old images. You can shrink them too per my instructions at http://xowa.org/home/wiki/Blog.html . Or you can just download the new 2017-02 full set

Also, I uploaded Wikidata today. Thanks.

[Ope30]

EN is 144 GB . Keep in mind I shrank the image databases to get rid of old images. You can shrink them too per my instructions at http://xowa.org/home/wiki/Blog.html . Or you can just download the new 2017-02 full set

Ah. It's okay.

Also, I uploaded Wikidata today

Again, thank you for the constant uploading! Nothing to talk about Wikidata. Still curious about your size regarding Wikidata. Mine is 41,0 GB (44.108.783.939 Bytes). What is yours?!

I do not want to start with this again to be honest, but, check out https://en.wikipedia.org/wiki/Germ%C3%A1n_Garmendia and https://de.wikipedia.org/wiki/HolaSoyGerm%C3%A1n, both are missing the same picture. It's not like I don't understand your explanation, it's just that fixing this is my biggest wish for XOWA. I'm aware enabling Web Access would fix this but I don't want to feel like enabling it everytime I occur an missing image(s), you know. No time pressure or anything. Is there anything I can do to help you?!

[gnosygnu]

Mine is 41,0 GB (44.108.783.939 Bytes). What is yours?!

Yup. Mine is 41.4 GB also.

I do not want to start with this again to be honest, but, check out https://en.wikipedia.org/wiki/Germ%C3%A1n_Garmendia

Yeah, the image was uploaded on 2017-02-03. https://commons.wikimedia.org/wiki/File:Germ%C3%A1n_Garmendia_2016.jpg

it's just that fixing this is my biggest wish for XOWA. I'm aware enabling Web Access would fix this but I don't want to feel like enabling it everytime I occur an missing image(s), you know. No time pressure or anything. Is there anything I can do to help you?!

Sorry about that. I had it on my list, but it fell off.

Let me work on it for the next round of wikis. I'm going to be busy with personal issues this week, but I'll put aside some time for this. Will let you know, and do keep me honest. :)

Also, just so you know I upload 2017-02 French Wikipedia tonight

[Ope30]

Sorry about that. I had it on my list, but it fell off.

Let me work on it for the next round of wikis. I'm going to be busy with personal issues this week, but I'll put aside some time for this. Will let you know, and do keep me honest. :)

👍
Also, take your time too!

Also, just so you know I upload 2017-02 French Wikipedia tonight

This is like the 4nd or 5nd wiki you uploaded in a row, right? Wow. Lol. I thank you for fulfilling my requests! Do I got anything to say about it? Nah. For some reason the last builds were pretty good. :P

Is it okay if I ask you to compare your wikis file size and my from now on?
FR is 54,3 GB (58.312.245.326 Bytes)

[gnosygnu]

Also, take your time too!

Yup. Will try to get it done soon but it may take a week

This is like the 4nd or 5nd wiki you uploaded in a row, right?

I think it's 4 but who's keeping count? ;)

Wow. Lol. I thank you for fulfilling my requests! Do I got anything to say about it? Nah. For some reason the last builds were pretty good. :P

Cool. Good to hear. 👍

Is it okay if I ask you to compare your wikis file size and my from now on?
FR is 54,3 GB (58.312.245.326 Bytes)

Mine is 54.8. Out of curiosity, why are you so interested in file size? If you're downloading from Download Central, the files should always be correct. I don't think there's been an issue with missing files and Download Central, but do let me know

Thanks!

[Ope30]

Mine is 54.8. Out of curiosity, why are you so interested in file size? If you're downloading from Download Central, the files should always be correct. I don't think there's been an issue with missing files and Download Central, but do let me know

Because I know your file sizes are legit, lol. Not only that, just want to be sure. Neither do I think there are issues in the Download Central.

[gnosygnu]

Because I know your file sizes are legit, lol. Not only that, just want to be sure. Neither do I think there are issues in the Download Central.

Okay. Just wanted to make sure I wasn't missing anything.

I'll try to add a file size total to the Wiki_setup urls in a future release. But in general, if Download Central completes without problems, then you really do have the same files as me. :)

[Ope30]

I'll try to add a file size total to the Wiki_setup urls in a future release. But in general, if Download Central completes without problems, then you really do have the same files as me. :)

I agree.

Thanks!

[gnosygnu]

I do not want to start with this again to be honest, but, check out https://en.wikipedia.org/wiki/Germ%C3%A1n_Garmendia

Yeah, the image was uploaded on 2017-02-03. https://commons.wikimedia.org/wiki/File:Germ%C3%A1n_Garmendia_2016.jpg

I looked at this issue more closely today, and this is not related to missing files. The image was added to the page after the dumps ran. More detailed info is available below the break

Let me work on it for the next round of wikis. I'm going to be busy with personal issues this week, but I'll put aside some time for this. Will let you know, and do keep me honest.

I spent several hours writing the code to try to handle the following scenario:

• The commons dump completes on 2017-02-02 00:00:00
• The enwiki dump completes on 2017-02-03 00:00:00
• Due to the timing differences, any new images added between 2017-02-02 and 2017-02-03 are not captured in the dump.

In short, this approach is complicated and involves downloading a lot of data. I'm tabling that change until it turns out that there are a lot of images that fall in that 2017-02-02 -> 2017-02-03 period. In my mind, there are probably no more than 3,000 (out of 5 million). If more are reported, I'll reinvestigate.

Note that this code would not have affected the image above as it was "missing" simply because it was added after the dump.

---

Part 1: timing differences

• On 2017-01-30, the page did not have an image: https://en.wikipedia.org/w/index.php?title=Germ%C3%A1n_Garmendia&diff=prev&oldid=762718796
• The 2017-02 dumps ran on 2017-02-02 and memorialized this version (2017-01-30, at 10:18)
• On 2017-02-03, after the dumps ran, the page was modified to include the image: https://en.wikipedia.org/w/index.php?title=Germ%C3%A1n_Garmendia&type=revision&diff=763438123&oldid=762718796

* I released the 2017-02 XOWA wikis on 2017-02-17

Part 2: why the code is complicated
The main problem is that there isn't a simple way to say "get the new images for enwiki between 2017-02-02 and 2017-02-03". I tried two different approaches, but one would involve downloading 500,000+ rows of data (call imageinfo only) and the other would involve downloading 200,000+ rows of data (call recentchanges and then call imageinfo). Both of these have scalability issues (I don't want to download that much data over the Wikimedia API)

In addition, it involves adding a non-insignificant amount of code (a few hundred lines) and time (20+ min) to the build process to try to find .05% of images

[Ope30]

I looked at this issue more closely today, and this is not related to missing files. The image was added to the page after the dumps ran. More detailed info is available below the break

Yeah, you're right. It's not an missing image, it just wasn't existing at that time.

I spent several hours writing the code to try to handle the following scenario:

The commons dump completes on 2017-02-02 00:00:00
The enwiki dump completes on 2017-02-03 00:00:00
Due to the timing differences, any new images added between 2017-02-02 and 2017-02-03 are not captured in the dump.

On 2017-01-30, the page did not have an image: https://en.wikipedia.org/w/index.php?title=Germ%C3%A1n_Garmendia&diff=prev&oldid=762718796
The 2017-02 dumps ran on 2017-02-02 and memorialized this version (2017-01-30, at 10:18)
On 2017-02-03, after the dumps ran, the page was modified to include the image: https://en.wikipedia.org/w/index.php?title=Germ%C3%A1n_Garmendia&type=revision&diff=763438123&oldid=762718796
I released the 2017-02 XOWA wikis on 2017-02-17

Part 2: why the code is complicated
The main problem is that there isn't a simple way to say "get the new images for enwiki between 2017-02-02 and 2017-02-03". I tried two different approaches, but one would involve downloading 500,000+ rows of data (call imageinfo only) and the other would involve downloading 200,000+ rows of data (call recentchanges and then call imageinfo). Both of these have scalability issues (I don't want to download that much data over the Wikimedia API)

I appreciate your big detailed explanation. It makes sense. I actually don't care about images that have not been added at that time, in other words, I would rather want this than images being completely missing. I mentioned it because I thought the image was missing. Sorry. Still, thanks for your time!

Do you remember the missing images regarding de.wikipedia.org/wiki/Duisburg_Hauptbahnhof (https://github.com/gnosygnu/xowa/issues/90)? What would be harder for you to fix? I assume both have the time differences, but I don't know the other facts.

I hope I understood this correctly! If I didn't, correct me.

Edit: Oh god, I really, really like the Night Mode! How comes I never requested things like that?
You said you improved the startup performance, I can confirm that. You added a new function called "full-text search", which, I believe, is to search through articles. If that is the case, I cannot wait for it to be more functioning.
What do you mean by "More support for French Wikipedia"? Is there any issue I don't know of?!

I want to ask you certain questions:
-How do I download images off the internet in Commons? I activated Web Access but that did not make a difference, or do I have to redownload?
-You can open images with Windows Photo Viewer, but that only counts for downloaded ones. What can I do to open existing images?
-Sorry, can you add the rest "Privacy Policy", "About Wikipedia" etc.? I don't know why it's not there.
-This is stupid, I know, can you change "Categories" into "Categories:"? I'm sure you know why. :-P
-Can you add a setting that, when you click on a page, it will open a new tab and automatically switch to it? You know, without having to click on the tab. Do you know how to bind the "Mouse Button 5" "key"? I would close tabs with it.

[gnosygnu]

I appreciate your big detailed explanation. It makes sense. I actually don't care about images that have not been added at that time, in other words, I would rather want this than images being completely missing. I mentioned it because I thought the image was missing. Sorry. Still, thanks for your time!

Cool. I was also worried after your report, but discovered that it wasn't really an issue after writing several hours of code. Oh well.

Do you remember the missing images regarding de.wikipedia.org/wiki/Duisburg_Hauptbahnhof (#90)? What would be harder for you to fix? I assume both have the time differences, but I don't know the other facts.

This was just a weird fluke in the dump processing for that month ("Short story: it looks like the dump excluded all files added between 2016-09-22 and 2016-10-01"). It shouldn't happen again.

Edit: Oh god, I really, really like the Night Mode! How comes I never requested things like that?

Cool! Glad you liked it. It was requested by a handful of users (for example: https://sourceforge.net/p/xowa/discussion/general/thread/7e8cca53/?limit=25#561b/4e30). I postponed it, because I needed to make other UI changes first. After I fixed the high DPI issues, it was a little easier to do the night mode feature.

You said you improved the startup performance, I can confirm that.

Cool here as well. I had a report from another user that clicking on the jar was faster than the exe. This build should make both exe and jar behave at same speed. Unfortunately I can't confirm it here as both happen pretty fast.

You added a new function called "full-text search", which, I believe, is to search through articles. If that is the case, I cannot wait for it to be more functioning.

Yeah, this is only for personal wikis now. I'm working on adding it to the full Wikipedia, but it will take at least a few weeks. But it should be the next major feature.

What do you mean by "More support for French Wikipedia"? Is there any issue I don't know of?!

The French Wikipedia layout for the Main Page is messed up. It shows 4 boxes side-by-side instead of a 2x2 grid.

This release fixes it for new users by upgrading to Xulrunner v31. I think existing users would have to make a manual patch (doing it automatically was a little complicated).

In short, go to fr.wikipedia.org in Wikipedia. If you get a 1 x 4 style layout instead of a 2 x 2 layout (like https://fr.wikipedia.orgthen you'll need the manual patch.

Hope this all helps. Thanks!

[Ope30]

Cool. I was also worried after your report, but discovered that it wasn't really an issue after writing several hours of code. Oh well.

I truly apologize.... I know it took you hours and hours to get a deep look....

Cool! Glad you liked it. It was requested by a handful of users (for example: https://sourceforge.net/p/xowa/discussion/general/thread/7e8cca53/?limit=25#561b/4e30). I postponed it, because I needed to make other UI changes first. After I fixed the high DPI issues, it was a little easier to do the night mode feature.

I'm glad that user(s) requested the Night Mode. It looks better than the original one, in my opinion.

The French Wikipedia layout for the Main Page is messed up. It shows 4 boxes side-by-side instead of a 2x2 grid.

In short, go to fr.wikipedia.org in Wikipedia. If you get a 1 x 4 style layout instead of a 2 x 2 layout (like https://fr.wikipedia.orgthen you'll need the manual patch.

I'm not 100% sure what you mean, but I just realized that FR is missing another line on the Navigation Bar, I don't know what it's called. Maybe it's supposed like that.

Yeah, this is only for personal wikis now. I'm working on adding it to the full Wikipedia, but it will take at least a few weeks. But it should be the next major feature.

I'm excited on how it will turn out! Take all the time you need. I don't give orders.

This was just a weird fluke in the dump processing for that month ("Short story: it looks like the dump excluded all files added between 2016-09-22 and 2016-10-01"). It shouldn't happen again.

Yep. Again, I really apologize for my comment. I feel bad regarding this, knowing that you put hours into the code, for "nothing", just because I was mistaken.

[gnosygnu]

I truly apologize.... I know it took you hours and hours to get a deep look....

Hey, nothing to apologize for. This was my mistake. I should've triaged the issue before trying to write code to fix it first.

I'm glad that user(s) requested the Night Mode. It looks better than the original one, in my opinion.

Honestly, I think there will still be some issues. XOWA uses a UI library called SWT, and it still has some annoying issues, such as a flashing white background. (See http://xowa.org/home/wiki/App/UI/Night_mode ) Unfortunately there is no way to fix these.

I'm not 100% sure what you mean, but I just realized that FR is missing another line on the Navigation Bar, I don't know what it's called. Maybe it's supposed like that.

Nah, it should be pretty obvious. See the following:

You won't see it if you're on Mac OS X machine (I thought you were on a Windows machine?)

I'm excited on how it will turn out! Take all the time you need. I don't give orders.

Well, it will definitely take a while. ;)

Yep. Again, I really apologize for my comment. I feel bad regarding this, knowing that you put hours into the code, for "nothing", just because I was mistaken.

Again, nothing to feel bad about. If anything, I'm sorry for making you feel that way. I was pretty annoyed with myself for not realizing it earlier, so my tone does sound cross. Sorry!

[Ope30]

Honestly, I think there will still be some issues. XOWA uses a UI library called SWT, and it still has some annoying issues, such as a flashing white background. (See http://xowa.org/home/wiki/App/UI/Night_mode ) Unfortunately there is no way to fix these.

Lol. I didn't notice the white flashing in XOWA until I read your comment. Sure, it's a bit annoying, but I can live with that. I still wouldn't mind a mix if there'd be one (I know there's no way to fix this).
I noticed that you only get the white flashing, best example here, the random page function, in big articles, long articles, you know, let's say "Angela Merkel", compare it to "Taschenrechner" and you'll notice that you only get flashes on long articles (Angela Merkel). Sometimes you get the flashing in "small" articles as well but only for, 5 milliseconds? However. Now regarding the tab one, I as well noticed that when you click on File The flashing is almost actually random, sort of.

You won't see it if you're on Mac OS X machine (I thought you were on a Windows machine?)

Hm. Fortunately I'm not having the same problem. Look: http://prnt.sc/ehu2il
Yep. I'm on Windows 10 64 bit. :P

I wanted to edit my comment above, but then I saw that you replied.
Ehm, can you check out de.wikipedia.org/wiki/Wikipedia:Quiz/Geschichte#Fragen and scroll down until you see a white picture? I'm curious, because, enabling Web Access doesn't matter. I'm assuming it's simply just missing.

[gnosygnu]

I noticed that you only get the white flashing, best example here, the random page function, in big articles, long articles, you know, let's say "Angela Merkel", compare it to "Taschenrechner" and you'll notice that you only get flashes on long articles (Angela Merkel).

Have you tried the following yet:

• Go to Special:XowaCfg?xowa.app.security
• Change Load HTML to browser method to url

When I have this set, it doesn't flash at all. I only get flashes when opening new tabs (Ctrl + T)

Now regarding the tab one, I as well noticed that when you click on File

Shouldn't make a difference. File -> Open Tab calls the same code as "Ctrl + T"

only at the first time you do this, afterwards you get nothing. But if you do it using the shortcut, then yeah, I'd say 50% of the time. The flashing is almost actually random, sort of.

Yeah, I spent a half hour looking at this. Basically, it's an SWT thing and it will be semi-random. Sometimes the first new tab will flash white, and then 5 new tabs will be fine, and then 2 new tabs will flash white. Unfortunately, not much to do

Hm. Fortunately I'm not having the same problem. Look: http://prnt.sc/ehu2il
Yep. I'm on Windows 10 64 bit. :P

That's really strange. When you installed 4.4, did you put it in a new directory? Or did you unzip it into your old 4.3 directory?

• If you installed it into a new directory, it would work. I fixed the issue for all new users.
• If you installed it into an existing directory, then it wouldn't work out of box. You would need to change one file to get it to work (which I don't automatically change for technical reasons)

Ehm, can you check out de.wikipedia.org/wiki/Wikipedia:Quiz/Geschichte#Fragen and scroll down until you see a white picture? I'm curious, because, enabling Web Access doesn't matter. I'm assuming it's simply just missing.

I tried now, and all images show correctly. See if you can post a screenshot or let me know which image is white, and I'll do a little more investigation.

Thanks.

[Ope30]

When I have this set, it doesn't flash at all. I only get flashes when opening new tabs (Ctrl + T)

Yeah, right. I don't get flashes at all neither.

Shouldn't make a difference. File -> Open Tab calls the same code as "Ctrl + T"

Not really. I get more flashing when I use the shortcut. Sometimes I get no flashing.

Yeah, I spent a half hour looking at this. Basically, it's an SWT thing and it will be semi-random. Sometimes the first new tab will flash white, and then 5 new tabs will be fine, and then 2 new tabs will flash white. Unfortunately, not much to do

No problem mate. I don't care about the flashing. 😋

That's really strange. When you installed 4.4, did you put it in a new directory? Or did you unzip it into your old 4.3 directory?

I did make a new folder, then I put it in. I also then moved my old settings into the new folder. That's all I actually do, everytime.

I tried now, and all images show correctly. See if you can post a screenshot or let me know which image is white, and I'll do a little more investigation.

http://image.prntscr.com/image/b4155efc80744c0bb8d1f1a43634cdce.png
This here is the link of the image:
de.wikipedia.org/wiki/Datei:The_story_of_the_greatest_nations,_from_the_dawn_of_history_to_the_twentieth_century_-_a_comprehensive_history,_founded_upon_the_leading_authorities,_including_a_complete_chronology_of_the_world,_and_(14578677039).jpg
I don't know why it's not there. Maybe because I'm missing a few bytes? My DE wiki is 57,5 GB, yours is 57,8. Let me know.

en.wikipedia.org/wiki/Seven_Wells_Hill, what is wrong with that article?
Can you see the error message in this picture above? http://prnt.sc/eib42o

[gnosygnu]

Yeah, right. I don't get flashes at all neither.

Cool

Shouldn't make a difference. File -> Open Tab calls the same code as "Ctrl + T"

Not really. I get more flashing when I use the shortcut. Sometimes I get no flashing.

I just tried now, and do get flashing with both. It's hard to compare frequency as the phenomenon is random. That said, if File -> New tab works for you, stick with it. :)

Yeah, I spent a half hour looking at this....
No problem mate. I don't care about the flashing. 😋

Nah, I just meant I spent half an hour looking at this a week ago when I first put in the feature. From everything I saw, it's SWT and unavoidable.

I did make a new folder, then I put it in. I also then moved my old settings into the new folder. That's all I actually do, everytime.

Cool. That's always the correct way to do it.

Help -> Update App should also work. It's just that it didn't work for this last release because of the xulrunner change (the details are a little complicated)

This here is the link of the image: de.wikipedia.org/wiki/Datei:The_story_of_the_greatest_nation

This looks like a bug caused by the long file name. It works for me, because my folder is E:\xowa. Yours is something like E:\XOWA\XOWA 2016-12 64-bit 123.

Try shortening your folder to just E:\XOWA. The image should show.

en.wikipedia.org/wiki/Seven_Wells_Hill, what is wrong with that article? Can you see the error message in this picture above?

Thanks for the screenshot. I've only seen that error when I have two or more XOWA opened at the same time. Have you tried restarting and reloading the same page? I'd guess that it shouldn't reappear.

If it does, send me the log from Help -> System Data -> Session Log

Thanks!

[Ope30]

This looks like a bug caused by the long file name. It works for me, because my folder is E:\xowa. Yours is something like E:\XOWA\XOWA 2016-12 64-bit 123.

True. Changed the folder name to something shorter. It worked for me. Thank you!

Thanks for the screenshot. I've only seen that error when I have two or more XOWA opened at the same time. Have you tried restarting and reloading the same page? I'd guess that it shouldn't reappear.

If it does, send me the log from Help -> System Data -> Session Log

Damn. No matter what I do, the article doesn't work. I can't get a Session Log neither, I can't find the session.txt file, but the Log folder is there.

[gnosygnu]

True. Changed the folder name to something shorter. It worked for me. Thank you!

Cool. I'll work on fixing that for a future release as well.

Damn. No matter what I do, the article doesn't work. I can't get a Session Log neither, I can't find the session.txt file, but the Log folder is there.

Hmm... I see the problem now. It didn't happen to me because I have the wikitext databases. When I use just the HTML databases, it's an issue.

The problem is related to the category system. Let me debug that too and get back to you.

Thanks.

[gnosygnu]

I'm closing this issue for now. I created a new progress thread at https://github.com/gnosygnu/xowa/issues/139

I'll work on the other two bugs and post updates to the 2016-03 issue.

Thanks.

[gnosygnu]

I'm temporarily reopening this thread to cover the two bugs cited earlier.

Both of these will be fixed in the next version. When it gets released I'll mark this thread closed again. If anyone wants a preview version of the next release, let me know, and I'll upload something.

@Ope30 Thanks again for reporting the issues.

1: en.wikipedia.org/wiki/Seven_Wells_Hill fails to load

This bug occurs because of a bad 2017-02 English Wikipedia release. Approximately 172,000 pages (out of 5 million) had bad category databases ids which in turn causes the page to fail.

I honestly don't know how this happened, and have not seen it before. I also checked 2017-03 English Wikipedia and it is not affected. I'm going to treat this as a fluke for now.

In the meantime, the next XOWA will gracefully ignore bad category databases ids and continue to load the page

2: de.wikipedia.org/wiki/Wikipedia:Quiz/Geschichte does not show one image

The problem occurs because the image has a title name with 218 characters:

The_story_of_the_greatest_nations,_from_the_dawn_of_history_to_the_twentieth_century_-_a_comprehensive_history,_founded_upon_the_leading_authorities,_including_a_complete_chronology_of_the_world,_and_(14578677039).jpg

Windows has a maximum of around 250 characters. Combined with a long XOWA root dir (in the above case, it was something like C:\xowa\xowa_v4.3.1.1703), the path quickly exceeded the 250 limit and failed to load.

I had code in place to handle a long title name, but not a long XOWA root dir. I added some code to handle the latter for the next release.

[gnosygnu]

Both of the above issues were part of tonight's v4.5 release.

@Ope30 In addition, I added a feature to specify the url name like your "English Wikipedia/Earth". See the help for Url format at home/wiki/Special:XowaCfg?grp=xowa.gui.url_bar in tonight's release. If you have any questions, feel free to comment here or open up a new issue.

Otherwise, I'm going to close this issue. Thanks again for all the defect reporting.