Semanticmediawiki: Property "is locked to prevent accidental data modification while a change propagation update is run" even though no change propagation is pending

[Dinoguy1000]

Setup and configuration

• SMW version: 3.0.0
• MW version: 1.31.1
• PHP version: 7.2.13-1+ubuntu16.04.1+deb.sury.org+1 (fpm-fcgi)
• DB system (MySQL, Blazegraph, etc.) and version: MariaDB 10.1.31-MariaDB-1~xenial

Issue

A property page on our wiki (https://yugipedia.com/wiki/Property:Fuses_with) reports as being locked while a change propagation is run, in spite of no such run happening or being available. This has been the state of things at least since October 10, and possibly for much longer (the page was last edited in December 2018).

Steps to reproduce

• Visit https://yugipedia.com/wiki/Property:Fuses_with
• Click "view source"
• Observe the notice about the page being locked

[kghbln]

Use $smwgChangePropagationProtection = false; to get out of the situation. Perhaps the frantic editing on that page confused the software. Probably also better to use datatype Text rather than String.

[Nikerabbit]

Fyi: I'm also seeing this quite frequently with category pages on my wikis. JobQueue is empty and the message doesn't go away.

[kghbln]

Fyi: I'm also seeing this quite frequently with category pages on my wikis. JobQueue is empty and the message doesn't go away.

Hmm, never had this since it was introduced. I am using SMW rather heavily I believe. Thus I guess that we are facing reality here... We have a way out but still: @mwjames FYI

[Dinoguy1000]

The above workaround fixed this for the page on my wiki, but I agree that a better resolution would be favorable.

[JeroenDeDauw]

@Nikerabbit I've noticed that running the SMW tests on your wiki will cause this kind of error.

[Nikerabbit]

I'm not running SMW tests on my wikis though, to my knowledge.

[JeroenDeDauw]

Maybe we're so big on testing that the tests execute in a background thread for every web request? ;)

[Dinoguy1000]

We have another page doing this now, https://yugipedia.com/wiki/Property:Stars. I accidentally left the old text (including the old has type declaration) in place while updating the property type, and it's been locked since. The "1 job estimated" message is bogus; the job queue is clear of SMW-related jobs.

I've tried the above suggestion of $smwgChangePropagationProtection = false;, as well as assigning the smw-pageedit right to admins (also setting $smwgEditProtectionRight = "smw-pageedit";), and neither allows the page to be edited.

If necessary I can create a new task for this, but felt it was similar enough to continue in this task. One thing to note is that we are now using SMW 3.1.5, if that makes any difference at all.

[kghbln]

The "1 job estimated" message is bogus; the job queue is clear of SMW-related jobs.

You are sure that at the time the deadlock of the page occurred there was not SMW-releated job?

I accidentally left the old text (including the old has type declaration) in place while updating the property type, and it's been locked since. The "1 job estimated" message is bogus; the job queue is clear of SMW-related jobs.

I am not sure if I have done the same however I have this issue too with one property page, i.e. there are circumstances that allow a page to be locked whatever you do.

[Dinoguy1000]

You are sure that at the time the deadlock of the page occurred there was not SMW-releated job?

When I made the edit there were other SMW-related jobs in the queue, but not from previous modifications of that property. Since then, all SMW jobs have been done.

[kghbln]

I believe the issue here is that SMW still believes that a job is to be cleared and does not recognize what this is no longer the case. We have some kind of "artifact" job reference here and no way to clear it.

[Dinoguy1000]

Is there something I can do server-side to clear it? Or at least to gather more information for you guys to debug with?

[kghbln]

I have not found a way. Even deleting and restoring the property page does not change the situation. What I have not done is dropping the backend and get a fresh start on the property. This is overdoing things though and I am not sure if this will help either.

[Dinoguy1000]

Given the number of annotations we deal with, I don't think a complete rebuild just for this is a great idea either.

[mwjames]

Well, the lock relies on the jobqueue information about the state and deduce whether something is holding a reference to it or not. For details, see [0].

The reliance of the jobqueue as tracker is fragile and I have a different idea in mind on how to better track the state but I don't know when this is going to happen.

[0] https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/src/Property/DeclarationExaminer/ChangePropagationExaminer.php

[Dinoguy1000]

Sometime since I reported the new page lock issue, the phantom job cleared, and now the page is just locked. The lock message mentions the smw-pageedit right, and Special:ListGroupRights shows admins have that right, but I am still unable to edit the property. (For the sake of convenience and clarity, here's a link to the page again: https://yugipedia.com/wiki/Property:Stars.)

[mwjames]

Sometime since I reported the new page lock issue, the phantom job cleared, and now the page is just locked. The lock message mentions the smw-pageedit right, a

But this has nothing to do with the change propagation, this is message that comes from [0] and I'm guessing someone has protected to the property manually.

As a side note:

"The property contains competing type specifications which may result in invalid value annotations therefore it is expected that a user assigns one appropriate type." defining the property as number and type, what is this?

[0] https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/src/Property/DeclarationExaminer/ProtectionExaminer.php

[kghbln]

I think this has nothing to do with the smw-pageedit permission. This is just the natural thing to look at first when trying to overcome the lockdown.

I'm guessing someone has protected to the property manually.

The lockdown occurs even it the page was not protected from editing manually. At least in my case this is not what happened thus not causing the issue. Probably also true for Dino.

[mwjames]

The lockdown occurs even it the page was not protected from editing manually. At least in my case this is not what happened thus not causing the issue. Probably also true for Dino.

I have to say that I'm unable to understand the issue, if the change propagation is ongoing and the property contains the a Change propagation property then the property is locked for said reasons (see the documentation for details). If the the Change propagation property is removed (means the dispatch job was able to find a related subjects) then the lock caused by Change propagation is lifted.

[kghbln]

The issue boils down to that we have a property page that was not protected manually with no propagation job left since the last edit that still cannot be edited manually, no matter what.

Edit: @Dinoguy1000 Try to edit page using the ReplaceText extension. I now remember that I was able to edit my property page this way.

[mwjames]

As I said, the message your seeing is from [0] which has nothing to do with the change propagation and is being shown because the smw-pageedit permission is expected. The relevant code to check for the permission is in [1]. As of now, I unable to verify the issue on my local wiki so unless someone tells me a reliable way to replicate to issue, don't expect further comments from my side.

[0] https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/src/Property/DeclarationExaminer/ProtectionExaminer.php

[1] https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/3.1.x/src/MediaWiki/PermissionManager.php#L145-L204

[Dinoguy1000]

But this has nothing to do with the change propagation, this is message that comes from [0] and I'm guessing someone has protected to the property manually.

No one's touched the protection level of the property, as can be seen from its edit history and logs.

As a side note:

"The property contains competing type specifications which may result in invalid value annotations therefore it is expected that a user assigns one appropriate type." defining the property as number and type, what is this?

The issue that led to this happening in the first place. I was updating the property using a copy-paste of its then-current text (well, actually a copy of the updated text of a closely-related property), and forgot to delete the old text before saving.

I have to say that I'm unable to understand the issue, if the change propagation is ongoing and the property contains the a Change propagation property then the property is locked for said reasons (see the documentation for details). If the the Change propagation property is removed (means the dispatch job was able to find a related subjects) then the lock caused by Change propagation is lifted.

There is no change propagation running for this or any other property; they all already finished. There is also no Change propagation property in the text of this property (I'm guessing that's what you're talking about). If the Change propagation property applies without being in the text of a property, and there's a way to manually remove it, I'd love to know.

Try to edit page using the ReplaceText extension. I now remember that I was able to edit my property page this way.

I'll give it a shot, thanks for the pointer. =)

As I said, the message your seeing is from [0] which has nothing to do with the change propagation and is being shown because the smw-pageedit permission is expected.

As I showed with my link to our S:ListGroupRights page, admins have smw-pageedit assigned (and I am an admin on the wiki); this was one of the things I tried to fix this issue before reporting it here.

As of now, I unable to verify the issue on my local wiki so unless someone tells me a reliable way to replicate to issue, don't expect further comments from my side.

The only thing I can suggest to attempt replication is to set up a test wiki with some relatively widely-used property (if you don't have such a wiki handy already) and add a new has type declaration to the property that changes its type, without replacing or removing the first one. This is the only time I've experienced this particular problem, and that is the condition that seems most obvious about it to me.

If you still can't replicate it (or if that's what you tried when you attempted replication) then it's perfectly reasonable that you wouldn't comment here more, if you have nothing else to add. Sometimes you just run out of things to say. =X

[Dinoguy1000]

Edit: @Dinoguy1000 Try to edit page using the ReplaceText extension. I now remember that I was able to edit my property page this way.

It worked! Thanks a ton for the suggestion. =D

Now to see if this turns out to be enough to also clear the protection...

[kghbln]

Now to see if this turns out to be enough to also clear the protection...

Actually it was. I was able to edit the page as an IP. I'd prevent this.

[Dinoguy1000]

I was able to edit the page as an IP. I'd prevent this.

Is this a general suggestion for all property pages? Is there a page on the SMW wiki which advises best practices for SMW-related page permissions?

[verdy-p]

Fyi: I'm also seeing this quite frequently with category pages on my wikis. JobQueue is empty and the message doesn't go away.

Yes, this happens repeatedly in translatewiki.net. Even after a page (a category page) was just created once (with very basic content, nothing displayed at all, only a parent category given which already existed), and never edited later, it was locked permanently since the first creation time).

And there's no way to reedit it later to change some bits, such as add a more relevant description, or recategorize it better.

Forcing the page refresh also has no effect, the page is still locked and not editable.

A few examples (pages created recently, never reedited after their initial creation more than 15 days ago; both were created when the wiki was absolutely not "busy" at all, when the job queue was initially empty but possibly a concurrent action from another user perturbed them):

• https://translatewiki.net/wiki/Category:Salishan_languages (now we cannot change its categorization sort key)
• https://translatewiki.net/wiki/Category:Translatewiki.net/ann (now we cannot remove the "Disabled language" template after this language was enabled)

Apparently this occurs only when a new page is created: a scheduled job is lost, or is not posted to unlock the page later, something unexpectedly "clears" the scheduled job remaining only in memory (but still not commited), most probably a false assertion check causing some exception in transitory states (possibly depending on other concurrent activities on the wiki server).

This may happen notably when creating a page while there was some other jobs in the queue, or could be caused by some stress condition: initially the job queue is empty, the job commit is delayed, some other works is done, then the fional commit uses unsafe code that still assumes that the job queue is empty but in fact some other job was commited in the middle.

The probable cause is a failure to initialize a new job queue correctly: a single test is performed once, but not rechecked later ans the code to create a new job in an empty job queue is very different from adding a job to the unempty queue.

Or the code incorrectly assumes that if the job queue was initially tested as empty, there's no need to commit the job in the database as it can run alone immediately at end of rendering the current page without fetching it from the database. It looks like an incorrect optimization whose pupose was to avoid using SQL INSERT while redering the currenty page, just to SELECT and DELETE/UPDATE it immediately after. I may just be safer to use INSERT always, (a concurrent INSERT may occur later adding to the queue, and the first job that will run after parsing the current page should still be fetched by a SELECT+UPDATE/DELETE; it would not be really costly in terms of SQL access as this row will most often still be in the SQL data cache)

[kghbln]

Thanks a lot for reporting our experience and ideas about this. Ever since I set $smwgChangePropagationProtection = false; this did no longer happen to me though I cannot tell if I just was lucky. Probably one should only do this on a non-publicly editable wiki or a wiki with lower likelyhood that wrong doing can occur.

What happened to me once or twice was that I edited on a page while heaps of change propagation jobs were still open for another page. This triggered a lockdown for the edited page.

now we cannot

Use the ReplaceText extension to edit these pages. I cannot test on twn since I can no longer use the extension.

[verdy-p]

Thanks a lot for reporting our experience and ideas about this. Ever since I set $smwgChangePropagationProtection = false; this did no longer happen to me though I cannot tell if I just was lucky. Probably one should only do this on a non-publicly editable wiki or a wiki with lower likelyhood that wrong doing can occur.

What happened to me once or twice was that I edited on a page while heaps of change propagation jobs were still open for another page. This triggered a lockdown for the edited page.

now we cannot

Use the ReplaceText extension to edit these pages. I cannot test on twn since I can no longer use the extension.

There's no such extension in Translatewiki.net. I think this can only be solved for now by an SQL admin, removing the locked status manually in the SQL database, or running some admin maintenance script to check and cleanup the incorrect locked status after check the locked page is not referenced by any pending job.

I don't know if SemanticWiki provides such maintenance script for wiki admins.

[kghbln]

There's no such extension in Translatewiki.net

Sure there is. However you have to be translation administrator to use it: https://translatewiki.net/wiki/Special:ReplaceText And yes, so far in it worked for me in all case, i.e. the page was editable after applying this trick.

I don't know if Semantic MediaWiki provides such maintenance script for wiki admins.

No and this appears just to help fixing the result of an issue rather that the issue itself.

[verdy-p]

There's no such extension in Translatewiki.net

Sure there is. However you have to be translation administrator to use it: https://translatewiki.net/wiki/Special:ReplaceText

So I cannot use it...

I don't know if Semantic MediaWiki provides such maintenance script for wiki admins.

No and this appears just to help fixing the result of an issue rather that the issue itself.

Standard Mediawiki provides a lot of such maintenance scripts, notably for recovery after some server crash (or power outage, or hardware failure on one disk, where soem data may be restored from backups/mirrors), or for version checking and upgrading some status/tables when a new database schema is published, or for reindexing tables with new collation data for a newer Unicode version or a new CLDR dataset, or for various extensions.

These scripts are maintained as part of the distribution, and updated when needed. Some scripts may be removed or replaced by other more powerful scripts making safer checks, or doing the same job faster and more simply. Some can only run during offline maintenance, newer scripts may run in the background while keeping the database online (where it may work in "degraded" mode or in slower mode with additional checks or additional commits to tracing tables so that the maintenance script can take the newer data also into account or so that a new version can still run with data in older/unupgraded data)

Offline maintenance is something that all wikis want to avoid, or it should take minimum time, allowing most of the work to be done while the database is back online and running in possibly slower mode. For very large wikis like English Wikipedia or Wikimedia Commons, most maintenance occurs online (with a compatiblity layer which may degrade the performance temporarily) as they want minimum downtime (and online maintenance made in the background for hours or days does not require the permanent presence of an admin, problems can be logged and delayed for being resolved for later maintenance).

Many of these maintenance scripts can run in the administrator's wiki console from their web browsers, where they can be launched without logging directly on the server with a *nix command line shell via SSH.

[kghbln]

Is this a general suggestion for all property pages? Is there a page on the SMW wiki which advises best practices for SMW-related page permissions?

@Dinoguy1000 I must have overlooked you post. I just added a recommendation to the respective page.

@verdy-p You have left the scope of this issue. General pros and cons for maintenance scripts, how and when to use them, etc. do not belong here. Please do not get me wrong but there is no further need to comment about this.

[verdy-p]

I just added a recommendation to the respective page.

This does not apply to the bug seen translatewiki.net: there was no change in datatypes, only the initial creation of a new very basic page with a category, and while the server had very low activity (no job pending initaiolly when the pages were submitted, this is easy to see in the wiki history log).

The only "maintenance" task was for the "changes" in categories, when it fact there was no change at all, it was an addition of a new page into an existing category (something that does not require any privilege and is a basic action performed by any user editing any page).

If you look at the (public) wiki history log you'll see that the "locked" page was created at approximately the same time as another page created by the same or another user, not seeing any "maintenance" warning at top before submitting his creation. Such condition is then completely unpredictable and can happen at any time. There was no abuse at alls of any privilege, no stress at all on the server it is only bound to very short period of times where two actions may occur simultaneously on a non-busy/idle server.

[kghbln]

@verdy-p I am on the brink of locking this thread. Of course this does not apply to what you wrote. This is an answer to @Dinoguy1000 and not to you.

[verdy-p]

Sorry. You took the previous comments positively and I confirmed the bug comment by @Nikerabbit seen on TWN with newer examples. All these were on topic. I suggested that this could be false assumption in the code that could trigger this lockdown situation, then requiring admin maintenance as you suggest. And such maintenance is not so easy, so adding a maintenance script could certainly help these admins doing things safely and fast (because the bug is still present and occuring quite often).

[DSwissK]

Had the same problem with 2 attributes on my website.

Just confirming that using ReplaceText works fine : https://dicoado.org/wiki/index.php?title=Attribut%3ALien&type=revision&diff=49842&oldid=49789

Edit : Both pages are still on lockdown though. I rebuilt the data and it didn't change its status.

[Rififia]

I've managed to clear the error on dicoado (the site of DSwissk above) just by purging all the cache. I've just ran update.php, it purged the cache, and done.

[verdy-p]

No progress for now:

Still this page is incorrectly "''is locked to prevent accidental data modification while a change propagation update is run''" even though no change propagation is pending since its initial creation (never edited since then):

https://translatewiki.net/wiki/Category:Translatewiki.net/bh (locked since more than one year)
https://translatewiki.net/wiki/Category:Salishan_languages (already signaled above 23 days ago, but locked since 22 Jun, nearly 6 weeks ago)

Another one, locked down immediately after initial creation just today (no new edit needed for now but same problem):
https://translatewiki.net/wiki/Category:Saraiki

Others (exactly the same problem):
https://translatewiki.net/wiki/Category:Banjar
https://translatewiki.net/wiki/Category:Translatewiki.net/fkv
https://translatewiki.net/wiki/Category:Luba-Katanga
https://translatewiki.net/wiki/Category:Aromanian
https://translatewiki.net/wiki/Category:Picard

It makes no sense at all, and SemanticMediaWiki really needs to develop a cleanup tool that site admins can run more simply, without having to search for specific locked pages that Semantic Mediawiki kept without creating any job to remove these page from the queue. This is definitely a bug of Semantic Mediawiki when it attempts to create a job (which is never commited or throws an error (may be because of concurrent queries on the DB, such as when creating temporary tables: the SQL schema should not be modified like this, it should be serialized but not made within the concurrent page parsing/rendering threads, and the separated serialization should use its own independant transactions, only for jobs in the job queue, but never for page editing/rendering).

The problem seems to be occuring systematically when we create a category page and the initial creation categorizes the page itself. So we need to create the category page first without categorizing it itself in ANY existing categorty, wait for propagation, hope this lock will be removed, then reedit it to add the parent category. But sometimes even in this case, if the category page is created and already contained member categories, the new caegory page will also be locked down, and then cannot be reedited to add its own parent category/ies.

SemanticMediawiki just breaks the basic categories of Mediawiki. I think there's no use to replicate the Mediawiki categories as a hierarchy for SemanticMediaWiki. If needed Semantic Mediawiki should just be hooking to lookup this hierarchy directly in MediaWiki, without reimporting them in its hierarchy (this is unnecessary work). Apparently SMW assumes that all category pages are created in order (i.e. the parent before all children); this assumption is false. And the creation of category pages does not change the relationshipo of the children that may already reference it, so the chain of triggers is incorrectly checked by SMW.

Note that the "solution" of using the admin-only "ReplaceText" extension is not really one: it requires forcing a change in the page, by overwriting it (adding at least one space or newline somewhere where it is "safe", or adding a dummy HTML-comment). The real solution would be to allow null-edits on these pages, or refreshes should trigger a check to reinsert the page in the job queue (because the last attempt just failed, but the page was still marked with a pending job that has NEVER been submitted and most probably NEVER run, as can be seen if we #ask for the properties that should have been reflected). Using ReplaceText apparently will not trigger the propagation in SMW properties, so this proposed solution is in fact useless (and the SMWK properties for parent categories are still incorrectly populated for these pages).

Most of the time the bug occurs when creating the category page which is already populated with member pages. You can preview the page before saving or not, but when saving you see instantly the rendered page with the "red box" saying that the page has been locked. But actually the background job has NOT been registered correctly, and will then never run. The page remains deadlocked in this state. you can press F5 key to refresh the page, the red box is no longer there, but the page is protected and not editable. You can try "purging" the cache, it has no effect.

And this bug occurs even when creating the category page without any parent category specfied in it, or with all its parent categories correctly set, themselves preexisting or not.

All seems to indicate that there's an uncommited transaction, which is cleared prematurely at end of the session; only the transaction for creating/modifying the category page is complete. The transaction to register the background job in the queue is lost even if the page status lock was set and registered (as part of the transaction for the page creation but not as part of the transaction for registering the background job). So the two transactions are not separated, two transaction are started but one of them is flushed and committed too early. SemanticMediawiki does not use the correct sessions for these separate transactions and this causes this recurrent bug.

[verdy-p]

I found a way to get a list of the affected page. In fact this is caused by the special property "Change propagation" being added to the affected pages (all of them categories) but never removed.

In fact this property is added AFTER the background job has already run and finished its job (this job did not find any property to delete, it silently terminated itself).

See this report on Translatewiki.net (various are caused by my own basic edits of pages to add/remove/change a category, some occurs immediately after creating a new category with a single edit. Some are caused by other users doing the same thing.
So this affects wikis whose job queue is empty when an edited page is saved to add/change/remove any categorization.

https://translatewiki.net/w/i.php?title=Special:Ask&q=%5B%5BChange+propagation%3A%3A%2B%5D%5D&p=format%3Dbroadtable%2Flink%3Dall%2Fheaders%3Dshow%2Fsearchlabel%3D...-20further-20results%2Fclass%3Dsortable-20wikitable-20smwtable%2Fsep%3D%2C-20&po=%3FModification+date%0A&sort=Modification+date&order=asc&eq=no&offset=0&limit=50

or:

{{Columns |width=26em |gap=2px |margin-top=1em |1=
{{#ask:[[Change propagation::+]]
 |?Modification date
 |sort=Modification date |order=asc
 |offset=0 |limit=500
 |link=all
 |format=table |class=wikitable sortable smwtable
 |headers=show
 |searchlabel=more... 
 }}
}}

with custom printouts, and sort order.

None of these pages have any job pending. They are locked since very long (more than one year for some).

And I don't see how we can remove these spurious special properties (which are not auditable with the common special pages, without knowing the internal special properties (that are hard to find, as the helpers in Special:Ask do not propose them and there's no simple report listing the defined special properties in the default installation Semantic Mediawiki; such hidden "magic" should never occur; it must be better documented; all we can see is the SQL report for debugging which just displays the internal SQL table names and some cryptic fields that no wiki admin can reasonable guess or manage without being himself a SMW developer and knowing perfectly the internal complex schema used, and poorly documented)

The suggestion above (using [[Special:ReplaceText]]) is not necessarily installed, or not granted implicitly to translation administrators (e.g. those admins in MetaWiki or Wikimedia Commons don't have this extension, normally integrated in MediaWiki 1.31+, it is not listed or accessible to them or displayed as an inexistant page, even if SemanticMediawiki is not installed on these Wikimedia wikis: by default, only members of the 'sysop' user group have access to the Replace Text functionality as it is extremely dangerous): this Special extension (not supported as well by Wikimedia on its public wikis, except possibly for a few sysops with severe restrictions and not granted to any bot and not at all for translation admins) is independant of SemanticMediaWiki.

Also I'm not convinced it can really solve the problem, due to the fact [[Special:ReplaceText]] is not maintained for this purpose; using it may just ''eventually'' and ''indirectly'' trigger some behavior in SemanticMediaWiki, to force it to reevaluate its own conditions and properties on pages modified bhy ReplaceText, but I'm not even sure that this would apply to dummy-edits, i.e. where there's actually no effective change to the wiki content of the page (Maybe it is needed to insert some dummy space or newline in the middle of the wiki content, or temporary change one character and revert it a bit later once SemanticMediawiki has finished its triggered updating work).

We still need a stable solution supported by SemanticMediawiki itself (i.e. some admin tool or script to remove these damned permanent locks, that continue to grow the list at unpredictable times), plus some fix in the SemanticMediawiki code for what really looks like an unchecked race condition.

[verdy-p]

See also
Unexpected permanently locked pages (bug occuring randomly, caused by SemanticMediawiki for its "change propagation" in Translatewiki.net.

Note that all these pages are ONLY categories, WITHOUT ANY custom semantic properties, they ONLY have the synthetic (automatic) properties tracking the standard hierarchy of Mediawiki categories. These are ONLY "special" builtin properties, generated by default by SemanticMediawiki.

NOTHING has changed with your fix, you did not not fix anything. And the bug continues with some pages added randomly to the list of locked page.

This is definitely a race condition between the time the jobn is setup in one thread (but the status is kept in a cache that will be stored later) and the background job that can run and terminate earlier, trying to delete the ```[[Change propagation::+]]```` pseudo-property for this page from the database '''before''' the status (kept in a cache of the main process) was actually commited to the database by the main thread. It seems to occur because the main thread may want to service other incoming requests from other visitors of the website.

[verdy-p]

I've found a workaround that works for some pages:
On those pages that don't have an "Edit" tab but a "View source" tab,

• click on it: this opens the editor in readonly mode.
• change end of the URL from "action=edit" to "action=purge", validate the modified URL
• the page reload a form where you have to click to "confirm" the purge, if you are requested. After that confirmation the page content will be shown again in its new "state".
• This displays the page again, but still with the "View source" tab instead of "Edit", but if you click it again immediately, the editor ''may'' finally run in normal edit mode.
• Save the page immediately as is (i.e. make a null edit without changing anything, notably not any referenced category)
• Now the page is unlocked.
  The trick is to force the purge and edit it fast enough before the cache is refilled with an edit-lock after a few seconds (or minutes on small servers with low ressources).

This does not work on all pages (apparently this occurs when there are two (duplicated) locking properties in SemanticWiki for the same page (the Mediawiki "purge cache" only removes the first one, the second one is kept, and it still locks the page and the special property will be added again as a duplicate; most probably this duplicate property has another internal difference in some of its tags, probably an old one)

This also does not work to fix the lock on page that cause a reference loop (e.g. a hard redirect of categories to any other one or to itself because of a mistake while editing the page, or saving it too soon before changing the redirect, e.g. by pressing the TAB key by error and pressing Enter to save it, and there's no prompt to confirm the save as there was an edit comment and no other warning detected about such reference loops).

Now there remains the main bug of locks left permanently on category redirects (hard redirects with #REDIRECT [[]] to any other page in any namespace) or to the current page itself. Such mistakes are possible and generally made unvoluntarily, but can't be corrected by the user seeing that.

This can occur at any time, from any user (confirmed or not), and can cause a nightmare on wikis as it allows some form of denial of service attacks that will require a site admin to fix it, as such edits (made unvoluntarily, or not) can't be easily reverted and important pages (or templates, modules) could be blanked and locked and generate lot of problems on the wiki if this affects a widely referenced page: being a simle translation admin, or wiki admin, or reviewer is not enough, this can only be solved by a site admin with a manual update in the database (using the SQL console with full access to the database, so this fix can't be open to "normal" wiki admins.

The SQL access privilege often requires access on a system shell console, and can expose data protected by privacy, so only site owners: very few "super-admins" (or just a single one in "small" sites) that can install Mediawiki on their own hosting server can do that; such correction using direct SQL can also be quite dangerous, the SQL request could delete many things, leaving nothing in the wiki history, and if there's any error, the site admin will have to restore the database from a backup dump, loosing all recent edits made after this last database dump; restoration will also require putting the whole site in read-only mode, for long time if there's the need to reload tens or hundreds of gibabytes). For this reason, a correction script should be created for such maintenance, to perform checks and avoid damaging or dangerous errors, and to avoid direct access to private user data (which should remain secret).

With the trick above, I could finally fix all the remaining locks that were left in Translatewiki.net (without having any need to reload the database or any admin privilege: you just need the edit permission on pages and the possibility to make a standard "action=purge"). For some cases, you'll need to find which other page was referencing the locked page or category, and force a purge and null-edit that other page, so that its spurious property entry will be removed from the cache and will no longer cause the lock on the other affected page.

This is visibly caused by a synchronization issue when the background job is trying to clear the cache of properties: sometimes this cache keeps an old record that has been deleted in the database by the background job, but it remains indefinitely (for months or years...) in the cache used by the front wiki server, even if there remains nothing in the SQL database. When trying to edit a page however, that locking property is still read only from this cache, without actually querying the SQL backend, and this raises the locking top banner displayed and the fact that the "Edit" tab shows "View source" only.

The cause of that bug seems to be that the background job sends a "cache clear" request before the SQL removing the spail Semantic Property is effectively commited to the database. If there are concurrent queries on the wiki trying to read the cache, as it is already flushed, it will forward an SQL request to refill the cache with the record that is still not deleted and commited by the background job. The background job then terminates, commits its DELETE, but the cache is left "as is", keeping the old record that should have been purged from the cache only at that time.

In summary, there's a problem of coordination of transactions between the SQL backend (SQL DELETE effectively committed) and the cache (purge old record in "memcached" used by concurrent SELECT queries coming from any user of the wiki), with incorrect of order of execution of commits on both sides...

A "fast" resolution of this (before the fix is made in the code) consists in just flushing "memcached" completely (forcing all other requests on the wiki to perform an actual SQL query to refill the cache). This will slow down the wiki only for a few minutes. No change is needed in the SQL database itself. Such resolution may be easily applied at hours of low usage of the server, and hsould be done at least once a week. This an be done simply by shutting down and restarting "memcached" immediately, without even the need to reboot the server.

Note that this bug does not seem to occur if MediaWiki is configured to store its cache in the same DB, it occurs if there's a separate cache server (like memcached or Wincache, or multiple cache instances configured; on large wikis with solid DB engines, the cache is in the same DB, and this is the case of most large wikis of Wikimedia, that use solid DB instances with lot of memory configured to handle the cache itself for SQL requests and all I/Os to any core storage tables, or caching tables to store intermediate computing results).

[Seb35]

I observed this issue on a newly-installed MW 1.36 + SMW 4.0.0-alpha and investigated, and have a reasonably-plausible hypothesis involving the cache in ChangePropagationDispatchJob.

Summary

The short answer is: whether you experience or not this issue depends on your $smwgMainCacheType:

• memcached, Redis, DB are immune;
• APCu has the issue.

Details

The theory is: ChangePropagationDispatchJob caches for 24h the number of jobs related to the given page, then when the jobs are executed the cache key for this page is deleted.

A focus about the cache type: it is a composite cache composed of OnoiCacheFixedInMemoryLruCache and a MW cache fixed by $smwgMainCacheType (and if the default CACHE_ANYTHING is used, depending on $wgMainCacheType or other cache parameters, see ObjectCache::newAnything for the exact logic). When a value is written in this composite cache, it is written on all sub-caches. When a value is fetch from this composite cache, the first value existing in a cache is returned. The first sub-cache FixedInMemoryLruCache is not significant here because it is very short term (discarded at the end of the request).

The core issue: In the case where the SMW main cache is shared between Web and CLI (memcached, Redis, DB) there is no issue because, when the job is executed in CLI, the cached value is deleted in the SMW main cache and the Web can access this state (deleted cached value); but if the SMW main cache is not shared between Web and CLI, the cached value is initially saved in the Web-accessible cache, and deleted (by the job) in the CLI-accessible cache, and in this case remaining in the Web-accessible cache.

---

Appendix: the actions which leads me into this hypothesis:

1. on my test MW 1.36, the MW installer set up $wgMainCacheType = CACHE_ACCEL; (APCu in-process cache)
2. I created a page property, setting its type and got a Change propagation updates are pending (3 jobs estimated) warning
3. I ran php maintenance/runJobs.php
4. The warning on the page was still present
5. I changed $wgMainCacheType = CACHE_DB; : the warning on the page was removed
6. I changed back to $wgMainCacheType = CACHE_ACCEL; : the warning came back
7. By exploring with php maintenance/eval.php:

> $applicationFactory = SMW\ApplicationFactory::getInstance();
> $subject = new SMW\DIWikiPage( 'Prop1', 102 );
> var_dump( SMW\MediaWiki\Jobs\ChangePropagationDispatchJob::hasPendingJobs( $subject ) );
0

1. I added the following code at the end of ChangePropagationDispatchJob::hasPendingJobs just before the return:

var_dump( $applicationFactory->getCache()->getName() );
var_dump( $applicationFactory->getCache()->fetch( $key ) );

• in CLI it is displayed

string(100) "Onoi\Cache\CompositeCache::Onoi\Cache\FixedInMemoryLruCache::Onoi\Cache\MediaWikiCache::SqlBagOStuff"
bool(false)

• in Web it is displayed

string(100) "Onoi\Cache\CompositeCache::Onoi\Cache\FixedInMemoryLruCache::Onoi\Cache\MediaWikiCache::APCUBagOStuff"
int(3)

(it can be noted that the CLI cache is SQL even if $wgMainCacheType = CACHE_ACCEL;, this is because ObjectCache::makeLocalServerCache() prevents using APCu in CLI, but even if it would authorise it, the corresponding cache would be discarded at the end of the CLI execution and not shared with the Web anyway.)