Smf2.1: Old BBC Removal

[jdarwood007]

It was brought to light that the upgrader process for old BBC removal is a killer during the upgrade process. A quick team discussion and a suggestion has arose that seems to be a good course of action.

The problem comes down to using a LIKE against a text field in a table containing lots of data. In all databases, this is a slow process. Even with fulltext indexes this can still be slow. Not all forums run full text. In addition some upgrade scenarios can cause the fulltext to be removed to facilitate other upgrade needs. Thus it is unsafe to trust a fulltext index to save the day.

The suggestion is as follows:

• Revert change to run the upgrader changes to remove the "old" bbc
• Add a section in the bbc parser for old bbc codes handling
• Add a hook for old bbc codes, allow accessing the entire list of "old" bbc
• Any bbc in the old bbc list will basically parse out as straight text.

The idea here is to silently just parse out as text any bbc in the old list. This prevents the need for the upgrader to remove them.

Mod authors will need to use 2 hooks if they wish to revive a removed bbc. One hook to add it to the bbc list, the second hook to modify the "old" bbc list. Mod authors would read the array of "old" bbc and remove the one they want to support.

If anyone wants to work on this, they are welcome to at any time. I'm marking it as RC1 issue as this is a show stopper and will kill large forums during the upgrade process.

[albertlast]

Can you please provide a mysql dump with a realistic amount of data for this issue?
Othetwise we can't reproduce this issue and test different solution

[LexArma]

There are forums out there with million upon millions of posts, I know of a certain 2.0 installation with more than 15 million posts. Also, one part of the discussion on this was the fact that we should avoid tampering with the message bodies in any case, because we are then altering contents of forums we have no authority over.

[albertlast]

Than anonymize then.
When we can't validate a issue,
then the issue didn't exists.

The population script for smf didn't do any bbc stuff and
eaven it would do this, the realistic amount of bbc code per row would be not right.

[albertlast]

When you got a database issue,
why not to try to solve this on database side?
I did an example implmentation for pg:
https://github.com/SimpleMachines/SMF2.1/pull/4874
end of the day it do same stuff like the other convert with in mind that the data amount could be "high".

And you could place this as a cronjob after the upgrade to convert the board after upgrade when you like.

[MissAllSunday]

@albertlast yes it is a DB issue but an incidental one. We will no longer offer a way to remove/upgrade old BBC tags in the upgrade itself but we might still offer a separate tool for those users who want to use it and thats where your approach could be used.