Pkp-lib: Fix genre assignment for upgrades

[asmecher]

The upgrade script from OJS 2.x currently sets the genre_id to 1, assuming that this will be SUBMISSION default per registry/genres.xml. (See https://github.com/pkp/ojs/blob/ojs-3_0_2-0/dbscripts/xml/upgrade/3.0.0_update.xml#L92.)

This may not refer to a genre belonging to the right context_id, and assuming knowledge of autoincrement columns is brittle. Fix the upgrade to use entry_key, and if possible, find and correct bad data introduced by earlier upgrades.

[asmecher]

@bozana, mind taking a look at this one? Thanks!

[bozana]

PRs:
ojs master: https://github.com/pkp/ojs/pull/1407

later fix for genre ids in file names:
https://github.com/pkp/ojs/pull/1613

[bozana]

@asmecher, could you please take a look at the PR above?

[bozana]

I will open this issue, because here I fixed the genres in the DB but not the file names in the files_dir...

[Vitaliy-1]

Hi @bozana and @asmecher
In my case I get an error upon test upgrading on my local machine that is possibly related to this issue:
(OJS 3.0.2 to OJS 3.1)

PHP Fatal error:  Uncaught Error: Call to a member function getId() on null in C:\xampp\htdocs\site2\classes\install\Upgrade.inc.php:2476
Stack trace:
#0 C:\xampp\htdocs\site2\lib\pkp\classes\install\Installer.inc.php(415): Upgrade->fixGenreIdInFileNames(Object(Upgrade), Array)
#1 C:\xampp\htdocs\site2\lib\pkp\classes\install\Installer.inc.php(265): Installer->executeAction(Array)
#2 C:\xampp\htdocs\site2\lib\pkp\classes\install\Installer.inc.php(186): Installer->executeInstaller()
#3 C:\xampp\htdocs\site2\lib\pkp\classes\cliTool\UpgradeTool.inc.php(88): Installer->execute()
#4 C:\xampp\htdocs\site2\lib\pkp\classes\cliTool\UpgradeTool.inc.php(64): UpgradeTool->upgrade()
#5 C:\xampp\htdocs\site2\tools\upgrade.php(34): UpgradeTool->execute()
#6 {main}
  thrown in C:\xampp\htdocs\site2\classes\install\Upgrade.inc.php on line 2476

[asmecher]

@bozana, would you mind taking a look?

[Vitaliy-1]

If I understand right, GenreDao is a Class with database methods for genres table.
There is not any STYLE value for entry_key in any row of this table. So in this expression on line 2467 (Upgrade.inc.php):
$styleGenre = $genreDao->getByKey('STYLE', $context->getId());
$styleGenre is null. And eventionaly on line 2467:
if ($submissionFile->getGenreId() != $styleGenre->getId()) {

[martinpub]

I can verify this update of Upgrade.inc.php solves the upgrade issues we were experiencing (#2934).

[bozana]

@Vitaliy-1 and @martinpub, how come you do not have those STYLE keys in your DB table genres? -- there are those default genres installed with OJS 3.x for each journal... How does your DB table genres look like? THANKS!

[bozana]

@asmecher, should those default genres be deletable?

EDIT: ah sorry, forget it, I've figure it out: https://github.com/pkp/pkp-lib/issues/2945 :-)

[Vitaliy-1]

CREATE TABLE `genres` (
  `genre_id` bigint(20) NOT NULL,
  `context_id` bigint(20) NOT NULL,
  `seq` bigint(20) DEFAULT NULL,
  `sortable` tinyint(4) NOT NULL DEFAULT '1',
  `enabled` tinyint(4) NOT NULL DEFAULT '1',
  `category` bigint(20) NOT NULL DEFAULT '1',
  `dependent` tinyint(4) NOT NULL DEFAULT '0',
  `supplementary` tinyint(4) DEFAULT '0',
  `entry_key` varchar(30) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

--
-- Dumping data for table `genres`
--

INSERT INTO `genres` (`genre_id`, `context_id`, `seq`, `sortable`, `enabled`, `category`, `dependent`, `supplementary`, `entry_key`) VALUES
(1, 1, 0, 0, 1, 1, 0, 0, NULL),
(2, 1, 1, 0, 0, 3, 0, 1, 'RESEARCHINSTRUMENT'),
(3, 1, 2, 0, 0, 3, 0, 1, 'RESEARCHMATERIALS'),
(4, 1, 3, 0, 0, 3, 0, 1, 'RESEARCHRESULTS'),
(5, 1, 4, 0, 0, 3, 0, 1, 'TRANSCRIPTS'),
(6, 1, 5, 0, 0, 3, 0, 1, 'DATAANALYSIS'),
(7, 1, 6, 0, 0, 3, 0, 1, 'DATASET'),
(8, 1, 7, 0, 0, 3, 0, 1, 'SOURCETEXTS'),
(9, 1, 8, 0, 0, 1, 1, 1, 'MULTIMEDIA'),
(10, 1, 9, 0, 1, 2, 1, 0, NULL),
(11, 1, 10, 0, 1, 1, 1, 0, NULL),
(12, 1, 11, 1, 1, 3, 0, 1, NULL);

We haven't yet performed update on our production system. Should we somehow alter this table or updating with our modifications for Update.inc.php would be just fine?

[bozana]

Hmmm... I do not understand why your entry_keys are NULL :-O

[Vitaliy-1]

My installation comes from OJS 3.0.
So, what do you recommend? :)
Leave the table as it is and perform an upgrade with the changes in the upgrade script?
Or somehow alter this table?
And I am afraid that our journal is not the only one experiencing this problem...

[bozana]

Hmmm.... yes... Apparently this could happen for other journals upgrading from OJS 3. However the correct situation is to have all those entry_keys in the DB table. -- They all should have been installed correctly and if then/later one deletes a default component/genre (Settings > Workflow > Components) the component/genre is not removed from the DB -- just the column "enabled" is set to 0. -- So I would guess that those NULLs are somehow wrong in the DB table, maybe from an earlier bug.
Thus I believe that in your case it would be good to repair the table to contain those keys. But before you do that, could you please tell me what do you see in the backend, as journal manager, going to Settings > Workflow > Components -- what components/genres are listed there?

[Vitaliy-1]

Name - Designation:
Article Text - 99Z
Image - ##
HTML Stylesheet - ##
Other - ##

[bozana]

Hmmm... @Vitaliy-1, what OJS version are you using?
Ah, ok, ok, it makes sense... Sorry...

[bozana]

So it looks like you do only have the default components/genres. Could you then try to fix those NULLs: the first would be SUBMISSION, and the last three IMAGE, STYLE, OTHER ? And for each journal, if you have several.

[Vitaliy-1]

OJS 3.0.2 at the moment. OK. Will report results.

[Vitaliy-1]

Thanks! Update of a production system was performed successfully without any problems.

[SuperDomek]

Hello, I believe there are more people having an issue with the upgrade from 3.0.2 to 3.1.0 with the function fixing the Genres throwing error on NULL. The thread in the forum https://forum.pkp.sfu.ca/t/upgrade-from-ojs-3-0-2-to-ojs-3-1-problem/34592. Could you please sum up what needs to be fixed in the data? Thanks

[bozana]

According to this configuration file https://github.com/pkp/ojs/blob/ojs-3_1_0-0/registry/genres.xml, all default genres have to exist for each journal in the DB table genres. The column entry_key should contain the appropriate key for all default genres.
Here an example of the DB table genres, when only one journal exist:

It is possible that the users already deleted i.e. disabled some components/genres (Settings > Workflow > Components) in the earlier OJS 3.x version or that they added some custom components/genres.
They will then also exist in the DB table genres, where the deleted i.e. disabled components/genres will have enabled = 0 and the custom genres will have no entry_key (which is correct and should not be changed).

Thus, the missing entry_keys for default genres have to be correctly entered in the DB.

[bozana]

@SuperDomek, is the comment above clear enough? THANKS!!!

[SuperDomek]

I understand. Yes, I think we have been deleting some genres or at least adding. The problem is right now for some reason all of these are set to NULL, see pic.

EDIT: This is the state in version 3.0.2 on the production server.
So right now I don't have a way of telling which is which. Maybe if I try to match the genre_id with row id in the source code in the list in Settings > Workflow > Components...

[bozana]

@SuperDomek, yes you can math the genre_id from Settings > Workflow > Components or from the DB table genre_settings...

EDIT: maybe I could try to write a script that would consider the entries in the DB table genre_settings... The requirement would be that they are correct. How are they in your case?

[carzamora]

@bozana thanks for the explanation!

I want to show you my table and how I see the components in the backend. Sorry, maybe is redundant but I need it for understanding the problems.

In OJS backend I see ## in Designation, so I thought that this was a missed word, so I replace some I need for my journal:

But when now I review the table in DB I see the problem:

For understanding what is happening, I return to the backend and change other Designation, in this case for Multimedia I assign MULTI and for Source Texts I assign SOUR:

And this happened in table genre in the DB:

I think that Multimedia was changed before and now not change, but the changes in Source Texts make disappear texts in the table genre.

Ok, I could use your table and replicate the same entry_key in my table, but I have questions:

1. What happens with my row number 13? @SuperDomek have 13 rows too!
2. What will happen, if in the next days I change this Designation again?
3. Or if I need to add another Component? Or delete a Component?

or this issue is resolved after installation OJS 3.1?

sorry for the large of this but I am trying to understand... thank you!!

[bozana]

@t4x0n, thanks a lot! Your situation and explanation is surely helpful for the others as well :-)

Apparently, there is a bug in OJS 3.0.2, when updating a component/genre -- this line is not correct: https://github.com/pkp/pkp-lib/blob/ojs-3_0_2-0/classes/submission/GenreDAO.inc.php#L172 -- instead of setKey the function getKey is used and thus the Genre object that is edited has no key any more.
I didn't know where those NULL values came, but now I figured it out. Unfortunately that line has to be fixed in OJS 3.1 too. The fix is coming. So thanks a lot for helping figure that out!!!
After that fix is applied there will be no problems any more when editing a component/genre.
(Designation does not exist any more in OJS 3.1)

I think your row number 13 is a custom component/genre, that you have created, correct? It can stay as it is -- the custom i.e. newly created components/genres do not have keys.

[bozana]

PRs:
pkp-lib master: https://github.com/pkp/pkp-lib/pull/2999
ojs master: https://github.com/pkp/ojs/pull/1656 (only submodule update)

pkp-lib ojs-stable-3_1_0: https://github.com/pkp/pkp-lib/pull/3000
ojs ojs-stable-3_1_0: https://github.com/pkp/ojs/pull/1657 (only submodule update)

[bozana]

@t4x0n and @SuperDomek, please apply the patch for pkp-lib ojs-stable-3_1_0 above in your 3.1 installation... THANKS!!!

[carzamora]

thanks to you @bozana!

[carzamora]

@bozana I change the line "getKey" to "setKey" and now works, but if I delete a Component and later I use the "Restore defaults" button nothing happened. Maybe you are changing other files to solve that?

another very important question: is recommended download all OJS from GitHub? or, is better idea wait and download the zip file from the official site?

[bozana]

@t4x0n, here is the patch fr the restore of default genres: https://github.com/pkp/pkp-lib/commit/3056ce843b765ceead6d0cd933abf6c6de303cb8

If you do not work with git, for using OJS, then it is better/easier to get the release package from here: https://pkp.sfu.ca/ojs/ojs_download/ -- it contains all submodules too... Working with git would make it easier to apply the new fixes/commits.

[carzamora]

thanks! at the moment I don't know how to work with git, and I am using the official packages but maybe in the near future I learn more for apply commits directly, I am just enthusiastic coder xD

thanks for all @bozana

[SuperDomek]

Hello, I'm sorry I've been away. Thank you very much for the patch. I'll try it and get back with the result.