Pkp-lib: Improve file upload during submission

[NateWr]

Describe the problem you would like to solve
Uploading files during submission involves lots of steps. It can be cumbersome and confusing, and authors are often asked to provide metadata that is not clear to them. In addition, the current UI is not accessible and will need to be revised before we can commission an audit of our submission process.

Describe the solution you'd like
A new UI component for uploading files should support the following:

• Ability to upload multiple files at once, by drag-and-drop or file selection.
• Provide useful defaults to minimize the number of fields that an author must complete.
• Abandon use of the file upload wizard.
• Identify the file component (eg - Article Text) and the file type (eg - .doc).
• Distinguish primary components (eg - Article Text) from supplemental components (eg - Data) and make selection of the correct component easier.
• Be fully accessible (keyboard controls and assistive technology).

Ideally, it will also take the following into consideration:

• Ability to reuse this file component in other parts of the workflow.
• Ability to be integrated into a new form when the submission process is revised.

Who is asking for this feature?
The ability to upload multiple files at once, by drag-and-drop or file selection, has emerged as a major request from our usability tests. This is also a requirement in order to be able to build and audit an accessible submission form.

[NateWr]

One of the questions I have is about the file metadata that we support.

Does it make sense to request this information _during submission_? My feeling is that it doesn't, because...

• authors are unlikely to know the correct data and are unreliable providers of metadata
• many of these fields are multilingual, which increases the likelihood that the author will be unable to provide accurate information
• we should design a submission process that minimizes the unnecessary labour required from submitting authors.

A description is probably important to include so that editorial staff can more easily identify the files. But otherwise can we push the metadata back to later in the process?

One potential hiccup is with OPS, where the distance between submission and publication is pretty short. But I suspect that could be better handled by a pre-publication check that required such information where necessary.

[willinsky]

I like the idea of a pre-publication request for metadata, rather than submission request, as authors are more motivated given it is ready to be published. We currently ask for "abstract" on submission of a paper, which is easy for authors and expected and valuable in selecting reviewers (along with reading the paper).

I'm not sure where the form displayed above is from, but would suggest that (a) we start to use Crossref's "Funder Registry" as standardized names will help in our various funding models for open access (and copyright reform proposal), (b) watch for when the topic for a box could be read as applying below or above because of spacing, and (c) we use to have a way for editors to suggest examples relevant to their field for each metadata category that then appeared in the appropriate box in faint gray suggesting it was to be written over. This was a compromise on the lack of controlled vocabularies in most fields. As a final thought, I can imagine a really conscientious journal might want to have a "designated indexer" on their editorial team whom authors could alert when stuck in selecting a metadata term, who would then step in read the abstract and populate the missing metadata field(s) so that the paper could be published.

[alexxxmendonca]

Should we also be discussing here integration with data repositories? Presumably and ideally these data have all been already filled in a data repository prior to the writing of the manuscrit itself therefore for the author to be able to simply transfer data + metadada from one place to another would make their life a lot easier.

[NateWr]

@willinsky this issue is specifically talking about file uploads, rather than the general submission metadata. But I think we'll get some good recommendations for submission metadata from the metadata working group soon, and implementing support for controlled vocabs like Crossref's Funder Registry should be pretty easy to do with the new metadata forms introduced in 3.2.

Should we also be discussing here integration with data repositories?

I think we'd need to deal with this on a repository-by-repository basis. Each case will require its own setup, configuration and transfer code. If you have some repositories in mind that you think should be supported, file a separate issue for them so that we can track that request.

[alexxxmendonca]

Thanks!

The one I'm interested is already filed #1822 (I had forgotten)

[carzamora]

Does it make sense to request this information during submission? My feeling is that it doesn't, because...

Hi @NateWr, I can say that in my journals nobody fills the metadata for files that upload, even when some pictures have information and copyright they include the info inside the manuscript but not in the metadata.

And, some things you proposed are very interesting, and they will really improve the usability:

Ability to upload multiple files at once, by drag-and-drop or file selection.
Provide useful defaults to minimize the number of fields that an author must complete.
Abandon use of the file upload wizard.

[NateWr]

Thanks @carzamora! After talking with a few people, it seems the file metadata is not widely used outside of OMP. And where it is used, it is only needed on publication rather than during submission. Some of the feedback I've gotten:

• No one knows of its use in OJS, but we would like to better support some fields, like licensing information, for supplementary files in the future.
• Part of OAI and maybe Google Scholar tags in OMP, due to the way that files are used directly with Publication Formats.
• Being able to modify the filename is important in OMP, where a single submission may contain more than 100 files.

Given this, the plan is to remove file metadata during the submission process. Later, when we look at refactoring file handling within the editorial workflow, we may move it out of there too, making it only available in galleys. But this is not yet decided and that work is not scheduled in the near future.

[NateWr]

The following PRs implement a new UI to upload files during submission. This control now supports uploading multiple files at once and uploading by drag-and-drop, and reduces the number of fields for the author to set in order to make submission easier.

These PRs also implement a major refactor of how submission files are handled, as discussed in https://github.com/pkp/pkp-lib/issues/6079. This refactor abandons the fileId/revision system and no longer duplicates files when a submission file is promoted from one file stage to another.

PRs:
https://github.com/pkp/pkp-lib/pull/6292
https://github.com/pkp/ui-library/pull/118
https://github.com/pkp/ojs/pull/2887
https://github.com/pkp/omp/pull/871
https://github.com/pkp/ops/pull/73
https://github.com/pkp/pdfJsViewer/pull/58
https://github.com/pkp/quickSubmit/pull/51
https://github.com/pkp/usageStats/pull/34
https://github.com/pkp/reviewReport/pull/37

@asmecher I still need to work on OMP/OPS, and I have not looked at the PR diffs yet to find stupid mistakes. But I'd appreciate it if you could start your review so that we can identify any major problems soon.

cc @amandastevens and @kaitlinnewson this will impact documentation.

[asmecher]

@NateWr, thanks, I've taken a review of those PRs -- some much-needed foundational improvements, and all that at a net removal of LOCs. Very impressive. I reviewed with emphasis on the PKP-lib pull, and in particular didn't look much at the front-end aspects (ui-library) or details like import/export. I'm ready for these to get ported to the other apps and then merged, once you're satisfied that you've gone through my comments, but please do check in with @ajnyga about timing as he's working on some OPS stuff that we'd like to be demoable in November.

[NateWr]

All merged! :tada: