Documentation: Meta-Issue: Paged Content

[dannylamb]

This is a meta issue to track the development of paged content features. Please refer to this issue in any subsequent issues to link them.

The inclusion of paged content will require several other features:

• OCR Derivatives
• Multi-page viewer
• Page ordering through UI
• Reconciliation of Newspaper vs. Books vs. Serials in 7.x (they really should all work the same way)

@rangel35 has started working on this for UT Austin, and may have a working solution soon.

[kayakr]

@dannylamb @rangel35 I'm involved in a project that is looking for this kind of functionality. Has there been any progress? Have decisions been made about how to model page objects and their relations?

[seth-shaw-unlv]

@kayakr, I've done a bit of work with this, but not extensively. Mostly I'm relying on the child object's member_of field plus a weight field (from the weight module) for ordering. A simple 'component parts' block with an appropriate display mode makes the display simple enough, although I haven't worked with the GUI-based reordering yet.

This seemed the most Drupal-ly way to do it. I attempted to use the book module, but that felt really awkward and clunky to work with.

[rangel35]

I’ve been working on a project that utilizes paged content…I use the drupal book module, with a custom content-type, it takes care of the pagination, I use a drush file for ingesting multi page objects that updates the book table in the database with the book nid and the page nid. It also adds the media and connects it to the page

The batch script uses a CSV to ingest the metadata and pages, this allows for the page files to retain their original names (important for preservation so the librarians requested this)…

The basic pagination is handled by the book module, but I am also adding the iana:next, iana:prev, iana:first, iana:last to my rdf mapping to be used in the future with SPARQL queries to create a gallery style banner below the book for jumping pages

The plan is to share my code with the team after the initial release so they can determine what if any they can use to create the “claw” version of the paged content.

I am happy to answer questions and bounce ideas

/Minnie

From: J Hunt [e-mail]
Reply-To: Islandora-CLAW/CLAW [e-mail]
Date: Sunday, February 3, 2019 at 5:44 PM
To: Islandora-CLAW/CLAW [e-mail]
Cc: Minnie Rangel [e-mail], Mention [e-mail]
Subject: Re: [Islandora-CLAW/CLAW] Meta-Issue: Paged Content (#932)

@dannylambhttps://github.com/dannylamb @rangel35https://github.com/rangel35 I'm involved in a project that is looking for this kind of functionality. Has there been any progress? Have decisions been made about how to model page objects and their relations?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/Islandora-CLAW/CLAW/issues/932#issuecomment-460100186, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AD3SW0xKi9vGQQ_nURQ04um3gm9kYQDZks5vJ3RXgaJpZM4WwkKW.

[kayakr]

@seth-shaw-unlv @rangel35 Thanks both for your responses. I'm keen to try the Drupal Book approach first - I think we'd gain some useful UI behaviour for page re-ordering out-of-the-box, even though I imagine book module to be a bit crusty these days.

@rangel35 Are you able to share your RDF mapping config yml for books?

Have any of you looked at decomposing a PDF into individual pages, or compositing a PDF from individual page scans (e.g. TIFF files)? Presumably this would need a new Crayfish micro-service wrapper for ghostscript or similar.

@dannylamb Is there an outline of how to go about building & installing a new PHP-side micro-service?

[dannylamb]

@kayakr There's no big outline per se, though that is a great idea for the documentation sprint. I'll be sure to add that to the list of tasks :smile:

Long story short, your microservice must

• Authenticate using JWT
• Expect the Apix-Ldp-Resource header, which should contain the url of the resource to perform the operation on
• Optionally use a X-Islandora-Args header which lets you pass extra arguments through to the underlying command line utility
• Expose an endpoint for OPTION requests, which will serve rdf describing the service for API-X

If you're looking for something to copy/paste as a starting point, I think Hypercube is the smallest and most straightforward. app.php and the HypercubeController are the most relevant files. Also, a lot of what we do in Crayfish-Commons handles most of the functionality outlined above.

There's more to it, though. You'll have to also set up some drupal actions to publish messages to the queue and wire up a listener, which would be a good test of some recent changes I committed. But it's probably easiest just to start with the microservice and go from there.

@Natkeeran and @whikloj are also pretty qualified on this subject and may have more to add.

[whikloj]

I'd agree with @dannylamb's points and suggest that you worry about making a Silex microservice that does what you want first. Then move on to fitting it in to Drupal environment.

@kayakr I'm not clear what the microservice you are considering would do?

[mjordan]

@dannylamb any preference for a framework/language to write a microservice in? Also, when you say "Authenticate using JWT", does that mean it must require Drupal to authenticate against it using JWT, or the other way around (microservice -> Drupal)?

[whikloj]

@mjordan we are (seemingly) fans of the Silex (now Symfony Flex) system and that is what all of Crayfish is currently using.

If you stick to a PHP framework you could use Crayfish-Commons and @jonathangreen's JWT authentication code therein.

However you can use whatever you like.

[dannylamb]

@mjordan Well, we started out with a software that's now EOL and we have to move everything forward at some point in the future, with Symfony Flex being the most obvious choice. We have no official policy around it, although a ways back we decided to stick to PHP for community code (I was writing things in Java and I don't think anybody really wanted to maintain that).

As for authentication, it's JWT for all the things. The microservice should validate a JWT token that gets passed in, and the microservice should use the supplied token when making requests to other services (like fedora or drupal).

[dannylamb]

ninja'd by @whikloj

I shoulda refreshed....

[seth-shaw-unlv]

@rangel35 I'd love to see the code/configs you have right now since I've been working on the same thing. I would rather follow along with what you are doing then have us both develop something similar but incompatible.

[rangel35]

@seth-shaw-unlv let me clean up my code a bit and create a github repo, currently it is only on UTAustin enterprise github so I can't share that.

it is a combination of a couple of modules so if you have ideas to combine them into one I would be happy to hear them.

@kayakr once I create the github repo I will post to here so that you can see what I have on my mapping file so far...I don't have everything mapped yet...the iana stuff for example I haven't added since that is for a future plan that I hadn't devoted time to it yet.

[rangel35]

Update: I ran into a snag on another issue but I am working on this today, I am off tomorrow but I hope to be able to upload and post in time for the claw call on Wednesday.

[antbrown]

Hi @rangel35, I'm also keen on learning how paged content might work in Islandora 8.

Hoping you get some time to post what you've got so far, or detail the kind of apprach you are taking?

I'll be looking to implement the Internet Archive BookReader for a project we're working on, so this is of great interest to me :)

[rangel35]

sorry for the delay...I've been sick lately but finally got it done and added to github

find it here https://github.com/rangel35/ladi_book_batch

[antbrown]

Hope you're feeling better! I'll check it out over the weekend :)

[rangel35]

I'll be adding some documentation during the weekend, as well as a link to the content type/form I use to maybe make it easier to test...but I think it should be general enough to use with repository item..

[seth-shaw-unlv]

I'm looking forward to seeing the content type you defined @rangel35.

[rangel35]

I'll try to get those uploaded by this weekend...I got sidetracked working on another issue and didn't find the time to do the documentation

[rangel35]

I've uploaded the content type I created...https://github.com/rangel35/cidca_book

I added to the ladi_book readme to describe the necessary table for the batch form...please feel free to ask me question on things that need clarifying, I can flesh out the readme from your questions..

[dannylamb]

Thx @rangel35, can't wait to check this out.

[rangel35]

I've made some changes to the batch module...it now works with islandora_object as it's contenttype (hardcoded, haven't added code to pick contenttype for allowed list)..to make it easier for folks who try testing it so they don't have to install two different modules...I also included a pre-prepared batch with two books as a sample (stole the idea from @mjordan hehehe)

/Minnie

[dannylamb]

^^ When these last three PRs are merged, paged content is a done deal!