Metadatamanagement: Clarification needed for metadata of pdf attachments

[rbirkelbach]

Why does it say in the documentation that metadata of pdfs should be deleted prior to uploading the pdfs? Imho they are very valuable, e.g. for SEO.
Maybe someone from @dzhw/fdz-sowis knows why this is in the documentation?

[UteH]

@rbirkelbach I think we have metadata ("Dokumenteigenschaften") of the pdf in mind. These are often wrong because they are generated automatically, such as

• "HIS" as company (because it is based on the DZHW template)
• a wrong title ("Lorem ipsum ..." because it is based on the DZHW template)
• or the person who started the document as "author"

[rbirkelbach]

Okay, I would propose to change it so that instead of being deleted, data providers should provide correct metadata. Thx for the clarification. I was a bit hasty in closing it.

---

Von: Ute H [e-mail]
Gesendet: Mittwoch, 10. Juli 2019 15:32
An: dzhw/metadatamanagement
Cc: Birkelbach, Robert; Mention
Betreff: Re: [dzhw/metadatamanagement] Clarification needed for metadata of pdf attachments (#2292)

@rbirkelbachhttps://github.com/rbirkelbach I think we have metadata ("Dokumenteigenschaften" of the pdf) in mind. These are often wrong because they are generated automatically, such as

• "HIS" as company (because it is based on the DZHW template)
• a wrong title ("Lorem ipsum ..." because it is based on the DZHW template)
• or the person who started the document as "author"

-
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/dzhw/metadatamanagement/issues/2292?email_source=notifications&email_token=AAIM526YGYC6F75X7WDZN5TP6XQGJA5CNFSM4H7PK4Y2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZTO5ZA#issuecomment-510062308, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAIM52ZNVKCC7MUE5EJADODP6XQGJANCNFSM4H7PK4YQ.

[rbirkelbach]

We discussed it and dataproviders need to provide correct attachment metadata.

[UteH]

ok, that's definitely more burdensome than deleting it. If so we have to spead the information (which attributes?) and write this down in our wiki or something.

[rbirkelbach]

I propose the following:
Title: mandatory
Subject: optional
Author: mandatory
Keywords: mandatory (edit Ute: freestyle?)

[rbirkelbach]

@AndyDaniel1 if I do not hear a veto from you till your holiday I'll send an email to the fdz-team which fields are now mandatory and update the documentation accordingly.

[fbaillet]

@rbirkelbach @UteH @AndyDaniel1 @dzhw/fdz-sowis

I agree with Uti that this would be more time consuming. I suggest you talk this over with the other data curators

[svietgen]

@rbirkelbach If we have to provide correct metadata in the future and author is mandatory, we have to think about the coding schemes - I wouldn't be sure what to write.

[rbirkelbach]

I think @AndyDaniel1 already agreed that it would be good to do it (I don't remember what kind of meeting it was), but I still had to make a proposal list of which metadata fields should be required. It's his decision. If you, @AndyDaniel1 , don't want it after all, just close the issue.

---

Von: fbaillet [e-mail]
Gesendet: Dienstag, 30. Juli 2019 10:48
An: dzhw/metadatamanagement
Cc: Birkelbach, Robert; Mention
Betreff: Re: [dzhw/metadatamanagement] Clarification needed for metadata of pdf attachments (#2292)

@rbirkelbachhttps://github.com/rbirkelbach @UteHhttps://github.com/UteH @AndyDaniel1https://github.com/AndyDaniel1 @dzhw/fdz-sowishttps://github.com/orgs/dzhw/teams/fdz-sowis

I agree with Utihttps://github.com/dzhw/metadatamanagement/issues/2292#issuecomment-510384102 that this would be more time consuming. I suggest you talk this over with the other data curators

-
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/dzhw/metadatamanagement/issues/2292?email_source=notifications&email_token=AAIM527LSF7LULJXDFJRWGDQB757LA5CNFSM4H7PK4Y2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3DIRLQ#issuecomment-516327598, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAIM52ZUCLHFELQ3MGLGAUTQB757LANCNFSM4H7PK4YQ.

[rbirkelbach]

@svietgen It's not yet mandatory. You're right that the coding schemes wouldn't need the author, but the title and keywords should be possible: Title: "Kodierliste für x", Keywords: "coding scheme, Kodierliste, Fächergruppen [...]"

[AndyDaniel1]

@rbirkelbach @UteH @svietgen @fbaillet I think that this topic is important, but we should discuss the efforts it will take us. I don't have a real clue how demanding this will be. We should postpone the decision on this issue until the End of August. In the meantime it would be great if you all can refine the demands and the efforts we have to invest to fullfil the demands.

[rbirkelbach]

I just did it for the gra2005 data&method report. It took me a minute. And it took 9 minutes including installing the software required on GNU/Linux to edit the pdf's metadata. I don't know by heart how difficult it is on windows, but if you're able to delete them it should be possible to go the same route to enter the correct metadata. So if we estimate it very conservatively it should not take longer than 10 minutes max per PDF, unless something is really unclear.

[UteH]

can we assign the keywords freestyle 😇 ?

[rbirkelbach]

In my opinion -- the product owner of course may chime in -- I'd use the tags of the study and if it's a concept attachment I'd use the concept tags plus the document type (e.g. Daten- und Methodenbericht) and maybe synonyms, e.g. for Datensatzreport I'd also add Codebook. I'll have a look at the different kinds of attachments we have and compile a list of what would make sense in my opinion. The reason I'm stressing this is that it would help a lot for a) the search within the mdm and b) when we have prerendering that it would help for finding the relevant documents quickly through google/ecosia/duckduckgo whatever search engine you like to use.

[rbirkelbach]

Method report

Title: Title of the document
Subject: optional
Authors: Mandatory
Keywords: Daten- und Methodenbericht, Method report, study title, study tags

Overviews

Title: Title of the document
Subject: optional
Authors: Mandatory
Keywords: Overview, Studienübersicht, summary, Zusammenfassung, study title, study tags

Questionnaire

• Title: Title of the document
• Subject: optional
• Authors: Mandatory, if there are authors mentioned in the document, use the authors, otherwise use the institution
• Keywords: Fragebogen, Erhebungsinstrument, Instrument, questionnaire, tags of the study

Questionnaire flow diagram

• Title: Filterführungsdiagramm Studytitle
• Subject: optional
• Authors: If known use the authors, otherwise the institution that made it, e.g. fdz.DZHW
• Keywords: flow diagram, Filterführungsdiagramm, studytitle, study tags

Coding lists

• Title: Kodierliste für X
• Subject: optional
• Authors: If known, use the authors, otherwise the institutions that made it, e.g. fdz.DZHW, destatis
• Keywords: coding list, Kodierliste, content description such as "Studienabschlüsse"

Datasetreport

• Title: Title of the document
• Author: fdz.DZHW
• Subject: optional
• Keywords: Datasetreport, Codebook, Variablenreport, variable report, code book, title of the subdataset

Have I missed something?

[rbirkelbach]

@AndyDaniel1 I made a list (see above). If you're happy with it, I'd put it in the documentation and announce it.

[AndyDaniel1]

@rbirkelbach I will give feedback until the end of the next sprint.

[AndyDaniel1]

@rbirkelbach thank you very much for the proposed metadata-scheme!!! If I get it right from the previous posts, you already did some testing on the meta-scheme. After discussing the issue with @UteH I think it is necessary to test the scheme on some more studies. I propose that you use the last six releases for the tests. You can use te help of one SHK.

Regarding the question which language should be used for the metadata I propose the following:
german-only-pdf: use german for the metadata
in any other case: use english

[rreitmann]

@rbirkelbach can we close this?

[rbirkelbach]

@rreitmann no, because I still have to time how long it takes to fill it in for the last 6 releases..

[rbirkelbach]

@dzhw/fdz-shks is anybody available to help me time how long it takes to fill in metadata of the pdfs? Thanks.

[annegaertner]

@liedtke-dzhw could you please take care of this? i hope the task will become clear by reading the comments above. otherwise please contact @rbirkelbach. thx!

[rbirkelbach]

@liedtke-dzhw please use: gra2005; gra2009; gsl2012; ssy21; phd2014; lib2016
edit: please let us also know if you encounter any problems. thx! :)

[rbirkelbach]

@liedtke-dzhw what's the status on this issue?

[liedtke-dzhw]

@rbirkelbach
for one document I need 3-5 minutes. I need 30-40 minutes for each study.
The duration depends on the number of documents.

[rbirkelbach]

Thank you, @liedtke-dzhw

[rreitmann]

@AndyDaniel1 and @rbirkelbach what is blocking the progress of this issue?

[McWallis]

Statement of Mrs. Cordes on the topic: File properties in PDFs published by DZHW.

generell sollten alle PDFs, die im Namen des DZHW erstellt wurden, zumindest die Anforderungen, was die Eigenschaften „Titel“ und „Verfasser“ der PDF-Datei angeht, erfüllen. Die Konventionen zur Benennung der Dateien (Sonderzeichen, Leerzeichen, Kleinschreibung, Datum) beziehen sich auf den ZOPE-Server, ist aber auch jenseits davon sinnvoll. Außerdem müssen die PDFs barrierefrei sein. Ansprechpartnerin dafür ist Frau Nölle, [email protected].

in general, all PDFs created in the name of the DZHW should at least meet the requirements regarding the properties "Title" and "Author" of the PDF file. The conventions for naming the files (special characters, spaces, lower case, date) refer to the ZOPE server, but make sense beyond that. Furthermore, the PDFs must be barrier-free. Contact person for this is Mrs. Nölle, [email protected].

Checkliste_zur_Erstellung_von_PDF.pdf

[AndyDaniel1]

The issue will be discussed fruther in https://github.com/dzhw/FDZ_Allgemein/issues/543