Hugo: {{ merge }} .Params dictionary parameters coverts all existing keys forcefully to lowercase

[hollowaykeanho]

What version of Hugo are you using (hugo version)?

$ hugo version
Hugo Static Site Generator v0.74.1-15163266/extended linux/amd64 BuildDate: 2020-07-13T19:02:45Z

Does this issue reproduce with the latest release?

Latest is v0.74. Yes.

Description

Given a data processing with the following key where the key is datePublished:

{{- $ret = merge $ret (dict "datePublished" (index $dataList "PageDate")) -}}

and the rendering code is as follows:

{{- $notFirst := "" -}}
{{- range $label, $value := .ldJSON -}}
       {{- if $notFirst -}}, {{ end -}}
       "{{- $label -}}": {{ $value -}}
       {{- $notFirst = "true" -}}
{{- end -}}

The generated `datePublished output got converted to full lowercase instead of as it is, as follows:

"datepublished": "2019-03-10T13:16:02+08:00"

The expected output should be:

"datePublished": "2019-03-10T13:16:02+08:00"

Deployed Location

This is used in <application/ld+json> partial layout rendering.

[hollowaykeanho]

Added above: jsonify function is not usable as it turns the entire dict data structure into a single string instead.

[moorereason]

Are you sure it's dict that is the culprit? You're example also uses merge. Please simplify your examples and confirm.

[hollowaykeanho]

Are you sure it's dict that is the culprit? You're example also uses merge. Please simplify your examples and confirm.

My apology for the previous identification. I carefully narrowed down the root cause analysis. Here is the full algorithm (with the problem):

{{/* ==ld+JSON=========== */}}
{{- $ret := dict "@context" "https://schema.org/" -}}
{{- $ret = merge $ret (dict "@type" "WebPage") -}}
{{- $ret = merge $ret (dict "name" (index $dataList "Title")) -}}
{{- $ret = merge $ret (dict "author" (index $dataList "PageCreators")) -}}
{{- $ret = merge $ret (dict "image" (index $dataList "PageThumbnailURL")) -}}
{{- $ret = merge $ret (dict "description" (index $dataList "PageDescription")) -}}
{{- $ret = merge $ret (dict "datePublished" (index $dataList "PageDate")) -}}
{{- if .Site.Params.schema -}}
        {{- $ret = merge $ret .Site.Params.schema -}}
{{- end -}}
{{- if .Params.schema -}}
        {{- $ret = merge $ret .Params.schema -}}
{{- end -}}
{{- $dataList = merge $dataList (dict "ldJSON" $ret) -}}

You are right, dict is not the culprit. It's the TOML dictionary merge that causes the problem. If I remove the following:

{{- if .Site.Params.schema -}}
        {{- $ret = merge $ret .Site.Params.schema -}}
{{- end -}}
{{- if .Params.schema -}}
        {{- $ret = merge $ret .Params.schema -}}
{{- end -}}

Then datePublished is then shown as expected. From the testing, both .Site.Params.schema and .Params.schema can independently alters the key.

The config/_default/params.toml content for .Site.Params.schema is as follow:

# schema.org fields
[schema]
"@type" = "WebPage"

For the specific page front-matter content for .Params.schema is as follow:

[schema]
"@type" = "SoftwareApplication"
"name" = "Bissetii Hugo Theme and Go Template Module"
"operatingSystem" = [ "Windows", "Debian", "OSX" ]
"applicationCategory" = [ "Hugo Theme Module", "Go Template Module" ]

[schema.offers]
"@type" = "Offer"
"price" = "0"
"priceCurrency" = "USD"

I'm re-titling the issue with the new findings. Please advise for next step.

[moorereason]

Ok, one more step. Are you sure it's merge? :smile: Strip as much complexity out of your example as possible. We don't need to see tons of template code and TOML data. Less is more here. For example, does {{ printf "%#v" .Site.Params.schema }} show lowercase keys? You're sure that the keys are correct prior to calling merge?

[hollowaykeanho]

Yeap. Affirmative with reproducible results. The prime suspects are:

{{- if .Site.Params.schema -}}
        {{- $ret = merge $ret .Site.Params.schema -}}
{{- end -}}
{{- if .Params.schema -}}
        {{- $ret = merge $ret .Params.schema -}}
{{- end -}}

From this code snippet, I highly doubt if is the culprit so only sensible suspect left is merge with TOML dictionary:

{{- $ret = merge $ret .Params.schema -}}
{{- $ret = merge $ret .Site.Params.schema -}}

I can't reduce the condition wrapping yet because most of the publications are yet to be ported with their specific schema and my hands are tight due to the late release schedule for creating a dedicated test sample for you guys. =(

When the TOML merges are removed, the output is correct:

When either or both of them show up, the output then goes into lowercase:

Site-only:

Page-only:

both site and page:

[bep]

@hollowaykeanho all the .Params maps are _lower case keys_ (put your data inside /data or a bundle or /assets if that is a problem), so you really need to provide an example. I just checked, and we do have tests for this.

[hollowaykeanho]

all the .Params maps are lower case keys (put your data inside /data or a bundle or /assets if that is a problem)

Noted. This is the problem when merging from .Params and it's aligned with the finding.

Question: can the limitation be lifted?

Per my understanding ld+json is one of the metadata related to content rather than the raw data serving content itself (e.g. numerical data for pie chart). Although /data and /assets can workaround it, it does not make sense from content writing point-of-view. Both of the Params are best served via front-matter and params.toml since they are indeed configurations.

Alternatively, can I create an independent config.toml file such as schema.toml for direct parsing without the lower case key conversion?

so you really need to provide an example

Don't quite get you. Is the following you're looking for (trimmed down version)?

{{- $ret := dict "@context" "https://schema.org/" -}}
{{- $ret = merge $ret (dict "description" "Description Value" -}}
{{- $ret = merge $ret (dict "datePublished" "Date") -}}
{{- if .Site.Params.schema -}}
        {{- $ret = merge $ret .Site.Params.schema -}}
{{- end -}}
{{- if .Params.schema -}}
        {{- $ret = merge $ret .Params.schema -}}
{{- end -}}

<html>
<body>
{{- $notFirst := "" -}}
{{- range $label, $value := $ret -}}
       {{- if $notFirst -}}, {{ end -}}
       "{{- $label -}}": {{ $value -}}
       {{- $notFirst = "true" -}}
{{- end -}}
</body>
</html>

Then observe datePublished like I did.

If you're querying about the ld+json deployment: here are a bunch of approaches:

1. https://developers.google.com/search/docs/guides/search-gallery
2. https://developers.google.com/search/docs/data-types/software-app

[hollowaykeanho]

After some experimentation and considering that Schema.org is very dynamic in terms of data and its data structure, using data directory makes more sense compared to using .Params.

Hence, there is no point relying on params.toml to facilitate ld+json data as it can quickly bloats the file to an unmaintainable state.

As for the merge .Params dictionary issue specified by @bep, it is expected to force all existing keys into lowercase. As an alternative, use data directory instead.

Closing this issue.