Racket: string-titlecase differences in CS/BC

[willghatch]

What version of Racket are you using?
7.8 cs and bc (also previous versions)

What program did you run?

#lang racket/base
(string-titlecase "ꭌaꭌA ᮾaᮾA 1b2c3e åbåb ÅbÅb")

What should have happened?
It should give the same output for Racket BC and CS. It doesn't.

This is perhaps two bugs.

• CS capitalizes the first letter after a number, BC uppercases all
  of them. The same thing happens with “ᮾ”.
• CS capitalizes the first letter after a “ꭌ” but only the first
  one, BC lowercases all of them.

The “small letter script R with ring” is weird because it doesn't
seem to have a capital counterpart, but clearly it's intended as a
lowercase letter. There don't seem to be any other similar “latin
script with ring” characters big or small.

Perhaps CS views it as case-ignorable while BC views it as having
case but where the uppercase transformation is identity?

Sudanese script on the other hand doesn't have casing (I believe).
So it makes sense that Sudanese script is treated like numerals for
casing. But it seems like CS and BC have a different interpretation
of what should happen when there is a mix of non-space case-ignorable
characters.

This bug is brought to you by Xsmith.

If you got an error message, please include it here.

Please include any other relevant details
Multiple Linux distros.

[willghatch]

Oh, I guess I didn't include the bit in my file that mentions that “ꭌ” is “small letter script r with ring” and “ᮾ” is “sudanese letter final k”. U+AB4C and U+1BBE respectively.

[mflatt]

The problem is that BC never implemented the Unicode rules for string-titlecase. It doesn't even pass two tests in the R6RS test suite, as below — a fact that I used to know, but had forgotten.

Expression:
 (string-titlecase "r6rs")
Result:
 "R6Rs"
Expected:
 "R6rs"

Expression:
 (string-titlecase "R6RS")
Result:
 "R6Rs"
Expected:
 "R6rs"

[rfindler]

Does Xsmith have a reducer? Maybe this can be a test case for Xsmith's reducer! :)

[willghatch]

On Tue, Oct 13, 2020 at 03:25:48PM -0700, Robby Findler wrote:

Does Xsmith have a reducer? Maybe this can be a test case for Xsmith's reducer! :)

Not yet, but we would like to build one.

[jackfirth]

@willghatch I built a Racket port of the Minithesis reducer (a simplified version of the core test case reducer in the Hypothesis property testing library). It works scarily well despite me putting very little effort into it. That might be a useful starting point to look at.

[samth]

My understanding from reading their paper is that xsmith is based on a similar random generation approach to hypothesis already, and thus this should hopefully be possible.

[willghatch]

Thanks, I'll have a look at your reducer. Adding a reducer to Xsmith should be relatively straightforward, we just haven't done it yet. However, as I run some more fuzzing and find more bugs I'm sure I'll be increasingly motivated to write one.

[mbutterick]

The “small letter script R with ring” is weird because it doesn't
seem to have a capital counterpart, but clearly it's intended as a
lowercase letter.

It’s part of the Teuthonista phonetic alphabet. Like other characters in that Unicode block, it does not have an uppercase mapping, even though it is classified as lowercase. (Strictly speaking, phonetic alphabets are symbolic and thus uncased. IIRC Unicode treats them as lowercase because they are sometimes officially adopted into written languages.)

Sudanese script

Sundanese (FWIW not Sudanese) script is uncased, so its characters are not classified as lowercase.

More broadly, string-titlecase is, like the top level, hopeless because capitalization is specific to languages, orthography, and context. (See also #1781)