Chapel: Declaring array of c_string literals results in string array

[ben-albrecht]

Summary of Problem

Declaring array of c_string literals results in string array instead of a c_string array.

I suspected that some c_string -> string coercion was happening, but neither of these casts are getting called when running the program:

// String.chpl
inline proc _cast(type t, cs: c_string) where t == bufferType { ... }

proc _cast(type t, cs: c_string) where t == string { ... }

Steps to Reproduce

Source Code:

// I want an array of c_string's but I get an array of strings
var arr = ['a'.c_str(), 'b'.c_str(), 'c'.c_str()];
writeln(arr[1].type:string);
writeln(arr[1].c_str().type:string);
writeln(arr.type:string);

Execution command:

> ./cStrArray
string
c_string
[domain(1,int(64),false)] string

Configuration Information

• Output of chpl --version: chpl version 1.21.0 pre-release (a5c89b0c85)

[bradcray]

Given that we're hoping to deprecate c_string completely, can you provide more rationale for why you want to create a c_string array in Chapel? (I.e., I'd be inclined to make this a "won't fix" since it requires putting effort in the opposite direction from where I think we want to go).

[e-kayrakli]

Reference: c_string deprecation issue

[ben-albrecht]

Given that we're hoping to deprecate c_string completely, can you provide more rationale for why you want to create a c_string array in Chapel?

I wanted to return an array of hard-coded strings from an exported Chapel function for Python consumption. Could either of you point me to the preferred way to write this without c_string?

I'd be inclined to make this a "won't fix" since it requires putting effort in the opposite direction from where I think we want to go

That seems reasonable. I will mark as "won't fix" and close this issue once I establish/document the preferred work-around for this use case.

[bradcray]

What do you mean by hard-coded?

I'd think what we'd want to do here is to extend Chapel-Python interop for strings and arrays to work together such that you could simply return a Chapel 1D array of strings and have it show up in Python as a 1D (NumPy?) array of strings. Would that accomplish what you're looking for?

Tagging @lydia-duncan and @dlongnecke-cray on this since they've worked on related tasks.

[lydia-duncan]

Given that strings are supported (albeit as "bytes" on the Python side), I would expect arrays of strings to have fallen out, though there may be some additional transformations that are required as a result of their intersection. If that's the case, ideally we'll fix it in such a way that we don't have the same problem later with arrays of more complicated types (like classes, as an example)

[ben-albrecht]

b/c @e-kayrakli asked, here's a few other variants and their results:

Using actual c_string literals:

var arr = [c'a', c'b', c'c'];
writeln(arr[1].type:string);
writeln(arr[1].c_str().type:string);
writeln(arr.type:string);

> ./cStrArray
string
c_string
[domain(1,int(64),false)] string

[ben-albrecht]

I would expect arrays of strings to have fallen out, though there may be some additional transformations that are required as a result of their intersection. If that's the case, ideally we'll fix it in such a way that we don't have the same problem later with arrays of more complicated types (like classes, as an example)

Here is a quick test with arrays of strings:

// This works:
export proc f() {
  return 'foo';
}

// This does not compile (see error below)
export proc g(): [] string {
  var A: [0..1] string = ['foo', 'bar'];
  return A;
}

chpl retString.chpl --library-python
Error compiling Cython file:
------------------------------------------------------------
...
        ret = chpl_f()
        return ret

def g():
        cdef chpl_external_array ret_arr = chpl_g()
        cdef numpy.ndarray [string, ndim=1] ret = numpy.zeros(shape = ret_arr.num_elts, dtype = string)
                          ^
------------------------------------------------------------

retString.pyx:41:27: Invalid type.
Traceback (most recent call last):
  File "retString.py", line 12, in <module>
    libraries=["retString"] + chpl_libraries + ["retString"])))
  File "/Users/balbrecht/.local/lib/python3.7/site-packages/Cython/Build/Dependencies.py", line 1096, in cythonize
    cythonize_one(*args)
  File "/Users/balbrecht/.local/lib/python3.7/site-packages/Cython/Build/Dependencies.py", line 1219, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: retString.pyx

I have opened an issue here: https://github.com/chapel-lang/chapel/issues/14500

[e-kayrakli]

This one actually works:

var arr: [1..3] c_string = [c'a', c'b', c'c'];
writeln(arr[1].type:string);
//writeln(arr[1].c_str().type:string);
writeln(arr.type:string);

The error is the 3rd line calling c_str on a c_string. This, of course, has the drawback of keeping track of the array size..

[ben-albrecht]

While @e-kayrakli found a work-around for the issue posted here, I guess I really care about returning arrays of strings from Chapel to python. I'll open a separate issue for that (#14500). Thanks everyone!