Chapel: Dynamic Bounding Boxes for (Inherited) Sparse Domains

[Tshimanga]

I have a blocking issue as follows:

In Numsuch we have the following classes that allow one to create matrices with string indices and well as regular indices,

  class BiMap {
    var keys: domain(string),
        ids:  [keys] int,
        idxkey: domain(int),
        idx: [idxkey] string;

class NamedMatrix {
  var D: domain(2),
      SD = CSRDomain(D),
      X: [SD] real,  // the actual data
      rows: BiMap = new BiMap(),
      cols: BiMap = new BiMap();



md5-5c449d4cf0b5f0329ef984fa09f9de56



  class Graph: NamedMatrix {
    var name: string,
        directed: bool = false,
        bipartite: bool = false,
        verts: BiMap,
        uVerts: BiMap,
        vVerts: BiMap;



md5-6cb6f804909eb53a031479a054cacd87



use LinearAlgebra,
       LinearAlgebra.Sparse,
       Chingon,  // feature/refactor-test branch
       Assert;


       var nv: int = 8,
          D: domain(2) = {1..nv, 1..nv},
          SD: sparse subdomain(D),
          X: [SD] real;

       var vn: [1..0] string;
       vn.push_back("star lord");
       vn.push_back("gamora");
       vn.push_back("groot");
       vn.push_back("drax");
       vn.push_back("rocket");
       vn.push_back("mantis");
       vn.push_back("yondu");

       SD += (1,2); X[1,2] = 1;
       SD += (1,3); X[1,3] = 1;
       SD += (1,4); X[1,4] = 1;
       SD += (2,2); X[2,2] = 1;
       SD += (2,4); X[2,4] = 1;
       SD += (3,4); X[3,4] = 1;
       SD += (4,5); X[4,5] = 1;
       SD += (5,6); X[5,6] = 1;
       SD += (6,7); X[6,7] = 1;
       SD += (6,8); X[6,8] = 1;
       SD += (7,8); X[7,8] = 1;

       var nm = new NamedMatrix(X=X);
       writeln(nm.X.domain);

       var g = new Graph(N=nm);   // THIS FAILS
       writeln(g.X);

As per conversation with @ben-albrecht this seems to be blocked by the bounding box expansion issue in tickets #7544 chapel-lang/chapel#7538 chapel-lang/chapel#7732 by @buddha314 and @bradcray .

Assistance/guidance in how to get this working would be greatly appreciated as this is negatively impacting productivity. Thank you!

[bradcray]

At a glance, this looks akin to issue #8525, but I'll need to take a better look to know. What is the best way to reproduce? Would you be able to create a NumSuch-free standalone test case demonstrating the same error? Thanks.

[Tshimanga]

I think so. I'll try to draft a reproducer really quick @bradcray

[Tshimanga]

Hmm, I don't know about minimal but this is essentially the test I ran with all external dependencies brought "in-house" should be able to just copy/paste/run as is so long as you have LinearAlgebra(.Sparse). Just ran it and got the same error @bradcray

use LinearAlgebra,
    LinearAlgebra.Sparse;


    class BiMap {
      var keys: domain(string),
          ids:  [keys] int,
          idxkey: domain(int),
          idx: [idxkey] string;

      /*
      Create an empty BiMap.
      */
      proc init() {
        super.init();
      }

      proc uni(b: BiMap) {
        this.keys += b.keys;
        this.ids += b.ids;
        this.idxkey += b.idxkey;
        this.idx += b.idx;
        return this;
      }
    }

    class NamedMatrix {
      var D: domain(2),
          SD = CSRDomain(D),
          X: [SD] real,  // the actual data
          rows: BiMap = new BiMap(),
          cols: BiMap = new BiMap();

       proc init() {
         super.init();
       }

       proc init(X) {
         this.D = {X.domain.dim(1), X.domain.dim(2)};
         super.init();
         this.loadX(X);
       }
    }

    proc NamedMatrix.loadX(X:[], shape: 2*int =(-1,-1)) {
      if shape(1) > 0 && shape(2) > 0 {
        this.D = {1..shape(1), 1..shape(2)};
      }
      for (i,j) in X.domain {
        this.SD += (i,j);
        this.X(i,j) = X(i,j);
      }
    }

    class Graph: NamedMatrix {
      var name: string,
          directed: bool = false,
          bipartite: bool = false,
          verts: BiMap = new BiMap(),
          uVerts: BiMap = new BiMap(),
          vVerts: BiMap = new BiMap();


      proc init(N: NamedMatrix) {
        super.init();
        this.verts = super.rows.uni(super.cols);
        this.uVerts = super.rows;
        this.vVerts = super.cols;
        this.loadX(N.X);
          }
        }


    var nv: int = 8,
        D: domain(2) = {1..nv, 1..nv},
        SD: sparse subdomain(D),
        X: [SD] real;

    var vn: [1..0] string;
    vn.push_back("star lord");
    vn.push_back("gamora");
    vn.push_back("groot");
    vn.push_back("drax");
    vn.push_back("rocket");
    vn.push_back("mantis");
    vn.push_back("yondu");

    SD += (1,2); X[1,2] = 1;
    SD += (1,3); X[1,3] = 1;
    SD += (1,4); X[1,4] = 1;
    SD += (2,2); X[2,2] = 1;
    SD += (2,4); X[2,4] = 1;
    SD += (3,4); X[3,4] = 1;
    SD += (4,5); X[4,5] = 1;
    SD += (5,6); X[5,6] = 1;
    SD += (6,7); X[6,7] = 1;
    SD += (6,8); X[6,8] = 1;
    SD += (7,8); X[7,8] = 1;

    var nm = new NamedMatrix(X=X);
    writeln(nm.X.domain);

    var g = new Graph(N=nm); // THIS FAILS
    writeln(g.X.domain);

[bradcray]

Happily, I don't believe the problem here is with issue #7544 (nor with #8525 as I'd feared). Instead, I think this is just user error (or at least has an obvious user-level fix). Specifically, this call:

var g = new Graph(N=nm); // THIS FAILS                                          

calls Graph's initializer:

  proc init(N: NamedMatrix) {
    super.init();
    this.verts = super.rows.uni(super.cols);
    this.uVerts = super.rows;
    this.vVerts = super.cols;
    this.loadX(N.X);
  }

which calls NamedMatrix's 0-argument initializer (in the call to super.init() above):

  proc init() {
    super.init();
  }

but the 0-argument initializer does nothing to establish the D and SD fields, so they're empty domains. Then Graph's initializer calls loadX() which assumes that D is non-empty, so you get the out-of-bounds error that you reported. If, instead, the N.X argument were passed to super.init() within Graph's initializer as follows:

  proc init(N: NamedMatrix) {
    super.init(N.X);
    this.verts = super.rows.uni(super.cols);
    this.uVerts = super.rows;
    this.vVerts = super.cols;
  }

then the 1-argument initializer for NamedMatrix is called:

  proc init(X) {
    this.D = {X.domain.dim(1), X.domain.dim(2)};
    super.init();
    this.loadX(X);
  }

which establishes D explicitly (and SD implicitly) in addition to calling loadX() (so note that I've removed the call to loadX in my rewrite of the Graph.init() call above) such that by the time loadX() is called, D and SD have been defined and are non-empty.

Please let me know if this resolves this issue for you, or at least makes it no longer a gating issue.

[Tshimanga]

Yes! I am so psyched that this is just a user error. @bradcray I could hug you ^_^

[buddha314]

I'm not going to approve the travel expense for that.