Cargo-raze: Expected behavior for path dependencies

[UebelAndre]

Right now this seems unsupported but I'm wondering what the expected behavior would be for path dependencies when running cargo-raze. Currently the tool fails with

Loaded override settings: RazeSettings {
    workspace_path: "//lib_b/cargo",
    target: "x86_64-unknown-linux-gnu",
    crates: {},
    gen_workspace_prefix: "raze",
    genmode: Remote,
    output_buildfile_suffix: "BUILD.bazel",
    default_gen_buildrs: false,
}
Error: Error during execution of `cargo metadata`: error: failed to get `lib_b` as a dependency of package `lib_b v0.1.0 (/private/var/folders/hf/phjl9q7501gb5wt_qr4ltn3m0000gn/T/cargo_raze_metadata_dir.poOrKhwblTg5)`

Caused by:
  failed to load source for dependency `lib_b`

Caused by:
  Unable to update /private/var/folders/hf/phjl9q7501gb5wt_qr4ltn3m0000gn/T/lib_a

Caused by:
  failed to read `/private/var/folders/hf/phjl9q7501gb5wt_qr4ltn3m0000gn/T/lib_a/Cargo.toml`

Caused by:
  No such file or directory (os error 2)

Any input/feedback would be greatly appreciated 😄

Example

Below is a small example case of what I used to produce the error above
repro.zip

├── lib_a
│   ├── BUILD.bazel
│   ├── Cargo.toml
│   ├── crates.bzl
│   └── remote
│       ├── BUILD.bazel
│       ├── proc-macro2-1.0.19.BUILD.bazel
│       ├── quote-1.0.7.BUILD.bazel
│       ├── syn-1.0.39.BUILD.bazel
│       └── unicode-xid-0.2.1.BUILD.bazel
└── lib_b
    └── Cargo.toml

3 directories, 9 files

lib_b/Cargo.toml

[package]
name = "lib_a"
version = "0.1.0"
authors = ["UebelAndre"]
edition = "2018"

[lib]
path = "fake/cargo_raze_only.rs"

[dependencies]
syn = "1.0.38"
quote = "1.0.7"

[raze]
workspace_path = "//lib_a/cargo"
genmode = "Remote"
output_buildfile_suffix = "BUILD.bazel"

lib_b/Cargo.toml

[package]
name = "lib_b"
version = "0.1.0"
authors = ["UebelAndre"]
edition = "2018"

[lib]
path = "fake/cargo_raze_only.rs"

[dependencies]
syn = "1.0.38"
quote = "1.0.7"
lib_b = { path = "../lib_a" }

[raze]
workspace_path = "//lib_b/cargo"
genmode = "Remote"
output_buildfile_suffix = "BUILD.bazel"

[TheButlah]

In my case, I'm hoping that cargo raze with path dependency support would enable a workflow where developers in an existing multi-lang bazel mono-repo can develop using cargo and the associated cli-niceities that it provides, and then build those files with bazel via cargo-raze to talk to the rest of the monorepo.

[UebelAndre]

@TheButlah Can you explain your setup a bit more? Maybe a tree dump of some top level directories and the location of the BUILD.bazel/Cargo.toml files you have?

[TheButlah]

Sure. We have a bazel monorepo that for the most part is all one main bazel workspace (although there are a few projects that are in their own bazel workspaces). The codebase is predominantly C++ and python. Suppose I want to have my project called "tensor_cache". Here is the monorepo, expanded to just be the areas of interest:

├── BUILD.bazel
├── WORKSPACE
├── control
├── data
│   ├── BUILD.bazel
│   ├── __init__.py
│   ├── h5
│   ├── tensor_cache
│   │   ├── BUILD.bazel
│   │   ├── cargo
│   │   │   ├── Cargo.toml
│   │   │   └── src
│   │   │       └── lib.rs
│   │   ├── raze
│   │   │   ├── BUILD
│   │   │   ├── Cargo.lock
│   │   │   ├── Cargo.toml
│   │   │   ├── crates.bzl
│   │   │   ├── fake_lib.rs
│   │   │   └── remote
│   │   └── tensor_cache_py.rs
│   ├── utils
│   └── ycb
├── datastructures
├── pyproject.toml
└── third_party

I had hoped that I would be able to have tensor_cache/cargo have the same layout as a regular cargo project, with src/lib.rs and the rest of those conventions, and I could work in the tensor_cache/cargo directory and run cargo check, cargo doc, etc. Then I would have a separate raze folder that holds another Cargo.toml file with a path dependency to tensor_cache. That would allow me to have all the niceness that is cargo, but also be able to build my cargo project with bazel for the purposes of getting it checked by CI and to get used by the the rest of the monorepo. I might even have tensor_cache_py.rs which is entirely built with bazel, that has a dependency on the raze-ifieid tensor_cache package.

Note for clarity's sake I named the raze related files under the "raze" folder and the "cargo" folder is instead a regular cargo package layout.

[quasicomputational]

There are at least two cases here:

1. The path points into the workspace. This is the example given above. cargo-raze ought to know to DTRT here without further instruction, because it already knows where all of the packages involved are and can construct a dependency tree.

2. The path points outside the workspace, into another. I think that this will need additional configuration, because cargo-raze won't know what the dependency's label is (which will be determined by how Bazel is configured - e.g., with new_local_repository(name = "foo", path = "../foo") it'd be something like @foo//packages/foo-rs).

[UebelAndre]

The case I'm most interested in solving is the example above (your traditional cargo workspaces). I think if there's a solution for that, then it becomes easier to solve for other examples of path dependencies. If I can get some other PRs through, I'll likely tackle this challenge next 😄

[TheButlah]

Sorry, I'm slightly confused - is the example I gave the "traditional cargo workspace"?

[UebelAndre]

Sorry, I'm slightly confused - is the example I gave the "traditional cargo workspace"?

Yeah, sorry, I was referring to your 1st example in which you also mentioned the example above. I think the 2nd is not directly related to cargo workspaces, it's just another way to define a dependency, right?

[TheButlah]

Yep - to summarize my use case:

I'd like to give a path dependency in cargo raze to another cargo package. I'm agnostic to whether the cargo raze package and the "real" cargo package are in the same cargo workspace (I suspect that they would be since in a monorepo you probably want to share one global cargo.lock file).

[UebelAndre]

One case of path dependencies should be handled as part of #262