Dvc.org: basic concepts: pipeline stages

[jorgeorpinel]

Other than explaining what a pipeline stage is, this can help us group 2 terms that are used extensively and sometimes confusingly throughout docs: .dvc files (formerly DVC-files) and dvc.yaml — which already have their own guide so maybe this can just use that page instead of a new basic concepts one. Not sure

See more context: https://github.com/iterative/dvc.org/pull/1408#issuecomment-643121300:

Mentioning dvc.yaml and .dvc (everywhere) is too noisy

Automatic linking

Maybe should be a separate issue

• [x] dvc.yaml could be like a command name (done in #1370), which automatically links to its user guide. #1137
• [x] Not sure about .dvc file — I think here we don't want to link automatically. #1137
• [x] dvc.lock ?
• "stage" could have a tooltip and glossary entry. Extracted to #1579
• [x] remove all existing manual links from dvc.yaml, dvc.lock, and .dvc file.
• "orphan stage" we could link manually to dvc add (term will be removed)
• "import stage" to dvc import (term will be removed)

Jump to https://github.com/iterative/dvc.org/issues/1431#issuecomment-658384344 for an update.

[shcheklein]

@rogermparent let's start on this next, when you have time

[jorgeorpinel]

Would be nice to decide on #1395 first too! I guess I should prioritize #550 to facilitate all this...

[rogermparent]

It seems like the most streamlined way to do this is add dvc.yaml as a glossary abbr target, giving the Tooltip a small entry that links to the full page.

For example, the Tooltip content could be:

The primary configuration file of DVC.

Read the full syntax here

I don't know how true that is, but I believe it gets the idea across.

If this is the kind of implementation we want, this change should be pretty simple. I can also use ripgrep to go through all existing content and change all the existing references to dvc.yaml to abbrs.

I can get a PR up soon showcasing this implementation.

[shcheklein]

I don't like the idea of using abbr - we'll have a lot of noise in the Markdown files. Let's link dvc.yaml (and other) inline code blocks instead automatically.

[rogermparent]

That's a reasonable concern, I'll check if we have an existing Remark plugin that has that behavior and make a new one if we don't.

[shcheklein]

@rogermparent yes, we do this already for dvc add kind of links, there should be code around for this.

[rogermparent]

From what I understand, there's three currently proposed entries for the automatic linker

• [x] dvc.yaml could be like a command name, which automatically links to its user guide
• "orphan stage" we could link manually to dvc add
• "import stage" to dvc import?

There's also

• "stage" could have a tooltip and glossary entry. Extracted to #1576

which I can do, but is notably separate from the rest and may need another Issue as Glossary entries need to wait for tooltip content where auto-linker entries have everything they need in the description.

I'll repost this in the PR thread as well, where it'll be easier to keep track of.

[jorgeorpinel]

"orphan stage" we could link manually to dvc add

Don't worry about that one. We got rid of the term.

"import stage" to dvc import

We won't be using that term either in the future, just import .dvc file possibly. So probably also no special work needed there.

"stage" could have a tooltip and glossary entry.

That's the doc-content part of this issue. I guess it should be a separate issue, let's just leave it unchecked for now? And we also need:

remove all existing manual links from dvc.yaml, dvc.lock, and .dvc file.

[jorgeorpinel]

Chnged to p1 since all the doc-engine work here has been done by @rogermparent from what I understand. Thanks Roger!!! BTE, I also added to the issue description (content related):

remove all existing manual links from dvc.yaml, dvc.lock, and .dvc file.

Is dvc.lock also auto-linked now? If not where do we add it? Thanks

[jorgeorpinel]

"stage" could have a tooltip and glossary entry.
which I can do, but is notably separate

Extracted this to #1579 Roger, so that your PR closes this one 🙂