Dvc.org: docs: Major DVC 1.x discrepancies

[jorgeorpinel]

Mission critical

• [x] Get Started! See https://github.com/iterative/dvc.org/issues/599#issuecomment-559233708 - Ivan is on it iterative/example-repos-dev/pull/14,
• [x] Installation
• [x] dvc add — mainly .dvc file examples #1411 + merge #1414
• [x] DVC-file guide(s) and maybe cache guide - included in #1370
• [x] dvc run PR #1420 + merge #1387
• [x] DVC-file guide(s) and maybe cache guide - included in #1370
• [x] document dvc.lock in user guide (see #1422) - #1448
• [x] dvc metrics - #1360
• [x] dvc plots - #1382

Finally

• document run cache? (#1289) #1464 #1655 (Will happen when experiments are more stable)
• [x] BLOCKER: Figure out auto linking for dvc.yaml and .dvc files to avoid manual management of hundreds of instances (see #1431). - #1576
• NEW: Address https://github.com/iterative/dvc.org/issues/1663 to simplify dvc.yaml/lock and .dvc file mentions (extracted)
• [x] Remove term Dvcfile (see #1445)
• 
  
  • Full sync of cmd ref usage strings and option names (see iterative/dvc/issues/4024)
  
  

Next priorities

• [ ] Review all .dvc file examples throughout docs (in YAML code blocks) — note we require YAML 1.2 (se iterative/dvc/pull/4415 and #1764) - #1608,
• [ ] ^ Also push/pull examples
• [ ] Review DVC cache info everywhere, especially in the internal files/dirs guide.
• Replace all instances of term "DVC-file(s)" 😱 as well as "stage file(s)". Use dvc.yaml and/or dvc.yaml as appropriate. (Review stage concept itself, see #1366) - #1372 + being addressed in https://github.com/iterative/dvc.org/pull/2047#issue-543854582 (not so relevant anymore though)
• Update/remove Katacoda courses accordingly (e.g. fix #1586, see also #1400, #670) Not 1.x related
• Update all other command references 😱 (make a list of which ones are pending first...) e.g. #1456, #1701 Extracted to #1824
• [x] Review all dvc add examples throughout docs. Avoid having to include the command output unless it's important given the context, definitely don't include progress bars - #1605, #1608
• [x] Let's create a list of places where the concept of stage file(s) and dvc run/checkout/repro are used - #1743:
  We should review all those docs to make sure the use of dvc run/etc and YAML examples are updated to 1.0
• [x] Update dvc repro - #1572
• [x] dvc pipeline -> dvc dag #1383 Need to update pipeline links, for now they just redirect to dag. - #1584
• [x] Pending target argument updates (see #1574)
• Review all dvc run examples throughout docs Extracted to #1824
• [x] Double check all the bullets in 1.0.0 pre-releases/candidates in https://github.com/iterative/dvc/releases [a0, a1, etc.] (at least up to a7)

[kurianbenoy]

@jorgeorpinel can I work on helping with small part with this issue or can you point to some other issues for the moment?

[jorgeorpinel]

Hi Kurian! I understand you're interested in contributing in the context of GSoD right?

Yes, you can probably help here. Are you familiar with dvc plot on v0.94 yet? That's the only item from the description I'm not already working on. It's a little complicated though but why don't you give it a shot? Learn the current version well first and then see what changed in that particular group of commands from the release logs as explained in the description, to see how the docs should change (for starters the name will be updated from plot -> plots).

[kurianbenoy]

I will explore first by using dvc plot command first, and understand how to use it. After that, I can start working in the docs. I had seen this tweet from Pared showing this demo of using dvc plot command

[jorgeorpinel]

I had seen this tweet from Pared showing this demo of using dvc plot command

Good find @kurianbenoy! Yes good plan but please if you find this too complicated to do within this week actually maybe it's best to work on something else instead because this particular epic issue right now has some immediate deadlines.

Notice I just added 2 checkboxes to the description, so consider working on the (currently) 2nd-to-last:

Remove all instances of term "DVC-file(s)" when referring to stage files...
(and create list for the last checkbox)

Or even #1366 (related to that 2nd-to-last subtask as well).

[kurianbenoy]

@jorgeorpinel for the plots command, what all should I check for? Will changing the usage and optional arguments of each command ie plot - show and diff necessary? I will review the changes in plots command in the new release and see the changes.

I can try to complete this task by Friday. If I can't complete it, then I will work on the new issues you have just added. Thanks again.

[jorgeorpinel]

Based on your question I don't think it's a good idea that you work on plots directly @kurianbenoy. Please do the alternative I suggested instead:

Notice I just added 2 checkboxes to the description, so consider working on the (currently) 2nd-to-last:

Remove all instances of term "DVC-file(s)" when referring to stage files...
(and create list for the last checkbox)

Or even #1366 (related to that 2nd-to-last subtask as well).

[jorgeorpinel]

@kurianbenoy here's another idea of something very specific you could help with if you wish to provide great value during this process: https://github.com/iterative/dvc.org/pull/1360#pullrequestreview-420734838 🙂 It would require that you are familiar with dvc metrics in DVC 1.0 though. Again, whatever is easier/quicker fro you from all the options above. Thanks

[pared]

@jorgeorpinel do you keep track of which .dvc examples are being done throughout the code? if there is some list, maybe we could help?

[jorgeorpinel]

Not keeping track of that at the moment @pared but they're all up for grabs! Thanks 🙂

[jorgeorpinel]

document run cache! - draft by @efiop?

Hi Ruslan, any chance the engineering team will be able to submit a draft for this soon? Thanks

[efiop]

@jorgeorpinel Run cache doesn't have much sense on itsown. We will likely describe it when experiments are ready. You could remove that checkbox from this ticket and reopen https://github.com/iterative/dvc.org/issues/1289 , since it is not going to happen very soon.

[jorgeorpinel]

Right, we don't need a stand-alone document for it but it should be mentioned at least in a few places like run and repro. It's definitely a major discrepancy between the released DVC version and the published docs though. Let's continue the conversation in #1289 (I added it to the checkbox and removed your mention for now). Thanks

[jorgeorpinel]

Update all other command references 😱 (make a list of which ones are pending first...)

@imhardikj can you work on that list please? You can extract this one to it's own epic issue with check boxes for pending commands. add, run, metrics? (see #1175), and plots? (see #1800) were taken care of per this issue. We also recently updated move, and I did some others in recent weeks but can't remember which ones. Also look in the existing issues please, as some of them have a ticket e.g. version (#1635) which is an easy one you can take care of soon.

A high priority one would be status for example, and BTW there's code samples of this command in many other places (same for other commands) so that's something we also need to check for updates, as some of those examples may be outdated — could be a separate epic though...

2 issues I found that could probably be merged into the new epic you'll create: #404, #1574 (but there's probably more). Thanks

[jorgeorpinel]

I've reviewed all the pending items and checked most of it. Only a few are left which were extracted to other issues/PRs (mainly #2026) so I'm closing this 🎉

Many thanks to everyone who helped here, especially @imhardikj !