V3: Evolving the specification of this repo

[iHiD]

Hello @exercism/track-maintainers :wave:

As we're getting closer to launching the beta of the tracks part of v3, we've been working on unifying the UI designs and UX flows, with the data that we have within the v3 repo, and considering more about how we merge in the data from the v2 language repos.

It's become clear some areas need to evolve slightly. This issue outlines the decisions we've made in line with this, and the changes required to the v3 repo as a result.

Concepts

So far within v3, we've talked about Concepts being the link between Concept Exercises, and not treated them as their own components. It's become clear as we've designed the UI, that Concepts should very much be first-class citizens of v3, with their own pages, etc. It's also become clear that the content that lives within the current after.md document really be content that belongs to the Concepts, not the exercises.

To accomodate this we'll be making the following changes.

Adding Concepts to config.json

The config.json will have the array of all track concepts added to it as follows:

concepts: [
  {
    "uuid": "...", # A permanent uuid, which allows for strings to change, and correlates this data to the database
    "slug": "strings", # a slug used within config and URLs, in kebab-case
    "name": "Strings", # The name of the concept as it will appear around the site
    "blurb": "A series of unicode characters held together with magic glue in a floating array" # One sentence that explains the concept within the language
  }
]

Adding a concepts directory

Each track will have a concepts directory (e.g. https://github.com/exercism/v3/tree/master/languages/csharp/concepts).

Each concept will have a folder in that directory, named with its slug (e.g. https://github.com/exercism/v3/tree/master/languages/csharp/concepts/strings).

Within that folder there will be two files:

• about.md: A file containing comprehensive information about this concept.
• resources.md: A file containing any external resources or reading on the subject

The about.md replaces an exercise's after.md.

Changes to Concept Exercises

We will be making several changes to Concept Exercises to help this migration:

Removing after.md files

As the after.md files are becoming part of the concepts, we will remove them from exercises.

Splitting up introductions

Introductions should be subheadinged with the slug of the concepts using ##

e.g.

## strings

Some intro to strings

## booleans

Some intro to booleans

This information will be used in various places and will be pre-parsed before shown to a user.

It remains important that the introduction concept shows only the minimum necessary information about a concept to allow a student to complete the exercise.

Changing the exercise's names and slugs to be based on the stories

Initially we named exercises based on their concepts. We've now realised this was a bad decision. The exercises should be named (both with slugs and titles) to match the story, not the concept taught.

Rename the concepts key to teaches

To help clarify it's purpose, we will be renaming the concepts key in each practice exercise to teaches.

Practice Exercises

We're going to start merging the practice exercises' configs into the v3 repo. For now we will not merge the exercise content, just the config.jsons.

These will live within "exercises": {"practice": [...] }}

• Each practice exercise should be copied into the config.json.
• A prerequisites key should be added to each exercise.
• Remove the following keys:
  
  
  
  • core
  
  
  • auto_approve
  
  
  • unlocked_by
  
  
• We will also be removing the topics key before launch, but will leave it in place for now to give maintainers a basis for their prerequisites list.

Migrating

• We will script adding the concepts into the config.json based on existing concepts and prerequisites
• We will script the change from concepts to teaches in the config.json
• We will explode the after.md files into about.md files
• We will ensure the correct headings are added to the introduction.md
• We will add a CI checks that headings are correct in introduction.md
• We will add a CI checks that the concepts in the exercise configs are present in the concepts array, and that the directories and about.md are present.
• We will script-migrate all your practice exercise configs.

Once this is all done:

• You will need to add the prerequisites key to each practice exercise.

---

We'll be doing all this next week, so you have this week to tell us any bits that we've got wrong :wink: We'll also aim to put an example PR together for one exercise this week just so people have clarity on that.

[angelikatyborska]

Some questions:

1. Given that resources.md will exist for a concept, can about.md (migrated from a concept exercise's after.md) also be full of links?
2. The idea for after.md was that it will be shown to the student right after they solve the concept exercise. Is this still going to happen, but for all the concept's about.mds?

[iHiD]

Given that resources.md will exist for a concept, can about.md (migrated from a concept exercise's after.md) also be full of links?

It can have links integrated, yes, but if we want to just provide some "more reading" list that will fit into resource. I suggest waiting until we have a design for you to see before starting changing the about.md or the resources.md, just so we have proper clarity.

The idea for after.md was that it will be shown to the student right after they solve the concept exercise. Is this still going to happen, but for all the concept's about.mds?

Yeah. So when they complete an exercise we'll highlight the concepts they've "learnt" and link to those. Then on the Concepts page (which will be a primary page on the Track and show the concepts in a dependency graph) there will be a notification that there is more for the more read there.

[SaschaMann]

Removing after.md files

As the after.md files are becoming part of the concepts, we will remove them from exercises.

I don't like this at all. after.md is a great way to show students commented example solutions and other info that has nothing to do with the concept(s), but the exercise in question. As @cmcaine often points out, reading annotated expert solutions is a great way to improve learning as well. In v2 all (?) Julia mentoring notes already include such a section that is copy-pastable and it would be nice if that was properly embedded in the website, rather than tacked on via an analyzer or w/e.

about.md: A file containing comprehensive information about this concept.

That seems quite redundant if the external resource is already comprehensive. I feel that in many cases this would result in a pupil-style exercise where you try to rephrase your source material just enough that it counts as your own work, or you'd copy the source 1:1 (if it has an appropriate licence) in which case you might as well just link to it so you don't risk going out of sync.

Splitting up introductions

I feel like this might be easier to maintain if it was just split into one file per concept-intro and then stiched together afterwards but I'm not sure.

We've now realised this was a bad decision.

Why?

[iHiD]

I don't like this at all. after.md is a great way to show student's commented example solutions and other info that has nothing to do with the concept, but the exercise in question

That's specifically not the purpose of the after.md though. The after.md is about the concept, not the exercise. To quote the existing docs:
Purpose: Provide information about the concept(s) for a student to learn from.

As @cmcaine often points out, reading annotated expert solutions is a great way to improve learning as well. In v2 all (?) Julia mentoring notes already include such a section that is copy-pastable and it would be nice if that was properly embedded in the website, rather than tacked on via an analyzer or w/e.

This is "Approaches". This is a whole different thing to the after.md. There will be a whole area on the exercise that shows different ways of approaching them, linking to concepts, other solutions, etc, and that can be written by maintainers etc. That's being handled entirely differently and wouldn't fit in the after.md even under the existing spec.

I feel like this might be easier to maintain if it was just split into one file per concept-intro and then stiched together afterwards but I'm not sure.

Yeah, I considered that as well, and wouldn't be against it. The reason I decided to go for the headings approach is that I felt that personally I'd find it easier to write the introduction.md if I could see how it pieced together as one thing, than trying to hold the different bits in my head. I think as long as we have CI for it, it doesn't hugely matter, but if there was big consensus for the multiple files, I'd be ok with that.

That seems quite redundant if the external resource is already comprehensive. I feel that in many cases this would result in a pupil-style exercise where you try to rephrase your source material just enough that it counts as your own work, or you'd copy the source 1:1 (if it has an appropriate licence) in which case you might as well just link to it so you don't risk going out of sync.

Maybe so. But certainly for Ruby, I think most Concepts don't have great single pages out there that explain just that concept. Normally they're intertwined with loads of other information, embedded in other docs, etc.

I feel that having all the content written in a consistent voice and style has lots of value.

[SaschaMann]

That's specifically not the purpose of the after.md though. The after.md is about the concept, not the exercise. To quote the existing docs:
Purpose: Provide information about the concept(s) for a student to learn from.

If you quote the whole thing:

Once the student completes the exercise they will be shown this file, which should provide them with a summary of what the exercise aimed to teach. If the exercise introduced new syntax, syntax samples should be included. At a minimum, this file should contain all information that is introduced in the .docs/introduction.md document.

This document can also link to any additional resources that might be interesting to the student in the context of the exercise, such as:

Popular usages for a feature
Common pitfalls in a feature's use (e.g. casual use of multiple threads)
Limitations on use that may catch out the unsuspecting developer
Alternative approaches addressed in other exercises
Compromises made for ease of learning or to accommodate the Exercism environment, e.g. multiple classes in single file
Similar features with which the concept may be confused
Performance characteristics and memory usage

Several of those don't fit into a concept doc.

This is "Approaches". This is a whole different thing to the after.md. There will be a whole area on the exercise that shows different ways of approaching them, linking to concepts, other solutions, etc, and that can be written by maintainers etc. That's being handled entirely differently and wouldn't fit in the after.md even under the existing spec.

Then I completely misunderstood what Approaches will be. I understood them basically as "community-mentored student solutions" rather than annotated solutions specifically created for teaching. Will have to rewatch the video I guess :D

I feel that having all the content written in a consistent voice and style has lots of value.

I agree, hence most concepts in Julia would likely just link to the relevant section in the Manual :)

[iHiD]

Several of those don't fit into a concept doc.

Maybe. But then those bits can go into the approaches stuff when that's ready.

I agree, hence most concepts in Julia would likely just link to the relevant section in the Manual :)

In that case, you'd just copy the introduction back out, as the spec says that "at a minimum, this file should contain all information that is introduced in the .docs/introduction.md document." 🙂

[BethanyG]

Maybe so. But certainly for Ruby, I think most Concepts don't have great single pages out there that explain just that concept. Normally they're intertwined with loads of other information, embedded in other docs, etc.

I think I am struggling with the opposite of this in Python. The docs are .... _copious_, and have been written at specific levels, and organized in different ways depending on how a programmer wants to interact with them:

• The Python Tutorial organizes much as we've said we want for track concept exercises -- in fact, it is very difficult for me to not just cut and past from this.
• The Standard Library organizes around types, classes, and methods.
• The Language Reference is very formal, and approaches things from a language design perspective.

The above three are backed up by a glossary.

So for me, I struggle to know where I might be adding any value at all, beyond giving a student a more "practical" or "focused" exercise to solidify a concept or type or method. That's not to say it is not worth doing -- just to say that it is hard for me to know what information belongs where -- and when to talk about things in "my own" voice -- and when to simply guide the student to the places where someone says it better than I ever can.

[neenjaw]

@BethanyG

Agreed it's hard to not just copy and paste sometimes. From an outsider perspective to python, the one thing that is always quite apparent to me that information is almost always out of date as soon as its written down. So there are a plethora of old blogs/posts/github code which are completely non-idiomatic (or worse, broken). So even an about that helps a person to organize the concept around current information would be useful.

There is so much noise when googling most programming languages that finding good documentation/blogs/articles/resources is really tough.

The curse is that as soon as we write it down, it is, also, out of date 🙂

[iHiD]

I think part of this is probably a lack of guidance on what Exercism is trying to achieve with this generally (as in Concepts both in this context, but also in the wider Concept Exercise situation). That's primarily because it's not easy to write down in a way that people coming from different programming language backgrounds (with very different experiences of documentation etc) all find sensical, and because I've never had to write it down, so I've not necessarily found the right words to do it yet 🙂

I'll try to expound my thinking more.

---

We're not trying to be is documentation. We're not trying to be comprehensive about the what - the details of every function call or technique . We're caring more about the why and focussing more on the differences to what a programmer might already know. We're trying to give people a mental scaffolding of how to piece the parts of a programming language together in the way that an experienced developer of that language would see them.

We want people to know how to reason about a Concept in an idiomatic way, and when using that concept is appropriate. Most concepts are used quite differently in differently languages - I want people to know the differences between how a Julia developer thinks about Strings to how a Ruby one thinks about them. That's the niche that we fill.

For example, in Ruby, one of the most useful things is the plethora of helper methods that do absolutely everything (like calling .first, .second on enumerables, or things like .upcase or .swapcase on Strings. I wouldn't want a concept on upcase or swapcase, or first, second, etc. I see Exercism's job as explaining that these sorts of methods exist and that as a Ruby developer, you should always be reaching to those methods. As part of that I'd be pointing to the docs etc.

So for a document on strings, I'd be saying things like:

- Strings can be created in many ways but normally just by using `""` or `''`. The main difference is whether they support interpolation or not [Link to docs to show the various ways to create strings].
- Strings have lots of helper methods for comparing, enumerating, and manipulating strings. For example `.ends_with?`, `.each_char` and `switch_case`. Remember to always check for helpful methods before writing your own. [Strings docs are here]
- Strings can be mutable or immutable depending on the version of Ruby you use and the settings you choose. Check your version here [docs] to find out. If you want to make a string immutable you can always freeze it using `.freeze`

That fleshed out in a friendly tone would be extremely valuable to most people learning Ruby, and doesn't exist anywhere really on the web (certainly no-where that's across languages). My aim would be to give people the key scaffolding around a concept, and to give them directions to useful places to write things. None of that information above has really changed in the 10 years I'm been programming in Ruby so it shouldn't go out of date quickly. And it doesn't massively replicate other things that are out there, it compliments them.

Some language might have good resources for these things (e.g. Python Tutorial), and we shouldn't afraid to copy+paste those key bits (licence permitting) and then let them get fleshed out or improved over time, while also pointing back to those more in depth resources (e.g. Python Tutorial) and saying "look here for a more in depth great version of this".

---

If y'all can help find a way to explain this that's clear and makes sense to everyone, then that would be appreciated 🙂

[ErikSchierboom]

I don't like this at all. after.md is a great way to show students commented example solutions and other info that has nothing to do with the concept(s), but the exercise in question. As @cmcaine often points out, reading annotated expert solutions is a great way to improve learning as well. In v2 all (?) Julia mentoring notes already include such a section that is copy-pastable and it would be nice if that was properly embedded in the website, rather than tacked on via an analyzer or w/e.

We'll have approaches to do this. Also note that concept exercises should be designed to preferrably have one idiomatic solution, which means that there are likely less different approaches to show (taking into account what the student knows at that point).

To me, where the after.md document really shines is in giving _context_ to the student. When should the concept be used, but also: when not? Are there conventions in using the concepts? In other words: what are the dos/donts? What we can do with the after.md documents is something that I find the IT world in general is severely lacking: proper, real-world tips on how to use a concept in a language. To me this lack of resources that shows how to use something in the real-world and what its trade-offs are is where we can help.

There is so much noise when googling most programming languages that finding good documentation/blogs/articles/resources is really tough.

☝️ I've seen this over and over again, where some basic concepts are don't have any standalone, focused documentation but instead are either incomplete, overly complicated or mixed in with other concepts. This is an area where I think we _can_ make a real difference. Some languages are even completely missing documentation for certain concepts.

[cmcaine]

I don't really understand the bit about concepts under headings in introduction files, but other than that this sounds okay to me 👍

Eager to hear more about how "Approaches" will work and possibly feed back on that design :)

Finally, I'm not sure there will be a single idiomatic solution to many of the Julia concept exercises: we can't rely on the students only knowing what we've taught them, so students are going to be coming to us with functional or procedural or vectorised approaches based on their prior experience.

Some concept exercises will be simple enough to only admit one sensible answer, others won't be.

[TheLostLambda]

While I understand the issues that @SaschaMann has with this proposal, I think that this affects the different tracks in very different ways. To that end, I suppose I'll be explaining my support of these changes.

Personally, I've _always_ had significant gripes with the after.md. Ultimately, I feel that it was intended to do far too many things. As a reference, it's not great because many exercises (particularly the early ones) involve several, often only loosely related concepts. In the current system, if I wanted to know more about numbers in Common Lisp, for example, I would see that there isn't an exercise for the concept, but it's actually grouped in with arithmetic's after.md. Currently, however, discovering that relies on viewing exercise metadata or manually checking every existing exercise.

As a recap of the exercise, I've always taken issue with the near copy-pasting from the introduction.md because the document needed to stand alone. I much prefer the proposed solution that moves concepts to their own home and leaves the introduction.md unduplicated.

Finally, as a syntax reference, you hit the same issue as with the concepts (different syntax is mashed together depending on the exercise, not any more logical organisation). The "Approaches" seems like a good way to replace this often quite dry section.

Kinda to @iHiD and @BethanyG , I agree that Exercism can fill a very unique niche that scaffolds learning via well-defined and connected network of concepts. I don't think we should be writing documentation, but I think that Exercism is the perfect place for actually maintaining an up to date index of idioms and how they all fit together. Common Lisp, for example, has an incredible online specification that's also incredibly intimidating; with such a niche language, tutorials are few and far between. I think our roll at Exercism should be building the scaffolding that students need to actually navigate the documentation and books available on their own. Idioms and perspective (between languages) should be the focus of our concepts.

Ultimately, I'm incredibly pleased with these suggested changes and am looking forward to refining things as we go along.

PS: @iHiD , while we are at it, did we ever update the test-runner spec? We got several working prototypes of test-runners generating test metadata from both C# and Common Lisp. :wink:

[SleeplessByte]

Then I completely misunderstood what Approaches will be. I understood them basically as "community-mentored student solutions" rather than annotated solutions specifically created for teaching. Will have to rewatch the video I guess :D

This is the key thing @iHiD should re-clarify, because I think that, if done right (and approaches != implementations), this is basically what after.md is WITHOUT the concept info.

[SleeplessByte]

(Trying to edit my previous post, but it's showing "Sorry something went wrong")

I don't really understand the bit about concepts under headings in introduction files, [...]

Same. I actually hate that part.

The rest is all 👍 from me.

[iHiD]

This is the key thing @iHiD should re-clarify, because I think that, if done right (and approaches != implementations), this is basically what after.md is WITHOUT the concept info.

Yeah - Approaches is fuzzy still, which is why I've not written/discussed much on it (beyond bouncing ideas around on that call).

My current thinking is that each exercise (concept or practice) has a page, which is markdown, where maintainers can expound some high-level thoughts on the exercise - be that what Sascha/Colin were thinking for the after.md, or more like the "mentor-notes" of v2 (but student-facing). These can link out to other solutions, etc, but will be currated.

The reasons it's still fuzzy is that I think there is value in having blog-post-length articles on particular approaches that people take, and so I'm not sure how this fits in with the more general overview.

We've not started the UX work on this yet, so I'm still quite 🤷 on how it all fits together.

I don't really understand the bit about concepts under headings in introduction files, [...]

It just makes it easier to see what students actually see as an introduction in the exercise. But as I said, I don't mind change that.

That would leave us with something like:

/exercises/concept/lasagna/introductions/strings.md
/exercises/concept/lasagna/introductions/bools.md

[neenjaw]

That would leave us with something like:

/exercises/concept/lasagna/introductions/strings.md
/exercises/concept/lasagna/introductions/bools.md

With those introduction files exploded out, do they really belong to the lasagna exercise any more? It seems that the idea is moving towards something like

/exercises/concept/lasagna/.docs/instructions.md
/exercises/concept/lasagna/.docs/hints.md

/concepts/<concept slug>/introduction.md
/concepts/<concept slug>/about.md
/concepts/<concept slug>/resources.md

[neenjaw]

_Github won't let me edit the last post to add on, sorry for two posts_

and then as a consequence, rather than exercise teaches a concept, it is almost as though concepts are taught_by exercises if that nuance comes across. (I think we saw this as well moving from an exercise map to a concept map)

[cmcaine]

I don't really understand the bit about concepts under headings in
introduction files, [...]

I don't dislike it, I just don't understand where it is used yet. Only on
the intro page? Is it ever combined with the other concept info? Or are the
slugs just there for automatic links and CI checks?

On Thu, 8 Oct 2020, 17:00 Jeremy Walker, [e-mail] wrote:

This is the key thing @iHiD https://github.com/iHiD should re-clarify,
because I think that, if done right (and approaches != implementations),
this is basically what after.md is WITHOUT the concept info.

Yeah - Approaches is fuzzy still, which is why I've not written/discussed
much on it (beyond bouncing ideas around on that call).

My current thinking is that each exercise (concept or practice) has a
page, which is markdown, where maintainers can expound some high-level
thoughts on the exercise - be that what Sascha/Colin were thinking for the
after.md, or more like the "mentor-notes" of v2 (but student-facing). These
can link out to other solutions, etc, but will be currated.

The reasons it's still fuzzy is that I think there is value in having
blog-post-length articles on particular approaches that people take,
and so I'm not sure how this fits in with the more general overview.

We've not started the UX work on this yet, so I'm still quite 🤷 on how
it all fits together.

I don't really understand the bit about concepts under headings in
introduction files, [...]

It just makes it easier to see what students actually see as an
introduction in the exercise. But as I said, I don't mind change that.

That would leave us with something like:

/exercises/concept/lasagna/introductions/strings.md

/exercises/concept/lasagna/introductions/bools.md

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/exercism/v3/issues/2293#issuecomment-705666354, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABNZA6KYCXQQMFF2R42WHULSJXOYBANCNFSM4SHPJTWA
.

[iHiD]

The introductions will be used in two places.

1. They form the first part of the README.md when a user downloads an exercise via the CLI and are the first part of what a user reads when solving the exercise in the web-editor (I'm vague on this because we've not finalised the UI for exactly how this bit looks yet). They are therefore very exercise-specific.
2. When the Concept Exercise has not been completed, they appear on a Concept page with a CTA (call to action) of "Complete Lasagna to Learn about Strings". Once the exercise has been completed that disappears and is replaced with the more detailed about.md

---

With those introduction files exploded out, do they really belong to the lasagna exercise any more? It seems that the idea is moving towards something like...

I've toyed with this too.

I think they belong in the lasagna exercise. The key requirement of the introduction is that it gives enough info to a student to use that concept to complete the exercise. I fear if they go into the concept folder, they'll lose that connection to the exercise and edge towards bloatedness, rather than maintaining that requirement of being the minimal (but enough-to-solve the exercise) content.

[iHiD]

it is almost as though concepts are taught_by exercises if that nuance comes across.

Yes! This is exactly the change we're making, and the change in language we've been using to describe it.

[iHiD]

Just to expand a little more, the reason for the headings is fundamentally because the primary purpose of the introduction is to be presented to the user a whole for the concepts that are being taught ((1) above). So it's normal state (in my mind) is being compiled with headings. The alternative state ((2) above) is them exploded out on pages, which is why my instinct is to explode them when needed but store them in their primary form.

But I'm not precious about it. The decision should come down to whichever is going to be easiest to work with, and reduces mistakes best.

[TheLostLambda]

But I'm not precious about it. The decision should come down to whichever is going to be easiest to work with, and reduces mistakes best.

I think my strong preference would be to keep them in one file with the sub headings. I think it would be slightly awful to jump between different introduction files when I'm going to write an introduction. I think it's easier to keep the intro specific to the exercise and concise when it's all in one file.

In any case, I do think that UI / UX mockups of what these heading enable would helpful to see :)

[cmcaine]

Maybe naming these fragments will help? I feel like they're not the introduction to the concept, they're the introduction within the context of that exercise.

They are introductions to the bit of the concept that we want to teach in exercise lasagna. Perhaps they're steps in a given concept's tutorial? Or concept-breadcrumbs? Or concept-factoid? I don't know.

Anyway, my slight preference is to have them in the files they will be merged into. I don't really like all the split up description, hints, etc. files as it is :)

[verdammelt]

I'm trying to understand what information should be in about.md and what should be instructions.md.

I'm currently working on transitioning my lists concept exercise to this new format. My after.md actually didn't have much info in it other than an interesting tid-bit about the functions available, however my introduction.md had lots of information about the data type. But in that discussion was mention of functions to create and manipulate lists. Should those all be moved to the about.md leaving my introduction.md nearly empty? Or should about.md be more high-level and discussion of particular functions for the data type be left for the various introduction.md on exercises that might use this concept?

[verdammelt]

Another question - Is there a way to link between concepts? For example the concept of list is built on top of the concept of cons cells in Common Lisp. I'd like to state this fact and link to the cons concept in the list concept about.md. Is this possible?

[ErikSchierboom]

Another question - Is there a way to link between concepts? For example the concept of list is built on top of the concept of cons cells in Common Lisp. I'd like to state this fact and link to the cons concept in the list concept about.md. Is this possible?

Not yet, but this is on our radar!

[ErikSchierboom]

I'm currently working on transitioning my lists concept exercise to this new format. My after.md actually didn't have much info in it other than an interesting tid-bit about the functions available, however my introduction.md had lots of information about the data type. But in that discussion was mention of functions to create and manipulate lists. Should those all be moved to the about.md leaving my introduction.md nearly empty? Or should about.md be more high-level and discussion of particular functions for the data type be left for the various introduction.md on exercises that might use this concept?

The general rule is that the introduction.md should contain _just_ enough information for the student to solve the exercise. That does not mean one has to mention all the functions the student can use. If possible, it is preferrable to mention a module/class name and have the student figure things out by themselves. If they get stuck, they can use the hints.

The about.md can contain many things. At a minimum it should contain a description of the concept, but having specific code examples to help explain things makes a lot of sense there too.

[ErikSchierboom]

@verdammelt I've opened an issue to discuss how to support linking between documents.