Mkdocs: site_dir contains folders (and subfolders) from doc_dir though not specified in mkdocs.yml

[Tset-Noitamotua]

After doing mkdocs build I expected to find in the site_dir only the rendered .html files in according folder structure as defined by pages: in mkdocs.yml + some mkdocs specific assets. Instead I find there every file and folder/subfolder from docs_dir.

Am I doing something wrong?

I am building test documentation for several projects which are all in my local workspace. So I'm kind of using mkdocs not as supposed to be used to build doculmentation per single project but instead for test docu only but for multiple projects at once. But the only relevant raw .md files are in project_folder/src/test/robotframework/TESTS (subsubsubfolder of each project)
e.g. ugly_app/src/test/robotframework/TESTS/test_doc.md.

My docs_dir tree looks akin like:

│   ├───ugly_app
│   │   └───src
│   │       ├───main
│   │       │   ├───java
│   │       │   │   └───com
│   │       │   │       └───mycompany
│   │       │   │           ├───...
│   │       │   │           │   └───...
│   │       │   │           │       └───...
│   │       │   │           └───....
│   │       │   │               └───...
│   │       │   │                   ├───...
│   │       │   │                   ├───...
│   │       │   │                   │   ├───...
│   │       │   │                   │   │  
│   │       │   │                   │   └───..
│   │       │   │                   ├───...
│   │       │   │                   ├───...
│   │       │   │                   │   └───...
│   │       │   │                   │       └───...
│   │       │   ├───...             ...
│   │       │   │   ├───...
│   │       │   │   │   └───..
│   │       │   │   │       └───...
│   │       │   │   ├───...
│   │       │   │   ├───...
│   │       │   │   └───.
│   │       │   └───...
│   │       │       ├───..
│   │       │       └───.
│   │       └───test
│   │           ├───java
│   │           │   └───com
│   │           │       └───mycompany
│   │           │           └───...
│   │           │               └──...
│   │           │                   ├───...
|   |           |                   ...
│   │           └───robotframework
│   │               ├───...
│   │               ├───...
│   │               │   └───...
│   │               ├───TESTS
│   │               ...
│   │               
│   ├───SCArchive_20111111Test
│   │   ├───robotframework
│   │   │   ├───...
│   │   │   ├───...
│   │   │   └───TESTS
│   │   │       └───...
│   │   └───...
... ...

My mkdocs.yml looks akin like:

site_name: KiSQA
site_description: mycompany
site_dir: _KiSQA_Site
docs_dir: PROJECTS
pages:
  - index.md
  - QA-Dashboard: index.md
  - Acceptance Tests:
    #- Absence Administration: AbsenceAdministration\src\test\robotframework\TESTS\README.md
    #- Activity Report: ActivityReport\src\test\robotframework\TESTS\README.md
    #- Offer Management: OfferManagement\src\test\robotframework\TESTS\README.md
    - uggly_app: uggly_app\src\test\robotframework\TESTS\test_doc.md
    - Vaadin7 Theme Demo: Vaadin7ThemeDemo\src\test\robotframework\TESTS\README.md
    - ...


site_author: 'Mr. Butch'
site_url: 'http://www.mycompany.com/'
theme: material
theme_dir: _KiSQA_Source
markdown_extensions:
  - admonition
  - footnotes
  - codehilite(linenums=true)
  - toc(permalink=true)
  - pymdownx.superfences
  - pymdownx.tasklist(custom_checkbox=true)
  - pymdownx.emoji:
      emoji_generator: !!python/name:pymdownx.emoji.to_svg

extra_javascript:
  - 'KiSQA_MKDocs_Extensions\javascripts\jenkins.js'
extra_css:
  - 'KiSQA_MKDocs_Extensions\stylesheets\buttons.css'

extra:
  palette:
    primary: 'red'
    accent: 'red'
  social:
    - type: 'github'
      link: 'https://github.com/mycompany'

[waylan]

No this is normal behavior. A decision was made early on (before I was involved) to allow you to include other assets in your docs_dir and to copy those over to the site_dir. Obviously images make sense, but but we don't actually check the file type. Therefore, the complete directory structure of your docs_dir is copied. The expectation is that nothing except what actually goes into the site_dir should ever be in the docs_dir.

Also note that settings like extra_css do not necessary ensure the CSS file is copied (that happens regardless), but it ensures the CSS file gets listed in the HTML head of each page. A subtle but important difference.

Finally, if we were to change the current behavior, it would break a lot of users existing sites. So while I can see an argument for an adjustment here, it is not likely to overcome the backward compatibility argument to leave things as-is.

[Tset-Noitamotua]

Oh no, that seems like a deal breaker then in my case(?). Is there no way to exclude folders so that they are completly ignored?

May be something like:

pages:
  - ignore: ugly_app/scr
  - ignore: ugly_app/test/java
  - ...

[waylan]

No, at this time we don't even allow pages to be ignored (see #699).

If it helps, the simplest use case we support is to define a config file with a site_name and a docs_dir and that's it. MkDocs automatically populates the pages and copies everything in the docs_dir over. Everything else in the config only builds on that (adding, not taking away). There is no "exclude" of any kind. It was never anticipated that non-documentation would be in the docs_dir... And quite frankly, why would anyone actually want that?

[Tset-Noitamotua]

Ok, thank you. I think I got the point. As I mentioned I am using MkDocs in a way it was not meant to be used :)
I think the easiest solution for me will be to clean up site_dir folder and wipe out everything that is not needed for the static pages.

Actually my raw MarkDown files are not "just" documentation but executable test specification (Robot Framework code in fenced code blocks) along with additional documentation (everything outside of code blocks). In combination with MkDocs this aims to become kind of single source of truth for our (acceptance) test automation.

Cheers
Tset

[apenwarr]

I created a mkdocs-exclude plugin that uses the new plugin API to do what's requested in this bug:
https://github.com/apenwarr/mkdocs-exclude

Hope this helps!