Mkdocs-material: yaml.constructor.ConstructorError: could not determine a constructor for the tag

[csandanov]

Starting today I'm getting the following error during a build, same for python 2 and 3.

https://app.wercker.com/Wodby/php-docs/runs/build/5b3365e4a296ce0001c90842?step=5b3365ec37b4370001c4f556

Collecting mkdocs-material
  Downloading https://files.pythonhosted.org/packages/38/3c/f22621b0d5b349e7ee6b711255ebdf02425ebd775267a0e959947eb40325/mkdocs_material-2.9.1-py2.py3-none-any.whl (570kB)
Collecting markdown-include
  Downloading https://files.pythonhosted.org/packages/ef/44/eb6e9b4fa1110b719abb876c9b6dd8b46af886a94536ec4e9117fe5e7b97/markdown-include-0.5.1.tar.gz
Collecting pygments
  Downloading https://files.pythonhosted.org/packages/02/ee/b6e02dc6529e82b75bb06823ff7d005b141037cb1416b10c6f00fc419dca/Pygments-2.2.0-py2.py3-none-any.whl (841kB)
Collecting pymdown-extensions>=4.10 (from mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/7d/0b/8a645733fc6ad20c1217808d2bf26eb80996901d173e623f886b75f62239/pymdown_extensions-4.11-py2.py3-none-any.whl (205kB)
Collecting tornado<5 (from mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/e3/7b/e29ab3d51c8df66922fea216e2bddfcb6430fb29620e5165b16a216e0d3c/tornado-4.5.3.tar.gz (484kB)
Collecting mkdocs>=0.17.1 (from mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/66/05/5a9e9f661bfd554950d71ead196231f15fe744bf7d1d2431ee1daa32841d/mkdocs-0.17.4-py2.py3-none-any.whl (1.2MB)
Collecting markdown (from markdown-include)
  Downloading https://files.pythonhosted.org/packages/6d/7d/488b90f470b96531a3f5788cf12a93332f543dbab13c423a5e7ce96a0493/Markdown-2.6.11-py2.py3-none-any.whl (78kB)
Collecting singledispatch (from tornado<5->mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/c5/10/369f50bcd4621b263927b0a1519987a04383d4a98fb10438042ad410cf88/singledispatch-3.4.0.3-py2.py3-none-any.whl
Collecting certifi (from tornado<5->mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl (150kB)
Collecting backports_abc>=0.4 (from tornado<5->mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/7d/56/6f3ac1b816d0cd8994e83d0c4e55bc64567532f7dc543378bd87f81cebc7/backports_abc-0.5-py2.py3-none-any.whl
Collecting livereload>=2.5.1 (from mkdocs>=0.17.1->mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/dd/b4/213daced3ff1b4e02a1f700748e20e9a7481f5bfef57d11ae9babfd4aa2f/livereload-2.5.2-py2.py3-none-any.whl
Collecting PyYAML>=3.10 (from mkdocs>=0.17.1->mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/10/7d/6efe0bd69580fecd40adf47ebaf8d807238308ccb851f0549881fa7605aa/PyYAML-4.1.tar.gz (153kB)
Collecting click>=3.3 (from mkdocs>=0.17.1->mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl (71kB)
Collecting Jinja2>=2.7.1 (from mkdocs>=0.17.1->mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl (126kB)
Collecting six (from singledispatch->tornado<5->mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from Jinja2>=2.7.1->mkdocs>=0.17.1->mkdocs-material)
  Downloading https://files.pythonhosted.org/packages/4d/de/32d741db316d8fdb7680822dd37001ef7a448255de9699ab4bfcbdf4172b/MarkupSafe-1.0.tar.gz
Building wheels for collected packages: markdown-include, tornado, PyYAML, MarkupSafe
  Running setup.py bdist_wheel for markdown-include: started
  Running setup.py bdist_wheel for markdown-include: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/af/4c/e3/599182bf5f723d76154ecf793632fc05f65250ba55a4a5bdce
  Running setup.py bdist_wheel for tornado: started
  Running setup.py bdist_wheel for tornado: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/72/bf/f4/b68fa69596986881b397b18ff2b9af5f8181233aadcc9f76fd
  Running setup.py bdist_wheel for PyYAML: started
  Running setup.py bdist_wheel for PyYAML: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/57/8e/30/4ef5a500e763c11371ee16ec7f68043efc832ed1ff0533adff
  Running setup.py bdist_wheel for MarkupSafe: started
  Running setup.py bdist_wheel for MarkupSafe: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/33/56/20/ebe49a5c612fffe1c5a632146b16596f9e64676768661e4e46
Successfully built markdown-include tornado PyYAML MarkupSafe
Installing collected packages: markdown, pymdown-extensions, six, singledispatch, certifi, backports-abc, tornado, pygments, livereload, PyYAML, click, MarkupSafe, Jinja2, mkdocs, mkdocs-material, markdown-include
Successfully installed Jinja2-2.10 MarkupSafe-1.0 PyYAML-4.1 backports-abc-0.5 certifi-2018.4.16 click-6.7 livereload-2.5.2 markdown-2.6.11 markdown-include-0.5.1 mkdocs-0.17.4 mkdocs-material-2.9.1 pygments-2.2.0 pymdown-extensions-4.11 singledispatch-3.4.0.3 six-1.11.0 tornado-4.5.3
Traceback (most recent call last):
  File "/usr/local/bin/mkdocs", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/mkdocs/__main__.py", line 155, in build_command
    site_dir=site_dir
  File "/usr/local/lib/python2.7/site-packages/mkdocs/config/base.py", line 177, in load_config
    cfg.load_file(config_file)
  File "/usr/local/lib/python2.7/site-packages/mkdocs/config/base.py", line 123, in load_file
    return self.load_dict(utils.yaml_load(config_file))
  File "/usr/local/lib/python2.7/site-packages/mkdocs/utils/__init__.py", line 71, in yaml_load
    return yaml.load(source, Loader)
  File "/usr/local/lib/python2.7/site-packages/yaml/__init__.py", line 74, in load
    return loader.get_single_data()
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 39, in get_single_data
    return self.construct_document(node)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 48, in construct_document
    for dummy in generator:
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 398, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 208, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 133, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 88, in construct_object
    data = constructor(self, node)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 414, in construct_undefined
    node.start_mark)
yaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/name:pymdownx.emoji.to_svg'
  in "/pipeline/source/mkdocs.yml", line 49, column 24

My build command:

pip install mkdocs-material markdown-include pygments
mkdocs build

mkdocs.yml:

site_name: PHP stack documentation
site_description: PHP stack documentation
site_author: Wodby, Inc.
site_url: https://wodby.com/stacks/php/docs

repo_name: wodby/php-docs
repo_url: https://github.com/wodby/php-docs

copyright: Wodby, Inc.

extra:
  social:
    - type: globe
      link: https://wodby.com
    - type: github-alt
      link: https://github.com/wodby
    - type: twitter
      link: https://twitter.com/WodbyHQ
  stack_url: https://cloud.wodby.com/stackhub

theme:
  name: material
  feature:
    tabs: true
  palette:
    primary: 'indigo'
    accent: 'indigo'
  custom_dir: 'theme'

extra_css: ["assets/wodby.css"]
extra_javascript: ["assets/intercom.js"]

markdown_extensions:
  - markdown.extensions.admonition
  - markdown.extensions.codehilite:
      guess_lang: false
  - markdown.extensions.def_list
  - markdown.extensions.footnotes
  - markdown.extensions.meta
  - markdown.extensions.toc:
      permalink: true
  - pymdownx.arithmatex
  - pymdownx.betterem:
      smart_enable: all
  - pymdownx.caret
  - pymdownx.critic
  - pymdownx.details
  - pymdownx.emoji:
      emoji_generator: !!python/name:pymdownx.emoji.to_svg
  - pymdownx.inlinehilite
  - pymdownx.magiclink
  - pymdownx.mark
  - pymdownx.smartsymbols
  - pymdownx.superfences
  - pymdownx.tasklist:
      custom_checkbox: true
  - pymdownx.tilde

pages:
    - Home: index.md
    - Backups: backups.md
    - Deployment: deployment.md
    - Import: import.md
    - Instance types: instance-types.md
    - Mail delivery: mail-delivery.md
    - Containers:
      - Overview: containers/index.md
      - Apache: containers/apache.md
      - AthenaPDF: containers/athenapdf.md
      - Blackfire: containers/blackfire.md
      - Cron: containers/cron.md
      - Elasticsearch: containers/elasticsearch.md
      - Kibana: containers/kibana.md
      - Mailhog: containers/mailhog.md
      - MariaDB: containers/mariadb.md
      - Memcached: containers/memcached.md
      - Nginx: containers/nginx.md
      - OpenSMTPD: containers/opensmtpd.md
      - PHP: containers/php.md
      - PostgreSQL: containers/postgres.md
      - Rsyslog: containers/rsyslog.md
      - Redis: containers/redis.md
      - Solr: containers/solr.md
      - SSH: containers/ssh.md
      - Webgrind: containers/webgrind.md
    - Local environment:
      - Overview: local/index.md
      - Quick start: local/quick-start.md
      - Domains: local/domains.md
      - Multiple projects: local/multiple-projects.md
      - Permissions issues: local/permissions.md
      - Import and export: local/import-export.md
      - Make commands: local/make-commands.md
      - For macOS users: local/docker-for-mac.md
      - Xdebug: local/xdebug.md

[squidfunk]

@facelessuser could this be related to some upstream change in the Pymdown Extensions package? See:

yaml.constructor.ConstructorError: could not determine a constructor for the tag 
    'tag:yaml.org,2002:python/name:pymdownx.emoji.to_svg'
  in "/pipeline/source/mkdocs.yml", line 49, column 24

[csandanov]

btw, line 49 is

      emoji_generator: !!python/name:pymdownx.emoji.to_svg

I have no idea whether I need it, just copied from this repo

[facelessuser]

I'm in the woods camping at least till the end of the week. I won't be able to look at this until I get back.

I will say you can't use the svg output unless you are using the old emojione assets or the twemoji assets. But we do test for svg and it is currently passing: https://github.com/facelessuser/pymdown-extensions/blob/master/tests/extensions/emoji/tests.yml#L48.

[squidfunk]

@facelessuser no hurry, enjoy the time off! I'm a bit jealous :-)

[facelessuser]

I cannot reproduce this issue. I tried using emoji_generator: !!python/name:pymdownx.emoji.to_svg locally, I tired copying and pasted the issue creator's markdown_extensions, and I could not reproduce the yaml error.

Something is wrong with either your yaml config or something with your local setup. Maybe even with your pyyaml installation. Maybe your installation of pymdown-extensions is corrupted, I have no idea. But from what I can see everything is working as it should.

I did not install markdown-include as I was mainly trying to evaluate if pydmown-extensions was broken in regards to to_svg, and it isn't.

[squidfunk]

Hmm, maybe a corrupted cache then? Had some fun with this problem on Travis.

[facelessuser]

Could be... there is a way to install without using the cache with pip.

As far as pymdown-extensions is concerned, I've made no changes in regards to to_svg. All tests are passing as well. Maybe this is a cache issue for the issue creator?

[csandanov]

I build it via wercker on official alpine-based python 3 image, the build file can be found at https://github.com/wodby/php-docs/blob/master/.wercker.yml. The issue has been resolved after I deleted emoji_generator

[squidfunk]

Okay, so as you say your issue is resolved (and it sounds like a cache inconsistency problem), I'm closing this issue.

[DSpeichert]

Actually, the latest automated build from https://hub.docker.com/r/squidfunk/mkdocs-material/ (squidfunk/mkdocs-material@sha256:b408c84f3662f987718daadd71f2b8436d35aeb85ef0fb1c4f01e2bfb8764aa0) also throws the same error for me - it started happening 3 days ago, exactly when that Docker image was automatically built. I'd say there's still some error somewhere.

[facelessuser]

This may be an issue with the docker image then, but I don't see an issue directly with pymdown-extensions unless some new evidence that shows otherwise surfaces. I'll let @squidfunk investigate this for now.

[squidfunk]

Will investigate asap.

[squidfunk]

I can confirm this issue with the latest Docker image. However, when I rebuild the image locally, everything works as expected. I will now try to rebuild the Docker image on Travis, maybe it was just some kind of hiccup / inconsistency.

[squidfunk]

The image is fixed now. I will re-open this issue to remember checking after the next release build if the Docker image works - I just pushed my local build because I lack the time for debugging the Travis build at the moment.

[squidfunk]

BTW @facelessuser the bug seems to be entirely related to the YAML parser, because my mkdocs.yml fails here:

google_analytics:
  - !!python/object/apply:os.getenv ["GOOGLE_ANALYTICS_KEY"]
  - auto

results in:

yaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/object/apply:os.getenv'

However, thanks for your time investigating!

[squidfunk]

I debugged the build - this is how PyYAML was installed in build 2.9.1:

this is how it was installed in 2.9.2

Could be related. However, locally on my machine it installed version 4.2b4!? I'm really confused.

[squidfunk]

@csandanov could you try and wrap the constructor functions in double quotes? See this comment for an explanation, example: (see @facelessuser comment below)

markdown_extensions:
  - pymdownx.emoji:
      emoji_generator: !!python/name:pymdownx.emoji.to_svg

becomes:

markdown_extensions:
  - pymdownx.emoji:
      emoji_generator: "!!python/name:pymdownx.emoji.to_svg"

[facelessuser]

Using quotes will make it pass, but not treat the entry as a function, it will be treated as a string.

[squidfunk]

Hmm okay, I don't really know much Python to debug this properly. For what it looks like it is almost certainly related to different versions of PyYaml. Sometimes 3.10 gets installed, sometimes 3.12, locally I have >4.1. Very weird.

[facelessuser]

Hmm, I'll play with it and see what I see when I get a chance.

[facelessuser]

So it seems some things have changed with pyyaml and safe mode (safe mode would restrict things like !!python/name: from being run): https://github.com/yaml/pyyaml/issues/187. I'm not sure if this is related or not, but interesting. I may try forcing the issue by specifically installing certain versions of PyYaml and see which causes the issue.

[facelessuser]

FYI, I don't see this issue with 3.12 pyyaml.

[facelessuser]

4.2b1 seems to be the one at fault. It's the one I can reproduce the issue on. 4.2b2+ seems fine. Is there a dependency forcing a beta of pyyaml? 3.12 should be the current stable version being installed. It looks like there is nothing stable above 3.x currently. I would not force pyyaml >4.1. Maybe there was a stable 4.1 briefly, and they might have released it and then took it down when they discovered issues...

[squidfunk]

Thanks for taking the time to debug this. It's really strange that the best got installed on my local system, while Travis keeps installing 3.10 or 3.12. I fixed the Docker image yesterday by building and pushing it from my machine, but I'll leave this issue open until the next release and check again if the issue still persists.

[squidfunk]

Seems to be fixed, just rolled out 2.9.4 and checked the Docker image - it's working.