Salt: require_in formula cannot extend ID error
Description of Issue/Question
I have an include on an external formula and require its execution to happen after the execution of some other state:
include:
- nginx.ng
download-openresty-more-headers:
archive.extracted:
- name: /tmp/
- source: https://github.com/openresty/headers-more-nginx-module/archive/v{{ openresty_headers_more['version']
}}.tar.gz
- source_hash: sha256={{ openresty_headers_more['hash'] }}
- archive_format: tar
- if_missing: /tmp/openresty_headers_more-{{ openresty_headers_more['version'] }}.tar.gz
rename-openresty-more-headers:
file.rename:
- name: /tmp/headers-more-nginx-module
- source: /tmp/headers-more-nginx-module-{{ openresty_headers_more['version'] }}
- require:
- archive: download-openresty-more-headers
- sls: nginx.ng
The following SLS file works, but the relationship on the require is the incorrect and I need to inverse the dependency such that nginx.ng requires the rename-openresty-more-headers state:
include:
- nginx.ng
download-openresty-more-headers:
archive.extracted:
- name: /tmp/
- source: https://github.com/openresty/headers-more-nginx-module/archive/v{{ openresty_headers_more['version']
}}.tar.gz
- source_hash: sha256={{ openresty_headers_more['hash'] }}
- archive_format: tar
- if_missing: /tmp/openresty_headers_more-{{ openresty_headers_more['version'] }}.tar.gz
rename-openresty-more-headers:
file.rename:
- name: /tmp/headers-more-nginx-module
- source: /tmp/headers-more-nginx-module-{{ openresty_headers_more['version'] }}
- require:
- archive: download-openresty-more-headers
- require_in:
- sls: nginx.ng
However, this results in the following error (which I understand from the docs should work as require_in is just the inverse of require):
local:
Data failed to compile:
----------
Cannot extend ID 'nginx.ng' in 'base:bc-nginx'. It is not part of the high state.
This is likely due to a missing include statement or an incorrectly typed ID.
Ensure that a state with an ID of 'nginx.ng' is available
in environment 'base' and to SLS 'bc-nginx'
Versions Report
Salt Version:
Salt: 2015.8.8.2
Dependency Versions:
Jinja2: 2.7.3
M2Crypto: Not Installed
Mako: Not Installed
PyYAML: 3.11
PyZMQ: 14.4.0
Python: 2.7.9 (default, Mar 1 2015, 12:57:24)
RAET: Not Installed
Tornado: 4.2.1
ZMQ: 4.0.5
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.2
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
libgit2: Not Installed
libnacl: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.2
mysql-python: 1.2.3
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
python-gnupg: Not Installed
smmap: Not Installed
timelib: Not Installed
System Versions:
dist: debian 8.4
machine: x86_64
release: 3.16.0-4-amd64
system: debian 8.4
ghost
All 13 comments
@grobinson-blockchain I am not able to replicate this error with a local filesystem using the following testcase:
include:
- state1
touch /tmp/state2:
cmd.run:
- creates: /tmp/state2
- require_in:
- sls: state1
state1:
touch /tmp/state1:
cmd.run:
- creates: /tmp/state1
➜ issues sudo salt 'ch3ll-mas*' state.highstate
ch3ll-master:
----------
ID: touch /tmp/state1
Function: cmd.run
Result: True
Comment: Command "touch /tmp/state1" run
Started: 18:28:25.048934
Duration: 18.212 ms
Changes:
----------
pid:
8194
retcode:
0
stderr:
stdout:
----------
ID: touch /tmp/state2
Function: cmd.run
Result: True
Comment: Command "touch /tmp/state2" run
Started: 18:28:25.067488
Duration: 11.87 ms
Changes:
----------
pid:
8195
retcode:
0
stderr:
stdout:
Summary for ch3ll-master
------------
Succeeded: 2 (changed=2)
Failed: 0
------------
Total states run: 2
salt 2015.8.8.2 (Beryllium)
Is there anything i'm missing from my test case? Also when you state "I have an include on an external formula " do you mean you are using gitfs? can you clarify please. Thanks
Ch3LL
on 10 May 2016
@Ch3LL I'm using https://github.com/saltstack-formulas/nginx-formula (which has a nginx.ng state) which I cloned and have configured via file_roots in /etc/salt/minion. I am not using gitfs.
ghost
on 10 May 2016
To provide a little more info I am using a tool called Packer (packer.io) to build machine images which I provision via salt.
I have the following file roots and pillar roots:
file_roots:
base:
- /srv/salt
- /srv/salt/nginx-formula
pillar_roots:
base:
- /srv/pillar
The pillar top.sls file is as follows:
base:
'*':
- nginx
and the nginx.sls file:
nginx:
ng:
install_from_source: True
source_version: 1.10.0
source_hash: 8ed647c3dd65bc4ced03b0e0f6bf9e633eff6b01bac772bcf97077d58bc2be4d
source:
opts:
- "--add-module=/tmp/headers-more-nginx-module"
openresty_heades_more:
version: 0.29
hash: 0a5f3003b5851373b03c542723eb5e7da44a01bf4c4c5f20b4de53f355a28d33
The state top file is as follows (and bc-nginx is the file in the snippet at the start of the thread)
base:
'*':
- bc-nginx
FWIW, the following works:
include:
- nginx
/tmp/demo1:
file.managed:
- require_in:
- sls: nginx
but this does not:
include:
- nginx.ng
/tmp/demo1:
file.managed:
- require_in:
- sls: nginx.ng
Is that because ng is a subfolder of nginx?
ghost
on 10 May 2016
Hi @Ch3LL were you able to reproduce this error?
ghost
on 12 May 2016
@grobinson-blockchain apologies for the wait. i'm looking through this now and will update when i'm done testing.
Ch3LL
on 13 May 2016
Okay I am now able to replicate the error and I think I have the smallest test case. I believe the issue is with the extends in your nginx/ng/init.sls file. Here is my test case:
/srv/salt/simple-test.sls
➜ salt cat simple-test.sls
include:
- test.test
touch /tmp/simple-test:
cmd.run:
- require_in:
- sls: test.test
➜ salt cat /srv/salt/test/test.sls
include:
- test.test1
extend:
create_test1_file:
cmd:
- creates: /tmp/testextend
➜ salt cat /srv/salt/test/test1.sls
create_test1_file:
cmd.run:
- name: touch /tmp/test1
➜ salt sudo salt 'ch3ll-mas*' state.highstate
ch3ll-master:
Data failed to compile:
----------
Cannot extend ID 'test.test' in 'base:simple-test'. It is not part of the high state.
This is likely due to a missing include statement or an incorrectly typed ID.
Ensure that a state with an ID of 'test.test' is available
in environment 'base' and to SLS 'simple-test'
ERROR: Minions returned with non-zero exit code
Weirdly enough if you add a state after the extends it works. Similar to the following:
➜ salt cat /srv/salt/test/test.sls
include:
- test.test1
extend:
create_test1_file:
cmd:
- creates: /tmp/testextend
touch-file:
cmd.run:
- name: touch /tmp/touch-file
➜ salt sudo salt 'ch3ll-mas*' state.highstate
ch3ll-master:
----------
ID: create_test1_file
Function: cmd.run
Name: touch /tmp/test1
Result: True
Comment: Command "touch /tmp/test1" run
Started: 17:33:03.006283
Duration: 39.291 ms
Changes:
----------
pid:
14426
retcode:
0
stderr:
stdout:
----------
ID: touch-file
Function: cmd.run
Name: touch /tmp/touch-file
Result: True
Comment: Command "touch /tmp/touch-file" run
Started: 17:33:03.045987
Duration: 23.553 ms
Changes:
----------
pid:
14428
retcode:
0
stderr:
stdout:
----------
ID: touch /tmp/simple-test
Function: cmd.run
Result: True
Comment: Command "touch /tmp/simple-test" run
Started: 17:33:03.069943
Duration: 14.141 ms
Changes:
----------
pid:
14429
retcode:
0
stderr:
stdout:
Summary for ch3ll-master
------------
Succeeded: 3 (changed=3)
Failed: 0
------------
Total states run: 3
Can you try appendng the following state to the nginx/ng/init.sls locally on your filesystem and report back whether it works or not:
touch-file:
cmd.run:
- name: touch /tmp/touch-file
@Ch3LL Thanks for taking the time to investigate this issue! When I add a state to end ngin/ng/init.sls it fixes the error message:
extend:
nginx_service:
service:
- watch:
- file: nginx_config
- require:
- file: nginx_config
nginx_config:
file:
- require:
{% if nginx.install_from_source %}
- cmd: nginx_install
{% else %}
- pkg: nginx_install
{% endif %}
nginx_test:
cmd.run:
- name: /usr/sbin/nginx -t
However the ordering of state execution is still incorrect because nginx.ng executes before download-openresty-more-headers and rename-openresty-more-headers states:
include:
- nginx.ng
download-openresty-more-headers:
archive.extracted:
- name: /tmp/
- source: https://github.com/openresty/headers-more-nginx-module/archive/v{{ openresty_headers_more['version']
}}.tar.gz
- source_hash: sha256={{ openresty_headers_more['hash'] }}
- archive_format: tar
- if_missing: /tmp/openresty_headers_more-{{ openresty_headers_more['version'] }}.tar.gz
- require_in:
- sls: nginx.ng
rename-openresty-more-headers:
file.symlink:
- name: /tmp/headers-more-nginx-module
- target: /tmp/headers-more-nginx-module-{{ openresty_headers_more['version'] }}
- require_in:
- sls: nginx.ng
What is interesting is that if I change the require_in from the nginx.ng state file to some named state within the nginx formula (i.e. nginx_configure) then it works as intended:
include:
- nginx.ng
download-openresty-more-headers:
archive.extracted:
- name: /tmp/
- source: https://github.com/openresty/headers-more-nginx-module/archive/v{{ openresty_headers_more['version']
}}.tar.gz
- source_hash: sha256={{ openresty_headers_more['hash'] }}
- archive_format: tar
- if_missing: /tmp/openresty_headers_more-{{ openresty_headers_more['version'] }}.tar.gz
- require_in:
- cmd: nginx_configure
rename-openresty-more-headers:
file.symlink:
- name: /tmp/headers-more-nginx-module
- target: /tmp/headers-more-nginx-module-{{ openresty_headers_more['version'] }}
- require_in:
- cmd: nginx_configure
I wonder if this could also be a bug?
ghost
on 16 May 2016
@grobinson-blockchain nice catch on it just fixing the error, but the ordering is wrong.
When you state " if I change the require_in from the nginx.ng state file to some named state within the nginx formula (i.e. nginx_configure) then it works as intended" Do you mean that the require_in ordering is correct when changing this?
Ch3LL
on 17 May 2016
@Ch3LL that's correct. The require_in ordering is (if I've understand it correctly from the docs) not applied when I have it on the sls state file, but it is applied when I have it on some arbitrary state _within_ the state file.
ghost
on 17 May 2016
Same problem here:
in state statex.tasks:
include:
- statex.tasks_before
- statex.tasks_after
[ states .. ]
ID_a:
file.directory:
- user: xxxx
- require_in:
- sls: tasks_after (or statex.tasks_after : same result)
=>
Cannot extend ID 'statex.tasks_after' in 'base:statex.tasks'. It is not part of the high state.
This is likely due to a missing include statement or an incorrectly typed ID.
Ensure that a state with an ID of 'statex.tasks_after' is available
in environment 'base' and to SLS 'statex.tasks'
-> it says that statex.tasks_after is an ID whereas it is a sls file, containing IDs.
gaetanquentin
on 1 Sep 2016
I am hitting the same issue :/ salt-ssh 2018.3.2 (Oxygen)
arno01
on 8 Jul 2018
it is a wrong "watch" clause.
see https://jorgar.tumblr.com/post/61650359127/cannot-extend-id-it-is-not-part-of-highstate
mobidyc
on 9 Jun 2019
Related issues
Oloremo
·
3Comments
saurabhnemade
·
3Comments
Inveracity
·
3Comments
allyunion
·
3Comments
qiushics
·
3Comments