Bert: ValueError: Assignment map with scope only name xxx should map to scope only xxx. Should be 'scope/': 'other_scope/'.

[yorkchu1995]

We add a attention layer in the end of model. It is okay when fine-tuning, but when I load saved model.ckpt again, it always fails as flows:

INFO:tensorflow:Error recorded from prediction_loop: Assignment map with scope only name bert/pooler_aic_attention/attention should map to scope only bert/pooler_aic_attention/attention/W_omega. Should be 'scope/': 'other_scope/'.
INFO:tensorflow:prediction_loop marked as finished
WARNING:tensorflow:Reraising captured error
Traceback (most recent call last):
File "../../bert-master_new/msp_run_classifier.py", line 1086, in
tf.app.run()
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "../../bert-master_new/msp_run_classifier.py", line 1051, in main
for (i, prediction) in enumerate(result):
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2437, in predict
rendezvous.raise_errors()
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/contrib/tpu/python/tpu/error_handling.py", line 128, in raise_errors
six.reraise(typ, value, traceback)
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2431, in predict
yield_single_examples=yield_single_examples):
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 551, in predict
features, None, model_fn_lib.ModeKeys.PREDICT, self.config)
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2186, in _call_model_fn
features, labels, mode, config)
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 1169, in _call_model_fn
model_fn_results = self._model_fn(features=features, *kwargs)
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2470, in _model_fn
features, labels, is_export_mode=is_export_mode)
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1250, in call_without_tpu
return self._call_model_fn(features, labels, is_export_mode=is_export_mode)
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1524, in _call_model_fn
estimator_spec = self._model_fn(features=features, *kwargs)
File "../../bert-master_new/msp_run_classifier.py", line 730, in model_fn
tf.train.init_from_checkpoint(init_checkpoint, assignment_map)
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/python/training/checkpoint_utils.py", line 187, in init_from_checkpoint
_init_from_checkpoint, ckpt_dir_or_file, assignment_map)
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/python/training/distribute.py", line 1040, in merge_call
return self._merge_call(merge_fn, args, *kwargs)
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/python/training/distribute.py", line 1048, in _merge_call
return merge_fn(self._distribution_strategy, args, *kwargs)
File "/home/zhuyuepeng01/.conda/envs/py36/lib/python3.6/site-packages/tensorflow/python/training/checkpoint_utils.py", line 243, in _init_from_checkpoint
scopes, tensor_name_in_ckpt))
ValueError: Assignment map with scope only name bert/pooler_aic_attention/attention should map to scope only bert/pooler_aic_attention/attention/W_omega. Should be 'scope/': 'other_scope/'.

[xwzhong]

if you want to map variable A to B, the name(contains scope) of A&B should has the same scope level, eg, the scope of A is xx/xx/xxx/name_a, then, B should be yy/yy/yyy/name_b.

[yorkchu1995]

if you want to map variable A to B, the name(contains scope) of A&B should has the same scope level, eg, the scope of A is xx/xx/xxx/name_a, then, B should be yy/yy/yyy/name_b.

But it has the same scope level, i don't know why.

[xyfZzz]

if you want to map variable A to B, the name(contains scope) of A&B should has the same scope level, eg, the scope of A is xx/xx/xxx/name_a, then, B should be yy/yy/yyy/name_b.

But it has the same scope level, i don't know why.

Hi，i got the same problem, did you solve this problem?

[yoquankara]

@xyfZzz
Can you test if below workaround is useful?

Workaround: in modeling.py/get_assignment_map_from_checkpoint

-     assignment_map[name] = name
+     assignment_map[name] = name_to_variable[name]

[jihunchoi]

@xyfZzz
Can you test if below workaround is useful?

Workaround: in modeling.py/get_assignment_map_from_checkpoint

-     assignment_map[name] = name
+     assignment_map[name] = name_to_variable[name]

I had the same problem, and for my case this workaround worked.

[HaodaY]

@xyfZzz
Can you test if below workaround is useful?

Workaround: in modeling.py/get_assignment_map_from_checkpoint

-     assignment_map[name] = name
+     assignment_map[name] = name_to_variable[name]

I have the same problem, and this workaround worked.

[mdmustafizurrahman]

I am facing the same problem. with with tf.variable_scope("loss"): I put a code in if else mode: one for a dense layer and another for a higway layer but I am also receiveing the same error
** File "C:\Users\t-mdrahm\code\bert-dlis\model\model.py", line 267, in build_tf_graph
self.num_input_bow_features_map)
File "C:\Users\t-mdrahm\code\bert-dlis\model\model.py", line 105, in __init__
self.load_checkpoint(init_checkpoint, self.known_task_names, within_scope=self.scope)
File "C:\Users\t-mdrahm\code\bert-dlis\model\model.py", line 125, in load_checkpoint
tf.train.init_from_checkpoint(init_checkpoint, assignment_map)
File "C:\Users\t-mdrahm\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\pythontraining\checkpoint_utils.py", line 190, in init_from_checkpoint
_init_from_checkpoint, args=(ckpt_dir_or_file, assignment_map))
File "C:\Users\t-mdrahm\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\distribute\distribute_lib.py", line 1516, in merge_call
return self._merge_call(merge_fn, args, kwargs)
File "C:\Users\t-mdrahm\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\distribute\distribute_lib.py", line 1524, in _merge_call
return merge_fn(self._distribution_strategy, *args, kwargs)
File "C:\Users\t-mdrahm\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\pythontraining\checkpoint_utils.py", line 246, in _init_from_checkpoint
scopes, tensor_name_in_ckpt))
ValueError: Assignment map with scope only name 61aef709/Passage/loss/highwayLayer/denseUnit0/denseUnit0Bias should map to scope only Passage/loss/highwayLayer/denseUnit0/denseUnit0Bias/Variable. Should be 'scope/': 'other_scope/'.

Workaround: in modeling.py/get_assignment_map_from_checkpoint IS NOT WORKING. please help
- assignment_map[name] = name
+ assignment_map[name] = name_to_variable[name]