Bert: When pretraining, what was the accuracy of the masked-LM task?

[bgshin]

When pretraining, what was the accuracy of the masked-LM task?

[batteryhp]

+1, I have the same question.

[space-N]

+1, I have the same question.

[bgshin]

May we can reference this? (I found this in the README.md)
So the accuracy is 0.985479?

Eval results
global_step = 20
loss = 0.0979674
masked_lm_accuracy = 0.985479
masked_lm_loss = 0.0979328
next_sentence_accuracy = 1.0
next_sentence_loss = 3.45724e-05

[johnmoberg]

That refers to training on ~30 sentences: "Note that since our sample_text.txt file is very small, this example training will overfit that data in only a few steps and produce unrealistically high accuracy numbers."