Spacy: Set the depth of the window in a NER model

[damianoporta]

Hello, i post the question again, maybe something has been changed during this period.
Previously i have added https://github.com/explosion/spaCy/issues/1532 that has been automatically closed. Is there a simple way to change the depth now?
Thanks

[honnibal]

The explanation in the previous thread is still good, although now it's possible to have arguments to nlp.begin_training() that reference specific components. Try:

nlp.begin_training(component_cfg={"ner": {"conv_depth": 8}})

I think that should control the convolution depth for the NER model, without changing the settings for anything else.

[alejandrojcastaneira]

Regards, would possible by this way to match also the "conv_depth, conv_width,etc.." settings from the pretrain component?

[honnibal]

@alejandrojcastaneira Here is a function I wrote for Prodigy recently that reads the pretrain weights and tries to get the hyper-parameters from them. It doesn't generalise to all architectures, only the built-in Tok2Vec function, but it may be useful for you:

def read_pretrain_hyper_params(loc, require=False):
    """Load a pretrain weights file and return certain hyper-parameters from it.

    This function relies on implementation details of spacy._ml.Tok2Vec.
    """
    if not require and (loc is None or not Path(loc).exists()):
        return {}
    msg = srsly.read_msgpack(loc)
    dims = {}
    for layer in msg[b"weights"]:
        for dim, value in layer.get(b"dims", {}).items():
            dims.setdefault(str(dim), []).append(value)
    if "nV" not in dims or "nP" not in dims or "nO" not in dims:
        raise ValueError(
            "Error interpreting pretrained weights. Perhaps this is an unexpected "
            "architecture? Or perhaps the weights file was produced by a different "
            "version of spaCy?"
        )
    output = {
        "embed_rows": max(dims["nV"]),
        "require_vectors": "nM" in dims,
        "cnn_maxout_pieces": dims["nP"][0],
        "token_vector_width": dims["nO"][0],
        "conv_depth": len(dims["nP"]) - 1,
    }
    return output

[alejandrojcastaneira]

Thanks you! for the detailed function.

After I get the hyper-parameters from a spacy weights file, If I'm training from a script, then where should I declare those, maybe only need to pass the return of this function to begin training:

nlp.begin_training(component_cfg={"ner": read_pretrain_hyper_params( "path/tok2vec/",True)})

[honnibal]

Yes, I think that should work.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.