Spark-nlp: Cannot set threshold in python YakeModel annotator to non-Integer

[tombriles]

Here is a simple test case:

`import sparknlp
from sparknlp.annotator import *
from sparknlp.base import *

spark = sparknlp.start()

keywords = YakeModel().setInputCols("token").setOutputCol("keyphrases").setThreshold(0.6)`

Description

Here's the bug in the code:
python/sparknlp/annotator.py line 2896
threshold = Param(Params._dummy(), "maxNGrams", "Keyword Score threshold", typeConverter=TypeConverters.toInt)

Expected Behavior

setThreshold() sets the threshold.

Current Behavior

TypeError: Invalid param value given for param "maxNGrams". Could not convert 0.6 to int

Possible Solution

Here's the bug in the code:
python/sparknlp/annotator.py line 2896
threshold = Param(Params._dummy(), "maxNGrams", "Keyword Score threshold", typeConverter=TypeConverters.toInt)

Steps to Reproduce

1. See test case above.

Context

Trying to set the threshold.

Your Environment

• Spark NLP version: 2.6.2
• Apache NLP version: 2.4.5
• Java version (java -version): 8
• Setup and installation (Pypi, Conda, Maven, etc.): conda
• Operating System and version: Databricks
• Link to your project (if any): N/A

[maziyarpanahi]

Hi @tombriles

This threshold is actuallythe maximum length of an extracted keyword. It is an integer value and not float. You need to set an integer value.

[tombriles]

Documentation says:
"setThreshold(float) Each keyword will be given a keyword score greater than 0. (Lower the score better the keyword) Set an upper bound for the keyword score from this method."

https://nlp.johnsnowlabs.com/docs/en/annotators#yakemodel-keywords-extraction

Also, the scala test at https://github.com/JohnSnowLabs/spark-nlp/blob/f3779e000c1620d3a9bdbebf59ea0918f5492440/src/test/scala/com/johnsnowlabs/nlp/annotators/keyword/yake/YakeTestSpec.scala sets the threshold to 0.6f.

[tombriles]

I now see where the confusion came from. ;)

[maziyarpanahi]

You are right, the Python API is totally wrong, it says:

threshold = Param(Params._dummy(), "maxNGrams", "Keyword Score threshold", typeConverter=TypeConverters.toInt)

It uses maxNGrams by mistake which made me think it was Int. This will be fixed in the upcoming release.

Thanks for reporting it :)