Compromise: Uninterrupted periods hang exponentially longer with each period

Created on 21 Aug 2018  路  4Comments  路  Source: spencermountain/compromise

If in the middle of a string, not the start or end, repeated uninterrupted periods cause compromise to hang exponentially longer with each successive period. I made a script to test this issue, and have provided that and the output in my own test below.

For reference, even a single non-space letter after each period nullifies this issue, offering the speed compromise is known for.

const nlp = require('compromise');

const ROUNDS = 50;
let text = 'Prefixing sentence';
let time;

for(let i=1; i<ROUNDS+1; i++) {
    text += '.';
    let temp_text = text +"Suffixing sentence";
    let temp_time;

    time = process.hrtime();
    nlp(temp_text)
    temp_time = process.hrtime(time);

    console.log(`${i} periods: ${temp_time[0]} seconds, ${temp_time[1]} nanosecs`);
}

Everything seems fine at first, with the output being far less than a second consistently:

1 periods: 0 seconds, 14482379 nanosecs
2 periods: 0 seconds, 2408267 nanosecs
3 periods: 0 seconds, 1872121 nanosecs
4 periods: 0 seconds, 2083712 nanosecs
5 periods: 0 seconds, 1833198 nanosecs

But by the end, the process takes an extreme amount of time. By 53 periods, it would take somewhere around an hour to parse the input.

40 periods: 7 seconds, 11823306 nanosecs
41 periods: 11 seconds, 924131543 nanosecs
42 periods: 18 seconds, 475012145 nanosecs
43 periods: 30 seconds, 544638540 nanosecs
44 periods: 49 seconds, 578529286 nanosecs
45 periods: 79 seconds, 614583931 nanosecs
46 periods: 130 seconds, 54811266 nanosecs
47 periods: 214 seconds, 796373699 nanosecs
48 periods: 343 seconds, 325737932 nanosecs
49 periods: 555 seconds, 488678599 nanosecs
50 periods: 881 seconds, 33217344 nanosecs

bug

Most helpful comment

ah! thank you Julia. Very nice issue.

I'm happy to take a look at this. It is likely a over-eager regular expression.
I will try to track it down.

All 4 comments

ah! thank you Julia. Very nice issue.

I'm happy to take a look at this. It is likely a over-eager regular expression.
I will try to track it down.

found it.

it's tripping on this one

here's a simpler reproduction:

console.time('reg');
let str = 'word......................................Suffixing';
str.match(/(\.\.+)+$/);
console.timeEnd('reg');

yeah, gonna need to take a walk now.

will try to fix it later.

ugh, this still isn't fixed. It's a hard one. I'm gonna punt this to v12, where a lot of this whitespace logic is being re-written.

hey Julia, thanks for your patience on this one. Finally got it fixed in 12.0.0.

console.time('reg')
let str = 'word..........................................Suffixing'
nlp(str).debug()
console.timeEnd('reg')
//reg: 14.908ms

馃槄

Was this page helpful?
0 / 5 - 0 ratings

Related issues

hboylan picture hboylan  路  7Comments

zzj0402 picture zzj0402  路  4Comments

zeke picture zeke  路  3Comments

douglance picture douglance  路  6Comments

bikibird picture bikibird  路  3Comments