Supercollider: .remove obstructs .do iterations in some Collection Subclasses. Depending on type/class and key of entry

[salkin-mada]

Environment

• SuperCollider version: 3.10.2, 3.10.4-rc1 and 3.11.0-beta1
• Operating system: Debian 10 / Windows 10

Steps to reproduce (minimal)

~set = IdentitySet.new;
~set.add(\mango)
~set.add(\roe)
~set.do{arg entry; entry.postln; ~set.remove(entry)}; ~set;
// -> Set[  ]
~set = IdentitySet.new;
~set.add(\mango)
~set.add(\banana)
~set.do{arg entry; entry.postln; ~set.remove(entry)}; ~set;
// -> Set[ banana ]
~set = IdentitySet.new;
~set.add(\mango)
~set.add(\banana)
~set.add(\litchi)
~set.add(\kartoffel)
~set.do{arg entry; entry.postln; ~set.remove(entry)}; ~set;
// -> Set[ banana ]
~set = IdentitySet.new;
~set.add(\mango)
~set.add(\banan) // removed last a in banana
~set.add(\litchi)
~set.add(\kartoffel)
~set.do{arg entry; entry.postln; ~set.remove(entry)}; ~set; // .. what??
// -> Set[  ]

please try same code but exchange IdentitySet.new with other OrderedIdentitySet.new
these behave differently with the given example above. The OrderedIdentitySet keeps bith banana and kartoffel and removing the last "a" in \banana makes no difference with an OrderedIdentitySet

also please try same code but exchange IdentitySet.new with other Collection subclasses and subclasses of those subclasses.
Dictionary and Array seems to work fine. So note that returned result varies with Class usage.
Set behaves differently compared to List. Set removes all but \banana, List removes half of entries going from 4 entries : to 2 entries : to 1 entry : to empty (maybe related to OrderedIdentitySet which behaves in the same way)

Expected vs. actual behavior

.do should iterate over all entries, it does not when using .remove inside the do function.
in addition this weird behavior seems to be related to the number of entries in the Collection and the names of these entries.. and what type/class the entries are. Also sometimes the key/name of variable that holds the Collection makes a difference...
Please try next part of Bug report.

Steps to reproduce (full mayhem)

~set = IdentitySet.new;
[\kickAddR, \snrR, \hats, \lowsub, \kicksR, \kickAddL, \snrL, \kicksL, \additivis].do{arg name; ~set.add(name)}
~set.do{arg name; ~set.remove(name)}
// -> IdentitySet[ snrL ]
~set.do{arg name; ~set.remove(name)}
// -> IdentitySet[  ]

// switching \snrL and \kickAddL placement in the array
~set = IdentitySet.new;
[\kickAddR, \snrR, \hats, \snrL, \kicksR, \kickAddL, \lowsub, \kicksL, \additivis].do{arg name; ~set.add(name)}
~set.do{arg name; ~set.remove(name.asSymbol)}
// -> IdentitySet[ kickAddL ]

// switching \snrL and \kickAddL placement in the array + changing \kickAddL to \kickAdd
~set = IdentitySet.new;
[\kickAddR, \snrR, \hats, \snrL, \kicksR, \kickAdd, \lowsub, \kicksL, \additivis].do{arg name; ~set.add(name)}
~set.do{arg name; ~set.remove(name.asSymbol)}
// -> IdentitySet[  ]

// try running these three lines several times, please note the .scramble
~set = IdentitySet.new;
[\kickAddR, \snrR, \hats, \snrL, \kicksR, \kickAddL, \lowsub, \kicksL, \additivis].scramble.do{arg name; ~set.add(name)}
~set.do{arg name; ~set.remove(name.asSymbol)}
// -> IdentitySet[ kickAddL ]
// or
// -> IdentitySet[ snrL ]

// here using same array as first examble but with IdentityBag
~set = IdentityBag.new;
[\kickAddR, \snrR, \hatss, \lowsub, \kicksR, \kickAddL, \snrL, \kicksL, \additivis].do{arg name; ~set.add(name)}
~set.do{arg name; ~set.remove(name.asSymbol)}
// -> IdentityBag[ hatss, snrL ]

~set = IdentityBag.new;
[\kickAddR, \snrR, \hatss, \lowsub, \kicksR, \kickAddL, \snrL, \kicksL, \additivis].scramble.do{arg name; ~set.add(name)}
~set.do{arg name; ~set.remove(name.asSymbol)}
// -> IdentityBag[ hatss, snrL ]
// -> IdentityBag[ snrR, snrL ]
// -> IdentityBag[ snrR, kickAddL ]
// etc.. always two left in IdentityBag

// doing exact same thing as above but giving IdentityBag another global variable key
~whatLOL = IdentityBag.new;
[\kickAddR, \snrR, \hatss, \lowsub, \kicksR, \kickAddL, \snrL, \kicksL, \additivis].scramble.do{arg name; ~set.add(name)}
~whatLOL.do{arg name; ~set.remove(name.asSymbol)}
// -> IdentityBag[  ]

// with IdentityDictionary .do seems to always iterate over all key entries
~set = IdentityDictionary.new;
[\kickAddR, \snrR, \hats, \lowsub, \kicksR, \kickAddL, \snrL, \kicksL, \additivis].do{arg name; ~set.put(name, 123.rand)}
~set.keys.do{arg name; ~set.removeAt(name)}; ~set;


// trying with other keys on a IdentitySet
~set = IdentitySet.new;
[\genericName1, \genericName2, \genericName3, \genericName4, \genericName5, \genericName6, \genericName7, \genericName8, \genericName9].scramble.do{arg name; ~set.add(name)}
~set.do{arg name; ~set.remove(name.asSymbol)}
// -> IdentitySet[ ]
// no problem here, always an empty IdentitySet after .do and .remove

// other keys, with same code as above
// try running these lines several times, note the .scramble
~set = IdentitySet.new;
[\name1, \name1, \name3, \name4, \name5, \name6, \name7, \name8, \name9].scramble.do{arg name; ~set.add(name)}
~set.do{arg name; name.postln; ~set.remove(name.asSymbol)}
// -> IdentitySet[ name5, name7, name3 ]
// -> IdentitySet[ name5, name1, name3 ]
// -> IdentitySet[ name8, name7, name3 ]
// etc.. always three left in IdentitySet

// printing what arrives inside the .do func
~set = IdentitySet.new;
[\name1, \name1, \name3, \name4, \name5, \name6, \name7, \name8, \name9].scramble.do{arg name; ~set.add(name)}
~set.do{arg name; name.postln; ~set.remove(name.asSymbol)}
/*
name4
name5
name7
name3
name6
*/
// -> IdentitySet[ name8, name1, name9 ]
~set.do{arg name; name.postln; ~set.remove(name.asSymbol)}
/*
name8
name1
name9
*/
// -> IdentitySet[  ]


// more weird/fishy behavior examples with IdentitySet
~set = IdentitySet.new;
['qwe','wer','ert','rty','tyu','yui','uio','iop','asd','sdf','dfg','fgh'].scramble.do{arg name; ~set.add(name)}
~set.do{arg name; ~set.remove(name.asSymbol)}
// seems to always return
// -> IdentitySet[ sdf ]
// or
// -> IdentitySet[ qwe ]


// try running the following lines over and over again.
// create random amount of random keys
~set = IdentitySet.new;
rrand(10,2000).do{~set.add(($a.ascii..$z.ascii).scramble.resamp0(rrand(1,26)).asAscii.asSymbol)};
~set.do{arg name; ~set.remove(name.asSymbol)}; // run this line until the ~set is empty
// here we see that the capability of .do to iterate over the full IdentitySet changes quite alot with amount of keys/entries in the ~set

// again .do iterates fully on IdentityDictionary, even with high numbers of entries
~set = IdentityDictionary.new;
200.do{~set.put(($a.ascii..$z.ascii).scramble.resamp0(rrand(1,26)).asAscii.asSymbol)};
~set.do{arg name; ~set.removeAt(name.asSymbol)};

// trying very first example but with String, again run these three lines several times and be amazed.
// completely random result.. sometimes empty IdentitySet sometimes one or several entries left. which ones is random.
~set = IdentitySet.new;
["kickAddR", "snrR", "hats", "lowsub", "kicksR", "kickAddL", "snrL", "kicksL", "additivis"].do{arg name; ~set.add(name.asString)}
~set.do{arg name; ~set.remove(name.asString)}

// trying without Identity
~set = Set.new;
rrand(10,2000).do{~set.add(($a.ascii..$z.ascii).scramble.resamp0(rrand(1,26)).asAscii.asSymbol)};
~set.do{arg name; ~set.remove(name.asSymbol)};
// same behavior. .do does not iterate fully

~set = Bag.new;
rrand(10,2000).do{~set.add(($a.ascii..$z.ascii).scramble.resamp0(rrand(1,26)).asAscii.asSymbol)};
~set.do{arg name; ~set.remove(name.asSymbol)};

[brianlheim]

this is expected behavior -- it's generally not good to modify a collection while you're iterating it. try using one of the methods that removes elements based on a condition, or collecting the elements you want to remove and removing them after do

[salkin-mada]

@brianlheim Is this behavior documented somewhere in the help docs? Cant see a mention of this in the Collection helpfile

[brianlheim]

feel free to add documentation if you think it would help!