Javaparser: When lexical preservation is enabled, removing the second instance of a cloned statement within a block fails with IllegalArgumentException

[SimonBaars]

I parse the following code:

public class Clone1 {

    public static void main(String[] args) {
        System.out.println("I'm a clone1");
        System.out.println("I'm a clone2");
        System.out.println("I'm a clone3");
        System.out.println("I'm a clone4");
        System.out.println("I'm a clone5");
        System.out.println("I'm a clone6");
        System.out.println("I'm a clone7");
        System.out.println("I'm a clone8");
        System.out.println("I'm a clone9");
        System.out.println("I'm a clone10");
        System.out.println("I'm not a clone!");
        System.out.println("I'm a clone1");
        System.out.println("I'm a clone2");
        System.out.println("I'm a clone3");
        System.out.println("I'm a clone4");
        System.out.println("I'm a clone5");
        System.out.println("I'm a clone6");
        System.out.println("I'm a clone7");
        System.out.println("I'm a clone8");
        System.out.println("I'm a clone9");
        System.out.println("I'm a clone10");
    }

}

I call Node.remove(Node) on the BlockStmt inside the method, passing the final Statement (System.out.println("I'm a clone10");) of the block statement as an argument. I then get the following exception:

java.lang.IllegalArgumentException: fromIndex(42) > toIndex(21)
    at java.util.SubList.<init>(AbstractList.java:624)
    at java.util.AbstractList.subList(AbstractList.java:484)
    at com.github.javaparser.printer.lexicalpreservation.LexicalDifferenceCalculator$CalculatedSyntaxModel.sub(LexicalDifferenceCalculator.java:44)
    at com.github.javaparser.printer.lexicalpreservation.DifferenceElementCalculator.calculate(DifferenceElementCalculator.java:123)
    at com.github.javaparser.printer.lexicalpreservation.LexicalDifferenceCalculator.calculateListRemovalDifference(LexicalDifferenceCalculator.java:94)
    at com.github.javaparser.printer.lexicalpreservation.LexicalPreservingPrinter$Observer.concreteListChange(LexicalPreservingPrinter.java:266)
    at com.github.javaparser.ast.observer.PropagatingAstObserver.listChange(PropagatingAstObserver.java:72)
    at com.github.javaparser.ast.NodeList.lambda$notifyElementRemoved$2(NodeList.java:470)
    at java.util.ArrayList.forEach(ArrayList.java:1257)
    at com.github.javaparser.ast.NodeList.notifyElementRemoved(NodeList.java:470)
    at com.github.javaparser.ast.NodeList.remove(NodeList.java:155)
    at com.github.javaparser.ast.stmt.BlockStmt.remove(BlockStmt.java:105)
    at com.simonbaars.clonerefactor.refactoring.ExtractMethodFromSequence.lambda$3(ExtractMethodFromSequence.java:80)
    at java.util.ArrayList.forEach(ArrayList.java:1257)
    at com.simonbaars.clonerefactor.refactoring.ExtractMethodFromSequence.removeLowestNodes(ExtractMethodFromSequence.java:80)
    at com.simonbaars.clonerefactor.refactoring.ExtractMethodFromSequence.lambda$0(ExtractMethodFromSequence.java:57)
    at java.util.ArrayList.forEach(ArrayList.java:1257)
    at com.simonbaars.clonerefactor.refactoring.ExtractMethodFromSequence.tryToExtractMethod(ExtractMethodFromSequence.java:57)
    at com.simonbaars.clonerefactor.refactoring.ExtractMethodFromSequence.refactor(ExtractMethodFromSequence.java:114)
    at com.simonbaars.clonerefactor.ast.CloneParser.parse(CloneParser.java:47)
    at com.simonbaars.clonerefactor.Main.cloneDetection(Main.java:53)
    at com.simonbaars.clonerefactor.Main.cloneDetection(Main.java:38)
    at com.simonbaars.clonerefactor.Main.cloneDetection(Main.java:34)
    at com.simonbaars.clonerefactor.helper.TypeTest.testProject(TypeTest.java:30)
    at com.simonbaars.clonerefactor.types.Type1Testcases.testSingleFile(Type1Testcases.java:144)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at junit.framework.TestCase.runTest(TestCase.java:154)
    at junit.framework.TestCase.runBare(TestCase.java:127)
    at junit.framework.TestResult$1.protect(TestResult.java:106)
    at junit.framework.TestResult.runProtected(TestResult.java:124)
    at junit.framework.TestResult.run(TestResult.java:109)
    at junit.framework.TestCase.run(TestCase.java:118)
    at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:124)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:41)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:541)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:763)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:463)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:209)

I think that this is caused by the fact that another equal statement exists in the block and I remove the second one.

I use the latest version of javaparser (3.14.5) and use the following parse configuration:

CombinedTypeSolver combinedTypeSolver = new CombinedTypeSolver(new ReflectionTypeSolver(), new JavaParserTypeSolver(sourceRoot));

        addLibrariesToTypeSolver(path, combinedTypeSolver);

        final ParserConfiguration config = new ParserConfiguration()
                .setLexicalPreservationEnabled(true)
                .setStoreTokens(true)
                .setSymbolResolver(new JavaSymbolSolver(combinedTypeSolver));

[edefazio]

This seems definitely related to the integration of the LexicalPreservingPrinter & Symbol Solver (parsing without these (2) things works fine):
````java
import com.github.javaparser.ast.stmt.Statement;
import draft.java._class;
import draft.java._method;
import draft.java.proto.$stmt;
import junit.framework.TestCase;

public class RemoveStmtsTest extends TestCase {

public static class Clone1 {

    public static void main(String[] args) {
        System.out.println("I'm a clone1");
        System.out.println("I'm a clone2");
        System.out.println("I'm a clone3");
        System.out.println("I'm a clone4");
        System.out.println("I'm a clone5");
        System.out.println("I'm a clone6");
        System.out.println("I'm a clone7");
        System.out.println("I'm a clone8");
        System.out.println("I'm a clone9");
        System.out.println("I'm a clone10");
        System.out.println("I'm not a clone!");
        System.out.println("I'm a clone1");
        System.out.println("I'm a clone2");
        System.out.println("I'm a clone3");
        System.out.println("I'm a clone4");
        System.out.println("I'm a clone5");
        System.out.println("I'm a clone6");
        System.out.println("I'm a clone7");
        System.out.println("I'm a clone8");
        System.out.println("I'm a clone9");
        System.out.println("I'm a clone10");
    }
}

public void testC(){
     _class _c = _class.of(Clone1.class);
    $stmt $s = $stmt.of( ()->System.out.println("I'm a clone10") );
    assertEquals(2, $s.count(_c));
    Statement st = $s.selectFirstIn(_c).astStatement;
    //manually
    _c.getMethod("main").getBody().ast().remove(st);
    assertEquals(1, $s.count(_c));//make sure I didnt remove the second one
    System.out.println( _c);
}

}
````

[matozoid]

I'd guess the lex pres printer, because the symbol solver doesn't modify the AST.

[tkoft]

I came across another case of this, in case it helps with debugging. I was trying to extract the contents of a then block into the parent block of the if statement. Here's a simplified example, splicing the contents of an inner block into its parent block:

    String contents =
        "public class Example {\n"
            + "  public Example() {\n"
            + "    System.out.println(\"f\");\n"
            + "    int anything = 0;\n"
            + "    {\n"
            + "      System.out.println(\"f\");\n"
            + "    }\n"
            + "  }\n"
            + "}\n";
    CompilationUnit compilationUnit = StaticJavaParser.parse(contents);
    LexicalPreservingPrinter.setup(compilationUnit);

    List<BlockStmt> blocks = compilationUnit.findAll(BlockStmt.class);
    BlockStmt outerBlock = blocks.get(0);
    BlockStmt innerBlock = blocks.get(1);

    // splice contents of inner block into outer block
    NodeList<Statement> siblings = outerBlock.getStatements();
    int myIndex = siblings.indexOf(innerBlock);
    NodeList<Statement> newStatements = new NodeList<>(siblings.subList(0, myIndex));
    newStatements.addAll(innerBlock.getStatements());
    newStatements.addAll(new NodeList<>(siblings.subList(myIndex + 1, siblings.size())));
    outerBlock.setStatements(newStatements);
    System.out.println(compilationUnit);

I found that the extraneous int anything = 0; line was required for the error to reproduce.

[matozoid]

See #2290