Ballerina-lang: METHOD_NOT_FOUND No such public method when tests are enabled

[BuddhiWathsala]

Description:
When I run the following piece of code without any tests, it will compile and even runs fine.

import ballerina/java as balJava;

public function newInstance(public handle class, int ...dimensions) returns handle = @balJava:Method {
    class: "java.lang.reflect.Array",
    paramTypes: ["java.lang.Class", {class: "int", dimensions:1}]
} external;

However, when I run with tests it returned the following error. Note that, even it has empty tests directory it gives the following error.

Creating balos
        target/balo/sample-2020r1-any-0.1.0.balo
error: buddhi/sample:0.1.0::main.bal:13:1: {ballerina/java}METHOD_NOT_FOUND 'No such public method 'newInstance' with '3' parameter(s) found in class 'class java.lang.reflect.Array''

Steps to reproduce:

Run the following code with and without tests.

import ballerina/java as balJava;

public function newInstance(public handle class, int ...dimensions) returns handle = @balJava:Method {
    class: "java.lang.reflect.Array",
    paramTypes: ["java.lang.Class", {class: "int", dimensions:1}]
} external;

Affected Versions:
Ballerina 2.0.0-Preview3

[kalaiyarasiganeshalingam]

Received the same kind of error by the following codes when compiling the task module

function attachExternal(Listener task, service s, any... attachments) returns ListenerError? = @java:Method {
    name: "attach",
    class: "org.ballerinalang.task.actions.TaskActions"
} external;

Equivalant java method in the TaskActions.java

public static Object attach(ObjectValue taskListener, ObjectValue service, Object... attachments) {}

Error:

Compiling source
        ballerina/task:1.1.0
error: ballerina/task:1.1.0::listener.bal:157:1: {ballerina/java}METHOD_NOT_FOUND 'No such public method 'attach' with '4' parameter(s) found in class 'class org.ballerinalang.task.actions.TaskActions''

[Dilhasha]

Update:

This happens when trying to generate test module jar during compile time. The reason why it happens only when tests are present is because test module generation is skipped otherwise. See [1].

[1] https://github.com/ballerina-platform/ballerina-lang/blob/v2.0.0-Preview3/compiler/ballerina-lang/src/main/java/org/wso2/ballerinalang/compiler/bir/codegen/CodeGenerator.java#L117

[Dilhasha]

Update:

The reason for this behavior is,
When generating test module jar at [2], the functions that have already been wrapped using BirFunctionWrapper at [3] are being processed again which leads to showing a wrong number of params for the function.

[2] https://github.com/ballerina-platform/ballerina-lang/blob/v2.0.0-Preview3/compiler/ballerina-lang/src/main/java/org/wso2/ballerinalang/compiler/bir/codegen/CodeGenerator.java#L126

[3] https://github.com/ballerina-platform/ballerina-lang/blob/v2.0.0-Preview3/compiler/ballerina-lang/src/main/java/org/wso2/ballerinalang/compiler/bir/codegen/JvmPackageGen.java#L770

Note:
This issue occurs when there are arrays in the method params.

[BuddhiWathsala]

@Dilhasha any fix that we can implement soon?. This is a blockage for most of the stdlibs that we started to migrate.

[Dilhasha]

I'm looking into skipping the non-test function processing at BirFunctionWrapper. I will update ASAP.

[Dilhasha]

@warunalakshitha If you have an idea on fixing this, please add.

[Dilhasha]

During the BirFunction wrapping for module jar generation, the non test functions in testable package functions also get updated. That leads to the params count being wrong leading to the mentioned issue.

I tried the possible solutions listed at [4] but was unable to get those working.

[4] https://docs.google.com/document/d/1gSAJePLQurXhXqmrkThysBukrU1RLU7eC70OiOC1Hmo/edit?usp=sharing

[BuddhiWathsala]

https://docs.google.com/document/d/1GuVrnsYo5u8uUa21pKNYB_MTQ944SaWA8ZhWB5WT5k4/edit?usp=sharing

Don't have permission to see the doc.

[Dilhasha]

https://docs.google.com/document/d/1gSAJePLQurXhXqmrkThysBukrU1RLU7eC70OiOC1Hmo/edit?usp=sharing

Don't have permission to see the doc.

Please check now. Sorry about that.

[BuddhiWathsala]

@Dilhasha, the ballerina/io module also getting a bad sad error when I add the tests directory with the following logs.

[2020-08-24 20:15:26,763] SEVERE {b7a.log.crash} - null 
java.lang.NullPointerException
    at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.validateBuiltinTypeAnnotationAttachment(SemanticAnalyzer.java:2931)
    at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.visit(SemanticAnalyzer.java:399)
    at org.wso2.ballerinalang.compiler.tree.BLangTypeDefinition.accept(BLangTypeDefinition.java:110)
    at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.analyzeNode(SemanticAnalyzer.java:2610)
    at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.analyzeNode(SemanticAnalyzer.java:2578)
    at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.analyzeDef(SemanticAnalyzer.java:2570)
    at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.visit(SemanticAnalyzer.java:256)
    at org.wso2.ballerinalang.compiler.tree.BLangPackage.accept(BLangPackage.java:161)
    at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.analyze(SemanticAnalyzer.java:230)
    at org.wso2.ballerinalang.compiler.CompilerDriver.typeCheck(CompilerDriver.java:322)
    at org.wso2.ballerinalang.compiler.CompilerDriver.compile(CompilerDriver.java:256)
    at org.wso2.ballerinalang.compiler.CompilerDriver.compilePackageSymbol(CompilerDriver.java:248)
    at org.wso2.ballerinalang.compiler.CompilerDriver.compilePackage(CompilerDriver.java:140)
    at org.wso2.ballerinalang.compiler.Compiler.compilePackages(Compiler.java:166)
    at org.wso2.ballerinalang.compiler.Compiler.compilePackage(Compiler.java:174)
    at org.wso2.ballerinalang.compiler.Compiler.compile(Compiler.java:91)
    at org.wso2.ballerinalang.compiler.Compiler.build(Compiler.java:99)
    at org.ballerinalang.packerina.task.CompileTask.execute(CompileTask.java:64)
    at org.ballerinalang.packerina.TaskExecutor.executeTasks(TaskExecutor.java:38)
    at org.ballerinalang.packerina.cmd.BuildCommand.execute(BuildCommand.java:433)
    at java.util.Optional.ifPresent(Optional.java:159)
    at org.ballerinalang.tool.Main.main(Main.java:57)

I'm not quite sure whether it is also related to this issue.

Steps to reproduce

Build this branch of the migrated module(https://github.com/ballerina-platform/module-ballerina-io/tree/bdev) using

./gradlew clean build

Please check this out too.

[Dilhasha]

@Dilhasha, the ballerina/io module also getting a bad sad error when I add the tests directory with the following logs.

[2020-08-24 20:15:26,763] SEVERE {b7a.log.crash} - null 
java.lang.NullPointerException
  at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.validateBuiltinTypeAnnotationAttachment(SemanticAnalyzer.java:2931)
  at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.visit(SemanticAnalyzer.java:399)
  at org.wso2.ballerinalang.compiler.tree.BLangTypeDefinition.accept(BLangTypeDefinition.java:110)
  at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.analyzeNode(SemanticAnalyzer.java:2610)
  at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.analyzeNode(SemanticAnalyzer.java:2578)
  at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.analyzeDef(SemanticAnalyzer.java:2570)
  at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.visit(SemanticAnalyzer.java:256)
  at org.wso2.ballerinalang.compiler.tree.BLangPackage.accept(BLangPackage.java:161)
  at org.wso2.ballerinalang.compiler.semantics.analyzer.SemanticAnalyzer.analyze(SemanticAnalyzer.java:230)
  at org.wso2.ballerinalang.compiler.CompilerDriver.typeCheck(CompilerDriver.java:322)
  at org.wso2.ballerinalang.compiler.CompilerDriver.compile(CompilerDriver.java:256)
  at org.wso2.ballerinalang.compiler.CompilerDriver.compilePackageSymbol(CompilerDriver.java:248)
  at org.wso2.ballerinalang.compiler.CompilerDriver.compilePackage(CompilerDriver.java:140)
  at org.wso2.ballerinalang.compiler.Compiler.compilePackages(Compiler.java:166)
  at org.wso2.ballerinalang.compiler.Compiler.compilePackage(Compiler.java:174)
  at org.wso2.ballerinalang.compiler.Compiler.compile(Compiler.java:91)
  at org.wso2.ballerinalang.compiler.Compiler.build(Compiler.java:99)
  at org.ballerinalang.packerina.task.CompileTask.execute(CompileTask.java:64)
  at org.ballerinalang.packerina.TaskExecutor.executeTasks(TaskExecutor.java:38)
  at org.ballerinalang.packerina.cmd.BuildCommand.execute(BuildCommand.java:433)
  at java.util.Optional.ifPresent(Optional.java:159)
  at org.ballerinalang.tool.Main.main(Main.java:57)

I'm not quite sure whether it is also related to this issue.

Steps to reproduce

Build this branch of the migrated module(https://github.com/ballerina-platform/module-ballerina-io/tree/bdev) using

./gradlew clean build

Please check this out too.

@BuddhiWathsala Can you please create a separate issue. These issues don't seem related.

[Dilhasha]

Update:
Had a call to review the fix with @Kishanthan and @warunalakshitha.

We discussed the following issues.

1. Rewrapping the non test functions again during test module jar generation.
2. Rewrapping introduce errors only when there are REST params in the function signature.

We decided to keep this PR https://github.com/ballerina-platform/ballerina-lang/pull/25388 and also, try to figure out why it introduces error only when trying to rewrap REST parameters.

For the PR https://github.com/ballerina-platform/ballerina-lang/pull/25388, the function mocking related tests are failing due to [1]. I'm looking into that as well.

[1] https://github.com/ballerina-platform/ballerina-lang/blob/master/misc/testerina/modules/testerina-core/src/main/ballerina/src/test/mock.bal#L368

[Dilhasha]

The rest parameters get updated at [2], adding an additional param for theparam list.. But, due to clone ref, all other linked functions get updated.
[3]https://github.com/ballerina-platform/ballerina-lang/blob/aster/compiler/ballerina-lang/src/main/java/org/wso2/ballerinalang/compiler/bir/codegen/interop/JMethodRequest.java#L81