Magento2: Jmeter error

[Grohotun]

Using benchmark.jmx from CE setup/performance-toolkit folder

jmeter.bat -n -t benchmark.jmx -j jmeter.log -l jmeter-results.jtl -Jhost=XXXXXX -Jbase_path=/ -Jadmin_path=backend -JsetupAndTearDownThread=10 -Jramp_period=300 -Jusers=1000 -Jorders=100000 -Jadmin_user=admin -Jadmin_password=XXXXXX -Jrequest_protocol=https

After small time test fails.

On jmeter.log I can see error:

WARN o.a.j.p.j.s.BeanShellSampler: Exception executing script. org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval Sourced file: inline evaluation of: ``import java.util.ArrayList; ArrayList categoryUrlKeysList; // If it is first ite . . .

Small investigation, error is here (and on other interation lists)

// If it is first iteration of cycle then recreate category url key list if (1 == Integer.parseInt(vars.get("category_url_key_counter"))) { categoryUrlKeysList = new ArrayList(); props.put("category_url_keys_list", categoryUrlKeysList); props.put("category_url_key", vars.get("category_url_key")); } else { categoryUrlKeysList = props.get("category_url_keys_list"); }

It seems, that if condition is logically correct, but in execution - it fails. (I'm not java developer, but error says, that categoryUrlKeysList isn't created)

If I add ArrayList categoryUrlKeysList; before if condition I'm do not get errors, but get test exit (due to no categories in benchmark workflow)

Same error, if using EE setup/performance-toolkit folder

Using 2.2.3 CE, but it's error/bug in benchmark.jmx
Using Jmeter 2/3/4

[mslabko]

Hi @Grohotun ,
Did you run php bin/magento setup:performance:generate-fixtures setup/performance-toolkit/profiles/ce/small.xml berfore run benchmark?

Provided benchmark.jmx assumes, that you have some data for benchmarking.

Additional info can be found in documentation: https://github.com/magento/magento2/tree/2.2.3/setup/performance-toolkit

[Grohotun]

Yes, products were generated via performance toolkit

[mslabko]

Do you run benchmark.jmx or benchmark_2015.jmx? Regarding "setupAndTearDownThread" in your example it looks like you use benchmark_2015.jmx.

Anyway could you please try to run a test with default paramters and 10 users for frontend pool:
frontendPoolUsers:

jmeter -Jhost=xxxx -JfrontendPoolUsers=10 -t setup/performance-toolkit/benchmark.jmx

After enable listener "View Results Tree" and run tests. In case of errors please provide information with "red" steps from Result Tree

[Grohotun]

@mslabko yes, at the time, when I wrote this issue I was using benchmark_2015.jmx.

I'm getting the same error

2018-03-21 23:10:40,520 ERROR o.a.j.u.BeanShellInterpreter: Error invoking bsh method: eval Sourced file: inline evaluation of: props.put("category_url_key", vars.get("category_url_key")); props.put("category . . . '' : Method Invocation props.put 2018-03-21 23:10:40,520 WARN o.a.j.p.j.s.BeanShellSampler: Exception executing script. org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval Sourced file: inline evaluation of:props.put("category_url_key", vars.get("category_url_key")); props.put("category . . . '' : Method Invocation props.put

[mslabko]

I've checked on 2.2.3 with benchmark.jmx and cannot reproduce it. Could you please enable "View Results Tree" and check results in "Extract categories" fragment. It looks like JMeter cannot find categories in Magento instance (step "SetUp - Get Categories Names and skus")

Also, it will be useful to enable "Debug PostProcessor" (rigght click on SetUp thread: Add -> Post Processors -> Debug PostProcessor) - results will be availabe in Result Tree

[Grohotun]

I'm getting the same error.
Can varnish cause this error?

[mslabko]

We run tests with Varnish too without errors. Do you have errors with disabled Varnish?

[paulie51]

We're getting this same error on 2.2.4, medium profile, benchmark.jmx with the only customisations to the command being overriding the host, and admin_(user|path|password).
Tested on Windows and Linux using jmeter 4.0

[mslabko]

@paulie51 , could you please test with jMeter 3.1?

[paulie51]

Hi @mslabko ,

Yes can confirm that works. I read @Grohotun initial comment about testing 2,3 and 4 and figured it'd be pointless me doing the same.

Thanks,

Paul.

[magento-engcom-team]

@Grohotun @paulie51 @mslabko Closing this ticket now, as from the thread it seems that issue is solved. Feel free to reopen if that's incorrect.

Thank you

[bnymn]

I am getting the same problem.

Magento 2.2.5
JMeter 5
Java 8

Thread Name: setUp Thread Group 1-1
Sample Start: 2018-10-23 11:43:33 BRT
Load time: 6
Connect Time: 0
Latency: 0
Size in bytes: 0
Sent bytes:0
Headers size in bytes: 0
Body size in bytes: 0
Sample Count: 1
Error Count: 1
Data type ("text"|"bin"|""): text
Response code: 500
Response message: org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval  Sourced file: inline evaluation of: ``import java.util.ArrayList;  // If it is first iteration of cycle then recreate  . . . '' : Attempt to resolve method: add() on undefined variable or class name: categoryUrlKeysList


SampleResult fields:
ContentType: 
DataEncoding: null

[magento-engcom-team]

Hi @TomashKhamlai. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• [ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• [ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento-engcom-team give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!
  

• [ ] 5. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento-engcom-team give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• [ ] 6. Add label Issue: Confirmed once verification is complete.

• [ ] 7. Make sure that automatic system confirms that report has been added to the backlog.

[TomashKhamlai]

JMeter 5 is not supported. Use JMeter 3.1

[QuentinFarizon]

Using benchmark.jmx from Magento EE 2.2.8, and Jmeter 3.1, issue is not solved, I'm getting the same error :

2019/04/08 15:58:31 ERROR - jmeter.util.BeanShellInterpreter: Error invoking bsh method: eval   Sourced file: inline evaluation of: ``props.put("category_url_key", vars.get("category_url_key")); props.put("category . . . '' : Method Invocation props.put 
2019/04/08 15:58:31 WARN  - jmeter.protocol.java.sampler.BeanShellSampler: org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval Sourced file: inline evaluation of: ``props.put("category_url_key", vars.get("category_url_key")); props.put("category . . . '' : Method Invocation props.put

[cmuench]

I have the same issue like @QuentinFarizonAfrimarket with the small profile.
Tested in Magento 2.3.1 Commerce Edition with JMeter 5.1.1 and JMeter 5.0.

019-04-17 10:34:00,968 ERROR o.a.j.u.BeanShellInterpreter: Error invoking bsh method: eval  Sourced file: inline evaluation of: ``import java.util.ArrayList;  // If it is first iteration of cycle then recreate  . . . '' : Attempt to resolve method: add() on undefined variable or class name: categoryUrlKeysList
2019-04-17 10:34:00,968 WARN o.a.j.p.j.s.BeanShellSampler: Exception executing script. org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval Sourced file: inline evaluation of: ``import java.util.ArrayList;  // If it is first iteration of cycle then recreate  . . . '' : Attempt to resolve method: add() on undefined variable or class name: categoryUrlKeysList
2019-04-17 10:34:00,968 INFO o.a.j.t.JMeterThread: Shutdown Test detected by thread: setUp Thread Group 1-1