Mocha: Passing data dynamically to Mocha

[OSP123]

Prerequisites

• [x] Checked that your issue hasn't already been filed by cross-referencing issues with the faq label
• [x] Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
• [x] 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
• [x] Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with: node node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend avoiding the use of globally installed Mocha.

Description

Hey ya'll, I'm currently working on an automated grader that takes in a Github repo and then passes the content to the test files. I'm finding it difficult to programmatically send information to the file before using mocha.addFile(). What's the best approach for solving this?

I've looked at vm, but it doesn't allow variables to be passed into it. Only other solution I can think of is having to build a file and then destroy it after mocha tests are run, but that could also run into weird race conditions.

I was messing with the loadFiles method, but again, the vm doesn't allow me to pass in variables to it. So, I can't pass the dynamically generated object for the test file to use.

Steps to Reproduce

1. Create Mocha test file.
2. Have a file such as runMocha.js that will execute the Mocha test programmatically.
3. Have Mocha test file dynamically take data from object passed in before executing Mocha.run()

Expected behavior: [What you expect to happen]

I'd like the files to be able to take in data like function arguments.

Actual behavior: [What actually happens]

I guess the functionality doesn't exist?

Reproduces how often: [What percentage of the time does it reproduce?]

Versions

• The output of mocha --version and node node_modules/.bin/mocha --version:
• The output of node --version:
• The version and architecture of your operating system:
• Your shell (bash, zsh, PowerShell, cmd, etc.):
• Your browser and version (if running browser tests):
• Any other third party Mocha related modules (with versions):
• The code transpiler being used:

Additional Information

[Bamieh]

Hello,

I will try to help you as much as possible, given that I do not fully understand your case without some code example or more details.

The best approach to unit testing is to provide fixtures rather than dynamic content. You are not testing if github is returning data or returning correct data (that is github's job) so saving their response in a fixture file and using that file to test your code is the best approach.
Fixtures also provide consistency which is key in unit testing (ie. if github was down your test will fail while they should not).

in your use-case, VMs will only complicate things, basically in your use-case you are only using VM's to avoid setting to the main nodeJS globals object, not the perfect excuse to use VMs since they complicate things _a lot_.

the vm doesn't allow me to pass in variables to it

It definitely does. you pass a context object to the node vm which is accessible to the vm script. Without running into much details:

const context = {
    Buffer,
    clearImmediate,
    clearInterval,
    clearTimeout,
    setImmediate,
    setInterval,
    setTimeout,
    console,
    process,
    myVar: 11,
};

context.global = context;
const compiledWrapper = vm.runInNewContext(wrapper, context, options);

[OSP123]

Thank you so much for your response!

So, the background of the application is that I'm creating an automated grader for student homeworks.

• The student homeworks are code homeworks that are hosted on GitHub public repos.
• I fetch these repos and then pass the base64 hash representation of each repo into an object.
• I created a test file that was using a test represent tation of the JSON file for testing

Ah, I do agree with the fixtures approach, but the data being sent back from GitHub is different for each student.

Here is some of the code to test an HTML file to make sure it matches our expectations:

const chai = require('chai');
const chaiSubset = require('chai-subset');
chai.use(chaiSubset);
const expect = chai.expect;
const base64 = require('base-64');
const HTML_CONTENT = require('../../00-sandbox-files/basic-portfolio-solution.json').html;
const CSS_CONTENT = require('../../00-sandbox-files/basic-portfolio-solution.json').css;
const decodedCSS = base64.decode(CSS_CONTENT[1].content);
const cheerio = require('cheerio');
const juice = require('juice');

let decodedHTMLcontact;
let decodedHTMLindex;
let decodedHTMLportfolio;

for (const obj in HTML_CONTENT) {
  if (HTML_CONTENT[obj].path == "contact.html") {
    decodedHTMLcontact = base64.decode(HTML_CONTENT[obj].content);
  } else if (HTML_CONTENT[obj].path == "index.html") {
    decodedHTMLindex = base64.decode(HTML_CONTENT[obj].content);
  } else if (HTML_CONTENT[obj].path == "portfolio.html") {
    decodedHTMLportfolio = base64.decode(HTML_CONTENT[obj].content);
  }
}

tests = function (html, css) {
  describe('HTML Elements tests that should pass for contact.html', function () {
    let $ = cheerio.load(decodedHTMLcontact);
    describe('HTML Elements that should exist in contact.html', function () {
      it('should contain a header element', function () {
        expect($('body').find('header').length).to.equal(1);
      });
      it('should contain a section element', function () {
        expect($('body').find('section').length).to.equal(1);
      });
      it('should contain several anchor elements', function () {
        expect($('nav').find('a').length).to.be.at.least(3, 'You need an additional anchor elements for your navigation elements');
      });
      it('should contain an h1 element', function () {
        expect($('body').find('h1').length).to.equal(1);
      });
      it('should contain a form element', function () {
        expect($('body').find('form').length).to.equal(1);
      });
      it('should contain a footer element', function () {
        expect($('body').find('footer').length).to.equal(1);
      });
    });

Note that the HTML_CONTENT was hard coded. My goal is to execute this programmatically from another file, runMocha.js, so that the test files use the Github data for HTML_CONTENT instead of local data we set up for our own testing. Here is the runMocha.js file so far:

const Mocha = require('mocha'),
         fs = require('fs'),
         vm = require("vm");
        tmp = require('tmp');


module.exports = {

  // Run the tests and have info about what can be returned
  runTests: function(contentObj, repo) {
    homeworkRouter(contentObj, repo);
    console.log("Content Object", contentObj);
    console.log("Repo", repo);
    mocha.run()
    .on('fail', function (test, err) {
      console.log('Test fail');
      console.log(err);
    })
    .on('end', function () {
      console.log('All done');
    });
  }
}

Also, I had no idea you could do that! That's so cool. OK, so considering I don't need to change the global JS object (I'm passing data that's constantly going to change), would it be better to just pass in the data as part of context?

[Bamieh]

@OSP123 awesome. Im excited every time i see some code being used for teaching!

The least code change is to collect all student data in an array and store it in a global. The context in a VM aliased by "global" is exactly as the global object in the main app, but inside a VM. So if your code works using the global object, it should work with globals in VMs, but i highly recommend staying away from VMs for this code (IMHO i believe it adds unnecessary complexity and no real added benefit for this use case).

fetchStudentsData().then(studentsData => {
   global.studentsData = studentsData;
}).then(_ => {
   mocha.run()
});

What I'd suggest you do the data fetching in a before and loop over the students to create the cases dynamically, this way you do not have to run mocha programmatically as well, just run it from the cli.

describe('Automated Grading', function() {
  let studentsData;
  before(async function() {
     const students = await getStudentsList();
     studentsData = Promise.all(students.map(student => {
        return getStudentData(student);
     }));
  });

  studentsData.forEach(student => {
     describe(`Grading for ${student.name}`, function() {
         it('...', function() {
           //...
         }); 
     })
  });
});

[OSP123]

Yeah, I'm really excited about it as well! The students and the TAs can pass in a github url to the the search bar and it comes back with grading information. I can't give away too much now (we are pushing for as much open source as possible, believe me), but this is going to impact a lot of people.

Unfortunately, because this is all run from a microservice, I don't think we can use the CLI unless we use exec() or something like that, no? This is all being run through an Express/Node application. So, the logic flow is like the following:

1. Client submits their Github url. Request is made to Express/Node application.
2. Express/Node application takes repo and username and makes request to Github API for data and adds the content of the files to an object as base64.
3. The object with the files are passed to the relevant test files and then executed.
4. The results are processed and preliminary grades are created. These are then sent back to the client.

Considering the logic flow, I'm thinking the easiest way to do this would be to use VM when running the runMocha.js file instead of running Mocha via exec() and executing files from cli, no?

[OSP123]

@Bamieh Any advice on execution?

[Bamieh]

@OSP123 try to follow the above hints, and avoid VMs. I'd point you to the wiki pages for instructions on how to run mocha programmatically https://github.com/mochajs/mocha/wiki/Using-mocha-programmatically

[Bamieh]

@OSP123 Buddy I will close this since this is not related to mocha per se, it's more of a programming challenge. Please feel free to open a new issue whenever you find something specifically related to mocha. I am on gitter if you need personal help with this.

[OSP123]

Yeah, I'd just like to avoid passing anything into a global, especially considering there are going to be thousands of different requests. But I do agree that it's more of a programming problem that just involves Mocha. However, perhaps it can be something that can be done open further for those that are using Mocha programmatically? I'll try and come up with something and then see if I can submit a PR. Thanks for the help!

[thusfar7]

@OSP123 Did you figure out a nice solution for this? Would you be willing to share it with the rest of us?

[OSP123]

Oh man, we came up with something, but I forgot what it was. We've had to shelve the project for other priorities.