Hydrogen: Build on Travis does fail due to tests

[theGreatWhiteShark]

Hydrogen version * : 538eaeb
Operating system + version : Linux & MacOS
Audio driver + version :

---

See https://travis-ci.org/hydrogen-music/hydrogen/jobs/483627041

I'm not quite sure what happened here since there wasn't a commit in a while. My lucky guess is that Travis did decrease permissions of the user running in the virtual box to strengthen their security. But I'm not into Travis (GitLab+Docker is so much easier to use) and have no time for at least a month to look into this issue.

As a quick fix, commenting out line 58 in .travis will stop the tests to be executed and to result in failed jobs. But this will ultimately defeat the entire purpose of having automated testing. (But having "build failed" on master isn't nice either.)

For affected pull requests see e.g. https://github.com/hydrogen-music/hydrogen/pull/673

[elpescado]

Hi,

Those tests were written by me. Those tests work by loading a song, exporting it to MIDI file and then comparing that file to reference MIDI file. Since there hasn't been any changes in codebase lately, I too think that the tests itself are faulty. I wouldn't blame security though, the files are still created but do not match reference renderings.

Result and reference files are compared bytewise. That's not 100% correct, since, as far as I know, one song can have more than one equivalent representation. For example, playing two notes at the same time is represented by two Note On messages, and those messages may appear in any order. I suspect that might be the case here.

Unfortunately, to confirm this, generated files should be examined but Travis environment is wiped after job finishes, so there's currently no way to do that. I have a solution, so I'll reach out to developers.

In the meantime, It's best to skip (comment out) offending test cases. I hope I can manage to submit PR with that soon.

[theGreatWhiteShark]

Nice idea.

Regarding the generated files: one could give Travis an encrypted SSH key so it has write access for the repo and tell it to push all artifacts of the tests to a dedicated branch. This way one can inspect the results later on in full detail. If the branch becomes to large, one can just prune it with e.g. bfg. It's the default way of using Github Pages + a static site generator to produce a blog. See here: .travis

[elpescado]

Travis CI can upload artifacts to Amazon S3.

[theGreatWhiteShark]

Sure. But it's yet another service to rely on.

[elpescado]

You're right. But that's shortcoming of Travis. All other CI solutions that I've used (Jenkins, AppVeyor, Atlassian Bamboo) provide artifact storage out-of-the-box.

[elpescado]

LOL, found it! ;)

Tests seem OK. The offending code is:

std::vector<char> SMFCopyRightNoticeMetaEvent::getBuffer()
{
    SMFBuffer buf;
    QString sCopyRightString;

    time_t now = time(0);
    tm *ltm = localtime(&now);                      // Extract the local system time.

    // Construct the copyright string in the form "(C) [Author] [CurrentYear]"
    sCopyRightString.append("(C) ");                // Start with the copyright symbol and a seperator space.
    sCopyRightString.append( m_sAuthor );           // add the author
    sCopyRightString.append(" ");                   // add a seperator space
    sCopyRightString.append( QString::number( 1900 + ltm->tm_year, 10 ) );  // and finish with the year.

    buf.writeVarLen( m_nDeltaTime );
    buf.writeByte( 0xFF );
    buf.writeByte( COPYRIGHT_NOTICE );
    buf.writeString( sCopyRightString );

    return buf.getBuffer();
}

Can you spot the bug? :)

Let's double check it.

1) test: FunctionalTest::testExportMIDI (F) line: 135 /home/travis/build/hydrogen-music/hydrogen/src/tests/functional_tests.cpp
Files differ at byte 42

Let's see what is on byte 42.

Indeed...

[mauser]

Hi everyone!

In my opinion this issue is fixed now, the issue with the year in the mid file has been solved and the tests are passing with current travis builds. Please re-open if this is not the case or if i misunderstood something.