Storybook: Angular-Storybook: a11y addOn does not show violations when it should do

[jameshughes7]

Describe the bug
When using the a11y storybook addon, scenarios which should return violations but are not doing so.

To Reproduce
Steps to reproduce the behavior:

1. Visit Storybook official netlify link
2. Note how there are _No a11y violations found_ with the Invalid contrast example
3. Attempting this myself within my own project I'm seeing similar results:
4. Configure angular project to use a11y addon by following the getting started doc:
   
   
   
   • npm install -D @storybook/addon-a11y
   
   
   • import '@storybook/addon-a11y/register';
   
   
5. Configure stories with a11y addon:
   
   
   
   • import { checkA11y, configureA11y } from '@storybook/addon-a11y';
   
   
6. Specify stories file with a11y configurations
7. Add a11y decorator to stories file
8. Set the following scss attributes of our button to the color white: background-color, background-color-hover and background-color-active.
9. Set the display attribute of our component to display: none

Expected behavior
Should see violations as component is not meeting the a11y configurations ['wcag2a', 'wcag2aa', 'wcag21aa']

Screenshots
Storybook official netlify link

My own project

Code snippets

button.stories.ts

import { storiesOf } from '@storybook/angular';
import { withNotes } from '@storybook/addon-notes';
import { action } from '@storybook/addon-actions';
import { checkA11y, configureA11y } from '@storybook/addon-a11y';

import { ButtonComponent } from '../projects/component-library/src/lib/buttons/button/button.component';

// a11y Configurations
const a11yConfigs = {
  runOnly: ['wcag2a', 'wcag2aa', 'wcag21aa'],
};
configureA11y(a11yConfigs);

storiesOf('Buttons/Button', module)
  .addDecorator(checkA11y)
  .addDecorator(withNotes)
  .add(
    'Primary',
    () => ({
      component: ButtonComponent,
      props: {
        btnClick: action('Button was clicked'),
        text: 'Prescribe',
      },
    }),
    { notes: 'Primary style button' }
  )
  .add(
    'Primary disabled',
    () => ({
      component: ButtonComponent,
      props: {
        text: 'Prescribe',
        disabled: true,
      },
    }),
    { notes: 'Primary style disabled button' }
  );

button.component.scss

$primary-color: #0033a0;
$primary-hover: #00277c;
$primary-disabled: #cac8c5;

button {
  display: flex;
  align-items: center;
  justify-content: center;
  cursor: pointer;
  height: 44px;
  letter-spacing: 0.01em;
  font-size: 14px;
  border-radius: 100px;
  min-width: 196px;
  padding: 0 16px;
  font-family: Helvetica;
  color: #fff;

  &:focus {
    outline: none;
  }

  &.primary {
    background-color: #fff;

    &:hover {
      background-color: #fff;
    }

    &:active {
      background-color: #fff;
    }

    &:disabled {
      cursor: default;
      background-color: $primary-disabled;
    }
  }
}

System:

• OS: [MacOS]
• Device: [Macbook Pro 2017]
• Browser: [chrome]
• Framework: [angular]
• Addons: [a11y]
• Version: [5.0.0-rc.5]

Additional context
n/a

[gaetanmaisse]

Storybook is not displaying any violations because in this case (BaseButton | Invalid contrast) axe-core, the library we are using for a11y, doesn't return any violations.

In fact this color-contrast rule is not in "passing" rules array neither but in incomplete.

As per doc:

The incomplete array (also referred to as the "review items") indicates which nodes could neither be determined to definitively pass or definitively fail. They are separated out in order that a user interface can display these to the user for manual review (hence the term "review items").

BUT for now only passes and violations array are displayed in SB ui. Maybe we should think about adding something to display incomplete rules array?

[shilman]

Thanks @gaetanmaisse . Opened a new feature request for this and closing this issue for now.