Elastalert: Add support to test rules with custom timeframe

[abhishekjiitr]

Currently, for testing any rule the _elastalert-test-rule_ script runs ElastAlert over the last 24 hours and prints out any alerts that would have occurred. It would be better if we don't ignore the _timeframe_ given in the rule's yml and run the test for only that period. If no _timeframe_ is given in the rule, then it may default to 24 hours.
see Testing Your Rule

[Qmando]

It doesn't ignore the timeframe in the rule.

For example, if you have a frequency rule with timeframe 1 hour, and you run elastalert-test-rule, you might generate a couple of alerts. Like if T-23 to T-22 hours had a ton of events, it would trigger an alert. It's basically simulating what would happen if you had started elastalert 24 hours earlier and left it running.

You can also change how long it runs for, but, only in 24 hours increments, using --days

[abhishekjiitr]

I understand that I can give the duration for testing in multiple of days, but I think If we are trying to evaluate a rule, it will be better if we have a way to evaluate its _hits_, _time_taken_ to execute, etc. for that duration only in addition to the command line option we already have in elastalert to give number of days as duration. If a user does not give the _timeframe_ in the rule and nor the command-line argument argument _days_ is present then we may take default duration for testing as one day / 24 hours.

We have a usecase that requires this and I am already writing some code like this. I will be more than happy to submit a Pull Request for the same:

Here I have the default value of days (args.days command-line option) as 0
Working Code:
File: test_rule.py, Class: MockElastAlert, Method: run_elastalert

# if days given as command line argument
if args.days > 0:
    starttime = endtime - datetime.timedelta(days=args.days)
else:
    # if timeframe is given in rule
    if 'timeframe' in rule:
        timeframe = rule['timeframe']
        starttime = endtime - timeframe
    # default is 1 days / 24 hours
    else:
        starttime = endtime - datetime.timedelta(days=1)

[Qmando]

Yeah it would be good to add more options for testing. You can always just run elastalert in debug mode and specify the exact time range, ie $ elastalert --debug --rule myrule.yaml --start 2018-10-28T12:00:00-07:00 --end 2018-10-28T13:30:00-07:00

elastalert-test-rule should probably be able to accept --start and --end too. Using the timeframe is smart too. I would accept a PR.