Robotframework: Wait Until Keyword Fails

[gkeishin]

There are use case test where we are expecting a test to fail if there is a failure in a loop which keep checking every x interval basically the inverse logic of Wait Until Keyword Succeeds.

So we ended up writing a similar code for it taking cue from existing Wait Until Keyword Succeeds .

If this new propose keyword Wait Until Keyword Fails could be added in the latter release version it would be really cool.

Sample Code currently we are using in our test

###############################################################################
def run_until_keyword_fails(retry, retry_interval, name, *args):

    r"""
    Execute a robot keyword repeatedly until it either fails or the timeout
    value is exceeded.
    Note: Opposite of robot keyword "Wait Until Keyword Succeeds".
    Description of argument(s):
    retry              Max timeout time in hour(s).
    retry_interval     Time interval in minute(s) for looping.
    name               Robot keyword to execute.
    args               Robot keyword arguments.
    """

    # Convert the retry time in seconds
    retry_seconds= DateTime.convert_time(retry)
    timeout = time.time() + int(retry_seconds)

    # Convert the interval time in seconds
    interval_seconds=  DateTime.convert_time(retry_interval)
    interval = int(interval_seconds)

    BuiltIn().log(timeout)
    BuiltIn().log(interval)

    while True:
        status= BuiltIn().run_keyword_and_return_status(name, *args)

        # Return if keywords returns as failure.
        if status==False:
            BuiltIn().log("Failed as expected")
            return  False
        # Return if retry timeout as success.
        elif time.time() > timeout > 0:
            BuiltIn().log("Max retry timeout")
            return  True
        time.sleep(interval)
        BuiltIn().log(time.time())

    return  True

###############################################################################

[chriscallan]

Seems like a logical extension of the keyword structure. I'm not sure how useful it would be for the general use cases. I've never needed anything like this but that doesn't mean that no one does (obviously at least one person does :-).
Only change I would suggest is to allow the log messages to be included as optional input arguments, so that the function could be just a little more flexible for the general use case.

[pekkaklarck]

1. This sounds somewhat special keyword for me. Could you clarify what kind of use cases you have?

2. Have you tested would using this Wait Until Keyword Succeeds with Run Keyword And Expect Error work instead? Something like this:

   *** Keywords ***
   Wait Until Keyword Fails
       [Arguments]    ${timeout}    ${retry}    ${keyword}    @{args}
       Wait Until Keyword Succeeds    ${timeout}    ${retry}     
       ...    Run Keyword And Expect Error    *
       ...    ${keyword}    @{args}
   

[gkeishin]

@pekkaklarck

• [x] 1. This sounds somewhat special keyword for me. Could you clarify what kind of use cases you have?

Yeah sort of, use cases while testing systems we got scenario.

a) where we need to wait on an error condition but doesn't necessarily fail the test if that error doesn't occur in X time. (GOOD PATH scenario)
- If err's within X time test fails
- If doesn't err's within X timed out, don't fail

b) where we know for sure that there will be an error due to error injection within X time. (BAD PATH scenario)

• [x] 2. Really appreciated. This works well for b) case but not really for a specific case a).

Any cleaner way to do to monitor a) failure scenarios.

[pekkaklarck]

To support your scenario a), you could wrap my example keyword into one more Run Keyword And Expect Error and expect to get the specific error about timeout. That get's a bit complicated, so having the logic in a custom library like you already have is probably a better idea. I still don't think this is useful and generic enough to be added into BuiltIn.

[gkeishin]

Thanks @pekkaklarck