Coveragepy: continue marked as not covered

[nedbat]

Originally reported by Anonymous

---

It seems that if:

• there is an if statement with two predicates separated by an or operator
• the second (last?) predicate is not always "reached" (due to the fact that the first was always evaluated to True)
• inside the if, there is only a continue statement (or a pass followed by continue)

the continue statement is marked as missing which is not true. Especially in a "if True or True:" case.

continue.py:

#!python
for i in (1, 2, 3, 4):
    if True or False:
        continue #  Missing

for i in (1, 2, 3, 4):
    if False or True:
        continue #  Run

for i in (1, 2, 3, 4):
    if True or True:
        continue #  Missing

for i in (1, 2, 3, 4):
    if True or True:
        print "test" #  Run
        continue     #  Run

print "End"

$ python --version
Python 2.7.2+

Run as:
$ coverage run continue.py && coverage html

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/198
• This issue had attachments: deadcode.py. See the original issue for details.

[nedbat]

Issue #598 was marked as a duplicate of this issue.

[nedbat]

Issue #594 was marked as a duplicate of this issue.

[nedbat]

Issue #593 was marked as a duplicate of this issue.

[nedbat]

Original comment by Peter Inglesby (Bitbucket: inglesp, GitHub: inglesp)

---

Is this limitation documented anywhere?

[nedbat]

Issue #497 was marked as a duplicate of this issue.

[nedbat]

Original comment by Sei Lisa (Bitbucket: Sei_Lisa, GitHub: Unknown)

---

@nedbat Yes, while I initially developed it trying to come up with some kind of solution for this issue, later I thought that it had some value by itself. It can be just a different category of line: no code, covered, uncovered, unreachable. Maybe I should instead open it as a feature request in a separate issue, as this one is related but not really the same thing?

[nedbat]

@Sei_Lisa I see what you mean about the unreachable blocks. I'll have to think about whether that is a good way to approach this or not.

For example, this code will also have an unreachable block:

for i in range(3):
    continue
    print("hello")  # unreachable

but I'm not sure I want to just write off that print statement. I think I'd like to flag it as uncovered.

[nedbat]

BTW: A recent thread in Python-Ideas (starting here: https://mail.python.org/pipermail/python-ideas/2014-May/027893.html) approved a possible change to Python to make disabling the optimizer possible.

[nedbat]

Original comment by Sei Lisa (Bitbucket: Sei_Lisa, GitHub: Unknown)

---

The attached is a proof of concept of what I meant. It successfully detects my test case and my real use case where I first ran into this issue, but not the OP's because the OP's jumps are conditional, so it's not known to be dead code unless the constants are analyzed.

[nedbat]

@schlamar I appreciate your support for the current behavior of coverage.py in this case, but it's pretty hard to argue that it is correct. The fact that it works out for your case doesn't change the fact that the continue line is executed, but is marked as not covered.

[nedbat]

Original comment by Marc Schlaich (Bitbucket: schlamar, GitHub: schlamar)

---

@nedbat Actually I find the behavior of Python in such a case pretty good regarding code coverage.

Consider

#!python

def test(a, b):
    for _ in xrange(5):
        if a or b:
           continue

If you run this with True, False it shows the continue as not covered. I think this is a good thing because you have not covered the case False, True so it helps you to (dis)cover all possible code paths.

IMO coverage should apply this even to all if conditions. If a if condition is only partly evaluated it should count as not covered (not sure if this is possible, though).

[nedbat]

If you have an idea of how to detect that, I'm all ears.

[nedbat]

Original comment by Peter Portante (Bitbucket: portante, GitHub: portante)

---

Could coverage detect the peephole optimization and mark it covered anyways?

[nedbat]

Issue #254 was marked as a duplicate of this issue.

[nedbat]

Unfortunately, this is not a bug in coverage.py. All of these examples are cases where CPython's peephole optimizer replaces a jump to a continue with a jump to the top of the loop, so the continue line is never actually executed, even though its effects are seen.

If you believe as I do that it would be useful to have a way to disable the peephole optimizer so that these sorts of analyses would give useful results, comment on this CPython ticket: http://bugs.python.org/issue2506 "Add mechanism to disable optimizations".

[nedbat]

Original comment by Ronald Oussoren (Bitbucket: ronaldoussoren, GitHub: ronaldoussoren)

---

And likewise for this code:

#!python



dct = {
    'a': {
        'src': 'B',
        'val': 'A',
    },
    'b': {
        'src': 'B',
        'val': 'B',
    },
    'c': {
        'src': 'C',
        'val': 'C',
    }
}

for k in ('a', 'c', 'd'):
    s = dct.get(k, {}).get('src')
    v = dct.get(k, {}).get('val')

    if s == 'A' or v == 'A':
        pass

    else:
        continue

    print "got",k

The continue is reached for 'c' and 'd'.

[nedbat]

Original comment by Daniel Black (Bitbucket: dan_black, GitHub: Unknown)

---

Similar to Jean-Tiare's example here's an isolated case:

continue is reached with example(4)

#!python

def example(a):
    for x in [1,2,3,4]:
        if x >= a:
            if x == 3:
                return
            continue
        b = a

example(1)
example(3)
example(4)

[nedbat]

Original comment by Jean-Tiare Le Bigot (Bitbucket: jtlebigot, GitHub: Unknown)

---

I ran into a similar use case with this snippet from ddbmock project

#!python
for fieldname, condition in expected.iteritems():
    if u'Exists' in condition and not condition[u'Exists']:
        if fieldname in self:
            raise ConditionalCheckFailedException(
                "Field '{}' should not exist".format(fieldname))
        # *IS* executed but coverage bug
        continue  # pragma: no cover
    if fieldname not in self:

I used "# pragma: no cover" as workaround for the stats after checking it is actually reached.

This is annoying but really not critical to me.

Thanks for this lib btw.

[nedbat]

I'm re-opening this issue so that people will find it instead of opening duplicates :) Maybe someday Python will have a way that we can fix it.

[skirpichev]

@nedbat, maybe this should go to the docs? Unless I miss something, there are no references to this issue in relevant documentation sections (i.e. "Things that cause trouble"). Same for https://github.com/nedbat/coveragepy/issues/772. Docs also could suggest better workarounds than adding "pragma: no cover", as people often do.

BTW, the original CPython issue is closed (as resolved) by PEP 626, which will be implemented in 3.10, I suspect.

[skirpichev]

It seems, this issue was solved (partially? there is no mention PEP 626 in the release notes yet) in the CPython 3.10.

For example:

$ cat continue.py 
#!python
for i in (1, 2, 3, 4):
    if True or False:
        continue  #  Missing

for i in (1, 2, 3, 4):
    if False or True:
        continue  #  Run

for i in (1, 2, 3, 4):
    if True or True:
        continue  #  Missing

for i in (1, 2, 3, 4):
    if True or True:
        print("test")  #  Run
        continue       #  Run

print("End")
$ python --version
Python 3.10.0a6+
$ coverage run continue.py && coverage report
test
test
test
test
End
Name          Stmts   Miss  Cover
---------------------------------
continue.py      14      0   100%
---------------------------------
TOTAL            14      0   100%

[nedbat]

You are right that this should be closed now. Python 3.10 gets it right, and actually, earlier versions did too, for the particular problem of "continue" in a for loop.

If you are still seeing this problem, it will go away when you use a newer version of Python.

(BTW: PEP 626 is mentioned as the last item in the 5.4 release notes)

[skirpichev]

Python 3.10 gets it right

@nedbat, I'm not sure. I didn't check all code snippets in this issue and linked issues.

As I said before, it seems that PEP 626 is not implemented fully yet.

[nedbat]

As I said before, it seems that PEP 626 is not implemented fully yet.

I'm not sure if you mean it isn't fully implemented in Python, or in coverage.py. We'd love to see examples of problems.

[skirpichev]

it isn't fully implemented in Python

Yes.

or in coverage.py

No, I doubt any changes are required.

We'd love to see examples of problems.

I'll try to run coverage tests in https://github.com/diofant/diofant/pull/1128 (there are ~20 workarounds for this issue).

But there is a lot of examples in the current issue and marked as "duplicates". It's worth to check out.

[nedbat]

But there is a lot of examples in the current issue and marked as "duplicates". It's worth to check out.

The CPython developers believe they have fully implemented PEP626. I don't know of cases that are still wrong. As far as I can tell, the problems reported here in this issue have been fixed. If you have specific examples of problems, please let us know.

[skirpichev]

The CPython developers believe they have fully implemented PEP626.

I'm not sure. Release notes still doesn't mention it.

If you have specific examples of problems, please let us know.

Sure, my biggest project has ~20 workarounds for this issue - I'll try run coverage tests on the 3.10 and report if problem persist.

[nedbat]

The What's New is being updated: https://github.com/python/cpython/pull/24892