When I toggle on header/footer sections for headers, I'm expecting those rows to be more visually distinguished from regular cells, and maybe even have placeholder labels. True, the text becomes bold, but you wouldn't even know that until you click into a header cell and start typing. It needs to be more explicit, IMO.
The original PR had some discussion on this (right from the start of the conversation) - https://github.com/WordPress/gutenberg/pull/15409.
I think the bold text just comes from the theme styling, so that might not be guaranteed.
Placeholders could help a lot, and adding a simple 2px border below the header and above the footer also help. Here's a mockup of what that could look like:

That helps a lot!
Can I pick this up? It will be my first contribution to WordPress
@dpkm95 That'd be awesome!
@shaunandrews I see there are no failing tests after after my change. Do you need me to add a snapshot test for table component? I see it's only added in block-library/embed. Is there any reason why snapshot tests are not added pervasively?
@dpkm95 Right now there's no visual snapshot tests. The snapshot tests you see in the code will test against the rendered output of a component. Since your changes only impact CSS, it would not be tested. Visual snapshot testing may be added in the future, but in the meantime, a visual verification of those changes should be sufficient for review.
Most helpful comment
Placeholders could help a lot, and adding a simple 2px border below the header and above the footer also help. Here's a mockup of what that could look like: