Tidyr: Add unfill(), the inverse of the fill() function

[huftis]

Here’s a request for an unfill() function, which should do the inverse of what fill() does. This is useful for formatting data frames for final output, where one often wants to avoid repeating values (see http://blog.darkhorseanalytics.com/clear-off-the-table for a nice illustration).

For example, it should turn x into x_blank (here NA values are shown as empty cells):

 x  x_blank
 A  A
 A        
 A        
 B  B
 B        
 C  C

Here’s a proof-of-concept implementation that works on _vectors_:

unfill_vec = function(x, direction = c("down", "up")) {
  if (direction[1] == "down") {
    x[which(x[-1] == x[1:(length(x) - 1)]) + 1] = NA
  } else {
    x = rev(unfill(rev(x), direction = "down"))
  }
  x
}

For the tibble df = tibble(x=c("A","A","A","B","B","C","C")), running df %>% unfill(x) should give the same output as:

df %>% mutate(x = unfill_vec(x))

Note that ‘unfilling’ is not the same as simple ’deduplication’, as shown here:

df2 = tibble(x = c("A", "A", "A", "B", "B", "C", "C", "A"))
df2 %>% mutate(x_unfill = unfill_vec(x),
               x_dedup = ifelse(!duplicated(x), x, NA))

This results in:

      x x_unfill x_dedup
  <chr>    <chr>   <chr>
1     A        A       A
2     A     <NA>    <NA>
3     A     <NA>    <NA>
4     B        B       B
5     B     <NA>    <NA>
6     C        C       C
7     C     <NA>    <NA>
8     A        A    <NA>

‘Unfilling’ is really a type of _untidying_, so perhaps there’s a better package than tidyr for it? On the other hand, tidyr _does_ contain spread(), which is also an untidying function and the inverse of gather().

[antoine-sachet]

A use case for this would be detecting a sensor measurement freeze. This is really only applicable if the sensor sensibility is so high (and/or the captures sufficiently spaced) that you would not expect consecutive measurements to have exactly the same value.

In this particular case, it is really _tidying_ and not _untidying_. I am sure you could come up with examples where removing repeated consecutive is tyding.

Side-note: I would not argue that spread is an untidying function... You should see some of the data I receive, spread is definitely super useful in tidying it up!

[hadley]

I really can't see how this could ever make data tidier (unlike spread()) so I'd prefer not to include in tidyr. However, there's a pretty simple implementation using lag():

unfill_vec <- function(x) {
  same <- x == dplyr::lag(x)
  ifelse(!is.na(same) & same, NA, x)
}

x <- c("A","A","A","B","B","C","C")
unfill_vec(x)