Covid-19-data: Enhancement: Non Cumulative Data

[murphystout]

Describe the feature you would like to see

I would love to see "NEW CASES" by Day and By State (and County). Right now the data seems to be cumulative which presents very annoying data transformation tasks if we are trying to calculate the change in New Cases (which seems to be the main data point being presented in the press).

Describe alternatives you've considered

Manually transforming the data, but this leads to potential for human error.

[raford]

@murphystout You can just write a quick script to do it. I saw that you have R listed as a language in one of your repos, so here's a quick and dirty way to do this in R. Note there are more efficient ways to do this in R, but this one clearly shows what is going on. You can adapt this from the national level data to the state/county level pretty easily.

dat <- read.csv("https://raw.githubusercontent.com/nytimes/covid-19-data/master/us.csv")
dead <- c()
case <- c()
k <- length(dat$deaths)
while (k >= 1) {
  if (k == 1) {
    dead[k] <- 0
  } else {
    dead[k] <- dat$deaths[k] - dat$deaths[k-1]
    case[k] <- dat$cases[k] - dat$cases[k-1]
  }
  k <- k - 1
}
case[1] <- 0
dat$new.cases <- case
dat$new.deaths <- dead
rm(k, case, dead)

[murphystout]

Wow, that's so helpful @raford , thanks so much!

I need the data by State, so I tinkered with your code and added another loop that cycles through the states, appending to a final dataframe:

`dat <- read.csv("https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-states.csv")
dead <- c()
case <- c()
datfinal <- data.frame(date=as.Date(character()),state=character(),fips=character(),cases=integer(),new.cases=integer(),new.deaths=integer()) #if names of columns of initial dat change, this needs to change, otherwise rbinds below wont work.

states <- unique(dat$state)
ks <- length(states)
while (ks >= 1)
{
datsub <- subset(dat,state==states[ks])
k <- length(datsub$deaths)
while (k >= 1) {
if (k == 1) {
dead[k] <- 0
} else {
dead[k] <- datsub$deaths[k] - datsub$deaths[k-1]
case[k] <- datsub$cases[k] - datsub$cases[k-1]
}
k <- k - 1
case[1] <- 0
datsub$new.cases <- case
datsub$new.deaths <- dead
}
datfinal <- rbind(datfinal,datsub)
ks <- ks - 1

}

View(datfinal)`

[tiffehr]

Thank you! Added this to our Collection of user-based additions and alternative formats. We hope this helps others find this conversation.

[rosshochwert]

In case anyone wants the county level data script:

rm(list = ls(all.names = TRUE))
dat <- read.csv("https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-counties.csv")
dead <- c()
case <- c()
datfinal <- data.frame(date=as.Date(character()),county=character(),state=character(),fips=character(),cases=integer(),new.cases=integer(),new.deaths=integer()) #if names of columns of initial dat change, this needs to change, otherwise rbinds below wont work.

fipss <- unique(dat$fips)
ks <- length(fipss)

while (ks >= 1) {
  datsub <- subset(dat,fips==fipss[ks])
  k <- length(datsub$deaths)
  dead <- c()
  case <- c()
  while (k >= 1) {
    if (k == 1) {
      dead[k] <- 0
    } else {
      dead[k] <- datsub$deaths[k] - datsub$deaths[k-1]
      case[k] <- datsub$cases[k] - datsub$cases[k-1]
    }
    k <- k - 1
    case[1] <- 0
    datsub$new.cases <- case
    datsub$new.deaths <- dead
  }

  datfinal <- rbind(datfinal,datsub)
  ks <- ks - 1

}

View(datfinal)

[tiffehr]

Awesome, thank you! Closing, but this remains on our Project board and we'll point people to it.