How to set conditions for filling specific gaps in panel data?

conditional-statements, missing-data, r, time-series

Solution

Here's a solution using `data.table`. The idea is to first create a `data.table` with just the missing entries `(dt.rest)` and then `rbind` them. I've written it in such a way that the output of each line (by copy/pasting and printing) should be fairly straightforward to follow. Let me know if something isn't clear.

require(data.table)
dt <- data.table(df, key="countrycode")
dt$year <- as.numeric(as.character(dt$year))
dt[J(unique(countrycode)), year2 := c(tail(year, -1), NA)]
dt.rest <- dt[, { tt <- which(year2-year-1 <=3); 
                  list(year = unlist(lapply(tt, function(x) 
                              seq(year[x]+1, year2[x]-1, by=1))), 
                       conflict = 0)
                }, by=countrycode]
setcolorder(dt.rest, c("year", "countrycode", "conflict"))

#    year countrycode conflict
# 1: 1996         135        0
# 2: 1990         750        0
# 3: 1991         750        0
# 4: 1992         750        0

Now, we just have to `rbind` them. This is done using `rbindlist` function within `data.table` that binds `data.frame` or `data.table` much more efficiently than `rbind`.

dt[, year2 := NULL]
dt <- rbindlist(list(dt, dt.rest))
setkey(dt, "countrycode", "year")

dt
#     year countrycode conflict
#  1: 1990         135        1
#  2: 1995         135        1
#  3: 1996         135        0
#  4: 1997         135        1
#  5: 1989         750        1
#  6: 1990         750        0
#  7: 1991         750        0
#  8: 1992         750        0
#  9: 1993         750        1
# 10: 1998         750        1

Problem

I have panel data in an R data.frame containing years from 1989-2008 for armed conflicts in various countries. However, only observations for countries which experienced armed conflict in a given year are included. The dataset is similar to this: ``` df <- data.frame(c("1989","1993","1998", "1990","1995","1997"), c(rep(c(750, 135), c(3,3))), c(rep(1,6))) names(df)<-c("year","countrycode","conflict") print(df) year countrycode conflict 1 1989 750 1 2 1993 750 1 3 1998 750 1 4 1990 135 1 5 1995 135 1 6 1997 135 1 ``` I now want to fill gaps in the panel data, BUT ONLY gaps which are not bigger than three years. For example, I want add rows between rows 1 and 2 and between rows 5 and 7 (the gap is 3 and 1 year respectively), but neither between rows 2 and 3 nor between rows 4 and 5 (the gaps is 4 years each). After this procedure, the data.frame above would look the following: ``` > df2 <- data.frame(c("1989","1990","1991","1992","1993","1998", + "1990","1995","1996","1997"), + c(rep(c(750, 135), c(6,4))), c(1,0,0,0,1,1,1,1,0,1)) > names(df2) <- c("year","countrycode","conflict") > print(df2) year countrycode conflict 1 1989 750 1 2 1990 750 0 3 1991 750 0 4 1992 750 0 5 1993 750 1 6 1998 750 1 7 1990 135 1 8 1995 135 1 9 1996 135 0 10 1997 135 1 ``` I have looked into the `plm` package (see here), but couldn't find any answer there. Also, I am relatively new to R, so I'll be happy for any hint.

Original source