Merge consecutive duplicate records including time

问题:

I have a very similar problem to the question asked here: Merge duplicate temporal records in database

The difference here is, that I need the end date to be an actual date instead of NULL.

So given the following data:

EmployeeId   StartDate   EndDate     Column1   Column2
1000         2009/05/01  2010/04/30   X         Y
1000         2010/05/01  2011/04/30   X         Y
1000         2011/05/01  2012/04/30   X         X
1000         2012/05/01  2013/04/30   X         Y
1000         2013/05/01  2014/04/30   X         X
1000         2014/05/01  2014/06/01   X         X

The desired result is:

EmployeeId   StartDate   EndDate     Column1   Column2
1000         2009/05/01  2011/04/30   X         Y
1000         2011/05/01  2012/04/30   X         X
1000         2012/05/01  2013/04/30   X         Y
1000         2013/05/01  2014/06/01   X         X

The proposed solution in the linked thread is this:

with  t1 as  --tag first row with 1 in a continuous time series
(
select t1.*, case when t1.column1=t2.column1 and t1.column2=t2.column2
                  then 0 else 1 end as tag
  from test_table t1
  left join test_table t2
    on t1.EmployeeId= t2.EmployeeId and dateadd(day,-1,t1.StartDate)= t2.EndDate
)
select t1.EmployeeId, t1.StartDate, 
       case when min(T2.StartDate) is null then null
            else dateadd(day,-1,min(T2.StartDate)) end as EndDate,
       t1.Column1, t1.Column2
  from (select t1.* from t1 where tag=1 ) as t1  -- to get StartDate
  left join (select t1.* from t1 where tag=1 ) as t2  -- to get a new EndDate
    on t1.EmployeeId= t2.EmployeeId and t1.StartDate < t2.StartDate
 group by t1.EmployeeId, t1.StartDate, t1.Column1,   t1.Column2;

However, this does not seem to work when you need the end date instead of just NULL.

Could someone help me with this issue?

回答1:

How about this?

create table test_table (EmployeeId int, StartDate  date, EndDate  date,   Column1 char(1),  Column2 char(1))
;
insert into test_table values
 (1000    ,     '2009-05-01','2010-04-30','X','Y')
,(1000    ,     '2010-05-01','2011-04-30','X','Y')
,(1000    ,     '2011-05-01','2012-04-30','X','X')
,(1000    ,     '2012-05-01','2013-04-30','X','Y')
,(1000    ,     '2013-05-01','2014-04-30','X','X')
,(1000    ,     '2014-05-01','2014-06-01','X','X')
;
SELECT EmployeeId, StartDate, EndDate, Column1, Column2 FROM 
(
    SELECT EmployeeId, StartDate 
    ,      MAX(EndDate) OVER(PARTITION BY EmployeeId, RN) AS EndDate
    ,      Column1 
    ,      Column2
    ,      DIFF
    FROM
    (
        SELECT t.*
        ,      SUM(DIFF) OVER(PARTITION BY EmployeeId ORDER BY StartDate ) AS RN
        FROM
        (
            SELECT t.*
            ,      CASE WHEN
                       Column1 = LAG(Column1,1) OVER(PARTITION BY EmployeeId ORDER BY StartDate)
                   AND Column2 = LAG(Column2,1) OVER(PARTITION BY EmployeeId ORDER BY StartDate)
                   THEN 0 ELSE 1 END AS DIFF
            FROM
                test_table t
        ) t
    )
)
WHERE DIFF = 1
;

回答2:

This is another solution (taken from How do I group on continuous ranges). It is simpler to code and also caters for NULL values (i.e. treats NULL = NULL unlike the simple LAG() comparison). However it might not be quite as efficient on large volumes of data due to the GROUP BY

SELECT EmployeeId
,      MIN(StartDate) AS StartDate
,      MAX(EndDate)   AS EndDate
,      Column1 
,      Column2
FROM
(
    SELECT t.*
    ,      ROW_NUMBER() OVER(PARTITION BY EmployeeId, Column1, Column2 ORDER BY StartDate ) AS GRN
    ,      ROW_NUMBER() OVER(PARTITION BY EmployeeId                   ORDER BY StartDate ) AS RN
    FROM 
           test_table t
    ) t
GROUP BY
       EmployeeId
,      Column1 
,      Column2
,      RN - GRN