利用SQL Server 2005的新功能NOW_NUMBER和CTE可以很好的實現.
舉例說明如下:
建立測試數據:
復制代碼 代碼如下:
create table Dup1
(
Col1 int null,
Col2 varchar(20) null
)
insert into Dup1 values
(1, 'aaa'),
(2, 'aaa'),
(2, 'aaa'),
(2, 'aaa'),
(3, 'bbb'),
(3, 'bbb'),
(4, 'ccc'),
(4, 'ddd'),
(5, 'eee')
select * from Dup1
可以查看到重復的數據有:
復制代碼 代碼如下:
SELECT Col1, Col2, COUNT(*) AS DupCountFROM Dup1GROUP BY Col1, Col2HAVING COUNT(*) > 1
接下來介紹如何delete掉重復的數據:
1.NOW_NUMBER:SQL Server 2005添加了很好用的RANKING函數(NOW_NUMBER,RANK,DENSE_RANK,NTILE),利用NOW_NUMBER()OVER(PARTITION GY)最為直接,也最為方便,不能修改表或者產生多余的列.
首先會分配一個列號碼,以Col1,Col2組合來分區排序.
復制代碼 代碼如下:
SELECT Col1, Col2,ROW_NUMBER() OVER (PARTITION BY Col1, Col2 ORDER BY Col1) AS rnFROM Dup1
得到的序號如下:
很明顯的是重復列都分組分割排序,只需要delete掉排序序號>1的即可.
復制代碼 代碼如下:
--用到CTE
WITH DupsD
AS (
SELECT Col1, Col2,
ROW_NUMBER() OVER (PARTITION BY Col1, Col2 ORDER BY Col1) AS rn
FROM Dup1
)
DELETE DupsD
WHERE rn > 1;
--或者
DELETE A FROM (
SELECT Col1, Col2,
ROW_NUMBER() OVER (PARTITION BY Col1, Col2 ORDER BY Col1) AS rn
FROM Dup1) A WHERE A.rn>1
2.創建一個標識鍵唯一的表記一列.
復制代碼 代碼如下:
ALTER TABLE dbo.Dup1
ADD
PK INT IDENTITY
NOT NULL
CONSTRAINT PK_Dup1 PRIMARY KEY;
SELECT *
FROM Dup1;
刪除找出與Col1,Col2相同并且比Dup1.PK大的記錄,也就是保留重復值中PK最小的記錄.
復制代碼 代碼如下:
DELETE Dup1
WHERE EXISTS ( SELECT *
FROM Dup1 AS D1
WHERE D1.Col1 = Dup1.Col1
AND D1.Col2 = Dup1.Col2
AND D1.PK > Dup1.PK );
3.select distant into,這種方法借助一個新的table,把不重復的結果集轉移到新table中.
復制代碼 代碼如下:
SELECT distinct Col1, Col2 INTO NoDupsFROM Dup1;select * from NoDups
建議采用第一種和第三種方法,第一種多見于T-SQL的編程中,第三種在ETL中常常使用.
