SQL Server Partitioning – How to Return All Rows from Multiple Partitions

partitioningsql serversql server 2014

I have a table that is manually partitioned into multiple filegroups and spread out over multiple tables by month. My partition scheme is using a date column. What I would like to do is something like the following:

SELECT a
, b
, c
FROM<table>
WHERE date between '2015-08-01' AND SYSDATETIME()

I am wondering if there is a way to do it without the need to use a UNION ALL on all the tables?

Best Answer

As long as you have the appropriate check constraints on the date column on each of the underlying tables, you can create a partitioned view:

CREATE VIEW dbo.2015Sales
AS
SELECT date, sales FROM dbo.Jan2015Sales
UNION ALL
SELECT date, sales FROM dbo.Feb2015Sales
...

This would allow the UNION ALL to be hidden behind the view, and SQL Server can eliminate irrelevant tables (based on your check constraints) from the query plan if the query explicitly specifies a date range.

However, partitioned tables, in which all of the data would be in one table and no UNION ALL would be needed, is the more common approach.

That said, there are a lot of potential performance implications for either approach, so it's best to test on your specific data and workload. For example, partitioned views frequently yield query plans with extraneous Concatenation operators, a situation that may reduce the accuracy of the Cardinality Estimator (especially in SQL 2012 and earlier). And partitioned tables, while very useful and probably what you want, do come with a couple potential downsides as well.

Related Solutions

SQL Server – Moving a Partition of Partitioned Table to Another File Group

There are a couple of issues here. First, the partition/filegroup mapping shown in your question does not match the DDL. A RANGE RIGHT function boundary is inclusive of the partition on the right so the results should be as below to match the DDL. Consider downloading the partition details custom report from codeplex that can run from an SSMS object explorer table context menu to show the boundaries and filegroups: http://ssmspartcustomrpt.codeplex.com/releases/view/30936 .

OnlineGeoPosition   1   FG_OlderData    less than                  2015-01-21 00:00:00.000
OnlineGeoPosition   2   FG_OlderData    greater than or equal to   2015-01-21 00:00:00.000
OnlineGeoPosition   3   FG_OlderData    greater than or equal to   2015-02-20 00:00:00.000
OnlineGeoPosition   4   FG_OlderData    greater than or equal to   2015-03-21 00:00:00.000
OnlineGeoPosition   5   FG_NewerData    greater than or equal to   2015-04-21 00:00:00.000
OnlineGeoPosition   6   FG_NewerData    greater than or equal to   2015-05-22 00:00:00.000
OnlineGeoPosition   7   FG_NewerData    greater than or equal to   2015-06-22 00:00:00.000
OnlineGeoPosition   8   FG_NewerData    greater than or equal to   2015-07-23 00:00:00.000

When you merge a RANGE RIGHT function, the removed partition is the one on the right. Data from the removed partition are moved to the retained one on the left. So merging a RANGE RIGHT sliding window partition always moves from newer to older partitions. However it is best to MERGE and SPLIT empty partitions as you are doing to avoid expensive data movement during partition DDL, which is about 4 times that of normal DML. Conversely, a SPLIT of a RANGE RIGHT function creates a new partition to the right of the specified boundary, and on the NEXT used filegroup, which is the main problem you are having.

Below is a sample script to achieve a sliding window with the new partition/filegroup mappings below. This script uses 2 staging tables, one for the latest partition on the old data filegroup and another for the oldest partition on the new data filegroup (the one to be moved). This allows the boundary to be safely merged with both adjacent partitions empty. You'll need either check constraints on the staging tables or another partition function/scheme to constrain the staging table data to match the target partition boundaries for the switch in. Data are moved efficiently using a CREATE INDEX...DROP_EXISTING.

OnlineGeoPosition   1   FG_OlderData    less than   2015-02-20 00:00:00.000
OnlineGeoPosition   2   FG_OlderData    greater than or equal to   2015-02-20 00:00:00.000
OnlineGeoPosition   3   FG_OlderData    greater than or equal to   2015-03-21 00:00:00.000
OnlineGeoPosition   4   FG_OlderData    greater than or equal to   2015-04-21 00:00:00.000
OnlineGeoPosition   5   FG_NewerData    greater than or equal to   2015-05-22 00:00:00.000
OnlineGeoPosition   6   FG_NewerData    greater than or equal to   2015-06-22 00:00:00.000
OnlineGeoPosition   7   FG_NewerData    greater than or equal to   2015-07-23 00:00:00.000
OnlineGeoPosition   8   FG_NewerData    greater than or equal to   2015-08-22 00:00:00.000

Sample script:

--switch out newest partition on FG_OldData
ALTER TABLE dbo.OnlineGeoPosition
SWITCH PARTITION $PARTITION.pf_Positions_LastSixMonth_MonthlyRange('2015-03-21')
TO dbo.OnlineGeoPosition_OlderTemp;

--switch out oldest partition on FG_NewData
ALTER TABLE dbo.OnlineGeoPosition
SWITCH PARTITION $PARTITION.pf_Positions_LastSixMonth_MonthlyRange('2015-04-21')
TO dbo.OnlineGeoPosition_NewerToOlderTemp;

--move empty partition from new data to old data
ALTER PARTITION FUNCTION pf_Positions_LastSixMonth_MonthlyRange()
    MERGE RANGE ('2015-04-21');

--create new empty partition on old data filegroup for moved data
ALTER PARTITION SCHEME ps_Positions_LastSixMonth_MontlyRange
          NEXT USED FG_OlderData;
ALTER PARTITION FUNCTION pf_Positions_LastSixMonth_MonthlyRange()
    SPLIT RANGE ('2015-04-21');

--rebuild each index with drop existing to move data from new to old filegroup
CREATE UNIQUE CLUSTERED INDEX PK_OnlineGeoPosition_NewerToOlderTemp
ON dbo.OnlineGeoPosition_NewerToOlderTemp(PartitioningColumn)
WITH(DROP_EXISTING=ON)
ON FG_OlderData;

--switch staging tables data back in
--staging tables must have a check constraint or be partitoned similarly as the target partition
ALTER TABLE dbo.OnlineGeoPosition_OlderTemp
SWITCH TO dbo.OnlineGeoPosition PARTITION $PARTITION.pf_Positions_LastSixMonth_MonthlyRange('2015-03-21')
ALTER TABLE dbo.OnlineGeoPosition_NewerToOlderTemp
SWITCH TO dbo.OnlineGeoPosition PARTITION $PARTITION.pf_Positions_LastSixMonth_MonthlyRange('2015-04-21')

--remove oldest partition
TRUNCATE TABLE dbo.OnlineGeoPosition_OlderTemp;
ALTER TABLE dbo.OnlineGeoPosition
SWITCH PARTITION $PARTITION.pf_Positions_LastSixMonth_MonthlyRange('2015-01-21')
TO dbo.OnlineGeoPosition_OlderTemp;
TRUNCATE TABLE dbo.OnlineGeoPosition_OlderTemp;
ALTER PARTITION FUNCTION pf_Positions_LastSixMonth_MonthlyRange()
    MERGE RANGE ('2015-01-21');

--create new partition on new data filegroup for future data
ALTER PARTITION SCHEME ps_Positions_LastSixMonth_MontlyRange
          NEXT USED FG_NewerData;
ALTER PARTITION FUNCTION pf_Positions_LastSixMonth_MonthlyRange()
    SPLIT RANGE ('2015-08-22');
GO

SQL Server – Aligned Index Column Order

You need to think about your partitions as if they are separate tables.

A clustered index on a partitioned table effectively has the partition number as its first key. For you, with a partition per unique PartitionKey, the clustered index will essentially have that column in the front of it. Or as good as...

I'd expect having Id first would feel better, in case you don't know the month and need to search all partitions. It'll seek on each partition rather than scan them.

Best Answer

Related Solutions

SQL Server – Moving a Partition of Partitioned Table to Another File Group

SQL Server – Aligned Index Column Order

Related Question