PostgreSQL – Repartition Tables Without Losing Data

partitioningpostgresql

I already have my table partitioned as follows:

CREATE TABLE customer
(
  c_custkey integer NOT NULL, 
  c_name character varying(25) NOT NULL, 
  c_address character varying(40) NOT NULL, 
  c_city character(10) NOT NULL, 
  c_nation character(15) NOT NULL, 
  c_region character(15) NOT NULL, 
  c_phone character(15) NOT NULL,
  c_mktsegment character(10) 
) PARTITION BY List(c_region) ;

create table customer_1 PARTITION of customer for values in ('EUROPE');
create table customer_2 PARTITION of customer for values in ('AFRICA');
create table customer_3 PARTITION of customer for values in ('AMERICA');
create table customer_4 PARTITION of customer for values in ('MIDDLE EAST');
create table customer_5 PARTITION of customer for values in ('ASIA');

My partitions are filled with data. I want to repartition my table without losing data. How to do that with PostgreSQL?

Best Answer

Essentially you create a new partitioned table and fill it with the data from the old one, for example with INSERT INTO ... SELECT. Then you can drop the old table and rename the new one. That requires downtime.

If you mind downtime, create a new database from a schema dump of the old one, except that your partitioned table is now partitioned differently. Then set up logical replication between the two databases and switch over as soon as the replica has caught up.

Related Solutions

Mysql – Partition by year and sub-partition by month thesql

I had to do the same thing and solved slightly differently. Basically as far as I understand by reading the docs MySQL subpartitioning does not support partition types besides HASH and KEY.

In MySQL 5.6, it is possible to subpartition tables that are partitioned by RANGE or LIST. Subpartitions may use either HASH or KEY partitioning. This is also known as composite partitioning.

This means that we can't determine in what subpartition a record will end up. It's up to MySQL. So I don't think it's wise giving such names to your subpartitions (i.e. january) because no way you're going to know if stuff created on that month will end up there. This because you can't subpartition by MONTH(nav_date) but only by HASH(MONTH(nav_date)) or KEY(MONTH(nav_date)).

So to solve the problem I decided to create a new month column in my table and then I added an index to it. Then I subpartioned by KEY(month) without caring about the subpartition names. This way in MySQL 5.6 I could select the main partition from the FROM clause and the subpartition by specifying the month in the WHERE clause.

Follows the full example:

CREATE TABLE `my_example` (
    `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
    `value` double DEFAULT NULL,
    `is_deleted` tinyint(1) NOT NULL DEFAULT '0',
    `timestamp` datetime NOT NULL,
    `last_modified` datetime NOT NULL,
    `month` tinyint(1) NOT NULL,
    PRIMARY KEY (`id`,`timestamp`, `month`),
    KEY `in_is_deleted` (`is_deleted`),
    KEY `in_last_modified` (`last_modified`),
    KEY `in_timestamp` (`timestamp`),
    KEY `in_month` (`month`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
PARTITION BY RANGE (YEAR(`timestamp`))
SUBPARTITION BY KEY (`month`) 
SUBPARTITIONS 12 (
    PARTITION p2011 VALUES LESS THAN (2012),
    PARTITION p2012 VALUES LESS THAN (2013),
    PARTITION p2013 VALUES LESS THAN (2014),
    PARTITION p2014 VALUES LESS THAN (2015),
    PARTITION p2015 VALUES LESS THAN (2016),
    PARTITION p2016 VALUES LESS THAN (2017),
    PARTITION p2017 VALUES LESS THAN (2018),
    PARTITION pmax VALUES LESS THAN MAXVALUE
);

EXPLAIN PARTITIONS SELECT * FROM my_example PARTITION (p2015); -- will go through all twelve subpartitions
EXPLAIN PARTITIONS SELECT * FROM my_example PARTITION (p2015) WHERE month = 9; -- will look only into one subpartition

MySQL Partitioning – Efficient Way to DELETE Whole Partition’s Data

To answer the update, the TRUNCATE PARTITION will not reset the auto_increment field. This makes sense since it would be sticky to handle in relation to the other partitions in the auto increment scenario.

However, a table truncate will reset the auto increment key.

Best Answer

Related Solutions

Mysql – Partition by year and sub-partition by month thesql

MySQL Partitioning – Efficient Way to DELETE Whole Partition’s Data

Related Question