How to replicate INSERT/UPDATE/DELETE statements u

Question

I would like to rename a PostgreSQL (9.6) table in a way that is recoverable for my system (A java app using JPA/Hibernate)

In my java code, a JPA entity would have the following annotations @Entity @Table(name="old_name") and the database would have an equivalent table called old_name.

I would like to rename the table to new_name in a way that I can incrementally update the database and java app, allowing for failure and rollback.

Typical steps would be

1. create copy of old_name in new_name
2. ensure read/writes available in both (i.e. data is replicated both ways)
3. update java app to use new table new_name
4. when confident system update complete, remove old_name

Effectively I would like a duplicate table in the same schema with the same data, both able to accept reads and writes, that can be read from JPA entities.

I am aware of the use of triggers, and would like to avoid that. I am hoping there is a technique I'm not aware of and haven't found that would make this less painful than using triggers.

I have tried to rename the table and create a "simple view" over it, however the JPA entity complains as it can't find a table with the name of the view. (Because it is a view, not a table :) and there seems no @View/@Table JPA annotation that will handle this)

I haven't yet tried the facilities listed here: http://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling as the majority seem to be about pooling, sharding, and I need a simple short term table replica, but I will be investigating these also.

Thanks - I would like the simplest option of course, preferring something built in to postgres/JPA but will seriously consider 3rd party options also.

Answer 1

Since this is a very common question, I wrote this article, on which this answer is based on.

Database tables

Assuming you have the following two tables:

CREATE TABLE old_post (
    id int8 NOT NULL,
    title varchar(255),
    version int4 NOT NULL,
    PRIMARY KEY (id)
)

CREATE TABLE post (
    id int8 NOT NULL,
    created_on date, 
    title varchar(255),
    version int4 NOT NULL,
    PRIMARY KEY (id)
)

JPA entity

The old_post table must be replicated with the newer post. Notice that the post table has more columns now than the old table.

We only need to map the Post entity:

@Entity(name = "Post")
@Table(name = "post")
public static class Post {

    @Id
    private Long id;

    private String title;

    @Column(name = "created_on")
    private LocalDate createdOn = LocalDate.now();

    @Version
    private int version;

    //Getters and setters omitted for brevity
}

Hibernate event listeners

Now, we have to register 3 event listeners to intercept the INSERT, UPDATE, and DELETE operations for the Post entity.

We can do this via the following event listeners:

public class ReplicationInsertEventListener 
        implements PostInsertEventListener {

    public static final ReplicationInsertEventListener INSTANCE = 
        new ReplicationInsertEventListener();

    @Override
    public void onPostInsert(
            PostInsertEvent event) 
            throws HibernateException {
        final Object entity = event.getEntity();

        if(entity instanceof Post) {
            Post post = (Post) entity;

            event.getSession().createNativeQuery(
                "INSERT INTO old_post (id, title, version) " +
                "VALUES (:id, :title, :version)")
            .setParameter("id", post.getId())
            .setParameter("title", post.getTitle())
            .setParameter("version", post.getVersion())
            .setFlushMode(FlushMode.MANUAL)
            .executeUpdate();
        }
    }

    @Override
    public boolean requiresPostCommitHanding(
            EntityPersister persister) {
        return false;
    }
}

public class ReplicationUpdateEventListener 
    implements PostUpdateEventListener {

    public static final ReplicationUpdateEventListener INSTANCE = 
        new ReplicationUpdateEventListener();

    @Override
    public void onPostUpdate(
            PostUpdateEvent event) {
        final Object entity = event.getEntity();

        if(entity instanceof Post) {
            Post post = (Post) entity;

            event.getSession().createNativeQuery(
                "UPDATE old_post " +
                "SET title = :title, version = :version " +
                "WHERE id = :id")
            .setParameter("id", post.getId())
            .setParameter("title", post.getTitle())
            .setParameter("version", post.getVersion())
            .setFlushMode(FlushMode.MANUAL)
            .executeUpdate();
        }
    }

    @Override
    public boolean requiresPostCommitHanding(
            EntityPersister persister) {
        return false;
    }
}

public class ReplicationDeleteEventListener 
        implements PreDeleteEventListener {

    public static final ReplicationDeleteEventListener INSTANCE = 
        new ReplicationDeleteEventListener();

    @Override
    public boolean onPreDelete(
            PreDeleteEvent event) {
        final Object entity = event.getEntity();

        if(entity instanceof Post) {
            Post post = (Post) entity;

            event.getSession().createNativeQuery(
                "DELETE FROM old_post " +
                "WHERE id = :id")
            .setParameter("id", post.getId())
            .setFlushMode(FlushMode.MANUAL)
            .executeUpdate();
        }

        return false;
    }
}

The 3 event listeners can be registered using a Hibernate Integrator:

public class ReplicationEventListenerIntegrator 
        implements Integrator {

    public static final ReplicationEventListenerIntegrator INSTANCE = 
        new ReplicationEventListenerIntegrator();

    @Override
    public void integrate(
            Metadata metadata,
            SessionFactoryImplementor sessionFactory,
            SessionFactoryServiceRegistry serviceRegistry) {

        final EventListenerRegistry eventListenerRegistry =
                serviceRegistry.getService(EventListenerRegistry.class);

        eventListenerRegistry.appendListeners(
            EventType.POST_INSERT, 
            ReplicationInsertEventListener.INSTANCE
        );

        eventListenerRegistry.appendListeners(
            EventType.POST_UPDATE, 
            ReplicationUpdateEventListener.INSTANCE
        );

        eventListenerRegistry.appendListeners(
            EventType.PRE_DELETE, 
            ReplicationDeleteEventListener.INSTANCE
        );
    }

    @Override
    public void disintegrate(
            SessionFactoryImplementor sessionFactory,
            SessionFactoryServiceRegistry serviceRegistry) {

    }
}

And, to instruct Hibernate to use this custom Integrator, you need to set up the hibernate.integrator_provider configuration property:

<property name="hibernate.integrator_provider"
          value="com.vladmihalcea.book.hpjp.hibernate.listener.ReplicationEventListenerIntegrator "/>

Testing time

Now, when persisting a Post entity:

Post post1 = new Post();
post1.setId(1L);
post1.setTitle(
    "The High-Performance Java Persistence book is to be released!"
);

entityManager.persist(post1);

Hibernate will execute the following SQL INSERT statements:

Query:["INSERT INTO old_post (id, title, version) VALUES (?, ?, ?)"], Params:[(1, The High-Performance Java Persistence book is to be released!, 0)]

Query:["insert into post (created_on, title, version, id) values (?, ?, ?, ?)"], Params:[(2018-12-12, The High-Performance Java Persistence book is to be released!, 0, 1)]

When doing another transaction that updates an existing Post entity and creates a new Post entity:

Post post1 = entityManager.find(Post.class, 1L);
post1.setTitle(post1.getTitle().replace("to be ", ""));

Post post2 = new Post();
post2.setId(2L);
post2.setTitle(
    "The High-Performance Java Persistence book is awesome!"
);

entityManager.persist(post2);

Hibernate replicates all actions to the old_post table as well:

 Query:["select tablerepli0_.id as id1_1_0_, tablerepli0_.created_on as created_2_1_0_, tablerepli0_.title as title3_1_0_, tablerepli0_.version as version4_1_0_ from post tablerepli0_ where tablerepli0_.id=?"], Params:[(1)]

 Query:["INSERT INTO old_post (id, title, version) VALUES (?, ?, ?)"], Params:[(2, The High-Performance Java Persistence book is awesome!, 0)]

 Query:["insert into post (created_on, title, version, id) values (?, ?, ?, ?)"], Params:[(2018-12-12, The High-Performance Java Persistence book is awesome!, 0, 2)]

 Query:["update post set created_on=?, title=?, version=? where id=? and version=?"], Params:[(2018-12-12, The High-Performance Java Persistence book is released!, 1, 1, 0)]

 Query:["UPDATE old_post SET title = ?, version = ? WHERE id = ?"], Params:[(The High-Performance Java Persistence book is released!, 1, 1)]

When deleting a Post entity:

Post post1 = entityManager.getReference(Post.class, 1L);
entityManager.remove(post1);

The old_post record is deletected as well:

Query:["DELETE FROM old_post WHERE id = ?"], Params:[(1)]
Query:["delete from post where id=? and version=?"], Params:[(1, 1)]

For more details, check out this article.

Code available on GitHub.