Saturday, April 6, 2019

WOW!!! What ? Now, I can get timestamp of created table and committed rows in PostgreSQL ? That's Awesome !!

I have seen people, DBA questioning whether there any way to check when the table was created in Postgres.
Till 9.5 there was NO appropriate answer for the question but after 9.5 we can say yes we can get create time of table because of new feature i.e Commit timestamps.

track_commit_timestamp

Record commit time of transactions. This parameter can only be set in PostgreSQL.conf file or on the server command line.
The default value is off. This value is a boolean value.
Change in value needs postmaster restart.

Test Case:

Without enabling track_commit_timestamp

SELECT pg_xact_commit_timestamp(xmin), * FROM pg_class where relname= 'tracking';
ERROR: could not get commit timestamp data
HINT: Make sure the configuration parameter "track_commit_timestamp" is set.
Enabling track_commit_timestamp
alter system set track_commit_timestamp='on';

Then restart the Postgres service(postmaster spinup).
create table tracking(i int);
To query that information, use the function pg_xact_commit_timestamp(transaction_id) to find out when rows were
INSERTed/UPDATEed, call the pg_xact_commit_timestamp function passing in the xmin system column
When a new table was created the new entry will be added to pg_class.

From pg_class we can take the last commit time for a table in our case its "tracking"(table name).
postgres=#
SELECT pg_xact_commit_timestamp(xmin), * FROM pg_class where relname= 'tracking';
-[ RECORD 1 ]------------+---------------------------------
relnamespace | 2200
pg_xact_commit_timestamp | 2016-10-08 00:41:57.824771+05:30
relname | tracking
reltype | 16508
Insert and update
postgres=# insert into tracking values (1);
INSERT 0 1
INSERT 0 1
postgres=# insert into tracking values (2);
postgres=#
SELECT pg_xact_commit_timestamp(xmin), * FROM tracking ;
   pg_xact_commit_timestamp | i
----------------------------------+---
2016-10-08 12:32:11.223785+05:30 | 1
2016-10-08 12:32:13.862489+05:30 | 2
postgres=# update tracking set i=10 where i=2;
UPDATE 1
postgres=#

SELECT pg_xact_commit_timestamp(xmin), * FROM tracking ;
   pg_xact_commit_timestamp | i
----------------------------------+----
2016-10-08 12:32:11.223785+05:30 | 1
2016-10-08 12:34:21.379262+05:30 | 10
Basically track_commit_timestamp records the committed row values(insert,update).
Note: Make sure that user must enable the track_commit_timestamp if he wants to record row's committed time.
The advantage of enabling this parameter can be used in multimaster(BDR) systems or pg_logical(replication setup) to over
comer from conflict resolution
Performance Impact:
This GUC added by the community to track the last committed timestamp of rows in tables.
Once enabled, commit timestamps are tracked for all transactions in the DB cluster, which causes a performance hit.If you only need it for one or a few tables, rather add timestamps to involved tables manually creating a trigger on tables and capturing the timestamp would be preferable .


Friday, March 22, 2019

Updated pg_rewind feature in PostgreSQL-9.6 for timeline switches

In this test case I am trying to make my previous Master as Master by rollbacking the changes happen into Standby.

Step 1:Install PostgreSQL-10 on both Master and Standby. Follow the Installation steps from Installation_Steps

Step 2: Now, Create Data Directory and make the changes to config files.

initdb -D /var/lib/pgsql/10/data/

vi /var/lib/pgsql/10/data/postgresql.conf

      port=5432
      listen_addresses = '*'
      wal_level = replica
      max_wal_senders = 10
      wal_keep_segments = 32
      wal_log_hints = on
      archive_mode    = on
      archive_command = 'cp %p /var/lib/pgsql/pg_archive/%f' 

vi /var/lib/pgsql/10/data/pg_hba.conf

host replication all 0.0.0.0 md5

Step 3: Start Master

pg_ctl -D /var/lib/pgsql/10/data/ start

Step 4: Upload the data in Master

/usr/pgsql-10/bin/pgbench -i -s 10

Step 5: Build the Streaming Replication

/usr/pgsql-10/bin/pg_basebackup -h 127.0.0.1 -p 5432 -U replication -D /var/lib/pgsql/10/slave-data/ -Xs -c fast -R

vi /var/lib/pgsql/10/slave-data/postgresql.conf

      port=6432
      listen_addresses = '*'
      hot_standby='on'

vi /var/lib/pgsql/10/slave-data/recovery.conf

         standby_mode = 'on'
         primary_conninfo = 'host=127.0.0.1 port=5432 dbname=postgres user=postgres'
         #recovery_target_timeline = 'latest'
         #restore_command = 'cp /var/lib/pgsql/pg_archive/%f %p'


Step 6:  Start the Standby by server and the promote it

/usr/pgsql-10/bin/pg_ctl -D /var/lib/pgsql/10/slave-data/ start
/usr/pgsql-10/bin/pg_ctl -D /var/lib/pgsql/10/slave-data/ promote

Step 7: Now load the data using pgbench into New Master(Which was earlier was standby) and Stop the New Master

/usr/pgsql-10/bin/pgbench -T 60 -p 6432 -d postgres
/usr/pgsql-10/bin/pg_ctl -D /var/lib/pgsql/10/slave-data/ -m fast stop

Step 8: Run pg_rewind to make the Previous Master and Rollback the changes happen in New Master

/usr/pgsql-10/bin/pg_rewind --target-pgdata /var/lib/pgsql/10/slave-data/ -n --source-server="port=5432 user=postgres dbname=postgres"
/usr/pgsql-10/bin/pg_rewind --target-pgdata /var/lib/pgsql/10/slave-data/ --source-server="port=5432 user=postgres dbname=postgres"

Step 9:  Edit postgres.conf file and create recovery.conf once it is done then start the server

vi /var/lib/pgsql/10/slave-data/postgresql.conf

      port=6432
      listen_addresses = '*'
      hot_standby='on'

vi /var/lib/pgsql/10/data/recovery.conf

    standby_mode = 'on'
    primary_conninfo = 'host=127.0.0.1 port=5432 dbname=postgres user=postgres'
    #recovery_target_timeline = 'latest'
    #restore_command = 'cp /var/lib/pgsql/pg_archive/%f %p'

/usr/pgsql-10/bin/pg_ctl -D /var/lib/pgsql/10/slave-data/ start

postgres=# select * from pg_stat_replication ;
 pid  | usesysid | usename  | application_name | client_addr | client_hostname | client_port |         backend_start         | bac
kend_xmin |   state   | sent_lsn  | write_lsn | flush_lsn | replay_lsn | write_lag | flush_lag | replay_lag | sync_priority | sync
_state
------+----------+----------+------------------+-------------+-----------------+-------------+-------------------------------+----
----------+-----------+-----------+-----------+-----------+------------+-----------+-----------+------------+---------------+-----
-------
 6011 |       10 | postgres | walreceiver      | 127.0.0.1   |                 |       55866 | 2019-03-21 11:50:00.193565+00 |
          | streaming | 0/D02BEE0 | 0/D02BEE0 | 0/D02BEE0 | 0/D02BEE0  |           |           |            |             0 | asyn
c
(1 row)

-bash-4.2$ psql
psql (10.7)
Type "help" for help.

postgres=# create table test(i int);
CREATE TABLE
postgres=# \q
-bash-4.2$ psql -p 6432    (Standby )
psql (10.7)
Type "help" for help.

postgres=# \dt
              List of relations
 Schema |       Name       | Type  |  Owner
--------+------------------+-------+----------
 public | pgbench_accounts | table | postgres
 public | pgbench_branches | table | postgres
 public | pgbench_history  | table | postgres
 public | pgbench_tellers  | table | postgres
 public | test             | table | postgres
(5 rows)

postgres=#

Predefined roles introduced in PostgreSQL 14 reduced my responsibilities

  User management is a crucial role in the database, and the DBA should assign suitable permissions to users. For example, some users requir...