pglogical | Logical Replication extension for PostgreSQL | Change Data Capture library
kandi X-RAY | pglogical Summary
kandi X-RAY | pglogical Summary
Logical Replication extension for PostgreSQL 15, 14, 13, 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of pglogical
pglogical Key Features
pglogical Examples and Code Snippets
Community Discussions
Trending Discussions on pglogical
QUESTION
I'm having trouble in setting up a task migrating the data in a RDS Database (PostgreSQL, engine 10.15) into an S3 bucket in the initial migration + CDC mode. Both endpoints are configured and tested successfully. I have created the task twice, both times it ran a couple of hours at most, the first time the initial dump went fine and some of the incremental dumps took place as well, the second time only the initial dump finished and no incremental dump was performed before the task failed.
The error message is now:
...ANSWER
Answered 2021-Jun-01 at 05:03Should anyone get the same error in the future, here is what we were told by the AWS tech specialist:
There is a known (to AWS) issue with the pglogical plugin. The solution requires using the test_decoding plugin instead.
- Enforce using the test_decoding plugin on the DMS Endpoint by specifying pluginName=test_decoding in Extra Connection Attributes
- Create a new DMS task using this endpoint (using the old task may cause it to fail due to dissynchronization between the task and the logs)
It sure did resolve the issue, but we still don't know what the problem really was with the plugin that is strongly suggested everywhere in the DMS documentation (at the moment).
QUESTION
I am trying to upgrade Amazon Aurora PostgreSQL from verion 9.6
to 10.14
via CloudFormation template command aws cloudformation update-stack --stack-name base --template-body file://cloudformation/base.yml
but every time update is getting rollback with one of the below messages:
Unmodifiable DB Parameter: wal_consistency_checking
Unmodifiable DB Parameter: ssl_dh_params_file
Here's the part of base.yml
looks like:
ANSWER
Answered 2021-Feb-15 at 11:29Those who are finding it difficult to upgrade, here's what I did:
- Updated the base template and added
AWS::RDS::DBParameterGroup
settings for Postgres 10 version - After that I executed
aws cloudformation update-stack --stack-name base --template-body file://cloudformation/base.yml --capabilities CAPABILITY_IAM
command to update the base stack - Then I executed:
aws rds modify-db-cluster
command from console
QUESTION
NOTE: a similar Q. was asked but was not answered here: How do I replicate a single schema in postgresql?
I am trying to replicate just one schema in a postgresql (13) database using pglogical to another server.
In my test setup on the provider node I have 2 schemas "shared" and "not_shared".
On the subscriber node I have one existing schema called "not_shared".
I want to replicate "shared" from the provider to the subscriber, I have already set up the nodes on both servers.
On the provider I create a replication set for just the "shared" schema (in theory) called "shared_repl":
...ANSWER
Answered 2020-Dec-05 at 02:16synchronize_structure just doesn't do what you want. You can copy the structure of just one schema from provider to subscriber yourself, using pg_dump -n shared -s
for example, then don't specify synchronize_structure := TRUE
when you create the subscription.
QUESTION
I'm following instructions from https://github.com/2ndQuadrant/pglogical to install pglogical on postgres 12 on Centos 8. The install seems be successful:
...ANSWER
Answered 2020-Oct-05 at 04:51Your session log tells the the server was installed as a prerequisite, but the "link" messages insinuate that there was already an incompatible client version in place. Probably you had installed PostgreSQL from the CentOS packages, but the pglogical RPMs pulled in the PGDG packages.
The error message probably means that shared_preload_libraries
contains pglogical
, but pglogical.so
could not be found in the lib
directory.
Presumably the installation process edited the configuration in your old server installation, but installed the shared object in the new one.
Upshot: you cannot use those pglogical binaries with your installation. Either switch to the PGDG RPMs or build pglogical from source.
You see that there is a certain amount of conjecture in my deductions, but that should help you solve the problem.
QUESTION
I'm trying to migrate and synchronize a PostgreSQL database using AWS DMS and I'm getting the following error.
...ANSWER
Answered 2020-Jan-31 at 06:28To enable logical decoding for an Amazon RDS for PostgreSQL DB instance
The user account requires the
rds_superuser
role to enable logical replication. The user account also requires therds_replication
role to grant permissions to manage logical slots and to stream data using logical slots.Set the rds.logical_replication static parameter to 1. As part of applying this parameter, we also set the parameters
wal_level
,max_wal_senders
,max_replication_slots
, andmax_connections
. These parameter changes can increase WAL generation, so you should only set therds.logical_replication
parameter when you are using logical slots.Reboot the DB instance for the static
rds.logical_replication
parameter to take effect.Create a logical replication slot as explained in the next section. This process requires that you specify a decoding plugin. Currently we support the test_decoding output plugin that ships with PostgreSQL.
The last item can be done with the following command:
QUESTION
I looked at the docs of pglogical.
But I could not find a way to list all replication sets, which I created with create_replication_set()
.
How to list all replication sets?
...ANSWER
Answered 2019-Sep-23 at 20:47According to the source, that should be as simple as
QUESTION
I am replicating data from Postgres 10.4 to another Postgres 10.4 instance using logical replication.
The subscriber has several triggers that log events to a single table. This table has a trigger that executes another function (that returns a trigger) to call NOTIFY for a downstream listener.
Trigger on the audit table looks like this:
...ANSWER
Answered 2019-Sep-18 at 14:42Update: It looks like this is a bug with PostgreSQL 10.4, and at least up to at least 11.4. There's an experimental patch available here.
According to this post on the PostgreSQL mailing list it looks like by default logical replication won't cause triggers to fire on replicas because tables generally have the "local" replication role and on logical replicas the data gets inserted with the "replica" role.It looks like you can alter your table to always fire triggers, including on replication by doing the following (see the documentation here):
QUESTION
Is there a way to convert a UUID to a UUID5 in an existing postgres data set?
I have a master/slave setup with pglogical replicating specific tables & columns. I need to give access to this replicated data to a third party, but I want to anonymise it as much as possible.
Every row in a table uses a UUID (generated in code) as primary key, but in order to ensure anonymity I need to ensure that the primary keys of the master & replicated tables are one-way transformed to make back-tracing harder. Some of the replicated tables refer to each other as well, meaning I can't just exclude those columns - values need to match between tables. Hopefully this makes sense so far...
Is there a way of using uuid-ossp
and uuid_generate_v5()
to essentially use the existing UUIDs as the namespace for a v5 UUID, returning those in the view to the third party (or even to use to transform the specific columns during replication)?
I'm a little stumped. If I can get this sorted entirely in postgres I won't have to muck about with additional tools to transform the data before allowing the third party access, I can just give them real-time data instead of batched. But if I need to involve some additional code then I guess that's not the end of the world...
Thanks in advance!
...ANSWER
Answered 2019-May-01 at 16:55Simply provide read-only views to a third-party (in a separate schema) and limit data access to them to those views (or schema).
A view can then be defined with a query such as:
QUESTION
I'm experimenting with a 2-node bidirectional replication using pglogical-9.6. I'm able to consistently cause a seg fault when I set pglogical.conflict_resolution = 'last_update_wins'.
I'm create a BEFORE DELETE TRIGGER to prevent insertion some values into replicated table.
...ANSWER
Answered 2019-Apr-04 at 15:21A segmentation fault is always a bug unless it is caused by hardware problems or data corruption.
You should file a bug with 2ndQuadrant; it is their software.
Make sure to include the reproducer with the bug report.
QUESTION
There are two PostgreSQL 9.6 nodes subscribed to each other via pglogical. If node A inserts a row into the replicated table then node B sees it and vice versa.
However, when I update a row on one node, then subsequent SELECT
queries on both nodes will keep returning different results - the current one and some of the previous ones.
Moreover, there are log entries about replication conflicts in the logs of both nodes.
Why does that happen and how do I fix that?
upd: setting pglogical.conflict_resolution to last_update_wins helps. Might consider other options of conflict resolution too
...ANSWER
Answered 2019-Mar-29 at 07:20Multi-master replication is difficult.
There are conflicts that are bound to occur unless your application is aware of and specifically tailored to multi-master replication:
Rows inserted on different nodes with the same (automatically generated primary key must conflict.
If you modify the primary key of a row on one node while updating or deleting it on another, the databases will “drift apart”, leading to future conflicts.
You will have to fix your application so that it avoids problems like the above, and you will have to manually find and resolve all the conflicts that occurred so far.
Here is an example of the second case:
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install pglogical
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page