bucardo | Bucardo multimaster and master/slave Postgres replication | Change Data Capture library
kandi X-RAY | bucardo Summary
kandi X-RAY | bucardo Summary
Bucardo - a table-based replication system.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of bucardo
bucardo Key Features
bucardo Examples and Code Snippets
Community Discussions
Trending Discussions on bucardo
QUESTION
I'm on an Ubuntu-derived system (specifically pop_os from System76). I am trying to get Bucardo installed, but it appears to be failing because plperlu
is not functional.
In a psql
prompt:
ANSWER
Answered 2021-Jun-22 at 22:13The issue is that there are multiple versions of Postgres running as servers. Running pg_lsclusters
will show what they are. The CREATE EXTENSION
fails as the extension code for the version of Postgres that the command is being run on has not been installed.
QUESTION
I am trying to migrate an Aurora cluster from one of our accounts to another. We actually do not have a lot write requests and the database itself is quite small, but somehow we decided to minimize the downtime.
I have looked into several options
- Use snapshot: cut off the mutation in source DB, take snapshot, share and restore in another account. This would definitely introduce some downtime
- Use Aurora cloning: cut off the mutation in source DB, clone the cluster in target account and switch to the target DB. According to AWS, the cloning is much faster than taking and restoring a snapshot, so the downtime should be shorter.
- I am not sure if I can use DMS to do this as I did not find useful doc/tutorials about moving Aurora across accounts. Also, I am not sure whether DMS will sync any write requests to target DB during migration.
- If DMS can not live sync, then I probably should use Bucardo to live migrate.
ANSWER
Answered 2021-Mar-13 at 19:01Looking at the docs, AWS Aurora with PostgreSQL compatibility is allowed as source & target endpoints. So, answering your question, yes it's possible.
Obviously, your source Aurora DB should be accessible from the target account. Check that the DB endpoint is public and the traffic is not restricted by ACLs rules or SGs rules.
Also, if you want to enable ongoing replication, you need to grant rds_replication
(or rds_superuser
) role to the source database user. Link to the docs.
QUESTION
Currently I am setting up Master/Master Replication with bucardo between 5 Nodes on different locations (should provide location transparency). The database holds ~500 Tables which should be replicated. I grouped them into smaller replication herds of 50 Tables at maximum based on their dependency on each other. All tables have primary keys defined and the sequencers on each node are set up to provide system wide unique identities (based on residue class)
To get an initial database on each node, I made a --data-only
custom format pg_dump into a File and restored this on each node via pg_restore
. Bucardo sync is setup with the bucardo_latest
strategy to resolve conflicts. Now when I start syncing bucardo is deleting all datasets in the origin database first and inserting it again from one of the restored nodes, because all restored datasets have a "later timestamp" (the point in time when I called pg_restore). This ultimately prohibits the inital startup as bucardo needs very much time and also fails, as there are lots of datasets to solve and timeouts often too short.
I also have 'last_modified' timestamps on each table which are managed by UPDATE triggers, but as I understand it, pg_dump inserts data via COPY, and therefore these triggers don't get fired.
- Which timestamp does bucardo use to find out who is
bucardo_latest
? - Do I have to call
pg_dump
with something likeset SESSION_REPLICATION_ROLE = 'replica';
?
I just want bucardo to keep track of every new change, not executing pseudo changes because of the restore.
EDIT: pg_restore has definitely fired several triggers at restore time...as said I keep track on user and last modification date in each table, and those values are set to the user and timestamp when the restore was done. I am aware, that I can set SESSION_REPLICATION_ROLE for a plain text format restore via psql
. Is this also possible for pg_restore somehow?
ANSWER
Answered 2020-Aug-06 at 07:40The common approach is make the dump/restore process before configure the replication.
So an option will be:
- drop the
bucardo
schema in each database - do a
bucardo remove
for each object (most of them allow useall
, likebucardo remove table all
- dump/restore your data
- Configure again the replication. Just make sure that when adding the sync, set the option
onetimecopy=0
. It's the default but I feel safer making it explicit.
Which timestamp does bucardo use to find out who is bucardo_latest?
bucardo
handles its own timestamp value. Each table should have a trigger named like bucardo.delta_myschema_mytable
that makes and insert in a table named like bucardo.delta_myschema_mytable
. This table has a column txntime timestamp with time zone not null default now()
and this is the timestamp used.
Do I have to call pg_dump with something like set SESSION_REPLICATION_ROLE = 'replica';?
AFAIK, if bucardo
triggers are already set in the tables, the option --disable-triggers
of pg_restore
should do the trick.
You can also check these articles about working with large databases and the use of session_replication_role
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install bucardo
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page