drbd | LINBIT DRBD kernel module

If you're doing Active/Passive (Active/Standby) clustering, there is no difference between a MongoDB on DRBD vs. MongoDB on any other block device.

If you had multiple active MongoDB's accessing a dual-primary (Active/Active) DRBD device, that's where the potential for corruption would come in.

LINSTOR manages storage in a cluster of nodes replicating disk space inside a LVM or ZFS volume (or bare partition I'd say) by using DRDB (Distributed Replicated Block Device) to replicate data across the nodes, as per the official docs:

"LINSTOR is a configuration management system for storage on Linux systems. It manages LVM logical volumes and/or ZFS ZVOLs on a cluster of nodes. It leverages DRBD for replication between different nodes and to provide block storage devices to users and applications. It manages snapshots, encryption and caching of HDD backed data in SSDs via bcache."

So I'd say yes, you really need to have the driver on every node on which you want to use the driver (I did see Docker's storage plugin try to mount the DRBD volume locally)

However, you do not necessarily need to have the storage space itself on the compute node, since you can mount a diskless DRBD resource from volumes that are replicated on separate nodes so I'd say your idea should work, unless there is some bug in the driver itself I didn't discover yet: your compute node(s) needs to be registered as being a diskless node for all the required pools (I didn't try this but remember reading it's not only possible but recommended for some types of data migrations).

Of course if you don't have more than 1 storage nodes you don't gain much from using LINSTOR/drbd (node or disk failure will leave you diskless). My use case for it was to have replicated storage across different servers in different datacenters, so that the next time one burns to a crisp 😅 I can have my data and containers running after minutes instead of several days...

OK, I solved it, changing the resource in pacemaker.

The first thing to note is that the high availability functionality (both shared-store and replication - options #2 & #3) configured via ha-policy is designed for use in a local data center with high speed, low latency network connections. It is not designed for disaster recovery.

The problem specifically with network-based data replication for you is that replication is synchronous which means there's a very good chance it will negatively impact performance since every durable message will have to be sent across the country from one data center to the other. Also, if the replicating broker fails then clients will automatically fail-over to the backup in the other data center.

Using a solution based on block-storage (e.g. Ceph or DRDB) is viable, but it's really an independent thing outside the control of ActiveMQ Artemis.

The mirror broker connection was designed with the disaster recovery use-case in mind. It is asynchronous so it won't have nearly the performance impact of replication, and if the mirroring broker fails clients will not automatically fail-over to the mirror.

Assuming you want to keep doing the check every 4 minutes, but suppress repeated mails for the same day or for 24 hours, you have to save the time when you sent the mail.

You can use the modification time of a file for this purpose.

1. # drbdadm disconnect # on both nodes
2. Change the IP address within the /etc/drbd.d/.res file on both nodes
3. # drbdadm adjust # on both nodes

When DRBD starts it runs through a series of steps, if any one of these fails it will skip the latter steps. One of these steps is to create a TCP socket. If it fails to do that, it will skip the latter steps, one of which is attaching to the disk.

I suspect in your case, that DRBD fails to find the IP address to use present on the system, and thus skips the latter steps of attaching to the disk, and thus starts up connectionless and diskless. Make sure the IP address you're changing DRBD to use is already present on the systems.