atomically | ActiveRecord extension | Object-Relational Mapping library

I have a relatively simple case where:

1. My program will be receiving updates via Websockets, and will be using these updates to update it's local state. These updates will be very small (usually < 1-1000 bytes JSON so < 1ms to de-serialize) but will be very frequent (up to ~1000/s).
2. At the same time, the program will be reading/evaluating from this local state and outputs its results.
3. Both of these tasks should run in parallel and will run for the duration for the program, i.e. never stop.
4. Local state size is relatively small, so memory usage isn't a big concern.

The tricky part is that updates need to happen "atomically", so that it does not read from a local state that has for example, written only half of an update. The state is not constrained to using primitives and could contain arbitrary classes AFAICT atm, so I cannot solve it by something simple like using Interlocked atomic operations. I plan on running each task on its own thread, so a total of two threads in this case.

To achieve this goal I thought to use a double buffer technique, where:

1. It keeps two copies of the state so one can be read from while the other is being written to.
2. The threads could communicate which copy they are using by using a lock. i.e. Writer thread locks copy when writing to it; reader thread requests access to lock after it's done with current copy; writer thread sees that reader thread is using it so it switches to other copy.
3. Writing thread keeps track of state updates it's done on the current copy so when it switches to the other copy it can "catch up".

That's the general gist of the idea, but the actual implementation will be a bit different of course.

I've tried to lookup whether this is a common solution but couldn't really find much info, so it's got me wondering things like:

1. Is it viable, or am I missing something?
2. Is there a better approach?
3. Is it a common solution? If so what's it commonly referred to as?
4. (bonus) Is there a good resource I could read up on for topics related to this?

Pretty much I feel I've run into a dead-end where I cannot find (because I don't know what to search for) much more resources and info to see if this approach is "good". I plan on writing this in .NET C#, but I assume the techniques and solutions could translate to any language. All insights appreciated.

In one of his great video, Jon Gjengset implements a mutex to notably understand the effect of std::sync::atomic::Ordering. The code is very simple : create a mutex that holds an integer and start many threads to add 1 to the integer concurrently and see the results. The code is here (I reproduce stricly Jon example) : https://github.com/fmassot/atomics-rust

When using correct ordering, we expect the program to make additions atomically and check the result as the sum of all added values. The code does several times on each thread the following actions :

• call compare_exchange_weak with Ordering::Acquire to get the lock
• on success increment the value by one
• release the lock with Ordering::Release

Unfortunately it does not seem to work on linux/x86_64 nor on macbook/arm64.

The results when running cargo r --release are sometimes correct, sometimes wrong like this:

Redis RPUSH docs here suggest that the return value of RPUSH is the length of the list after the push operation.

However, what's not clear to me is:

1. Is the result of RPUSH the length of the list after the push operation atomically, (so the result is definitely the index of the last item just added by RPUSH) or...
2. Is it possible other RPUSH operations from concurrent Redis clients could have executed before the RPUSH returns, so that you are indeed getting the new length of the list, but that length includes elements from other RPUSH commands?

Thanks!

I am creating a Smart Contract (BEP20 token) based on the BEP20Token template (https://github.com/binance-chain/bsc-genesis-contract/blob/master/contracts/bep20_template/BEP20Token.template). The public contructor was modified to add some token details. However all of the standard functions are giving compile time issues like Overriding function is missing.

** here is the source code **

The problem: I'm implementing a non-blocking data structure, where threads alter a shared pointer using a CAS operation. As pointers can be recycled, we have the ABA issue. To avoid this, I want to attach a version to each pointer. This is called a versioned pointer. A CAS128 is considered more expensive than a CAS64, so I'm trying to avoid going above 64 bits.

I'm trying to implement a versioned pointer. In a 32b system, the versioned pointer is a 64b struct, where the top 32 bits are the pointer and the bottom 32 is its version. This allows me to use CAS64 to atomically alter the pointer.

I'm having issues with a 64b system. In this case, I still want to use CAS64 instead of CAS128, so I'm trying to allocate a pointer aligned to 4GB (i.e., 32 zeros). I can then use masks to infer the pointer and version.

The solutions I've tried using alligned_malloc, padding, and std::align, but these involve allocating very large amounts of memory, e.g., alligned_malloc(1LL << 32, (1LL << 32)* sizeof(void*)) allocates 4GB of memory. Another solution is using a memory mapped file, but this involves synchronization that we're trying to avoid.

Is there a way to allocate 8B of memory aligned to 4GB that I'm missing?

So in rust, I want to have an atomically referenced count pointer, but I want that pointer to also be atomic. Meaning that for whatever pointer the Arc holds, it will free the memory pointed to by the pointer when the reference count is zero.

Say I have the following code in C++

How do I atomically check-and-set something in Ecto Repo? I want to make sure no other process changed any part of the struct in parallel, even if the two writes are not-overlapping and read the same data.

For example,

I'm using MariaDB 10.3 (I presume mySQL behaves similarly), to which I make several concurrent connections.

From one of those connections, I insert the same value (let's say 'xxx' below) into several different tables, like so:

Dual write is a problem when we need to change data in 2 systems: a database (SQL or NoSQL) and Apache Kafka (for example). The database has to be updated and messages published reliably/atomically. Eventual consistency is acceptable but inconsistency is not.

Without 2 phase commit (2PC) dual write leads to inconsistency.

But in most cases 2PC is not an option.

Transactional Outbox is a microservice architecture pattern where a separate Message Relay process publishes the events inserted into database to a message broker.

Multiple Message Relay processes running in parallel lead to publishing duplicates (2 processes read the same records in the OUTBOX table) or unordering (if every process reads only portion of the OUTBOX table).

A single Message Relay process might publish messages more than once also. A Message Relay might crash after processing an OUTBOX record but before recording the fact that it has done so. When Message Relay restarts, it will then publish the same message again.

How to implement a Message Relay in Transactional Outbox patterns, so that risk of duplicate messages or unordering is minimal and the concept works with all SQL and NoSQL databases?