mongock | Lightweight MongoDB migration tool for Java | Microservice library

In the new version 5, the basic change is that a ChangeUnit holds the unit of execution. That's normally done in the method annotated with @Execution, so the first approach is just doing the same you are doing but calling the @Execution method:

If I am not wrong, I see two parts in this problem

1. Where to store the user's configuration/how to pass it to Mongock
2. Based on that configuration, initialise your MongoDB database.

I guess we both agree the second step is clear: Mongock somehow takes the configuration and performs the migration.

Regarding of how to provide that configuration to Mongock, in your first option you mention to add a field for the environment, right? Are you saying you have the same database for production and dev environments? In my opinion, unless you have a really good reason for it, I wouldn't recommend that. In my ideal world, you would have a database per environment, so that field for environment is not needed. Mongock just needs to take the information from that collection that has been previously added.

Then the issue is how that configuration data about users is added to the database. If it can be hardcoded from your java code you can easily do it in Mongock. If you are using Spring you can use @Profile to talle which changeLog execute in which environment.

If not, another idea is passing those users as environment variables or configuration. If you are using kubernetes, ansible of that kind of tool, providing it by environment variable is not a bad idea. But I guess I don't have enough information to get more committed with any option.

That's due to incompatible versions.

As you can see in this compatibility table, the version of Mongock you are using is compatible with spring-data 3.x, but you are using 2.x instead.

Mongock provides two drivers for spring-data, mongodb-springdata-v3-driver, for spring-data 3.x and mongodb-springdata-v2-drver for spring-data 2.2.X

I have a couple of suggestions:

1. Either upgrade your spring-data dependency to 3.x, or use mongodb-springdata-v2-drver(first one recommended)
2. Use dependency management so you don't need to specify the version for each dependency. You can do it for spring and for Mongock

With that you should fix your issue ;)

As you have mentioned, MongoDB transaction model has some limitations. Please see official MongoDB documentation.

Having that into account, you cannot wrap that operation in a transaction, so you cannot use it in a standard ChangeLog, if you are using transactions.

However, the good new i that, as always :D, Mongock team thought about it and will provide a mechanism for pre-transactions operations.

We expect to have a release candidate ready around June. However, if this is an urgent need, please send us an email to dev@cloudyrock.io and we'll try to find a solution.

I am going to provide the short answer first and then the long one. I suggest you to read the long one too in order to understand it properly.

By default, Mongock guarantees that the ChangeLogs/changeSets will be run only by one pod at a time. The one owning the lock.

What really happens behind the scenes(if it's not configured otherwise) is that when a pod takes the lock, the others will try to acquire it too, but they can't, so they are forced to wait for a while(configurable, but 4 mins by default)as many times as the lock is configured(3 times by default). After this, if i's not able to acquire it and there is still pending changes to apply, Mongock will throw an MongockException, which should mean the JVM startup fail(what happens by default in Spring).

This is fine in Kubernetes, because it ensures it will restart the pods. So now, assuming the pods start again and changeLogs/changeSets are already applied, the pods start successfully because they don't even need to acquire the lock as there aren't pending changes to apply.

Now, assuming the lock and the mutual exclusion is clear, I'd like to point out a potential issue that needs to be mitigated by the the changeLog/changeSet design.

This issue applies if you are in an environment such as Kubernetes, which has a pod initialisation time, your migration take longer than that initialisation time an the Mongock process is executed before the pod becomes ready/health(and it's a condition for it). This last condition is highly desired as it ensures the application runs with the right version of the data.

In this situation imagine the Pod starts the Mongock process. After the Kubernetes initialisation time, the process is still not finished, but Kubernetes stops the JVM abruptly. This means that some changeSets were successfully executed, some other not even started(no problem, they will be processed in the next attempt), but one changeSet was partially executed and marked as not done. This is the potential issue. The next time Mongock runs, it will see the changeSet as pending and it will execute it from the beginning. If you haven't designed your changeLogs/changeSets accordingly, you may experience some unexpected results because some part of the data process covered by that changeSet has already taken place and it will happen again.

This, somehow needs to be mitigated. Either with the help of mechanisms like transactions, with a changeLog/changeSet design that takes this into account or both.

Mongock currently provides transactions with “all or nothing”, but it doesn’t really help much as it will retry every time from scratch and will probably end up in an infinite loop. The next version 5 will provide transactions per ChangeLogs and changeSets, which together with good organisation, is the right solution for this.

Meanwhile this issue can be addressed by following this design suggestions.

It's not really duplicated. It creates a changelog entry per execution.

However, we understand this is not the more common desired behaviour, we are releasing a bugfix(4.3.8) for version 4 in the next days, probably today.

In version 5, which is under development, we'll keep this by default, plus updating the last_execution field we'll add, and add the option to insert a new entry per execution if desired.

As discussed in this Github issue, the issue was resolved by upgrading to version 4.3.7.

Anything you can do with normal MongoTemplate, you can do it as well with MongockTemplate.

In this case, you can tune the UpdateOptions and some other stuff, but it can be as easy as

Your institution is right. Mongock's lock is just internal. Also it's worthy knowing that you can configure how long this lock is taken for, how long Mongock waits for it, if it cannot acquire it and how long it will be trying. All that it's in the Mongock's documentation

Speaking about the transactions, Mongock delegates completely the transaction to the underneath framework. I don't fully understand your question, but I can explain which type of transactions are offered by Mongock and you can have a better understanding of the impact.

• All or nothing: The entire Mongock process is wrapped in a transaction, if it fails at any point, the entire migration is rolled back.(currently provided)
• Per changeSet(method): Every changeSet successfully executed is committed. If it fails in the middle, it's rolled back.(Will be provided in version 5)
• Per changeLog(class): The entire changeLog class is wrapped in a transaction, so it will be committed only if all its changeSets are successfully executed.(Will be provided in version 5)

General speaking, the recommended way to organise your migrations is per changeLog. A changeLog should represent a migration step, it shouldn't be designed in a feature approach. For example if you have in your model Client and Bill, you shouldn't design a changeLog for Client and another for Bill, having in them the different migration you apply over the time. Instead, you should do it in such a way that a changeLog represent a migration step, affecting the entire model, so you can identify your migration history through your changeLogs. From this point of view, Mongock's best practice is to use per changeLog transactions