migrator | A backup solution and data migration utility for Android | Continuous Backup library

With this playbook

this is my model

Below is my JSON file

I am using redis_OM in order to make it easier to do aggregations using redisearch module. The thing is that I want to upload some data to redis everyday without keeping the one uploaded the day before, that is to say, I want either to make the old data expire or overwrite it with the new one.

I am doing it with the example of the Redis documentation and redis cloud. This is my model:

I have built a test app using nestjs + Sequelize ORM + docker database (as of now local). As per documentation, I am using umzug library and AWS Lambda SAM template and triggering lambda handler. Below is the code for it. Connection Pooling is implemented to reuse existing sequelize connection. Below is the lambdaEntry.ts file where I trigger umzug.up() function. It is triggering but not migrating files.

When done from command prompt node migrate up it works correctly. I am testing using sam invoke command to test it.

I have an app consisting of multiple services, each with its own postgres database. I want to deploy it to AWS. Kube is too complicated for me, so I decided to use AWS ECS for services + AWS RDS for DBs. And deploy everything using Terraform.

I have a CI/CD pipeline set up, which upon a merge to the staging branch, builds, tests, and deploys the app to the corresponding environment. Deploying basically consists of building and pushing docker images to AWS ECR and then calling terraform plan/apply.

Terraform creates/updates VPC, subnets, ECS services with tasks, RDS instances, etc.

This works.

But I'm not sure how to apply db migrations.

I have a separate console app whose only purpose is to apply migrations and then quit. So I can just run it in the CI/CD pipeline before or after applying terraform. However, before doesn't work because if it's the very first deployment then the databases wouldn't exist yet, and after doesn't work because I want to first apply migrations and then start services, not the other way around.

So I need some way to run this migrator console app in the middle of terraform deployment – after rds but before ecs.

I read an article by Andrew Lock where he solves this exact problem by using jobs and init containers in Kubernetes. But I'm not using Kube, so that's not an option for me.

I see in AWS ECS docs that you can run standalone tasks (one-off tasks), which is basically what I need, and you can run them with AWS CLI, but whilst I can use the cli from the pipeline, I can't use it in the middle of terraform doing its thing. I can't just say to terraform "run some random command after creating this resource, but before that one".

Then I thought about using AWS Lambda. There is a data source type in Terraform called aws_lambda_invocation, which does exactly what it says in the name. So now I'm thinking about building a docker image of migrator in the build stage of the pipeline, pushing it to AWS ECR, then in terraform creating an aws_lambda_function resource from the image and aws_lambda_invocation data source invoking the function. Make ECS depend on the invocation, and it should work, right?

There is one problem with this: data sources are queried both when planning and applying, but I only want the migrator lambda to run when applying. I think it could be solved by using count attribute and some custom variable in the invocation data source.

I think this approach might work, but surely there must be a better, less convoluted way of doing it? Any recommendations?

Note: I can't apply migrations from the services themselves, because I have more than one instance of each, so there is a possibility of two services trying to apply migrations to the same db at the same time, which would end badly.

If you are wondering, I use .NET 5 and GitLab, but I think it's not relevant for the question.

In this function I compile rem to px and em to px.

I'm new to NodeJs, and I'm following the full-stack tutorial made by Ben Awad https://www.youtube.com/watch?v=I6ypD7qv3Z8&t=7186s.

After setting up my server and everything works fine. I added express-session for session storage and linked it with Redis using Redis-connect and Redis client.

This is my index.ts:

I came across this error when modifying a DB first project (using fluent migrator) and scaffolding the EF context to generate models. I have reproduced it by making a code-first simplification. This means that I can't accept answers that suggest modifying the annotations or fluent configuration, because this will be deleted and recreated on the next migration and scaffold.

The simplified idea is that a device has:

• many attributes
• many histories representing changes to the device over time
  • each history entry has an optional location

IOW you can move a device around to locations (or no location) and keep track of that over time.

The code-first model I came up with to simulate this is as follows: