cloud-deploy | Governing Self-Service On-Demand Cloud Provisioning | GCP library

I have come pretty far in case of setting up everything necessary for building, deploying and running an docker image to AWS ECS.

So far I have acheived:

• docker image is built and pushed to my private gitlab container registry
• AWS ECS cluster, services and task definitions are setup, up and running. I have two task definitions within one cluster service. Both containers are running - on initial startup, ECS pulled images from gitlab registry successfully (React webapp, images differ in some frontend configurations only, thats why I have two containers/task definitions).

Now the problem: I am trying to follow official gitlab documentation for deploying image to ECS. Just to be clear, I am using Fargate, not EC2. I have problem especially with gitlab-ci.yml, because in documentation, they refer to template which should be used for deployment.

Template itself consists of another 2 templates:

1. build template
2. deploy template

Since I cannot rely on these templates as they are, not just because they didn't work when I configured everything as described in the documentation, but mainly because I have two task definitions which must be deployed witin one CD job.

I have managed to copy deployment scripts into my own gitlab-ci.yml file in my repo. This is current version, which has only one task definition:

I have an AWS Layer in aws account A and a Lambda function referencing that layer in account B. I create the layer from account A and permissioned account B to it using:

I am trying to run a spring batch with remote partitioning on K8s cluster using spring-cloud-deployer-kubernetes. Eventhough I have configured a service account and mentioned in my application properties the below way

I don't know if this question is appropriate for this forum.

I am developing a C# ASP.NET Core webservice and a client-side library that uses this webservice. At this time, both the code for the server-side part and the client-side part live in the same Git repository and Visual Studio solution. The server-part is intended to run in Azure.

I have a combination of unit tests and integration tests, including end-to-end integration tests that test both client and server at the same time. I have written tests that can run the client either (depending on configuration) against an instance of the server in Azure (with a URI that gets injected) OR against a local in-memory server instance using Microsoft.AspNetCore.TestHost.

This works great. The in-memory server instance is very convenient when coding. Our Continuous Integration (Jenkins) server runs the integration tests against a cloud-deployed server (i.e., an instance of our ASP.NET Core webservice that has been deployed to Azure). This testing-against-the-cloud provides vital value but is too cumbersome to do while coding locally.

Here comes the problem: The server is intended to run on .NET 5. The client part needs to be able to run on both .NET Framework 4.8 and .NET 5. We want to test that a client running 4.8 can work correctly against a server running .NET 5 (i.e., an instance of our ASP.NET Core webservice that has been compiled against .NET 5).

It is of course possible to run integration tests using .NET 4.8 against a cloud-deployed server that uses .NET 5.

It is probably not possible - or at least not convenient - to run these tests locally (since the same test process would have to run both .NET 4.8 and .NET 5). But I guess we can live with that.

Of course I can build the server part against .NET 5 and deploy it to the server and run ONE integration test against that. But that doesn't solve my long-term problem. I want to be able to regularly run all my integration tests. More specifically, what I would like to achieve is:

1. It should be easy to run the integration tests locally with a .NET 5 client against a locally hosted .NET 5 server (as today).
2. The Jenkins server should run the integration tests with a .NET 5 client against a cloud-deployed .NET 5 server (as today).
3. The Jenkins server should ALSO run the integration tests with a .NET 4.8 client against a cloud-deployed .NET 5 server.
4. I don't want to have to maintain two sets of tests. I want it to be easy to run all the same tests in all 3 scenarios.

I can achieve 1-3 by making a copy of my integration test project and letting this new project target .NET 4.8 and always run against the cloud. But that would mean maintaining two sets of tests.

Do you have any advice for how I can achieve all 4 objectives?

Thanks in advance!

EDIT 1: Regarding platforms: Our developer machines and Jenkins hosts run Windows. The cloud-deployed server also runs on Windows.

EDIT 2: Simply multi-targeting the test project (i.e., letting the test project compile against both .NET 4.8 and .NET 5) is not possible. The test project has a reference to the server part because it needs to be able to run the server locally. The server part needs .NET 5; it cannot compile against .NET 4.8 (it relies on libraries that don't exist for .NET 4.8). Hence the test project cannot compile against .NET 4.8 either.

I tried to do a simple deployment of nextcloud on a k8s cluster hosted using minikube on my local machine for learning purposes. This deployment doesn't have any database/storage attached to it. I'm simply looking to open the nextcloud homepage on my local machine. However, I am unable to do so. Here are my yamls.

Deployment yaml:

I'm trying to create a gitlab ci/cd pipeline able to upload on Amazon S3 only the edited files. It is a possible thing? If yes, how can I edit my current .gitlab-ci.yml so only the changed files are uploade to ASW S3:

I'm using Azure to host my Java Spring Boot application. I have created a trial to test this solution and I managed to deploy my application following this tutorial Deploy with Azure which is using the azure-webapp-maven-plugin and the command mvn azure-webapp:deploy.

My problem is that when I'm making changes inside my application and I want to apply them in my Azure Web App but nothing actually update when I reach the page, here is the message after using the deploy command :

I am playing with microk8s and I am trying to deploy nextcloud to get more familiar with it. However the deployment of nextcloud went fine, I am facing some issues with setting ingress for that. Maybe you could take a look at my manifests and ingress resource and help find me the problem.

This is the deployment file:

I have a Spring Cloud Dataflow installed on an Openshift cluster. I try to register an application, then to create a task, and everything is fine, but when I try to schedule the task I got the following exception:

java.lang.IllegalArgumentException: Unable to create directory for local repository: ?/.m2/repository at org.springframework.util.Assert.isTrue(Assert.java:118) ~[spring-core-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.cloud.deployer.resource.maven.MavenArtifactResolver.(MavenArtifactResolver.java:114) ~[spring-cloud-deployer-resource-maven-2.2.0.RELEASE.jar!/:2.2.0.RELEASE] at org.springframework.cloud.deployer.resource.maven.MavenResource.(MavenResource.java:122) ~[spring-cloud-deployer-resource-maven-2.2.0.RELEASE.jar!/:2.2.0.RELEASE] at org.springframework.cloud.deployer.resource.maven.MavenResource.parse(MavenResource.java:268) ~[spring-cloud-deployer-resource-maven-2.2.0.RELEASE.jar!/:2.2.0.RELEASE] at org.springframework.cloud.dataflow.registry.support.AppResourceCommon.getResource(AppResourceCommon.java:167) ~[spring-cloud-dataflow-registry-2.4.0.RELEASE.jar!/:2.4.0.RELEASE] at org.springframework.cloud.dataflow.registry.service.DefaultAppRegistryService.getAppResource(DefaultAppRegistryService.java:266) ~[spring-cloud-dataflow-registry-2.4.0.RELEASE.jar!/:2.4.0.RELEASE] at org.springframework.cloud.dataflow.registry.service.DefaultAppRegistryService$$FastClassBySpringCGLIB$$a8bae4.invoke() ~[spring-cloud-dataflow-registry-2.4.0.RELEASE.jar!/:2.4.0.RELEASE] at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769) ~[spring-aop-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747) ~[spring-aop-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366) ~[spring-tx-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99) ~[spring-tx-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747) ~[spring-aop-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) ~[spring-aop-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.cloud.dataflow.registry.service.DefaultAppRegistryService$$EnhancerBySpringCGLIB$$2e83e4f6.getAppResource() ~[spring-cloud-dataflow-registry-2.4.0.RELEASE.jar!/:2.4.0.RELEASE] at org.springframework.cloud.dataflow.server.service.impl.DefaultSchedulerService.getTaskLauncherResource(DefaultSchedulerService.java:423) ~[spring-cloud-dataflow-server-core-2.4.0.RELEASE.jar!/:2.4.0.RELEASE] at org.springframework.cloud.dataflow.server.service.impl.DefaultSchedulerService.schedule(DefaultSchedulerService.java:211) ~[spring-cloud-dataflow-server-core-2.4.0.RELEASE.jar!/:2.4.0.RELEASE] at org.springframework.cloud.dataflow.server.controller.TaskSchedulerController.save(TaskSchedulerController.java:154) ~[spring-cloud-dataflow-server-core-2.4.0.RELEASE.jar!/:2.4.0.RELEASE] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_232] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_232] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_232] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_232] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:888) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) [spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) [spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:660) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) [spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) [tomcat-embed-websocket-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.springframework.web.filter.ForwardedHeaderFilter.doFilterInternal(ForwardedHeaderFilter.java:158) [spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) [spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) [spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:108) [spring-boot-actuator-2.2.4.RELEASE.jar!/:2.2.4.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) [spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) [spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1598) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_232] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_232] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.30.jar!/:9.0.30] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_232]

I don't know what does it mean, I'm a newbie in Dataflow. I don't understand why he is trying to use Maven instead of a kubernetes deployer and why I get this error. Could someone provide me some more explanation?

I'm registering the applications as docker containers, by the way.