gorouter | Go Server/API micro framework | HTTP library

Ok, I'll answer this question myself, in case someone else may need that in the future.
Actually, I finally managed to implement it.

TLDR - the solution is here: https://github.com/TheFonz2017/Spring-Cloud-Netflix-Ribbon-CF-Routing

The solution:

• Allows to use Ribbon on Cloud Foundry, overriding Go-Router's load balancing.
• Adds a custom routing header to Ribbon load balancing requests (including retries) to instruct CF's Go-Router to route requests to the service instance selected by Ribbon (rather than by its own load balancer).
• Shows how to intercept load balancing requests

The key to understanding this, is that Spring Cloud has its own LoadBalancer framework, for which Ribbon is just one possible implementation. It is also important to understand, that Ribbon is only used as a load balancer not as an HTTP client. In other words, Ribbon's ILoadBalancer instance is only used to select the service instance from the server list. Requests to the selected server instances are done by an implementation of Spring Cloud's AbstractLoadBalancingClient. When using Ribbon, these are sub-classes of RibbonLoadBalancingHttpClient and RetryableRibbonLoadBalancingHttpClient.

So, my initial approach to add an HTTP header to the requests sent by Ribbon's HTTP client did not succeed, since Ribbon's HTTP / Rest client is actually not used by Spring Cloud at all.

The solution is to implement a Spring Cloud LoadBalancerRequestTransformer which (contrary to its name) is a request interceptor.

My solution uses the following implementation:

I have already found a solution to my problem (Selenium v3.14.0).

• Hub

As per their official documentation Configuring html5, AngularJS uses a “#” in it’s urls. HTML5Mode of AngularJS removes these “#” from URL.

Activate HTML 5 Mode

Create html5.mode.config.js file in webapp/app/blocks/config/ directory:

This has nothing to do with Node or your Express application. This is due do the gorouter that Cloud Foundry uses which sets a timeout of 60 seconds.

See:

• https://github.com/cloudfoundry/gorouter/issues/100
• https://cf-docs.jp-east-https://cf-docs.jp-east-1.paas.cloud.global.fujitsu.com/en/manual/faq/faq/topics/c-faq-14.html

There may be a way to override it: https://docs.cloudfoundry.org/services/route-services.html#timeouts

Reading CF documentation about HTTP routing I figured out that IBM cloud only uses ports 80 and 443 and then forwards the requests via HTTP to the ports the app is listening to (for example if my NodeJs is running on port 6001 and I call the cloud endpoint on port 443, the GoRouter will forward the request via HTTP to the correct port, adding the X-Forwarded-Proto header to pass the application the information of the original protocol used for the request.

Having this in mind (assuming this is correct), in my NodeJs code I cannot use something like https.createServer(opts, app) giving that all the requests coming to the App Container will be via HTTP.

That is correct.

First of all what is the difference between terminating TLS at Load Balancer or at GoRouter? What are the reasons behind this choice?

This only applies if you are operating your own Cloud Foundry platform. If you are deploying apps to a Cloud Foundry platform operated by someone else, they will make this decision and it won't affect you as a user.

As an operator, this is a choice you need to make. There are some trade-offs depending on what you choose to do.

1. You can terminate at the LB. This is typically the fastest as LB's are very efficient at handling TLS/SSL. The LB can then forward traffic onto the Gorouter unencrypted, which puts less work on the Gorouter, but comes at the cost of not encrypting traffic between the two (may not be OK, depending on security requirements). In this scenario, it is the responsibility of the LB to add the x-forwarded-* headers.

   browser -> HTTPS -> LB -> HTTP -> Gorouters -> HTTP -> Your App

2. You can use a layer-4 LB and have connections balanced across your Gorouters. This enables the Gorouters to terminate TLS/SSL. They are pretty efficient at doing this, but less so than most LBs. This also gets you encryption in the request path up to the Gorouter. In this scenario, it is the responsibility of the Gorouters to add the x-forwarded-* headers.

   browser -> HTTPS -> LB -> HTTPS -> Gorouters -> HTTP -> Your App

3. You can terminate at the LB, but open a new TLS/SSL session between the LB & the Gorouters. This is the least efficient option as it requires terminating two TLS/SSL sessions, but it provides for encryption in the request path up to the Gorouter. It also tends to be the most flexible working with non-layer-4 LBs and it can allow your LB to inspect the HTTP traffic, because you're terminating the session at the LB. In this scenario, it is the responsibility of the LB to add the x-forwarded-* headers.

   browser -> HTTPS (session A) -> LB -> HTTPS (session B) -> Gorouters -> HTTP -> Your App

Again, if you're not operating a Cloud Foundry platform, then you can ignore this.

My second question is which is the correct way of handling the certificates once they are forwarded to my application as HTTP headers? This is due to the fact the my NodeJs server will be an http server, created with express in the standard way http.createServer(app).

You don't need to do anything fancy with the way you create your server. All you need to do is look at the x-forwarded-* headers & use those to make your decisions.

1. Did the request come in over HTTPS? Look at either x-forwarded-proto, which should be set to https for HTTPS requests or look at x-forwarded-port, which should be set to 443 for HTTPS requests.

2. Was a client cert provided with the request? Look at X-Forwarded-Client-Cert. If it contains a cert, then the client provided the cert.

3. Is the client's cert valid? If your app gets the request, then the client's cert is valid. You know this because the platform handles that part for you. Since the platform (either LB or Gorouter) is terminating the TLS/SSL connection, it's the responsibility of it to validate the cert. If your app receives a request & x-forwarded-client-cert is set, then the cert is valid.

4. How do I make authorization decisions based on a client cert? This is a little trickier, but typically you'd pull the cert out of x-forwarded-client-cert, read/parse it and make decisions based on the cert content (which we know is valid thanks to the platform). Most likely you're going to look at the subject name and treat that like a user name. Then look up roles or permissions for that user. However, how you handle that is up to you as the developer.

Hope that helps!

Currently (May 2017), the closest thing is indeed a blue-green deployment, which does not allow you to control the percentage that is routed to the new app.
There has been discussion about supporting this kind of feature as part of the V3 API work, as an extension to native blue-green deployment (i.e., where the route switching is all done by Cloud Foundry and not by an API client like the CLI or plugin) but I don't believe anything has been scoped out or implemented yet.

Currently when you run apps on Cloud Foundry, they never listen for HTTPS requests. They always listen for HTTP requests on the assigned port. HTTPS is handled upstream either by the GoRouter or by your company's load balancer. The HTTPS information is propagated to your app via the x-forwarded-* headers. This means you will don't need to use server.ssl.enabled with your Spring Boot apps and you don't need to worry about providing any certs.

If you want to handle mTLS that get's a little trickier. You can let the upstream LB or GoRouter handle this for you. In a way similar to x-forwarded-*, it validate the client certificate and make sure it's trusted. If it's not trusted, then the connection will fail. If it's trusted, it will pass you the certificate information via an HTTP header, X-Forwarded-Client-Cert. At this point, you know the cert is valid & trusted, and you can go further to use the cert information to make choice about authorization and access.

More about this all here: https://docs.pivotal.io/pivotalcf/2-1/adminguide/securing-traffic.html#gorouter_mutual_auth

The platform also manages and assigns certificates for every application instances. You can use these certificates to identify your clients. More about these here: https://docs.pivotal.io/pivotalcf/2-1/devguide/deploy-apps/instance-identity.html

If your operator has set all this up for you, it makes doing mTLS a bit easier. If you're doing app-to-app communication, you don't really have to do anything with certs, the platform provides & validates them for you. If you have clients off platform that are connecting, it's trickier cause you have to give them certs and your operator has to configure the platform to trust them. It can still be done, though and your app doesn't have to change to support that.

Switching gears. If you really want to go with the traditional approach, you can do that. I know I hinted above that wouldn't be possible. The above information applies to HTTP routes and it's not possible for HTTP routes. If you use a TCP route for your application though, it can work. With a TCP route, your app just gets TCP packets so it can do whatever it wants with them. This includes doing HTTPS. If that's what you want, it's up to the application to handle TLS. The platform just routes the TCP packets to your app and the app does the rest.

That means you need to provide the server certs, client certs and configure your app/server to validate them. It's very similar to your setup locally or on a traditional server setup. With this approach, the platform does very little for you.

There's a section here which describes creating a TCP route. It's pretty easy and the same as an HTTP route, but just an additional flag to set the port.

https://docs.cloudfoundry.org/devguide/deploy-apps/routes-domains.html#create-route

You can also read more about the differences between HTTP & TCP routes here: https://docs.cloudfoundry.org/devguide/deploy-apps/routes-domains.html#http-vs-tcp-routes

Hope that helps!

I solved my problem. In my case mutual TLS was enabled on gourouter:

By default, Gorouter requests but does not require client certificates in TLS handshakes.

https://docs.cloudfoundry.org/adminguide/securing-traffic.html#gorouter_mutual_auth

Add a registry key:

UndertowEmbeddedServletContainerFactory has been renamed in 2.0. It's now UndertowServletWebServerFactory. I noticed that you're also using EmbeddedServletContainerCustomizer. Its equivalent in 2.0 is org.springframework.boot.web.server.WebServerFactoryCustomizer.