docker-mail | Docker container running dovecot and postfix | Continuous Deployment library

This is more of a network administration question, than a programming question, so may be considered off topic. That being said:

The SMTP protocol is used for two different, but similar purposes: Message Submission, and Message Transission.

Message Submission is done by an MSA, Message Submission Agent, generally on behalf of an end user, but perhaps on perhaps of a script or process. Traditionally, these are clients like Thunderbird, Apple Mail, or the email client on your phone. In modern practice, this is generally done authenticated (with user credentials) and encrypted on ports 465 or 587.

• Port 465, SMTPS (smtp-secure, by analogy with https) is technically deprecated, but widely used. It is used for SMTP over TLS, where the encryption is encrypted from connection until termination.
• Port 587, submission is generally used with STARTTLS, where the connection is first made unencrypted, but upgraded shortly thereafter using a special command.

Both these ports generally accept mail from a user with credentials, for any destination, and will hold and relay these for the user. For example, if you connect to smtp.gmail.com on port 465 or 587, and authenticate as user@gmail.com, it will allow you to submit email for anyone, as long as it is from user@gmail.com.

Message Transmission is done by an MTA, Message Transmission Agent, generally on behalf of all the users of a site or service. Relaying is done between sites on port 25, with opt-in STARTTLS encryption. Authentication is not generally done, but there is a complicated system of reputation tracking, firewalls, and blacklists generally used behind the scenes. Usually only mail for a specific site is accepted on this port. For example, if you connect to one of gmail.com's MX servers (for example, gmail-smtp-in.l.google.com as of this writing) on port 25, and it thinks you are a trustworthy IP, it will accept mail from anyone to any gmail address (subject to further scanning). It will refuse to relay to anyone offsite.

Message Retrieval is generally done by IMAP on ports 143 (with STARTTLS) or 993 (with TLS from connection). This is a pull service used by an end-user (generally) to retrieve emails being held by an MTA on their behalf. POP3 is also used (on 110 and 995) by some sites, but it is a much less capable protocol.

Traditionally, submission and transmission were both done on port 25 without authentication, but that's a no go on the modern internet. It was split into transmission and submission so network resources could be better controlled. As you may have discovered, many ISPs and cloud services restrict port 25 so end-users cannot act as transmitters without their consent, and so relaying happens either through their servers or some other service that will take responsibility.

This, through this model, gmail users can generally only submit via gmail's submission server, and other users must submit through their services server, and spammers can't just set up a server anywhere to transmit messages to gmail. If they do and their ISP hasn't firewalled it, their reputation will shortly be trashed and be placed on many blacklists.

Additionally, a lot of this doesn't even happen over the traditional protocols anymore. If you use Google services and clients, you will likely be using a custom protocol tunneled over HTTPS, or the public GMAIL REST protocol. If you're using Microsoft, they have no less than 3 email protocols: Exchange ActiveSync, Exchange Web Services, and Microsoft Graph/Outlook MAIL Rest API, all using HTTPS.

This is because you can't access service through 127.0.0.1. If your services on the same network you will able to access it by the links directive you passed, like change your address from 127.0.0.1 to mailcatcher. If your services in different networks - you can open ports on service that you need to call and then access it by machine local ip address (not the 127.0.0.1 but by the your physical/virtual machine ip address in local network)

By default container are isolated in docker. You can allow connection between containers in docker by adding link argument when creating your Container USAGE :

The solution for this one it to make sure both containers, RainLoop and Mail, share a bridged network. Then all the configurations can stay the same.

That is a very complex and philosophic topic, but I will give you my view on it and some facts to support it.

I think the best way is the second one - Create an additional cluster, and that's why:

1. You need a point which should be accessible from any of your environments. With a separate cluster, you can set the same firewall rules, routes, etc. in all your environments and it doesn't affect your current workload.

2. Yes, you need to pay a bit more. However, you need resources to run your shared applications, and overhead for a Kubernetes infrastructure is not high in comparison with applications.

3. With a separate cluster, you can setup a real HA solution, which you might not need for staging and development clusters, so you will not pay for that multiple times.

4. Technically, it is also OK. You can use Heapster to collect data from multiple clusters; almost any logging solution can also work with multiple clusters. All other applications can be just run on the separate cluster, and that's all you need to do with them.

Now, about your questions:

Is it a problem to have monitoring-data in a cluster because one cannot see the monitoring-data if the cluster is offline?

No, it is not a problem with a separate cluster.

Is it common practice to have an additional cluster for shared services that should not have an impact on other parts of the application?

I think, yes. At least I did it several times, and I know some other projects with similar architecture.

Is it (easily) possible to send metrics and logs from one kubernetes-cluster to another (we are running kubernetes in OpenTelekomCloud which is basically OpenStack)?

Yes, nothing complex there. Usually, it does not depend on the platform.

Try use networks:

(At the moment I have no where to try it, so I hope it works, or at least help you decipher the dilemma)

Specifically answering your questions first:

1. You should be able to have postfix listen on any port you specify by editing the main.cf configuration file and changing the smtp listener to a numbered port of your choice. Of course, if it isn't a "known" port, I'm not sure what/who will ever connect to it, but maybe you don't care in this situation as you are only using postfix as a relay?

2. It may depend some on the Linux distribution or setup of your host, but most systems will leave email in the local delivery "mail spool" if there is no system/daemon set up to move it anywhere else. Back when that was the normal way to handle multi-user mail on UNIX systems, a login user used a mail reader client to read through email in your local "spool", and of course if you don't have that, you can simply vi your mail file and read the raw contents if necessary. These mail files are normally located in /var/spool/mail on most systems.

Stepping away from your questions, I would guess you don't necessarily need postfix running on your host, especially as your containerized mailserver is handling the port 25 SMTP traffic for the host. Local email will stay local, I assume, without postfix, and be available through local means; and you might even find a simpler solution to external forwarding (e.g. a script that can parse mail spools and just connect to an SMTP relay and send it to an external address) if you want that.