file-storage | File storage abstraction for Yii2 | Web Framework library

The CSI Driver injects the secrets in the pod by placing them as files on the file system. There will be one file per secret where

• The filename is the name of the secret (or the alias specified in the secret provider class)
• The content of the file is the value of the secret.

The CSI does not create environment variables of the secrets. The recomended way to add secrets as environment variables is to let CSI create a Kubernetes secret and then use the native secretKeyRef construct

Why does this work for the PostgreSQL deployment but not my Django API, for example?

In you Django API app you set an environment variable POSTGRES_PASSWORD to the value /mnt/secrets-store/PG-PASSWORD. i.e you simply say that a certain variable should contain a certain value, nothing more. Thus the variable ill contaain the pat, not the secret value itself.

The same is true for the Postgres deployment it is just a path in an environment variable. The difference lies within how the Postgres deployment interprets the value. When the environment variables ending in _FILE is used Postgres does not expect the environment variable itself to contain the secret, but rather a path to a file that does. From the docs of the Postgres image:

As an alternative to passing sensitive information via environment variables, _FILE may be appended to some of the previously listed environment variables, causing the initialization script to load the values for those variables from files present in the container. In particular, this can be used to load passwords from Docker secrets stored in /run/secrets/ files. For example:

$ docker run --name some-postgres -e POSTGRES_PASSWORD_FILE=/run/secrets/postgres-passwd -d postgres

Currently, this is only supported for POSTGRES_INITDB_ARGS, POSTGRES_PASSWORD, POSTGRES_USER, and POSTGRES_DB.

Is there a way to add them in env: without turning them in secrets and using secretKeyRef? No, not out of the box. What you could do is to have an entrypoint script in your image that reads all the files in your secret folder and sets them as environment variables (The name of the variables being the filenames and the value the file content) before it starts the main application. That way the application can access the secrets as environment variables.

Am I understanding Azure storage correctly, it's either you fit into the Azure storage model or you don't?

You can use Azure Storage Model for your CMS Application. You can use either Blob Storage or File Share

Can anyone advise how I can achieve what's required by the CMS application as described below by using Blobs?

You can use Data Lake Gen 2 storage account if you want to use Azure Blob Storage.

Data Lake Gen 2 storage enables hierarchical namespace so that you can use subfolders in the Blob Storage as per your requirements

Problem with using Blob

Blob Storage allows subfolders if we use Data Lake Gen 2 storage account. You can enable Blob Public Anonymous access

The problem with using File Share

Azure File Share supports but does not allow public anonymous access. You can use Azure Managed Identity (System-Assigned) for your web app to access the Azure File Share.

Then your application would be able to access the Azure File Share without SAS token

Block storage is a traditional disk. In the old days, you would walk down to the computer store, buy a Hard Disk Drive that has a specific amount of storage (eg 1TB), plug it into the computer and then use it for storage. If it runs out of disk space, you had to delete files or buy additional disks.

This type of storage is called 'Block Storage' because the disk is divided into many blocks. The Operating System is responsible for managing what data is stored in each block and it also maintains a directory of those files. You'll see terms like FAT32 or exFAT for these storage methods.

Computers expect to have this type of directly attached disk storage. It's where the computer keeps its operating system, applications, files, etc. It's the C: and D: drives you see on Windows computers. When using services like Amazon EC2, you can attach Block Storage by using the Amazon Elastic Block Store service (Amazon EBS). Even though storage is virtual (meaning you don't need to worry about the physical disks), you still need to specify the size of the disk because it is pretending to be a traditional disk drive. Therefore, you can run out of space on these disks (but it is fairly easy to expand their size).

Next comes network-attached storage. This is the type of storage that companies give employees where they can save their documents on the network instead of a local disk (eg the H: drive). The beauty of network-attached storage is that it doesn't look like blocks on a disk -- instead, the computer just says "save this file" or "open this file". The request goes across the network to a file server, which is responsible for storage the actual data on the disk. This is a much more efficient way to store data since it is centralized rather than on everybody's computer and it is much easier to backup. However, your company still needs to have disk drives that store the actual data.

Then comes object storage which has become popular with the Cloud. You can store files in Amazon S3 (or MinIO, which is S3-compatible) without worrying about hard disks and backups -- they are the job of the 'cloud'. You simply store data and somebody else worries about how that data is stored. It is typically charged via pay-as-you-go, so instead of buying expensive hard disks up-front, you just pay for the amount of storage used. The data is typically automatically replicated between multiple disks so it can survive the failure of disk drives and even data centers. You can think of cloud-based block storage as unlimited in size. (It isn't actually unlimited, but it acts like it is.)

Services like S3 and MinIO also do more than simply store the data. They can make the objects available via the Internet without having to run a web server. They can have fine-grained permissions to control who (and what) can access the data. Amazon S3 is very 'close' to other AWS services, making it very fast to use data in Amazon EC2, Amazon EMR, Amazon RDS, etc. It's even possible to use query engines like Amazon Athena that allow you to run SQL commands to query data stored in Amazon S3 without needing to load it into a database. You can choose different storage classes to reduce cost while trading-off access speed (like the old days of tape backup). So think of Object Storage as 'intelligent storage' that is much more capable than a dumb disk drive.

Bottom line: Computers expect to have block storage to boot and run apps, but block storage isn't a great way to manage the storage of large amounts of data. Object Storage in the Cloud is as simple as uploading and downloading data without having to worry about how it is stored and how to manage it -- that is the cloud's job. You get to spend your time adding value to your company rather than managing storage on disk drives.

That's the same strange bug I encountered the first time. A possible solution is to put the script in the renderUI:

I tried with using the Azure storage explorer in my system

combine the URL from file properties , remove the / from the end the URL Form file with SAS token.

Example URL:

https://testsasex.file.core.windows.net/testsas/test.txt?sv=2020-04-08&ss=f&srt=sco

1) Click on the file ,select the properties next copy the URL from file properties

2) Generate SAS From Azure storage explorer .

Right click on the storage in Storage explorer and select the Generate shared access signature.

3) Select File and add permissions and click on Create.

3) A second Shared Access Signature dialog will then display that lists the blob container along with the URL(Connection String) and QueryStrings you can use to access the storage resource. Select Copy next to the URL you wish to copy to the clipboard

4) After that ,combine the URL (Remove / from URL) from file properties and SAS token

OUTPUT

Instead of IAsyncCollector we have multiple output binding in .net 5

On this function we will give the multiple output values.

After this problem disappearing one day and reappearing the next, I did some more digging, and found that this appears to be due to the default netty http client that azure is using.

Updating my POM to the following has resolved this issue:

Create a kubernetes secret using config.yaml file.