sif | A minimal ACK written in Go

As mentioned in a separate discussion, Xvfb is not supposed to be start through startx or /usr/bin/X but rather with the supplied run script.

It is not possible to build a singularity image based on a vagrant VM, though you can use a vagrant VM to do the actual building.

original answer:

You can just vagrant ssh to the VM and use singularity normally. The user guide docs are fairly comprehensive and searchable.

The Singularity admin guide has info specifically on using Singularity in Vagrant on a Windows host OS. It is not the first place I'd personally expect it to be, but it is there and should get you up and going anyway.

Depending on the CI software you're using, you can have certain jobs run only when specific files have changed. I use Gitlab CI, which has the only/except:changes rule. There is probably something similar for most other CI platforms, but you'll have to check their docs.

Otherwise, your solution is probably the simplest.

You cannot build a singularity image from a dockerfile. But you can build a docker image from the dockerfile and then convert that docker image to singularity format.

Short answer: yes

• SINGULARITY_BIND=$SSH_AUTH_SOCK ./myimage.sif
• singularity run -B $SSH_AUTH_SOCK myimage.sif

Long answer: yes, as long as the right directories and environment variables passed to the image.

You need:

1. git/ssh installed in the singularity image (may or may not be in a default docker image)
2. your $HOME directory mounted in. This is default, but may need to do it manually (-H /home/my_user) depending on configuration
3. ssh-agent environment variable SSH_AUTH_SOCK exported. By default your host environment is passed into the container, but this can be disabled with -e | --cleanenv
   • See also: docs on environment variables
4. the path set in $SSH_AUTH_SOCK mounted in. This is why we're using SINGULARITY_BIND or -B $SSH_AUTH_SOCK
   • See also: docs on bind paths and mounts

An easy way to test if your keys are being accessed correctly:

I've run into a similar issue. I was logged into Docker on the command line (so could bypass rate limits), but Singularity was invoking the Docker API without authentication.

You can tell Singularity to authenticate with Docker by prepending environment variables to your command:

SINGULARITY_DOCKER_USERNAME= SINGULARITY_DOCKER_PASSWORD= singularity build my_img.sif Singularity

See this for more detail.

Ok, here is one way to obtain what you want in about 2min. Some tickers are bad, that's why it crashes.

Here's the code. I use joblib for threading or multiprocess since it doesn't work in my env. But, that's the spirit.

I had this exact problem and I managed to fix it in Ubuntu 18.04 with singularity version 2.6.1 (I don't think the solution will be much different for you in 2.5.2 though).

Just first a small disclaimer: I'm only using Linux for two weeks now, so, sorry if I'm confusing any terms. I'm new ;)

Steps to take

1. Deinstall earlier versions of Singularity if needed.

2. download tarbell of the correct Singularity version by opening a Terminal and running the following commands:

   VERSION=2.6.1

   wget https://github.com/singularityware/singularity/releases/download/$VERSION/singularity-$VERSION.tar.gz

   tar xvf singularity-$VERSION.tar.gz

3. Now, there should be a folder called singularity-2.6.1 (for me it was placed in Home). Then open the file mount.c placed in /singularity-2.6.1/src/lib/image/squashfs/mount.c (I am not sure if this is the same for v2.5.2 but I assume not that much has changed. We might be out of luck though)

4. In this file find the line of code

   if ( singularity_mount(loop_dev, mount_point, "squashfs", MS_NOSUID|MS_RDONLY|MS_NODEV, "errors=remount-ro") < 0 )

   Change this to

   if ( singularity_mount(loop_dev, mount_point, "squashfs", MS_NOSUID|MS_RDONLY|MS_NODEV, "") < 0 )

   Don't forget to save!

5. Return to the Terminal to install Singularity from the edited installation files by running the commands

   cd singularity-$VERSION

   ./configure --prefix=/usr/local

   make

   sudo make install

For me, the problem was now fixed!

Problem rationale

Apparently, there was a bug in squashfs such that it simply ignored arguments if it didn't know it. When this was fixed earlier this year, it started giving errors when encountering an invalid argument, as happened in our case. (Also explained here https://www.mail-archive.com/debian-kernel@lists.debian.org/msg117435.html) Therefore, removing the unknown argument "errors=remount-ro" fixed the problem.

Since I'm honestly 100% new to this, I'm not sure if it will induce any new errors at other places. However, this is also how this error was basically fixed in a later release of Singularity (https://github.com/hpcng/singularity/pull/4802/files/3f3136be150d9a8f63e58a0b4e39dc8487f1c0dc).

Hope it helped!

You need to finagle it a bit to make it play nice with gitlab CI. The easiest way I found was to clobber the docker entrypoint and have script step be the full singularity build command. We're using this to build our singularity images with v3.6.4, but it should work with v3.7.0 as well.

e.g.,