grub2 | Ongoing downstream work on grub2 , including Fedora | Theme library

Found the answer

#https://ubuntu.com/blog/nvidia-cuda-inside-a-lxd-container

You created the filesystem on the entire volume (file), not on a partition. If you want it on a partition, then you would have to write a partition table in the file and create the filesystem in the partition.
Edit. Not sure if grub even has a way to boot with no GPT or MBR.

To use this image partitioned, create a 1MB larger file, write the GPT partition table on the first megabyte, use dd to copy your image to the rest.

Solution was to modify the instructions given in this answer for Fedora, which is to install the grub2 manually after OS installation

Install Fedora without bootloader

• Click Full disk summary and boot loader... at the bottom of Installation Destination screen.
• Select the hard disk that you going install Fedora and click Do not install boot loader button.
• And continue with installation process as usual.

Installing grub2 manually

CPU flag of 1gb hugepage support and guest OS support/enabling are not enough to get 1 gb hugepages working in virtualized environment.

The idea of huge pages both on PMD (2MB or 4 MB before PAE and x86_64) and on PUD level (1 GB) is to create mapping from aligned virtual region of huge size into some huge region of physical memory (As I understand, it should be aligned too). With additional virtualization level of hypervisor there are now three (or four) memory levels: virtual memory of app in guest OS, some memory which is considered as physical by guest OS (it is the memory managed by virtualization solution: ESXi, Xen, KVM, ....), and the real physical memory. It's reasonable to assume that hugepage idea should have the same size of huge region in all three levels to be useful (generate less TLB miss, use less Page Table structures to describe a lot of memory - grep "Need bigger than 4KB pages" in the DickSites's "Datacenter Computers: modern challenges in CPU design", Google, Feb2015).

So, to use huge page of some level inside Guest OS, you should already have same sized huge pages in physical memory (in your Host OS) and in your virtualization solution. You can't effectively use huge page inside Guest when they are not available for you host OS and Virtualization software. (Some like qemu or bochs may emulate them but this will be from slow to very slow.) And when you want both 2 MB and 1 GB huge pages: Your CPU, Host OS, Virtual System and Guest OS all should support them (and host system should have enough aligned continuous physical memory to allocate 1 GB page, you probably can't split this page over several Sockets in NUMA).

Don't know about ESXi, but there as some links for

• RedHat and some(?) linux virtualization solutions (with libvirtd). In "Virtualization Tuning and Optimization Guide" manual hugepages are listed for Host OS: ⁠8.3.3.3. Enabling 1 GB huge pages for guests at boot or runtime :

Procedure 8.2. Allocating 1 GB huge pages at boot time

1. To allocate different sizes of huge pages at boot, use the following command, specifying the number of huge pages. This example allocates 4 1 GB huge pages and 1024 2 MB huge pages: 'default_hugepagesz=1G hugepagesz=1G hugepages=4 hugepagesz=2M hugepages=1024' Change this command line to specify a different number of huge pages to be allocated at boot.

Note The next two steps must also be completed the first time you allocate 1 GB huge pages at boot time.

1. Mount the 2 MB and 1 GB huge pages on the host:

   # mkdir /dev/hugepages1G # mount -t hugetlbfs -o pagesize=1G none /dev/hugepages1G # mkdir /dev/hugepages2M # mount -t hugetlbfs -o pagesize=2M none /dev/hugepages2M

2. Restart libvirtd to enable the use of 1 GB huge pages on guests:

   # service restart libvirtd

1 GB huge pages are now available for guests.

• For Ubuntu and KVM: "KVM - Using Hugepages"

By increasing the page size, you reduce the page table and reduce the pressure on the TLB cache. ... vm.nr_hugepages = 256 ... Reboot the system (note: this is about physical reboot of host machine and host OS) ... Set up Libvirt to use Huge Pages KVM_HUGEPAGES=1 ... Setting up a guest to use Huge Pages

• For Fedora and KVM (old manual about 2MB pages): https://fedoraproject.org/wiki/Features/KVM_Huge_Page_Backed_Memory

• ESXi 5 had support of 2MB pages, which should be manually enabled: How to Modify Large Memory Page Settings on ESXi

• For "VMware’s ESX server" of unknown version, from March 2015 paper: BQ Pham, "Using TLB Speculation to Overcome Page Splintering in Virtual Machines", Rutgers University Technical Report DCS-TR-713, March 2015:

Lack of hypervisor support for large pages: Finally, hypervisor vendors can take a few production cycles before fully adopting large pages. For example, VMware’s ESX server currently has no support for 1GB large pages in the hypervisor, even though guests on x86-64 systems can use them.

• Newer paper, no direct conclusion about 1GB pages: https://rucore.libraries.rutgers.edu/rutgers-lib/49279/PDF/1/

We find that large pages are conflicted with lightweight memory management across a range of hypervisors (e.g., ESX, KVM) across architectures (e.g., ARM, x86-64) and container-based technologies.

• Old pdf from VMWare: "Large Page Performance. ESX Server 3.5 and ESX Server 3i v3.5". https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/large_pg_performance.pdf --- only 2MB huge pages are listed as supported

VMware ESX Server 3.5 and VMware ESX Server 3i v3.5 introduce 2MB large page support to the virtualized environment. In earlier versions of ESX Server, guest operating system large pages were emulated using small pages. This meant that, even if the guest operating system was using large pages, it did not get the performance benefit of reducing TLB misses. The enhanced large page support in ESX Server 3.5 and ESX Server 3i v3.5 enables 32‐bit virtual machines in PAE mode and 64‐bit virtual machines to make use of large pages.

grub-redhat.j2 has some typos.

Line 11: set supperusers="{{ grub_user }}"

Change to: set superusers="{{ grub_user }}"

Line 12: password_pbkdf2 {{ grub_user }} {{ grub_password_v2_passwd }}

Change to:password_pbkdf2 {{ grub_user }} {{ grub_password_v2_passwd }}

If you're going to use the same grub password on multiple machines, you might want to consider using ansible-vault encrypt_string to encrypt the raw password, then create an additional task which runs grub2-mkpasswd-pbkdf2 (part of grub2-tools-minimal) with the command module and pass it the vaulted variable. Registering the output of that task (e.g. as grub_password_v2_passwd) would result in every target that you run the playbook against receiving a unique hash even though the underlying password would be the same. The expect module works great for this:

Grub2 does not have builtin arithmetic support. You need to add Lua support if you want that, see this answer for details.

Write a GRUB driver module for your FS, like how it has modules for XFS, ext2/3/4, and so on. https://wiki.osdev.org/Writing_GRUB_Modules

https://www.gnu.org/software/grub/manual/grub/grub.html#Images explains the different components of GRUB2. The filesystem driver is loaded as part of core.img, which is built from other images.

Keep in mind that many new filesystems don't get GRUB support right away, and having a separate /boot partition that GRUB can read is not rare. e.g. on a single disk instead of a complicated RAID (although GRUB does have some md support), not encrypted, and with a long-established filesystem that GRUB does have a driver for. And that doesn't need a complicated log replay on dirty shutdowns before even reading.

Also, the earliest stage of GRUB booting needs to load core.img from a block list that's hard-coded into the small first thing the firmware loads, so filesystems where the blocks of a file's data can move around when the file wasn't modified aren't usually a good fit for /boot at all. You'd never know when you had to re-install GRUB after any change to anything on /boot.

But if you still want to make your FS bootable in GRUB, check the GRUB source code, there's probably some documentation.

https://www.gnu.org/software/grub/grub-documentation.html

https://www.gnu.org/software/grub/grub-development.html

Run with -vvvv to see what ansible is doing.

Have you setup ansible in sudoers for password less privilege elevation? you are getting a message that it is waiting for "escalation prompt". That means when you are running with become, you are failing to become since it needs the password. Make sure your test user is in /etc/sudoers AND you have it marked for that user to NOT need to enter a password when running sudo commands. The entry should end with :NOPASSWD on the line in that file.