Dev Ups

Published 2023-05-06 in storage

Writing memtest86.img to small disks in Linux

Who doesn't love a bargain? In this economy you have to be prepared to test quickly after enjoying these little flutters.

Testing stuff requiring a reboot is time-consuming, which is why I'm recording my findings here.

I recently binned a broken sub-GB USB stick. My smallest stick is now 1 GB. It's the first place I look for a RAM tester.

There are alternatives to the big 2, MemTest86 and Memtest86+. The pair have had their share of problems. Trusting and justifying new tooling is time-consuming. Getting an RMA on anything else would not be fun.

I'll look at how to resize partitions on a disk using GPT, and how MemTest86 works.

Legacy MemTest86 v9

I had MemTest86 v9.0 on my old 1 GB disk. It had a couple of folders named something like "recovery" which each contained 10,000 empty or small files that had literally filled the device. It would need 10 minutes from power on to starting to test memory. I couldn't delete those teeny files, it was taking too long. I dislike deleting lots of small files, particularly on stock images like MemTest86. It's way faster and gentler on the flash to format or re-partition.

I could only see the first partition, of 256 MB, but at least it held my screenshots. It couldn't make new screenshots because the disk was full. It was last flashed 2 years ago.

Memtest86+

Not being already on my test stick already weighed heavily against this option. I learned that it had been abandoned. I recall having long-running problems with one of these two memtests.

Someone recently took on maintenance of Memtest86+. I love Linux, but opensource doesn't pay. Actual coding and maintenance can feel like a charitable endeavour and hardly sustainable.

I installed it onto my stick. It was about 2 MB. I am not 100%, but I don't think it made the remaining space available for use. As you will see when I discuss installing MemTest86 to USB, I could have grown that partition and made the whole disk available. Read on to learn and try to apply the concepts to Memtest86+.

The concept of stress testing is incredibly simple. It shouldn't need more than 2 MB, or much maintenance.

Comparison

Re-booting takes about 20s, much longer if you still use spinning rust. My old MemTest86 stick added 10 minutes to this anxiety. My old stick was using a full 256 MB, that I knew about, possibly more clever stuff I didn't. I assumed the 256 MB was hidden somewhere, as the files on the top level looked just a few MB. Deleting the two folders with 20,000 sparse files felt like it broke my computer.

It was the first time I'd booted Linux in a while, and I think my Internet disappearing was the system updates demanding a reboot. More unproductive time. I cancelled the deletion and rebooted to start afresh.

MemTest86 v10

This is the recommended way to burn an image to disk:

$ sudo dd if=memtest86-usb.img of=/dev/sda status=progress
1007903232 bytes (1.0 GB, 961 MiB) copied, 426 s, 2.4 MB/s
dd: writing to '/dev/sda': No space left on device
1971201+0 records in
1971200+0 records out
1009254400 bytes (1.0 GB, 962 MiB) copied, 426.353 s, 2.4 MB/s

As you see, I ran out of space. 1 GB isn't that 1 GB, i.e. 1000 MB, and certainly isn't the 0.5 GB quoted as the minimum stick size.

$ ll -sh memtest86-usb.img
1.1G memtest86-usb.img

Trimming the bloat

Mount the image, so it is accessible to the file system as a loop device:

sudo losetup --find --partscan memtest86-usb.img

Find the partitions:

lsblk | grep part -B1
loop20        7:20   0     1G  0 loop
├─loop20p1  259:6    0   255M  0 part 
├─loop20p2  259:7    0   256M  0 part 
└─loop20p3  259:8    0   512M  0 part

Make sure to note down this loop device number exactly. You could end up removing one which you don't own. I did, and had to reboot to be sure it didn't break something important (loop0 was known as core something).

The first two partitions under loop20 are mountable, and their file systems are shrinkable. The file system must be shrunk so that gparted can shrink the parent partition, and free up disk space.

sudo mount /dev/loop20p1 /mnt
sudo fstrim -v /mnt
sudo umount /mnt
sudo mount /dev/loop20p2 /mnt
sudo fstrim -v /mnt
sudo umount /mnt

This will tell us:

/mnt: 246.8 MiB (258813952 bytes) trimmed
/mnt: 247.8 MiB (259862528 bytes) trimmed

We get an error attempting to mount the third partition:

$ sudo mount /dev/loop20p3 /mnt
mount: /mnt: wrong fs type, bad option, bad superblock on /dev/loop20p3, missing codepage or helper program, or other error.

The third partition

Investigating what the possible missing mount option could be revealed nothing new about the filesystem technology:

$ sudo fdisk -l /dev/loop20
Disk /dev/loop20: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: ABBC4666-F7B0-4937-B2B1-32BF0F4C7934

Device          Start     End Sectors  Size Type
/dev/loop20p1    2048  524287  522240  255M Microsoft basic data
/dev/loop20p2  524288 1048575  524288  256M EFI System
/dev/loop20p3 1048576 2097118 1048543  512M Microsoft basic data

$ lsblk -o name,size,type,fstype,mountpoint | grep part -B1
loop20          1G loop
├─loop20p1    255M part vfat
├─loop20p2    256M part vfat
└─loop20p3    512M part

The view of memtest86-usb.img from gparted

Possibly, it is encrypted.

The third, "DMA test" partition is for a relatively new DMA test, which bypasses CPU involvement. The test can also fail due to a bad flash disk. I would blame this for filling my last disk, but I later discovered that it is labelled experimental, and disabled in the settings by default. There's 50% bloat right there.

Having just run fstrim on the other two partitions pares the reported data content right back to 530 MB. Unfortunately the image itself is what dd reads and that is unchanged in size.

We need to remove the bloat inside of partitions and at the end of the image. We do so using the gparted GUI launched with:

sudo gparted /dev/loop20

Effects of the partitions

Manipulation in gparted is the experiment recorded in this post. I tried several things, and the final image would usually still boot. I began conservatively, listening to gparted warning me about how moving partitions could cause them not to be bootable. The thing is, the third partition isn't recognised as anything other than msftdata, there's no boot or EFI indicator.

We cannot resize the final partition. Not until we can mount and fstrim it. We must either delete or move it left. Being conservative allows MemTest86 to continue to persist screenshots (in wasteful BMP format) and its official looking, "certificates of testing".

Everything gets saved to the first partition. You could mount the second one, as we have already, to run fstrim. The second partition appears to be involved in EFI booting. By default, each of the first two partitions gets an entry in UEFI boot options.

It's interesting that gparted describes both partitions as FAT16. disks shows one as ext4 and the other as FAT, leading me to believe they were mirror copies, one for Linux and one for Windows. Windows shows the only partition it mounted, as FAT.

With all persistent storage going to the first, there is no need to move the second. I'm 95% sure I did move the second without losing OS-level access to the saved files. The only reason you would move it is if you delete the DMA partition and want the disk to only contain useful partitions. I lose OS-level access to logs and screenshots when I enlarge the first two partitions beyond 256 MB.

I can delete the third partition, but that denies me OS-level access to logs and screenshots.

What do I mean by OS-level access? The OS automatically mounts the first partition, where MemTest86 persists what we request it to. The files are under the EFI/boot directory. When we lose access, the disk still appears to write these files, as indicated by LED blinking, but the mounted partition appears empty.

I occasionally made builds that wouldn't boot. I recall these happened when extending the first partition beyond 256 MB. It happened just a couple of times. It's an odd one, but I accept my findings.

This was the ratio I settled on:

Golden ratio of patition sizes

All functionality remained, and I have a few MB spare if I want to try to make another partition.

Shoring up the partition scheme

With the changes applied in gparted, it is time to clear up the partition mounts:

sudo losetup --detach /dev/loop20

The image is larger than my disk because the unallocated space is still present in the file.

$ fdisk -l memtest86-usb.img
Disk memtest86-usb.img: 973.2 MiB, 1020281344 bytes, 1992737 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: ABBC4666-F7B0-4937-B2B1-32BF0F4C7934

Device              Start     End Sectors  Size Type
memtest86-usb.img1   2048  432127  430080  210M Microsoft basic data
memtest86-usb.img2 432128  862207  430080  210M EFI System
memtest86-usb.img3 862208 1910783 1048576  512M Microsoft basic data

The needed size is proportional to the end sector multiplied by the sector size:

sudo truncate --size=$[(1910783+1+33)*512] memtest86-usb.img

We add 1 because we want to end after, not on, the last sector.

Truncating removes the partition table backup from the end of the disk. We add 33 to allow us to recreate the partition table at the end of the disk.

If we remount the image as a loop device we can attempt to open it in gparted. Gparted will complain "The backup GPT table is corrupt, but the primary appears OK, so that will be used." Gparted will not provide further assistance or information.

Instead, leave the image unmounted and run sudo fdisk -l memtest86-usb.img to see the error:

GPT PMBR size mismatch (1992736 != 1910816) will be corrected by write.
Disk memtest86-usb.img: 933.2 MiB, 978338304 bytes, 1910817 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device             Boot Start     End Sectors  Size Id Type
memtest86-usb.img1          1 1910816 1910816  933M ee GPT

We can fix from gdisk memtest86-usb.img. gdisk will detect the missing Backup header. To fix it we go into extra mode with x. e to relocate backup data structures. w to write and quit.

Success

$ sudo dd if=memtest86-usb.img of=/dev/sda status=progress
975204864 bytes (975 MB, 930 MiB) copied, 419 s, 2.3 MB/s
1910817+0 records in
1910817+0 records out
978338304 bytes (978 MB, 933 MiB) copied, 443.335 s, 2.2 MB/s

Conclusion

We only ever see the first partition automatically mounted in the OS. We want it as large as possible.

I didn't dare delete the second partition. It can be moved, though I had no success moving it beyond the 256 MB default location. It could prevent the disk booting.

The third partition is waste, but in my experience, is required if we want saved screens and logs to appear to the OS.

I would be interested in minimising the first and second partitions and deleting the third. It's boring having to photograph screen dumps, but MemTest86 only offers limited opportunities to take a screen grab.

Apart from MemTest86's impressive certificates, the advantage is it has a long legacy of support. I want to record time spent in a known stress test. I also want to save the reported speeds of various RAM configurations. A photograph at the end quickly captures all the information needed, 3 numbers, which can be curated just as easily as if they had been saved to the stick.

If 64 MB disks still existed I am sure we could make it run off one.

More interesting would be to devote just 2 MB to the other memory tester, Memtest86+, and expand its partition to the full disk size. At least I could compare and contrast the two products.

Summary script

cd ~/Downloads/memtest86-usb
sudo losetup --find --partscan memtest86-usb.img
lsblk | grep part -B1
sudo mount /dev/loop20p1 /mnt
sudo fstrim -v /mnt
sudo umount /mnt
sudo mount /dev/loop20p2 /mnt
sudo fstrim -v /mnt
sudo umount /mnt
sudo gparted /dev/loop20
# Manual resizing in gparted
sudo losetup --detach /dev/loop20
fdisk -l memtest86-usb.img
# Use final sector from the above command:
sudo truncate --size=$[(1910783+1+33)*512] memtest86-usb.img
gdisk memtest86-usb.img

x, e, w, y

sudo dd if=memtest86-usb.img of=/dev/sda status=progress

Foolproof, apart from the resizing in gparted.