Dev Ups

Published 2022-09-03 in ci-cd

Azure VM creation process, for maximum value, quickly

I will describe setting up Azure VMs; what matters, not what doesn't. The disk is another factor that isn't reflected in the final price before you click create. I go into a bit more depth with disk specification. If your intention is to provision a VM quickly by script, you might be surprised by how little random access matters.

The first task from the Azure portal is to find the create VM option. This isn't always obvious, for example clicking +Create from the context of my resource group leads me to the marketplace where it appears Azure wants to sell me a proprietary OS/image. Ubuntu 20 is there, but it's September 2022 and Ubuntu 22 still isn't. Far better navigation is to be had from your Azure homepage, where you are just a hover and two clicks away from initiating the creation procedure.

Accepting the defaults in the VM creation wizard is the fastest way to create 3 VMs. Of course IaC scales better but at the cost of needing to write, understand, and maintain code. Some of the defaults depend on your previous usages.

Basics

The "Basics" tab first shown during VM creation is the most important. The region and VM size will determine the cost. Deploying in parallel to multiple availability zones would too but Kubernetes provides some high availability (HA) of its own so is not required here.

Resource groups

The resource group is a feature unique to Azure. It is required for all chargeable resources and provides a convenient way to manage the group in terms of cost control. It takes a region (is regional) but can contain resources from any region. I recommend not deleting it after use but instead deleting its contents, that way the fractional pennies that go to the different services it contains can be better identified.

Size

B1s is a 1GB VM and the cheapest suitable for running containers (worker nodes).

B1ms is a 2GB VM and the cheapest suitable for running the master node. Kubernetes has a minimum requirement of 1700MB currently, though third party Kubernetes providers are constantly looking to reduce that.

B1ls is a 0.5GB VM and the cheapest for testing arbitrary concepts that can squeeze into 512MB. You can run docker, and so basic containers, on one of these. Building images will probably exhaust the RAM. Some images already have huge RAM requirements. You'll learn the reprovisioning concept well picking B1ls.

Virtual Machine name

Filling this field auto-populates the resource group if left as default ("(New) Resource group").

Image

Choose free, and modern. I chose Ubuntu Server 22.04 LTS - Gen2.

Administrator account

Authentication type should remain as public key by default. By default Azure will provide this to you, once only, as the VM is being created.

I prefer to manage just one so choose "Use existing public key" as the key source and paste the public key in the space provided. The public key must be the RSA type, and not the newer ED25519 variety.

Disks

Choose anything but premium for disk type. Standard HDD should work out much cheaper than SSD so that is my preference. Both will weigh in at 30GB, despite what disk-types and disk-pricing say about SSDs starting from 4GB and HDDs from 32GB. sudo du -hs / shows just 2.1GB occupied. df -h says there's 1.4G in /.

The disk can feel like a bottleneck when provisioning from script. That's what I do, so I made some benchmarks (limited by my puny B1ls VM).

HDD synthetic quick checks

azureuser@testVM1:~$ sudo hdparm -Tt /dev/sda

/dev/sda:
 Timing cached reads:   18140 MB in  1.99 seconds = 9127.01 MB/sec
 Timing buffered disk reads: 122 MB in  3.03 seconds =  40.31 MB/sec

azureuser@testVM1:~$ dd if=/dev/zero of=/tmp/output bs=8k count=100k; rm -f /tmp/output
102400+0 records in
102400+0 records out
838860800 bytes (839 MB, 800 MiB) copied, 15.3096 s, 54.8 MB/s

SSD synthetic quick checks

azureuser@testVM1:~$ sudo hdparm -Tt /dev/sda

/dev/sda:
 Timing cached reads:   20078 MB in  1.99 seconds = 10092.95 MB/sec
 Timing buffered disk reads: 138 MB in  3.01 seconds =  45.81 MB/sec

azureuser@testVM1:~$ dd if=/dev/zero of=/tmp/output bs=8k count=100k; rm -f /tmp/output
102400+0 records in
102400+0 records out
838860800 bytes (839 MB, 800 MiB) copied, 15.66 s, 53.6 MB/s
azureuser@testVM1:~$ 

Installing docker times

26s for SSD vs 34.5s for HDD executing:

time sudo apt-get install docker.io -y

Flexible I/O tester

This is an external package, apt-get installed, for testing I/O. So that you can trust how I'm running it I copied the invocations verbatim from this SO answer by Mikko Rantalainen. The differences between each are in the value of the rw, fsync, and iodepth parameters. The blocksize you should already be familiar with; large blocks mean less head seeking/movement, ie. less randomness in access patterns.

Sequential READ speed with big blocks

fio --name TEST --eta-newline=5s --filename=fio-tempfile.dat --rw=read --size=500m --io_size=10g --blocksize=1024k --ioengine=libaio --fsync=10000 --iodepth=32 --direct=1 --numjobs=1 --runtime=60 --group_reporting

SSD:

Run status group 0 (all jobs):
   READ: bw=53.6MiB/s (56.2MB/s), 53.6MiB/s-53.6MiB/s (56.2MB/s-56.2MB/s), io=3250MiB (3408MB), run=60608-60608msec

HDD:

Run status group 0 (all jobs):
   READ: bw=53.6MiB/s (56.2MB/s), 53.6MiB/s-53.6MiB/s (56.2MB/s-56.2MB/s), io=3248MiB (3406MB), run=60577-60577msec

Sequential WRITE speed with big blocks

fio --name TEST --eta-newline=5s --filename=fio-tempfile.dat --rw=write --size=500m --io_size=10g --blocksize=1024k --ioengine=libaio --fsync=10000 --iodepth=32 --direct=1 --numjobs=1 --runtime=60 --group_reporting

SSD:

Run status group 0 (all jobs):
  WRITE: bw=53.5MiB/s (56.1MB/s), 53.5MiB/s-53.5MiB/s (56.1MB/s-56.1MB/s), io=3238MiB (3395MB), run=60549-60549msec

HDD:

Run status group 0 (all jobs):
  WRITE: bw=53.6MiB/s (56.2MB/s), 53.6MiB/s-53.6MiB/s (56.2MB/s-56.2MB/s), io=3247MiB (3405MB), run=60555-60555msec

Random 4K read QD1

fio --name TEST --eta-newline=5s --filename=fio-tempfile.dat --rw=randread --size=500m --io_size=10g --blocksize=4k --ioengine=libaio --fsync=1 --iodepth=1 --direct=1 --numjobs=1 --runtime=60 --group_reporting

SSD:

Run status group 0 (all jobs):
   READ: bw=4887KiB/s (5004kB/s), 4887KiB/s-4887KiB/s (5004kB/s-5004kB/s), io=286MiB (300MB), run=60014-60014msec

HDD:

Run status group 0 (all jobs):
   READ: bw=4899KiB/s (5016kB/s), 4899KiB/s-4899KiB/s (5016kB/s-5016kB/s), io=287MiB (301MB), run=60001-60001msec

Mixed random 4K read and write QD1 with sync

fio --name TEST --eta-newline=5s --filename=fio-tempfile.dat --rw=randrw --size=500m --io_size=10g --blocksize=4k --ioengine=libaio --fsync=1 --iodepth=1 --direct=1 --numjobs=1 --runtime=60 --group_reporting

SSD:

Run status group 0 (all jobs):
   READ: bw=370KiB/s (379kB/s), 370KiB/s-370KiB/s (379kB/s-379kB/s), io=21.7MiB (22.7MB), run=60007-60007msec
  WRITE: bw=372KiB/s (381kB/s), 372KiB/s-372KiB/s (381kB/s-381kB/s), io=21.8MiB (22.9MB), run=60007-60007msec

HDD:

Run status group 0 (all jobs):
   READ: bw=306KiB/s (314kB/s), 306KiB/s-306KiB/s (314kB/s-314kB/s), io=18.0MiB (18.8MB), run=60007-60007msec
  WRITE: bw=313KiB/s (321kB/s), 313KiB/s-313KiB/s (321kB/s-321kB/s), io=18.4MiB (19.3MB), run=60007-60007msec

Result

FIO doesn't show much differences between the technologies. Obviously an SSD will perform more consistently for hugely random access, but software and OS developers had been looking to reduce these before the advent of SSD. Why random reads are the same speed between the technologies probably has more to do with Azure optimisations. They do say the HDDs are locally redundant (as are SSDs) so maybe they have a huge resident memory cache in front of them.

Other tabs

I skip these. Tags is useful for cost monitoring, if you plan on having costs. Networking is useful for security but I get all I need from it by enabling ports in the Basics tab, only port 22 being open is usually enough for me.

Conclusion

Docker install times of 26s vs 34.5s present a potential time saving 25% and mirror what I'd observed before, installing Kubernetes

I can't get a price on 30GB of either standard SSD or HDD. The E4 SSD is 32GB and $2.40/month (so $2.25 for 30GB), the S4 HDD is 32GB and $1.54/month. With VMs starting at $3.80/month the choice of disk can increase price by upto 59% for SSD and 38% for HDD. Assuming there aren't any other hidden costs.