Sunday, January 29, 2012

Varying SCSI queue depth for VMware PVSCSI block devices.

I was toying with the block subsystem a bit in a Linux virtual machine running under ESX 5.0, when I realized
I could not change the SCSI queue depth. It turns out that the driver simply didn't implement the interface! It was, however, pretty easy to fix this.

Now you can do something like the following:
# for i in {a..z}; do  eval 'echo 1 > /sys/block/sd$i/device/queue_depth'; done

The patch should make it to mainline, when the current PVSCSI maintainer, Arvind Kumar, gets it integrated.

Why would you care? Because /sys/block/sda/device/queue_depth is quite different from /sys/block/sda/queue/nr_requests. nr_requests controls the request flow before the I/O scheduler, while queue_depth controls the flow of actual dispatch on I/O device. You might be interested in either of those if you run with multiple VMDKs, have an intensive I/O workload on one disk, and notice starvation on others.

1 comment:

  1. What about ring size of 32? To go with QD=255?

    VSA appliances need both.

    Can this be managed with ESXi 5.5 vmtools now?

    Anyone ever done a pepsi challenge (ESXi 5.5 versus Hyper-v 2012R2) to see how far each can push ?