I've solved this now through some iterative Googling but since it was frustrating me so much I wanted to post it here for people to find in the future.
Running "pacmd list-sinks" showed my HDMI output device in state "SUSPEND" with reason as "IDLE". It goes into this state after only a few seconds of no audio. Thanks to this old post I commented out the following line in /etc/pulse/default.pa -
Code: Select all
### Automatically suspend sinks/sources that become idle for too long
#load-module module-suspend-on-idle