Have tinkered with the different settings for a few hours now. Lost count on how many times I've rebooted the system... I tried the settings from your system files, fitzsteve. Didn't help I'm afraid. I tried setting low vram, high vram, nothing made a difference. Tried different options for glquake, low resolution, high resolution, colors, etc. Also tried smaller screen size in-game. Nothing made a difference. I came a cross a note in the documentation for the game saying that if the user experience high amounts of graphical corruption, try setting the Mediator jumper "Wait" as closed. Did that, but then the system would just freeze after booting into workbench.
I did however notice something. When running the Mediator configuration, if I selected "No" for MMU, then the workbench were smoother in a way. The text under the icons didn't appear rough and the shadow on the menu when right-clicking didn't have color speckles anymore. Quake didn't run when this was selected, of course. Just got a white screen that had slight flashing/flickering.
Still haven't tried a clean OS 3.9 install. Have some minor issues getting that working on my Amiga. Hopefully I'll sort that out tomorrow.