Yep that's exactly the situation you're in then, the host is still holding on to the audio and USB devices.
You can release those from the host's device drivers with a VM pre-start script like this:
echo 0000:0c:00.1 > /sys/bus/pci/devices/0000:0c:00.1/driver/unbind
echo 0000:0c:00.2 > /sys/bus/pci/devices/0000:0c:00.2/driver/unbind
echo 0000:0c:00.3 > /sys/bus/pci/devices/0000:0c:00.3/driver/unbind
Then if your VM manager doesn't do it for you, you can rebind them to vfio-pci by their PCI IDs (use lspci -nn to show the IDs). Here's an example with my hardware's IDs that you'd need to adapt for your device IDs.
echo 8086 1d2d > /sys/bus/pci/drivers/vfio-pci/new_id
echo 8086 1d26 > /sys/bus/pci/drivers/vfio-pci/new_id
echo 1b73 1100 > /sys/bus/pci/drivers/vfio-pci/new_id