It sounds like you need to take a look at the settings in the Audio MIDI Setup application in the Utilities folder. This allows you to set sample rate and word size, along with set the output type (where Encoded Digital Audio is generally one of the options), and setup the surround options using the Configure Speakers button.

Hope this helps!

Office: MacMini + Roon -> MS600 -> AR LS-9 -> M60s (1993 vintage)
Home: [AppleTV -> HD621] + [Roon -> 218] -> G65 -> DSP5200HC + DSP5200s + DSP3200s
Being a JAFA in Auckland, New Zealand