Sorry, but I can't believe that this is about timing. I don't see how the data packets, which are buffered, can impact DAC timing*.
I
believe part of the answer is that a varying clock signal introduces varying voltages into the ground (and, quite possibly, the +Ve power) plane. I suspect that the PLL having to continually adjust to keep the lock on this varying frequency is a possible culprit.
Meridian also discovered some time ago that noise entering the 818 through the ethernet cable had a significant impact on sound quality which led, ultimately, to them designing the ID41 to eliminate some of this noise. A more thorough answer is likely to be found within the EtherRegen designer's detailed posts:
John Swenson.
Whatever the reasons, on a high-end system, there is a clear difference between different ethernet switches (I can easily hear the difference in the depth of the soundstage between the SotM switch and the second Antipodes CX ethernet output).