You can sorta set the delay per source.

what you have to do is use the lipsync option in a preset. You can then create a custom preset per source. So when you select a source, it loads a specific preset with your lipsync.