I think there are two things... short term and long term term listening.

For me, short term is about a quick test of whether there is a difference.

Long term is about whether I like the difference.

Back to the proposed test... I suppose bias might get in the way. Placebos might be useful.
E.g. If a user with strong bias towards there being an effect might report a preference even if they were given 5 identical tracks... i.e. the same track 5 times.

So, it could be a really good study of both ripping tech and psychology. Anyone got a son/daughter heading to college to study brain stuff?

Keeping it simple I think the test proposed could still be useful or at least interesting.


Rgds,
Nick