How do you A/B test prompt changes when you can't measure quality automatically?

Classic A/B testing assumes you can measure the outcome. For most of our prompts the quality signal is ambiguous — "was this response helpful?" doesn't translate to a clean metric. Human rating is expensive and slow. LLM-as-judge has its own biases. How are product teams making confident decisions about prompt changes without either burning budget on labellers or flying blind?