In my experience, the house of cards is usually debugging, not building. I've had to do some pretty crazy debugging, stepping through deep call stacks frame by frame, keeping track of big data structures as they change along the way. This can take a huge amount of focus that would absolutely be ruined by even a 5 second conversation.