Well for one thing they can start figuring out who is not yet registered on Signal but would likely be in the phone contacts list of a rich person's number that they know. Social engineering attacks succeed with less.