I always thought the Butlerian Jihad was a convenient way to remove AI as a plot element. Same thing with shields and explosions; it made swordfighting a plausible way to fight in a universe with faster-than-light travel.