To me it seems like the main way they "make it harder" is that _other people_ like to talk about them a lot and might be inclined to take intolerances in general (including diagnosed ones) less seriously. Which I don't really understand -- it doesn't hurt me for someone else to avoid eating something, why should I fault them for that?