Not rinsing also leaves all the food residue, germs (dead or alive) and debris that has collected in your mouth throughout the day. Not rinsing my mouth after brushing feels like showering and leaving the soap on.
Germs will be in your mouth whether you rinsed or not. The bacteria matter, those are the reason you don't want food there in the first place, and those are killed better by not rinsing.
Think of the paste more like something you need to apply and let sit, and less as some kind of fancy soap to wash the teeth with.
