Yes. You want to offer a consistent user experience to your visitors. If they would see a different page / post every time they access your site they would quickly realize they are being your “guinea pigs” and this could influence their behaviour. Morevoer from a statistical point of view the validity of the experiment depends on keeping the two “samples” (i.e. the two or more groups of users, each one seeing a different version of the post/page) independent.