Robots.txt not updating on live site

Howdy! We had a code release this afternoon and noticed that the robots.txt file is not in sync with our latest changes on the live domain… it is fine on the vercel.app site. Not sure how long this should take as everything else from the code release is synced up.

https://famvia-site-famvia.vercel.app/robots.txt (looks fine - updated with our latest code release this afternoon)
famvia.com/robots.txt (outdated and not in sync with the .app site)

Tried to purge cache as well.

Hi, @famviatravel! Nice to have you join us in the Vercel Community :wave:

This is likely related to DNS propagation. When you make changes to your DNS settings or deploy updates, it can take some time for these changes to propagate across the internet. This process can take up to 24-48 hours in some cases.

It could also be due to caching. Vercel automatically caches static files, including robots.txt, to improve performance. These files are replicated and deployed to every region in Vercel’s global Edge Network after the first request .

When you redeploy your site, Vercel automatically invalidates the cache for changed files. However, this process might not be instantaneous across all regions .

Here are some steps you can take:

  1. Wait for Propagation: Given that your deployment was recent, it’s possible that the changes are still propagating. You might need to wait a bit longer for the robots.txt file to update on your custom domain .
  2. Verify Production Deployment: Ensure that your latest deployment is actually set as the production deployment for your custom domain in the Vercel dashboard.

Remember, Vercel’s infrastructure is designed for faster and more reliable global propagation, with cache purges and configuration updates now propagating globally in about 300ms. However, external factors like DNS propagation can still affect how quickly changes are visible on custom domains.

Let us know how you get on!

1 Like

Also, for what it’s worth, I just checked them side by side and they both are the same.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.