Image Optimization Requests coming through in Vercel even though I use a custom loader

Help
1

I have been trying to cutdown on my usage in Vercel and one of the areas what my image optimization. After chatting with a few folks at Vercel it was identified that some crawler somewhere was accessing a deployment preview link… fun! I enabled deployment protection on all of my projects making it so that you have to be authenticated with Vercel in order to see the preview URL. This kind of sucks especially if you are building something for a client and you want to show them progress… by I digress on that.

One of my projects is still getting image optimization requests even though I stopped using next/image a year ago (switching to sanity-image instead). I did just recently switch back to next/image with a custom image loader from next-sanity/image. Unfortunately, I still see requests to /_next/image coming through in my logs as well as usage for this particular project.

Looking at the request, a vast majority of them if not all of them come from facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php), or meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler) with some from Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0. The images that are being requested are years old at this point, so I’m not sure why they would still be coming in. Anyone have any ideas?

2

Hi, @jamesrsingleton!

Thanks for your patience :pray:

The fact that you’re still seeing requests to /_next/image suggests that these URLs are cached or indexed somewhere, possibly by Facebook’s crawler. It might be worth getting in touch with them, if possible.

One idea would be to look at your robots.txt and configure it to disallow crawling of the /_next/image path. This might help prevent future crawling.

User-agent: *
Disallow: /_next/image

You could also try to use Vercel Edge Middleware to intercept and handle these requests at the edge, potentially improving performance.

Would be curious to hear how this goes!

2 Likes
3

I have updated my robots.txt to look like

User-Agent: *
Allow: /
Disallow: /_next/image

So :crossed_fingers: that I see a reduction soon.

4

Thanks – keep us posted!

5

So I implemented this 3 days ago and still see image optimizations coming in for this particular site.

6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.