I store all boxcovers, pornstar headshots, avatars user image uploads in Varnish cache instance in the front-end web server. All that data is stored in the server's memory which is much faster than fetching from a drive somewhere in a server on the network. It's a little over 200GB of in-memory data at the moment. The prior cache purge method I was using for avatars was taking too long to complete. The CDN would re-fetch the image before it was purged from Varnish locally. The CDN would get stale data/old image which would then be cached at the CDN for a week or more. The images displayed in the forum come from the CDN.

When you upload an avatar the image you see comes directly from the server that processes the uploads and avoids the caching layers (Varnish and CDN). That's why you see that the upload was a success and the image exists on our end.

All of these layers of caching are part of what enables ADT's average page load speed to be under 2 seconds worldwide.

[Link]