Cache Freshness vs Hit Rate: Correctness vs Efficiency
Overview Cache freshness measures whether the data returned by a cache still matches the current state of its source of truth, while hit rate measures how often requests are answered directly from the cache instead of falling through to the origin. The two metrics pull in opposite directions: optimizing for freshness means shorter TTLs and more origin traffic, while optimizing for hit rate means longer TTLs and a higher chance of serving stale data. Tuning a cache well means choosing the right balance point for that specific data’s tolerance for staleness. ...