What mechanisms support our ability to estimate durations on the order of minutes? Behavioral studies in humans have shown that changes in contextual features lead to overestimation of past durations. Based on evidence that the medial temporal lobes and prefrontal cortex represent contextual features, we related the degree of fMRI pattern change in these regions with people's subsequent duration estimates. After listening to a radio story in the scanner, participants were asked how much time had elapsed between pairs of clips from the story. Our ROI analysis found that the neural pattern distance between two clips at encoding was correlated with duration estimates in the right entorhinal cortex and right pars orbitalis. Moreover, a whole-brain searchlight analysis revealed a cluster spanning the right anterior temporal lobe. Our findings provide convergent support for the hypothesis that retrospective time judgments are driven by 'drift' in contextual representations supported by these regions. 23 subjects were scanned in a 3T full-body MRI scanner while listening to a 25-minute (and 34-second) radio story called "Tunnel Under the World" Download the README.txt file for a detailed description of this dataset's content