Coarse particles are transported in a river as bed load, i.e., they move in frequent contact with and are supported by the granular bed. This movement is typically intermittent and may be described by a series of steps are rests, the distributions of which determine particle dispersion. Laboratory and field studies of bed load tracer dispersion have reported sub- and super-diffusive behavior, both of which have been successfully reproduced with stochastic transport models. Although researchers have invoked heavy-tailed step lengths as the cause of anomalous dispersion, most observations report thin-tailed distributions. Little attention has been paid to rest periods, and stochastic transport models have not been connected to the underlying mechanics of particle motion. Based on theoretical and experimental evidence, we argue that step lengths are thin-tailed and do not control the longterm dispersion of bed load tracers; they are determined by momentum balance between the fluid and solid. Using laboratory experiments with both marbles and natural sediments, we demonstrate that the rest time distribution is power law, and argue that this distribution controls asymptotic dispersion. Observed rest times far exceed any hydrodynamic timescale. Experiments reveal that rest times of deposited particles are governed by fluctuations in river bed elevation; in particular, the return time for the bed to scour to the base of a deposited particle. Stochastic fluctuations in bed elevation are describable by an Ornstein-Uhlenbeck (mean-reverting random walk) model that contains two parameters, which we show are directly related to the granular shear rate and range of bed elevation fluctuations, respectively. Combining these results with the theory of asymmetric random walks (particles only move downstream), we predict superdiffusive behavior that is in quantitative agreement with our observations of tracer dispersion in a natural river.