In this paper we present a variational technique that handles coarse-graining and passing to a limit in a unified manner. The technique is based on a duality structure, which is present in many gradient flows and other variational evolutions, and which often arises from a large-deviations principle. It has three main features: (a) a natural interaction between the duality structure and the coarse-graining, (b) application to systems with non-dissipative effects, and (c) application to coarse-graining of approximate solutions which solve the equation only to some error. As examples, we use this technique to solve three limit problems, the overdamped limit of the Vlasov-Fokker-Planck equation and the small-noise limit of randomly perturbed Hamiltonian systems with one and with many degrees of freedom.