We analyse discontinuous Markovian strategies for differential games. The best response correspondence uniquely maps almost all profiles of opponents’ strategies back to the strategy space. We thus make Markov-perfect equilibria in a wide class of differential games well-behaved, resolving a long-standing open problem. We provide a readily applicable necessary and sufficient condition for best responses and Markov-perfect Nash equilibria. We demonstrate our methods in a canonical model of non-cooperative mitigation of climate change. Our approach provides novel, economically important results: we obtain the entire set of symmetric Markov-perfect equilibria, and demonstrate that the best equilibria can yield a major welfare improvement over the equilibrium which previous literature has focused on. International climate negotiations can be seen as being about coordination on good equilibria, rather than about bargaining over the limited surplus available in a dynamic prisoner’s dilemma.