We quantify the role of geography and land quality for agricultural productivity differences across countries using high-resolution micro-geography data and a spatial accounting framework. The rich spatial data provide for each cell of land covering the entire globe, the potential yield for 18 crops, which measures the maximum attainable crop output given soil quality, climate conditions, terrain topography, and a given level of cultivation inputs. While there is considerable heterogeneity in land quality across space, even within narrow geographic regions, we find that low agricultural land productivity is not due to unfavourable geographic endowments. If countries produced current crops in each cell according to potential yields, the rich-poor agricultural yield gap would virtually disappear, from 214 percent to 5 percent. We also find evidence of additional aggregate productivity gains attainable through spatial reallocation and changes in crop production.