Exchanging scarce resources during execution among a group of agents is one way to improve the overall performance in multi-agent systems with limited shared resources, but implementing optimal sequential resource allocation is often a nontrivial problem in complex systems with uncertainties. In this paper, we present an MILP-based algorithm that can automatically break a large mission into multiple phases and make optimal resource (re)allocations at the entry of each phase. We illustrate our algorithms through several increasingly complex classes of sequential resource allocation problems, and show through experiments that our techniques can increase agents' rewards for varying levels of constraints on resources and constraints on exchanging resources.