Mixed-integer Programmingfor Control Arthur Richards and Jonathan How Motivation • What is Mixed-integer Programming? – MIP is an optimization method that combines continuous and discrete variables • Why is it useful? – MIP can model complex planning and control problems involving both continuous and discrete decisions • Why now? Is MIP new? – MIP is not a new concept, BUT online use has only arrived with fast computers and software Mixed-integer Programming for Control 2/63 Problem Classes Nonlinear Convex Non-convex Logical T F T F Mixed-integer Programming for Control 3/63 Problem Classes Nonlinear Convex Non-convex Logical T F T F Nonlinear opt. No MIP Mixed-integer Programming for Control 4/63 . No MIP MIP Mixed-integer Programming for Control 5/63 .Problem Classes Nonlinear Convex Non-convex Logical T F T F Nonlinear opt. No MIP Approximate by MIP Mixed-integer Programming for Control MIP 6/63 .Problem Classes Nonlinear Convex Non-convex Logical T F T F Nonlinear opt. Problem Classes Nonlinear Convex Non-convex Logical T F Two main goals of MIP approach: capture non-convexity and logic T F Nonlinear opt. No MIP Approximate by MIP Mixed-integer Programming for Control MIP 7/63 . No MIP • Then apply powerful integer Approximate MIP optimization tools to non-convex by MIP problems Mixed-integer Programming for Control 8/63 .Problem Classes Nonlinear Convex Non-convex Logical T F Two main goals of MIP approach: capture non-convexity and logic T • Encode non-convexity using F integers Nonlinear opt. Session Outline (1/2) 1. Introduction to MIP – MIP definition – Modeling • • • – – – – Assignment Non-convex constraints Piecewise affine systems Solving MIP Using optimization for feedback control Techniques for online solution of MIP Examples Mixed-integer Programming for Control 9/63 . MILP Assignment for Multi-Vehicle Systems 4. Projected Variable Metric Algorithm 3. Real-time Path-Planning for Tactical UAV 5. Receding Horizon Implementation of MILP for Vehicle Guidance Mixed-integer Programming for Control 10/63 .Session Outline (2/2) 2. edu/MILP seis.uk/~aeagr • Email arthur.edu Mixed-integer Programming for Control 11/63 [email protected] Resources • Slides and sample codes available online • Web resources acl.ac.uk jhow@mit. Formal Definition • Mixed-integer Linear Program (MILP) • Inherently non-convex • NP-complete BUT with good software. can find globaloptimum in many useful instances Mixed-integer Programming for Control 12/63 . Modelling using MIP . ) is PWA a (B5.A2) (B1. A(i+1)] Mixed-integer Programming for Control 14/63 .A3) (B4.A4) b Binary zi = 1 if b ∈ [Ai.MIP Modelling 1: PWA [Example from Bersimas and Tsitsiklis] • Constrain a = f(b) where f(.A5) (B2.A1) (B3. A(i+1)] Mixed-integer Programming for Control 15/63 .g.MIP Modelling 1: PWA [Example from Bersimas and Tsitsiklis] • Constrain a = f(b) where f(. choose z3 = 1 all other zi = 0 Binary zi = 1 if b ∈ [Ai.) is PWA e. choose z3 = 1 all other zi = 0 Binary zi = 1 if b ∈ [Ai.MIP Modelling 1: PWA [Example from Bersimas and Tsitsiklis] • Constrain a = f(b) where f(.g.) is PWA e. A(i+1)] Mixed-integer Programming for Control 16/63 . ) is PWA e. A(i+1)] Interpolation over [A3. A4] Mixed-integer Programming for Control 17/63 .g.MIP Modelling 1: PWA [Example from Bersimas and Tsitsiklis] • Constrain a = f(b) where f(. choose z3 = 1 all other zi = 0 Binary zi = 1 if b ∈ [Ai. M >> a2x .MIP Modelling 2: Disjunction [Bersimas and Tsitsiklis.b1.b2 • Known as “big-M” method Mixed-integer Programming for Control 18/63 . Morari. Brailsford and Williams] • Require either a1x ≤ b1 or a2x ≤ b2 • MILP form • M > 0. M >> a1x . g. choose z1 = 1 • MILP form requires z2 = 0 • M > 0.b1. M >> a2x . M >> a1x . Brailsford and Williams] • Require either a1x ≤ b1 or a2x ≤ b2 e. Morari.b2 • Known as “big-M” method Mixed-integer Programming for Control 19/63 .MIP Modelling 2: Disjunction [Bersimas and Tsitsiklis. g. Morari.b2 • Known as “big-M” method Mixed-integer Programming for Control 20/63 . M >> a1x . M >> a2x . choose z1 = 1 • MILP form requires z2 = 0 • M > 0.b1.MIP Modelling 2: Disjunction [Bersimas and Tsitsiklis. Brailsford and Williams] • Require either a1x ≤ b1 or a2x ≤ b2 e. b2 • Known as “big-M” method Mixed-integer Programming for Control 21/63 . M >> a2x .g. M >> a1x .MIP Modelling 2: Disjunction [Bersimas and Tsitsiklis. Brailsford and Williams] • Require either a1x ≤ b1 or a2x ≤ b2 e. Morari. choose z1 = 1 • MILP form requires z2 = 0 • M > 0.b1. Brailsford and Williams] • Require either a1x ≤ b1 or a2x ≤ b2 OR choose z2 = 1 • MILP form requires z1 = 0 • M > 0. M >> a2x . Morari.MIP Modelling 2: Disjunction [Bersimas and Tsitsiklis.b2 • Known as “big-M” method Mixed-integer Programming for Control 22/63 .b1. M >> a1x . MIP Modelling 3: Avoidance [Schouwenaars et al. Richards et al] • Similar to disjunction • Point (x. y) must be outside obstacle x − x max ≥ − Mc 1 and x min − x ≥ − Mc 2 c1=0 ⇒ and y − y max ≥ − Mc 3 c =0 ⇒ 2 and y min − y ≥ − Mc 4 and 4 ∑c k =1 k ≤3 c3=0 ⇒ y (xmax . ymax) (xmin . ymin) x c4=0 ⇒ Mixed-integer Programming for Control 23/63 . MIP Modelling 4: Assignment • Assign N tasks to N agents. • Cost of assigning agent i to task j is cij Binary zij = 1 if agent i assigned to task j • Special case – LP works – Constrain 0 ≤ zij ≤ 1 : all vertices are integer Mixed-integer Programming for Control 24/63 . MIP Modelling 4: Assignment • Assign N tasks to N agents. • Cost of assigning agent i to task j is cij Binary zij = 1 if agent i assigned to task j • Add resource constraint – MILP needed Mixed-integer Programming for Control 25/63 . MIP Modelling 5: Modes • System has two modes Mode 1 Mode 2 Mixed-integer Programming for Control 26/63 . MIP Modelling 5: Modes • System has two modes Mode 1 Mode 2 • MIP representation Mixed-integer Programming for Control 27/63 . needs binaries Mixed-integer Programming for Control 28/63 .MIP Modelling 6 : Speed Limits • 2-norm approximation ||v|| ≤ vmax : convex. easily handled ||v|| ≥ vmax : non-convex. Karimi. but look for something better – “Most existing MILP formulations that employ big-M constraints do suffer from the poor relaxation (relaxed MILP). AIChE. 2005 Mixed-integer Programming for Control 29/63 . which is a notorious feature of bigM.MIP Modelling: Remarks • Examples span many problem classes – Combinations and extensions possible • Joint assignment/path planning with avoidance • PWA systems with disjunction constraints • Logical constraints – “if A and B then C” • There are often multiple ways of expressing a problem using MIP – Rule of thumb: big-M is nearly always an option. A. Sundaramoorthy and I.” • Improving Mixed Integer Linear Programming Formulations A. Khurana. ” Automatica. Mixed-integer Programming for Control 30/63 . Beasley. Clarendon Press. N. Tsitsiklis. Bemporad and M. ``Computational Logic and Integer Programming. 1996. dynamics. Brailsford. E. Williams and S. and constraints. Introduction to Linear Optimization. 1995. • A. Floudas Nonlinear and Mixed-Integer Programming Fundamentals and Applications Oxford University Press. 35:407-427. • D. Morari “Control of systems integrating logic. 1997. 1999 • H. Editor J.Modelling References • C. pp." in Advances in Linear and Integer Programming.249-281. Bertsimas and J. Athena Scientific. MIP Solution . if relaxation – is infeasible. Choose an i and (branching) a) Fix zi = 0. • Recursive tree search of binary options • Can stop “early” by fathoming. b) Fix zi = 1. go to 2.1}→0 ≤ zi ≤ 1 2. Relax binary constraints zi ∈ {0. or – has worse cost than best binary so far Mixed-integer Programming for Control 32/63 .MIP Solution: Branch & Bound • Finds global optimum by tree search 1. go to 2. Solve relaxed problem (bounding) 3. or – gives binary result. if relaxation – is infeasible. Solve relaxed problem (bounding) 3. Choose an i and (branching) a) Fix zi = 0. Relax binary constraints zi ∈ {0. or – has worse cost than best binary so far. or – gives binary result. go to 2.1}→0 ≤ zi ≤ 1 2. go to 2. Mixed-integer Programming for Control 33/63 . • Recursive tree search of binary options • Can stop “early” by fathoming. b) Fix zi = 1.MIP Solution: Branch & Bound • Finds global optimum by tree search 1. subject to bcon: b = sum{i in 1. subject to acon: a = sum{i in 1..N}. var z{i in 1... subject to xi{i in 1. subject to xN: x[N] <= z[N-1].(N-1)}: x[i] <= z[i-1] + z[i]..(N-1)} = 1.N} B[i]*x[i]. subject to xpos{i in 1..N}: x[i] >= 0. subject to zsum: sum{i in 1.N} A[i]*x[i]. Mixed-integer Programming for Control 34/63 . subject to x1: x[1] <= z[1].MIP Solution: AMPL • AMPL easily translates models – A Mathematical Programming Language – Helps sort out indexing – Interfaced to many solver codes PWA example var x{i in 1... subject to xsum: sum{i in 1..N} x[i] = 1.(N-1)} binary. MIP Solution: CPLEX • Commercial solver code from ILOG • Implements branch-and-bound in conjunction with tried and tested branching heuristics • Interfaces – AMPL – Matlab MEX – C API Mixed-integer Programming for Control 35/63 . Mixed-integer Programming for Control 36/63 . Kernighar. W. Mixed-Integer. Dynamic. M. and B. and Mixed-Integer Nonlinear Optimization by Floudas et al. Fourer. AMPL. Online • Good comparison of non-commercial MILP software: • MINOPT: A Modeling Language and Algorithmic Framework for Linear. Gay. The Scientific Press.Other Software • Decision tree for Optimization with discrete variables Online • Modeling examples: MILP Model for Short Term Scheduling of Multistage Batch Plants by Grossmann et al. Online • The Hybrid Systems Group – Multi-Parametric Toolbox. D. • Interface Software and example (Matlab ÅÆ AMPL ÅÆ CPLEX) • AMPL: R. Nonlinear. 1993. A modeling language for mathematical programming. MIP Solution: Remarks • AMPL is good for prototyping. but more direct interfaces are faster • Other discrete optimization tools are available – Heuristic methods for general MIP • Genetic algorithms • Simulated annealing – Special cases • Dynamic programming for knapsack problem Mixed-integer Programming for Control 37/63 . MIP for Control . Go to 1. Mixed-integer Programming for Control 39/63 . Execute some initial portion of that sequence 3.Control using Optimization • • Have seen that MIP can find optimal solutions for complex planning problems “Control” also considers uncertainty – – • Introduce feedback to compensate Update plans to include new information Concept is the same as Model Predictive Control (MPC) 1. Use numerical optimization to design an open-loop control sequence for the future 2. ‘task done’) – Non-convex constraints (e.g. logic states (e. avoidance) – PWA cost Mixed-integer Programming for Control 40/63 .g.MIP and MPC • MPC optimization has three key features – Dynamics model – Constraints – Cost • MIP can enter all three – PWA dynamics. Properties of MPC • Start with a recursion – Solution is {u(k0|k0) u(k0+1|k0) … u(k0+N|k0)} at k0 – Then {u(k0+1|k0) … u(k0+N|k0) ???} feasible at k0+1 Mixed-integer Programming for Control 41/63 . Properties of MPC • Start with a recursion – Solution is {u(k0|k0) u(k0+1|k0) … u(k0+N|k0)} at k0 “The tail” Some control here – Then {u(k0+1|k0) … u(k0+N|k0) ???} feasible at k0+1 Mixed-integer Programming for Control 42/63 . Properties of MPC • Start with a recursion – Solution is {u(k0|k0) u(k0+1|k0) … u(k0+N|k0)} at k0 “The tail” Some control here – Then {u(k0+1|k0) … u(k0+N|k0) ???} feasible at k0+1 • Use recursion to prove cost decrease – J(k+1) ≤ J(k) – a(x(k)) • Use cost J as Lyapunov function – Stage cost must be positive definite in x Mixed-integer Programming for Control 43/63 . easily handling MIP – If notconstraints at setpoint.Properties of MPC • Start with a recursion – Solution is {u(k0|k0) u(k0+1|k0) … u(k0+N|k0)} at k0 “The tail” Some control here – Then {u(k0+1|k0) … u(k0+N|k0) ???} feasible at k0+1 • Very Use general recursion to prove cost decrease results. and cost • Use cost J as Lyapunov function – Stage cost must be positive definite in x Mixed-integer Programming for Control 44/63 . J(k+1) ≤ J(k) – a(x(k)) [Bemporad and Morari. 1999] dynamics. MIP/MPC Example: Modes • Recall earlier example Mode 1 Mode 2 Mixed-integer Programming for Control 45/63 . MIP/MPC Example: Modes 1. Repeat Mixed-integer Programming for Control 46/63 . Optimize 2. Apply u(k) = u*(k|k) 3. Optimize Cost Dynamics Initial condition Terminal Control limits Mode logic 2. Apply u(k) = u*(k|k) 3. Repeat Mixed-integer Programming for Control 47/63 .MIP/MPC Example: Modes 1. Using MIP Online Computation Time • Need to solve MIP online – Good computer and solver handle most cases – Still NP-hard problem – some instances of large problems can be slow • Tricks to accelerate solution time are often problem-specific – More to follow in example talk • Three general approaches – Use prior knowledge – Approximate cost-to-go – Multi-parametric integer programming Mixed-integer Programming for Control 49/63 Prior Knowledge I : Iteration Space Station Rendezvous with Plume Impingement (PI) Constraints • Firing expected only at start and end: “bang-off-bang” • Remove constraints on PI around center of maneuver • Check results for PI • Iterate, adding constraints back in, until no PI Mixed-integer Programming for Control 50/63 Prior Knowledge II : Grouping • Minimum fuel Space Station fly-by with PI constraints Mixed-integer Programming for Control 51/63 . Prior Knowledge II : Grouping • Minimum fuel Space Station fly-by with PI constraints – 9600 binary variables – Impractical to find optimal solution • Strategy: – Group time-steps Time (secs) No plume constraints 8 All plume constraints 1800 (limit) Groups of three 587 Mixed-integer Programming for Control 52/63 . Approximate Cost-to-Go • Replace tail of long plan with approximate plan – Re-plan online as goal approached Mixed-integer Programming for Control 53/63 . Multi-Parametric MIP • Calculate solution offline as a piecewise affine function of the initial condition – [Bemporad et al 2004] • Replace online MIP with PWA look-up Online Solve MIP Execute first step Offline Online Solve mpMIP PWA lookup Solution map Mixed-integer Programming for Control Execute first step 54/63 . Task Assignment Example . – SAM site visited before HVT – Synchronized strike of HVT Æ All can be expressed in a canonical form • Select assignments & trajectories to optimize desired mission objectives and satisfy dynamic/timing constraints – Costs for demonstration based on time.Example: Task Assignment • Different types of UAV’s • Different types of targets HVT SAM • Example timing constraints – BDA after Strike after Recon. but extends to scores / risks – Hard because problems are tightly coupled Mixed-integer Programming for Control 56/63 . Experimental Scenario • 4 vehicles • 16 tasks • Full horizon assignment approach not scalable for real-time assignment • Environment with dynamic changes – Original assignment – Vehicle death – Pop-up target – Vehicle death – Target location update Mixed-integer Programming for Control 57/63 . Experimental Scenario • 4 vehicles • 16 tasks • Full horizon assignment approach not scalable for real-time assignment • Environment with dynamic changes – Original assignment – Vehicle death – Pop-up target – Vehicle death – Target location update Mixed-integer Programming for Control 58/63 . UAV Example . UAV Example • Very simple trial example – One UAV avoiding one obstacle – Uses MPC and MIP • Problem – – – – – Matlab sim script AMPL data file AMPL state file AMPL script AMPL model file • Download: http://seis.bris.uk/~aeagr Mixed-integer Programming for Control 60/63 .ac. ac.bris.uk/~aeagr Mixed-integer Programming for Control 61/63 .UAV Example • Very simple trial example – One UAV avoiding one obstacle – Uses MPC and MIP • Implementation – – – – – Matlab sim script AMPL data file AMPL state file AMPL script AMPL model file • Download: http://seis. – Abdollah Homaifar (North Carolina A\&T State Univ. – Bijan Sayyarrodsari (Pavilion Tech.) • MILP Assignment Problems for Multi-Vehicle Systems – Matthew Earl (BAE) – Raffaello D'Andrea (Cornell Univ. of Pennsylvania). How (MIT) Mixed-integer Programming for Control 62/63 .) • Receding Horizon Implementation of MILP for Vehicle Guidance – Yoshiaki Kuwata (MIT) – Jonathan P.Session Outline • Projected Variable Metric Algorithm for Mixed Integer Optimization Problem – Ali Ahmadzadeh (Univ.) – Robert Miller (Northrop Grumman Corp. Inc).) • Real-Time MILP Path-Planning for Tactical UAV Applications – Cedric Ma (Northrop Grumman Corp. bris.edu/MILP seis.mit.edu Mixed-integer Programming for Control 63/63 .Questions? • Web resources acl.ac.uk [email protected]@bristol.uk/~aeagr • Email arthur.