Nonlinear programming convex optimization

This series of complementary textbooks cover all aspects of continuous optimization, and its connections with discrete optimization via duality. The two convex optimization books deal primarily with convex, possibly nondifferentiable, problems and rely on convex analysis. By contrast the nonlinear programming book focuses primarily on analytical and computational methods for possibly nonconvex differentiable problems. It relies primarily on calculus and variational analysis, yet it still contains a detailed presentation of duality theory and its uses for both convex and nonconvex problems.

The books share notation, style, and prerequisites, and are aimed at students, researchers, and practitioners, roughly at the first year graduate level. They rely on rigorous analysis, but they also aim at an intuitive exposition that makes use of geometric visualization. They are supported by substantial internet-accessible material, including exercises with solutions, lecture slides, and course content from MIT Open Courseware. Three other complementary and more specialized books of the author can also be freely downloaded: the 1998 Network Optimization book, the 1989 Parallel and Distributed Computation book (coauthored with J. Tsitsiklis), and the 1982 Constrained Optimization and Lagrange Multiplier Methods book.

Instructors that adopt any one book as a main textbook for a course may contact the publishing company Athena Scientific for a free adoption copy.

This book provides an up-to-date, comprehensive, and rigorous account of nonlinear programming at the first year graduate student level. It covers descent algorithms for unconstrained and constrained optimization, Lagrange multiplier theory, interior point and augmented Lagrangian methods for linear and nonlinear programs, duality theory, and major aspects of large-scale optimization.

The third edition of the book is a thoroughly rewritten version of the 1999 second edition. New material was included, some of the old material was discarded, and a large portion of the remainder was reorganized or revised. The new material covers a variety of topics, such as first order methods, proximal algorithms, alternating direction methods of multipliers, and conic programming; also large-scale optimization topics of much current interest, such as incremental methods, and distributed asynchronous computation, and their applications in machine learning, signal processing, neural network training, and big data applications.

Another objective of the third edition to integrate its contents with those of the other books, together with internet-accessible material, so that they complement each other and form a unified whole. To this end, bridges to the other books have been provided, with extensive references to generalizations, discussions, and elaborations of the analysis.

Click here to visit the book’s web site at Athena Scientific for exercise solutions, slides, and other instructional material, or to order the book directly from the publisher for faster service.

Click here to download the Table of Contents and the Preface.

Click here to order the book from amazon.com.

This book provides a comprehensive and accessible presentation of algorithms for solving convex optimization problems. It relies on rigorous mathematical analysis, but also aims at an intuitive exposition that makes use of visualization where possible. This is facilitated by the extensive use of analytical and algorithmic concepts of duality, which by nature lend themselves to geometrical interpretation. The book places particular emphasis on modern developments, and their widespread applications in fields such as large-scale resource allocation problems, signal processing, and machine learning.

The book is aimed at students, researchers, and practitioners, roughly at the first year graduate level. It is similar in style to the author’s 2009 Convex Optimization Theory book, but can be read independently. The latter book focuses on convexity theory and optimization duality, while the 2015 Convex Optimization Algorithms book focuses on algorithmic issues. The two books share notation, and together cover the entire finite-dimensional convex optimization methodology. To facilitate readability, the statements of definitions and results of the “theory book” are reproduced without proofs in Appendix B.

Click here to visit the book’s web site at Athena Scientific for exercises with solutions, slides, and other instructional material, or to order the book directly from the publisher for faster service.

Click here to download Preface and Table of Contents, and Chapter 1

Click here to order the book from amazon.com.

This book aims at an insightful, concise, and rigorous treatment of the basic theory of convex sets and functions in finite dimensions, and the analytical/geometrical foundations of convex optimization and duality theory. It provides much of the mathematical background needed for the in-depth reading of the Nonlinear Programming and Convex Optimization Algorithms books.

Convexity theory is first developed in a simple accessible manner, using easily visualized proofs. Then the focus shifts to a transparent geometrical line of analysis to develop the fundamental duality between descriptions of convex functions in terms of points, and in terms of hyperplanes. Finally, convexity theory and abstract duality are applied to problems of constrained optimization, Fenchel and conic duality, and game theory to develop the sharpest possible duality results within a highly visual geometric framework.

The book is aimed at students, researchers, and practitioners, roughly at the first year graduate level. It is similar in style to the author’s 2015 Convex Optimization Algorithms book, but can be read independently. The latter book focuses on algorithmic issues, while the 2009 Convex Optimization Theory book focuses on convexity theory and optimization duality. The two books share notation, and together cover the entire finite-dimensional convex optimization methodology.

Click here to download the Entire Book.

Click here for the Statements of all the Mathematical Results.

Click here to order the book from amazon.com.