P.-A. Absil, R. Mahony, R. Sepulchre Many problems in the sciences and engineering can be rephrased as optimization problems on matrix search spaces endowed with a so-called manifold structure. This book shows how to exploit the special structure of such problems to develop efficient numerical algorithms. It places careful emphasis on both the numerical formulation of the algorithm and its differential geometric abstraction—illustrating how good algorithms draw equally from the insights of differential geometry, optimization, and numerical analysis. Two more theoretical chapters provide readers with the background in differential geometry necessary to algorithmic development. In the other chapters, several well-known optimization methods such as steepest descent and conjugate gradients are generalized to abstract manifolds. The book provides a generic development of each of these methods, building upon the material of the geometric chapters. It then guides readers through the calculations that turn these geometrically formulated methods into concrete numerical algorithms. The state-of-the-art algorithms given as examples are competitive with the best existing algorithms for a selection of eigenspace problems in numerical linear algebra.

Optimization Algorithms on Matrix Manifolds offers techniques with broad applications in linear algebra, signal processing, data mining, computer vision, and statistical analysis. It can serve as a graduate-level textbook and will be of interest to applied mathematicians, engineers, and computer scientists.

Andreas Antoniou Practical Optimization: Algorithms and Engineering Applications provides a hands-on treatment of the subject of optimization. A comprehensive set of problems and exercises makes the book suitable for use in one or two semesters of a first-year graduate course or an advanced undergraduate course. Each half of the book contains a full semester’s worth of complimentary yet stand-alone material. The practical orientation of the topics chosen and a wealth of useful examples also make the book suitable as a reference work for practitioners in the field.

Advancements in the efficiency of digital computers and the evolution of reliable software for numerical computation during the past three decades have led to a rapid growth in the theory, methods, and algorithms of numerical optimization. This body of knowledge has motivated widespread applications of optimization methods in many disciplines, e.g., engineering, business, and science, and has subsequently led to problem solutions that were considered intractable not too long ago.

Key Features:

extensively class-tested

provides a complete teaching package with MATLAB exercises and online solutions to end-of-chapter problems

includes recent methods of emerging interest such as semidefinite programming and second-order cone programming

presents a unified treatment of unconstrained and constrained optimization

uses a practical treatment of optimization accessible to broad audience, from college students to scientists and industry professionals

provides a thorough appendix with background theory so non-experts can understand how applications are solved from point of view of optimization

Michael Berry, Tony F. Chan, James Demmel, June Donato, Jack Dongarra, Victor Eijkhout, Charles Romine, Henk van der Vorst In this book, which focuses on the use of iterative methods for solving large sparse systems of linear equations, templates are introduced to meet the needs of both the traditional user and the high-performance specialist. Templates, a description of a general algorithm rather than the executable object or source code more commonly found in a conventional software library, offer whatever degree of customization the user may desire.

Templates have three distinct advantages: they are general and reusable, they are not language specific, and they exploit the expertise of both the numerical analyst, who creates a template reflecting in-depth knowledge of a specific numerical technique, and the computational scientist, who then provides "value-added" capability to the general template description, customizing it for specific needs. For each template that is presented, the authors provide a mathematical description of the flow of the algorithm, discussion of convergence and stopping criteria to use in the iteration, suggestions for applying a method to special matrix types, advice for tuning the template, tips on parallel implementations, and hints as to when and why a method is useful.

Ake Bjõrck The method of least squares was discovered by Gauss in 1795. It has since become the principal tool for reducing the influence of errors when fitting models to given observations. Today, applications of least squares arise in a great number of scientific areas, such as statistics, geodetics, signal processing, and control.

In the last 20 years there has been a great increase in the capacity for automatic data capturing and computing. Least squares problems of large size are now routinely solved. Tremendous progress has been made in numerical methods for least squares problems, in particular for generalized and modified least squares problems and direct and iterative methods for sparse problems. Until now there has not been a monograph that covers the full spectrum of relevant problems and methods in least squares.

This volume gives an in-depth treatment of topics such as methods for sparse least squares problems, iterative methods, modified least squares, weighted problems, and constrained and regularized problems. The more than 800 references provide a comprehensive survey of the available literature on the subject.

Stephen Boyd, Lieven Vandenberghe Convex optimization problems arise frequently in many different fields. A comprehensive introduction to the subject, this book shows in detail how such problems can be solved numerically with great efficiency. The focus is on recognizing convex optimization problems and then finding the most appropriate technique for solving them. The text contains many worked examples and homework exercises and will appeal to students, researchers and practitioners in fields such as engineering, computer science, mathematics, statistics, finance, and economics.

Ke Chen Preconditioning techniques have emerged as an essential part of successful and efficient iterative solutions of matrices. Ke Chen's book offers a comprehensive introduction to these methods. A vast range of explicit and implicit sparse preconditioners are covered, including the conjugate gradient, multi-level and fast multi-pole methods, matrix and operator splitting, fast Fourier and wavelet transforms, incomplete LU and domain decomposition, Schur complements and approximate inverses. In addition, aspects of parallel realization using the MPI are discussed. Very much a users-guide, the book provides insight to the use of these techniques in areas such as acoustic wave scattering, image restoration and bifurcation problems in electrical power stations. Supporting MATLAB files are available from the Web to support and develop readers' understanding, and provide stimulus for further study. Pitched at graduate level, the book is intended to serve as a useful guide and reference for students, computational practitioners, engineers and researchers alike.

Thomas Cormen, Charles Leiserson, Ronald Rivest, Clifford Stein The updated new edition of the classic Introduction to Algorithms is intended primarily for use in undergraduate or graduate courses in algorithms or data structures. Like the first edition, this text can also be used for self-study by technical professionals since it discusses engineering issues in algorithm design as well as the mathematical aspects.

In its new edition, Introduction to Algorithms continues to provide a comprehensive introduction to the modern study of algorithms. The revision has been updated to reflect changes in the years since the book's original publication. New chapters on the role of algorithms in computing and on probabilistic analysis and randomized algorithms have been included. Sections throughout the book have been rewritten for increased clarity, and material has been added wherever a fuller explanation has seemed useful or new information warrants expanded coverage.

As in the classic first edition, this new edition of Introduction to Algorithms presents a rich variety of algorithms and covers them in considerable depth while making their design and analysis accessible to all levels of readers. Further, the algorithms are presented in pseudocode to make the book easily accessible to students from all programming language backgrounds.

Each chapter presents an algorithm, a design technique, an application area, or a related topic. The chapters are not dependent on one another, so the instructor can organize his or her use of the book in the way that best suits the course's needs. Additionally, the new edition offers a 25% increase over the first edition in the number of problems, giving the book 155 problems and over 900 exercises that reinforce the concepts the students are learning.

Lawrence C. Evans This text gives a comprehensive survey of modern techniques in the theoretical study of partial differential equations (PDEs) with particular emphasis on nonlinear equations. The exposition is divided into three parts: 1) representation formulas for solutions, 2) theory for linear partial differential equations, and 3) theory for nonlinear partial differential equations.

Included are complete treatments of the method of characteristics; energy methods within Sobolev spaces; regularity for second-order elliptic, parabolic, and hyperbolic equations; maximum principles; the multidimensional calculus of variations; viscosity solutions of Hamilton-Jacobi equations; shock waves and entropy criteria for conservation laws; and much more.

The author summarizes the relevant mathematics required to understand current research in PDEs, especially nonlinear PDEs. While he has reworked and simplified much of the classical theory (particularly the method of characteristics), he emphasizes the modern interplay between functional analytic insights and calculus-type estimates within the context of Sobolev spaces. Treatment of all topics is complete and self-contained. The book's wide scope and clear exposition make it a suitable text for a graduate course in PDEs.

R. Fletcher Fully describes optimization methods that are currently most valuable in solving real-life problems. Since optimization has applications in almost every branch of science and technology, the text emphasizes their practical aspects in conjunction with the heuristics useful in making them perform more reliably and efficiently. To this end, it presents comparative numerical studies to give readers a feel for possibile applications and to illustrate the problems in assessing evidence. Also provides theoretical background which provides insights into how methods are derived. This edition offers revised coverage of basic theory and standard techniques, with updated discussions of line search methods, Newton and quasi-Newton methods, and conjugate direction methods, as well as a comprehensive treatment of restricted step or trust region methods not commonly found in the literature. Also includes recent developments in hybrid methods for nonlinear least squares; an extended discussion of linear programming, with new methods for stable updating of LU factors; and a completely new section on network programming. Chapters include computer subroutines, worked examples, and study questions.

Gene H. Golub, Charles F. Van Loan Revised and updated, the third edition of Golub and Van Loan's classic text in computer science provides essential information about the mathematical background and algorithmic skills required for the production of numerical software. This new edition includes thoroughly revised chapters on matrix multiplication problems and parallel matrix computations, expanded treatment of CS decomposition, an updated overview of floating point arithmetic, a more accurate rendition of the modified Gram-Schmidt process, and new material devoted to GMRES, QMR, and other methods designed to handle the sparse unsymmetric linear system problem.

Michael T. Heath Heath 2/e, presents a broad overview of numerical methods for solving all the major problems in scientific computing, including linear and nonlinear equations, least squares, eigenvalues, optimization, interpolation, integration, ordinary and partial differential equations, fast Fourier transforms, and random number generators. The treatment is comprehensive yet concise, software-oriented yet compatible with a variety of software packages and programming languages. The book features more than 160 examples, 500 review questions, 240 exercises, and 200 computer problems.

Changes for the second edition include: expanded motivational discussions and examples; formal statements of all major algorithms; expanded discussions of existence, uniqueness, and conditioning for each type of problem so that students can recognize "good" and "bad" problem formulations and understand the corresponding quality of results produced; and expanded coverage of several topics, particularly eigenvalues and constrained optimization.

The book contains a wealth of material and can be used in a variety of one- or two-term courses in computer science, mathematics, or engineering. Its comprehensiveness and modern perspective, as well as the software pointers provided, also make it a highly useful reference for practicing professionals who need to solve computational problems.

C. T. Kelley Linear and nonlinear systems of equations are the basis for many, if not most, of the models of phenomena in science and engineering, and their efficient numerical solution is critical to progress in these areas. This is the first book to be published on nonlinear equations since the mid-1980s. Although it stresses recent developments in this area, such as Newton-Krylov methods, considerable material on linear equations has been incorporated. This book focuses on a small number of methods and treats them in depth. The author provides a complete analysis of the conjugate gradient and generalized minimum residual iterations as well as recent advances including Newton-Krylov methods, incorporation of inexactness and noise into the analysis, new proofs and implementations of Broyden's method, and globalization of inexact Newton methods.

Donald E. Knuth The bible of programming theory and practice is being updated for the first time in more than 20 years. The book is concerned with information structures—the representation of information within a computer, the structural interrelations between data elements and how to work with them efficiently, and applications to simulation, numerical methods and software design.

Donald E. Knuth The second book in a three volume set, this book provides a complete introduction to this topic. It presents a readable and coherent summary of the major paradigms and basic theory of semi numerical algorithms, providing a comprehensive interface between computer programming and numerical analysis.

Donald E. Knuth The first revision of this third volume is the most comprehensive survey of classical computer techniques for sorting and searching. It extends the treatment of data structures in Volume 1 to consider both large and small databases and internal and external memories. The book contains a selection of carefully checked computer methods, with a quantitative analysis of their efficiency. Outstanding features of the second edition include a revised section on optimum sorting and new discussions of the theory of permutations and of universal hashing.

Jorge Nocedal, Stephen Wright Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization. It responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems.

For this new edition the book has been thoroughly updated throughout. There are new chapters on nonlinear interior methods and derivative-free methods for optimization, both of which are used widely in practice and the focus of much current research. Because of the emphasis on practical methods, as well as the extensive illustrations and exercises, the book is accessible to a wide audience. It can be used as a graduate text in engineering, operations research, mathematics, computer science, and business. It also serves as a handbook for researchers and practitioners in the field. The authors have strived to produce a text that is pleasant to read, informative, and rigorous - one that reveals both the beautiful nature of the discipline and its practical side.

There is a selected solutions manual for instructors for the new edition.

William H. Press, Brian P. Flannery, Saul A. Teukolsky, William T. Vetterling The product of a unique collaboration among four leading scientists in academic research and industry, Numerical Recipes is a complete text and reference book on scientific computing. In a self-contained manner it proceeds from mathematical and theoretical considerations to actual practical computer routines. With over 100 new routines bringing the total to well over 300, plus upgraded versions of the original routines, the new edition remains the most practical, comprehensive handbook of scientific computing available today.

Yousef Saad Since the first edition of this book was published in 1996, tremendous progress has been made in the scientific and engineering disciplines regarding the use of iterative methods for linear systems. The size and complexity of the new generation of linear and nonlinear systems arising in typical applications has grown. Solving the three-dimensional models of these problems using direct solvers is no longer effective. At the same time, parallel computing has penetrated these application areas as it became less expensive and standardized. Iterative methods are easier than direct solvers to implement on parallel computers but require approaches and solution algorithms that are different from classical methods.

Iterative Methods for Sparse Linear Systems, Second Edition gives an in-depth, up-to-date view of practical algorithms for solving large-scale linear systems of equations. These equations can number in the millions and are sparse in the sense that each involves only a small number of unknowns. The methods described are iterative, i.e., they provide sequences of approximations that will converge to the solution.

This new edition includes a wide range of the best methods available today. The author has added a new chapter on multigrid techniques and has updated material throughout the text, particularly the chapters on sparse matrices, Krylov subspace methods, preconditioning techniques, and parallel preconditioners. Material on older topics has been removed or shortened, numerous exercises have been added, and many typographical errors have been corrected. The updated and expanded bibliography now includes more recent works emphasizing new and important research topics in this field.

Audience
This book can be used to teach graduate-level courses on iterative methods for linear systems. Engineers and mathematicians will find its contents easily accessible, and practitioners and educators will value it as a helpful resource. The preface includes syllabi that can be used for either a semester- or quarter-length course in both mathematics and computer science.

Contents
Preface to the Second Edition; Preface to the First Edition; Chapter 1: Background in Linear Algebra; Chapter 2: Discretization of Partial Differential Equations; Chapter 3: Sparse Matrices; Chapter 4: Basic Iterative Methods; Chapter 5: Projection Methods; Chapter 6: Krylov Subspace Methods, Part I; Chapter 7: Krylov Subspace Methods, Part II; Chapter 8: Methods Related to the Normal Equations; Chapter 9: Preconditioned Iterations; Chapter 10: Preconditioning Techniques; Chapter 11: Parallel Implementations; Chapter 12: Parallel Preconditioners; Chapter 13: Multigrid Methods; Chapter 14: Domain Decomposition Methods; Bibliography; Index.