Preface

course in linear algebra. We assume the students have had at least one

course in calculus.

Philosophy of the Text

1. Students had difficulty reading linear algebra textbooks. Frequently, they were too terse, especially where proofs of important results were concerned.
2. Students invariably ran into trouble as the largely computational first half of the course gave way to the more theoretical second half.  Students were suddenly asked to work on a much higher level of abstraction and had difficulty with such nontrivial concepts as span, linear independence, one-to-one, onto, etc.
3. Most textbooks contained few, if any, guidelines about reading and writing simple mathematical proofs. However, many instructors have traditionally used a first course in linear algebra as a vehicle for familiarizing students with proof techniques, or building upon the introductory material on proofs found in an earlier course in discrete mathematics.

Clarity: Above all, we have striven for clarity and used straightforward language throughout the book, occasionally sacrificing brevity for clear and convincing explanation. We strongly encourage students to take advantage of the book's presentation by reading it deeply and

thoroughly.

Smooth transition to abstraction: To make the transition to the second, more theoretical, half of the course easier, we have students working on proofs as quickly as possible. After a discussion of the basic properties of vectors, there is a special section (Section 1.3) on general proof techniques, with concrete examples using the material on vectors from

Sections 1.1 and 1.2.  The early placement of Section 1.3 helps to give students a strong foundation and build their confidence in the reading and writing of proofs.

Revisiting Topics: We frequently introduce difficult concepts by revisiting them frequently throughout the text. Students are initially exposed to abstract concepts through concrete examples, and then again in increasingly abstract settings as they progress through the book. Here are several examples:

• Students are introduced to the concept of linear combinations beginning in Section 1.1, long before linear combinations are defined for real vector spaces in Chapter 4.
• The row space of a matrix is introduced in Chapter 2, thereby preparing students for the more general concepts of subspace and span in Sections 4.2 and 4.3.
• The technique behind the first two methods in Section 4.6 for computing bases are introduced earlier in Sections 4.3 and 4.4 in the Simplified Span Method and the Independence Test Method, respectively. In this way, students will become comfortable with these methods in the context of span and linear independence before employing them to find appropriate bases for vector spaces.
• The concepts of eigenvalues and eigenvectors are initially studied in Chapter 3 in the context of matrices. Further properties of eigenvectors are included throughout Chapters 4 and 5 as the underlying vector space concepts are introduced. A full treatment of the subject for linear transformations appears at the end of Chapter 5. The more advanced topics of orthogonal and unitary diagonalization are covered in Chapters 6 and 7.

Major Changes for the Third Edition

True/False exercises: True/False exercises have been added at the end of each section in Chapters 1 through 10, as well as in Appendices B and C. These help students to check their understanding of the fundamental concepts introduced in each section. Full explanations of the answers to the true/false exercises appear in the Student Solutions Manual.

Gaussian elimination: Gaussian elimination is now the first method introduced for solving systems of linear equations (Section 2.1). This is more intuitive and computationally more efficient than row reducing completely to reduced row echelon form. However, we also discuss the Gauss-Jordan method for its theoretical applications (Section 2.2).

Equivalent linear systems, rank, and row space: The theoretical concepts of equivalent systems, rank, and row space have been streamlined and placed together in Section 2.3.

Linear Independence: A more straightforward definition and exposition of the concepts of linear independence/dependence is presented at the beginning of Section 4.4.

Complex Linear Algebra: For those who want to include a treatment of complex matrices, eigenvectors, and vector spaces, the corresponding material on these topics (formerly in Section 7.1) has been rearranged and segmented into four shorter sections, Sections 7.1 through 7.4. This allows complex linear algebra topics to be covered more easily in parallel with their corresponding real linear algebra topics.  For example, Section 7.1 on complex n-vectors and matrices can be covered directly after completing Chapter 1, thus allowing the integration of these complex structures throughout the course. Formal prerequisites for each section of Chapter 7 are given in a chart following this Preface.

New Applications: Two new application sections are included in this edition: Section 8.6, "Change of Variables and theJacobian," and Section 8.11, "Max-Min Problems in R³ and the Hessian Matrix." Also, the application "Computer Graphics" (Section 8.8) has been extensively revised. The material on "Function Spaces" and "Quadratic Forms" has been moved to Sections 10.2 and 10.3, respectively.

Elementary Matrices: By popular demand, the material on elementary matrices from the first edition has been restored in Section 10.1.

Revised Computational Methods: As mentioned earlier, two computational methods useful for finding bases which were formerly in Section 4.6 have been moved earlier in the text to Sections 4.3 and 4.4, respectively, and given new names (Simplified Span Method and Independence Test Method). Also, the methods for diagonalizing and orthogonallydiagonalizing a linear operator in Sections 5.5 and 6.3 have been streamlined to make their connection to the Diagonalization Method of Section 3.4 more apparent.

Computational Aid: The material in Appendix D ("Computers and Calculators") has been updated to reflect the current state of various software packages.

Help on the Web: Our web site, http://www.sju.edu/~dhecker/linalg.html, contains appropriate updates on the textbook as well as a way to communicate with the authors. It also contains information on earlier versions of some of the software packages mentioned in Appendix D, in case you are working with an older version.  We also expect the web site to have updates for future versions of the software as they are released.

Features

The text also contains an unusually large number of exercises. There are more than 830 numbered exercises, and many of these have multiple parts, for a total of more than 2135 questions. Some are purely computational. Many others ask the students to write short proofs, or to explore further consequences of the material. The exercises within each section are

generally ordered by increasing difficulty, beginning with basic computational problems and moving on to more theoretical problems and proofs. Answers are provided at the end of the book for approximately half the computational exercises; these problems are marked with a star. Full solutions to the star exercises appear in the new Student Solutions Manual.

Careful coverage of vector space topics: Many students have difficulties when abstract vector space topics are introduced. These concepts represent a sharp transition from concrete problem solving to theoretical conceptualizing. The material in Sections 4.1 through 5.4 (vector spaces and subspaces, span, linear independence, basis and dimension, coordinatization, linear transformations and their matrices, kernel and range, and isomorphism) constitutes the

"heart" of this linear algebra text, and we have taken great care to help students focus attention on these important concepts. As previously mentioned, we revisit several difficult topics in new settings in order to gently introduce them to students.

Early introduction of eigenvalues/eigenvectors:Eigenvalues and eigenvectors are introduced early in the text (Section 3.4), just before the introduction of abstract vector spaces. Students traditionally findeigenvalues to be a difficult topic. By introducing eigenvalues and eigenvectors early on, we have the opportunity to present these concepts initially on an elementary level, and then reinforce them throughout the course with appropriate examples in subsequent sections. This approach enables students to gain confidence with eigenvalues and eigenvectors before encountering a more thorough, detailed treatment in Section 5.5.

Emphasis on Proofs: We have written the proofs of the theorems in the text in a careful, concise manner to give students a model for writing their own proofs. We have also included a special section early in the text, Section 1.3, that reviews the most important proof techniques in the context of linear algebra. We have left the proofs of some elementary theorems to the student. However, for almost every nontrivial theorem in Chapters 1 through 6, we have either included a proof, or given detailed hints which should be sufficient to enable students to provide a proof on their own.  (Note: The only exception is Theorem 2.4 (uniqueness of reduced row echelon form).) Most of the proofs that are left as exercises can be found in the Student Solutions Manual. The exercises corresponding to these proofs are marked with a right-pointing triangle. We have avoided "clever" or "sneaky" proofs, in which the last line

suddenly produces "a rabbit out of a hat," because such proofs invariably frustrate students.

They are given no insight into the strategy of the proof or how the deductive process was used. In fact, such proofs tend to reinforce the students' mistaken belief that they will never become competent in the art of writing proofs.

In this text, proofs longer than one paragraph are often written in a "top-down" manner, a concept borrowed from structured programming. A complex theorem is broken down into a secondary series of results, which together are sufficient to prove the original theorem. In this way, the student has a clear outline of the logical argument and can more easily reproduce the proof if called on to do so.

Applications: Linear algebra is a subject with a multitude of practical applications, and we have included many standard ones so that instructors can choose their favorites. There is a chart following the Preface which lists the most important linear algebra applications in this

text.  Chapter 8 is devoted entirely to applications of linear algebra, but there are also several shorter applications in Chapters 1 to 6. Another chart following the Preface lists the prerequisites required for each of the application sections in Chapter 8. Instructors may choose to assign some of these applications as reading assignments outside of class.

Help with technology: Almost all students now have access to appropriate computer software or graphing calculators to reduce the amount of computational drudgery involved in a typical elementary linear algebra course. We believe that once a student has mastered the concepts of matrix multiplication and row reduction, there is no need to waste precious time in or out of class with rote computations. This frees both the instructor and the student to concentrate on the theoretical ideas of the subject without becoming unduly bogged down with calculations.

While the exposition of this text does not depend on the use of any particular technology, Appendix D provides a short introduction to several prominent computer packages: Maple 8®, 

Derive 5®, Mathematica 4.2®, MATLAB 6.5® and several graphing calculators from Texas Instruments: the TI-86®, TI-89®, TI-92®, TI-92 Plus®, and Voyage 200®. While this appendix does not give an in-depth treatment of these packages and calculators, it does illustrate how to perform several fundamental types of vector and matrix computations in each environment. Our

web site,

http://www.sju.edu/~dhecker/linalg.html,

will contain updates for new versions of these software packages, as well as details on

other calculators and older versions of the software packages.

Formal computational methods: There are 17 computational methods presented in step-by-step form, each illustrating a fundamental process in linear algebra. These have been placed in boxes for easier reference by the students.  Several additional numerical methods are presented, especially in Sections 9.1 and 9.2. A chart listing all of these methods appears after this Preface.

Subsections, Summary Charts, and Symbol Table: Almost every section of the text is divided into several manageable subsections to enhance clarity and readability. These subsections are individually titled to highlight the main themes of the section.  Condensed versions of some useful charts are printed on the inside front and back covers for easy reference.

Finally, for convenience, there is a comprehensive Symbol Table listing all of the major symbols employed in this text related to linear algebra together with their meanings.

Supplements: We have written a Student Solutions Manual that contains the full worked-out solution for each exercise marked with a star in the textbook (whose answer also appears at the back of the book). This manual also contains proofs for most of the theorems whose proofs were left as exercises. The exercises corresponding to these theorems are marked with a right-pointing triangle. There is also an Instructor's Manual that contains the answers to all computational exercises, and complete solutions to the theoretical and proof exercises. This manual also includes three versions of a sample test for each of Chapters 1 through 7. These can be used without change or as a test question bank for making tests. Answer keys for the sample tests are also included.

Additional information, as well as appropriate updates that become available

after the book is printed will be posted on our web site,

Chapter-by-Chapter Summary

elementary linear algebra courses:

• Chapter 1 (Vectors and Matrices) introduces vectors and matrices and their fundamental operations and properties. This chapter includes a special section (Section 1.3) on proof techniques, illustrating some of the most important methods of proof and pointing out some of the pitfalls.
• Chapter 2 (Systems of Linear Equations) begins with the solution of systems of linear equations using the Gaussian elimination and Gauss-Jordan row reduction methods.\ This is followed by a discussion of the uniqueness of reduced row echelon form, equivalent systems, rank, row space, and inverses of matrices.
• Chapter 3 (Determinants and Eigenvalues)  introduces the determinant (using a cofactor approach) and shows its usefulness in working with systems of linear equations.\ The chapter ends with an introductory treatment of eigenvalues and eigenvectors for matrices.
• Chapter 4 (Finite Dimensional Vector Spaces)  begins a treatment of the abstract concepts of vector spaces and subspaces.  Span, linear independence, basis and dimension, and coordinatization are covered. Several useful methods for finding bases are illustrated.
• Chapter 5 (Linear Transformations)  introduces linear transformations. The matrix, kernel, and range of a linear transformation are covered. One-to-one and onto linear transformations are treated in depth, and an isomorphism of any n-dimensional real vector space with Rⁿ is established.  The chapter ends with a more formal treatment of the concepts of eigenvalues and diagonalization in the context of linear transformations.
• Chapter 6 (Orthogonality)  begins with a study of orthogonal and orthonormal bases, and the Gram-Schmidt Process.\ Orthogonal matrices, orthogonal complements, and orthogonal projections are treated. The chapter ends with orthogonal diagonalization, a fitting culmination of the material in the first six chapters.

these chapters can be covered at any time after their stated prerequisites

have been met:

• Chapter 7 (Complex Vector Spaces and General Inner Products) generalizes the material of earlier chapters to complex vector spaces and general inner product spaces. The various sections of Chapter 7 are written so that they may be covered in tandem with the corresponding sections on real vectors and matrices. For this reason, there is no need to wait until the end of the course to discuss these "complex"  topics.
• Chapter 8 (Additional\ Applications)  is devoted to applications of linear algebra, including elementary graph theory, Ohm's Law, least-squares polynomials, Markov chains, Hill substitution, change of variables and the Jacobian matrix in two and three dimensions, rotation of axes, computer graphics, differential equations, least-squares solutions for inconsistent systems, and max-min problems in R³ involving the Hessian matrix.
• Chapter 9 (Numerical Methods)  discusses important considerations when using a computer or calculator to perform computations in linear algebra.\ Numerical methods such as partial pivoting, the Jacobi and Gauss-Seidel iterative methods, LDU decomposition, and the Power Method for calculating dominant eigenvalues are covered.
• Chapter 10 (Further Horizons)  covers three supplemental topics: elementary matrices, function spaces, and quadratic forms.

• Appendix A (Miscellaneous Proofs)  contains proofs of five results that were omitted in the main part of the text because of length or complexity.
• Appendix B (Functions)  includes a review of basic function terminology and properties, as well as a treatment of one-to-one, onto, inverse, and composite functions.
• Appendix C (Complex Numbers)  contains a review of the basic properties of complex numbers.
• Appendix D (Computers and Calculators)  includes a brief introduction to the use of several software packages and graphing calculators in performing basic vector and matrix operations.

Guide for the Instructor 

• Section 1.3 (An Introduction to Proofs) can be covered, in whole, or in part, at any time after Section 1.2.
• Section 3.3 (Further Properties of the Determinant) containssome material that can be omitted without affecting most of the remaining development. The topics of general cofactor expansion, (classical) adjoint matrix, and Cramer's Rule are used very sparingly in the rest of the text.
• Section 6.1 (Orthogonal Bases and the Gram-Schmidt Process) can be covered any time after Chapter 4, as can much of the material in Section 6.2 (Orthogonal Complements).

Two suggested timetables for covering the material in this text are presented below --- one for a 3-credit course, and the other for a 4-credit course. While all the material of Chapters 1 through 6, and some of Chapter 7, would be covered in the 4-credit course, the 3-credit course could de-emphasize portions of Sections 1.3, 2.3, 3.3, 5.5, 6.2, and 6.3, and

would not include Chapter 7.
 
 
 

	3-Credit Course	4-Credit Course
Chapter 1	5 classes	6 classes
Chapter 2	4 classes	5 classes
Chapter 3	3 classes	6 classes
Chapter 4	12 classes	12 classes
Chapter 5	8 classes	9 classes
Chapter 6	2 classes	5 classes
Chapter 7		3 classes
Chapters 8/ 9/ 10 (selection)	2 classes	4 classes
Review	3 classes	3 classes
Tests	3 classes	3 classes
Total	42 classes	56 classes

Acknowledgments

production designer.

We also want to thank those who have supported our textbook at various stages. In particular, we thank Agnes Rash, chair of the Mathematics and Computer Science Department at Saint Joseph's University for her continual support of this project. We also thank Paul Klingsberg and Richard Cavaliere of Saint Joseph's University, both of whom gave us many suggestions for improvements to the second edition.We thank those students who have classroom-tested versions of the earlier editions of the manuscript. Their comments and suggestions have been extremely useful, and have guided us in shaping the text in many ways.

We acknowledge those reviewers who have supplied many worthwhile suggestions. For reviewing the first edition, we thank the following:

    C. S. Ballantine,OregonStateUniversity

Yuh-ching Chen, FordhamUniversity

    Susan Jane Colley, OberlinCollege

    Roland di Franco, University of the Pacific

    Colin Graham, Northwestern University\

    K. G. Jinadasa,IllinoisStateUniversity

    Ralph Kelsey, DenisonUniversity

MasoodOtarod, University of Scranton

    J. Bryan Sperry, PittsburgStateUniversity

    Robert Tyler, SusquehannaUniversity

For reviewing the second edition, we thank the following:

    Ruth Favro,Lawrence Technological University

    Howard Hamilton, CaliforniaStateUniversity

    Ray Heitmann,University of Texas,Austin

    Richard Hodel, DukeUniversity

    James Hurley, University of Connecticut

    Jack Lawlor,University of Vermont

    Peter Nylen,AuburnUniversity

    Ed Shea,CaliforniaStateUniversity, Sacramento

For reviewing the third edtion, we thank the following: 

JohnLawlor, University of Vermont

Susan Jane Colley, OberlinCollege

Joel Robbin, University of Wisconsin

Ian Morrison, FordhamUniversity

Ali Miri, University of Ottawa

VaniaMascioni, BallStateUniversity

SergeiBezrukov, University of WisconsinSuperior

Don Passman, University of Wisconsin

Last, but most important of all, we want to thank our wives, Ene and Lyn, for bearing extra hardships so that we could work on this text. Their love and support has been an inspiration. We also thank Ene, who conveniently works at Saint Joseph's University for ferrying various revisions and files of the manuscript between us.

Coming to Terms with Linear Algebra

One characteristic that we expect students to manifest is a greater linear independence in problem-solving. After much reflection on the kernel of ideas presented in this book, the range of new methods available to them should be graphically augmented in a multiplicity of ways. An associative feature of this transition is that all of the new techniques they learn should become a consistent and normalized part of their identity in the future. In addition, students will gain a singular new appreciation of their mathematical skills. Consequently, the resultant change in their self-image should be one of no minor magnitude.

One obvious implication is that the level of the students' success is an isomorphic reflection of the amount of homogeneous energy they expend on this complex material. That is, we can often trace the rank of their achievement to the depth of their resolve to be a scalar of new distances. Similarly, we make this symmetric claim: the students' positive, definite growth is clearly a function of their overall coordinatization of effort. Naturally, the matrix of thought behind this parallel assertion is that students should avoid the negative consequences of sparse learning. Instead, it is the inverse approach of systematic and iterative study that will ultimately lead them to less error, and not rotate them into useless

dead-ends and diagonal tangents of zero worth.

Of course some nontrivial length of time is necessary to transpose a student with an empty set of knowledge on this subject into higher echelons of understanding. But, our projection is that the unique dimensions of this text will be a determinant factor in enriching the span of students' lives, and translate them onto new orthogonal paths of wisdom.

                Stephen Andrilli

                David Hecker

                August, 2003 

Prerequisite Chart for Chapters 7 through 10

*In addition to the prerequisites listed, each section in Chapter 7 requires

the sections of Chapter 7 that precede it, although most of Section

7.5 can be covered without having covered Sections 7.1 through 7.4.

**Section 8.6 uses the fact that the detrminant of a matrix equals the determinant of its transpose from Section 3.3, but we believe that it is more appropriate to cover this section directly after Section 3.1 to provide a deeper geometric understanding of the determinant.

***The techniques presented for solving differential equations in Section

8.9 require only Section 3.4 as a prerequisite. However, terminology from

Chapters 4 and 5 is used throughout Section 8.9.

****The Power Method in Section 9.3 requires only material from Section 3.4

for its implementation. However, topics from Chapters 4 and 5 are needed for

the justification of the Power Method and are used throughout Section 9.3.

*****The material in Section 10.2 requires only a knowledge of Section 4.4

(Linear Independence). However, several exercises in Section 10.2 involve

material from Sections 4.5, 4.6, and 4.7.

Applications

presented throughout the text. (See the Prerequisite Chart for the

applications in Chapters 8 and 10.)

 
 

Application	Section
Resultant Velocity	Section 1.1
Newton's Second Law	Section 1.1
Work	Section 1.2
Shipping Cost and Profit	Section 1.5
Curve Fitting	Section 2.1
Balancing Chemical Equations	Section 2.2
Areas and Volumes	Section 3.1
Large Powers of a Matrix	Section 3.4
Orthogonal Projections and     Reflections in Rn	Section 6.2
Distance from a Point to a Line	Section 6.2
Graph Theory	Section 8.1
Ohm's Law	Section 8.2
Least-Squares Polynomials	Section 8.3
Markov Chains	Section 8.4
Hill Substitution (Coding Theory)	Section 8.5
Change of Variables and the  Jacobian	Section 8.6
Rotation of Axes	Section 8.7
Computer Graphics	Section 8.8
Differential Equations	Section 8.9
Least-Squares Solutions for     Inconsistent Systems	Section 8.10
Max-Min Problems in R3 and     the Hessian Matrix	Section 8.11
Quadratic Forms	Section 10.3

Formal Methods

throughout the text:

 

Section	Formal Method
Section 2.4	Inverse Method (finding the inverse of a matrix)
Section 3.4	Diagonalization Method (diagonalizaing a square matrix)
Section 4.3	Simplified Span Method (determining span using row reduction)
Section 4.4	Independence Test Method (determining linear independence     using row reduction)
Section 4.6	Inspection Method (finding a basis by inspection)
Section 4.6	Enlarging Method (enlarging a linearly independent set to     a basis)
Section 4.7	Coordinatization Method  (coordinatizing a vector with     respect to an ordered basis)
Section 4.7	Transistion Matrix Method (calculating a transition matrix using     row reduction)
Section 5.3	Kernel Method (finding a basis for the kernel of a linear     transformation)
Section 5.3	Range Method (finding a basis for the range of a linear     transformation)
Section 5.5	GeneralizaedDiagonalization Method (diagonalizing    a linear operator)
Section 6.1	Gram-Schmidt Process (creating an orthogonal set from a     linearly independent set)
Section 6.3	Orthogonal Diagonalization Method (orthogonally diagonalizing    a symmetric operator)
Section 7.2	Generalized Gram-Schmidt Process (creating an orthogonal     set in an inner product space)
Section 8.8	Similarity Method (in computer graphics, finding a matrix for a     transformation centered at a point other than the origin)
Section 9.3	Power Method (finding the dominant eigenvalue of a     square matrix)
Section  10.3	Quadratic Form Method (diagonalizing a quadratic form)

Numerical Methods

text:
 

Section	Numerical Method
Section 2.1	Gaussian Elimination
Section 2.1	Back Substitution
Section 2.2	Gauss-Jordan Row Reduction
Section 2.4	Solving a system using the inverse of the     coefficient matrix
Section 3.1	Basketweaving (to find the determinant of     a 3×3 matrix)
Section 3.2	Finding a determinant by row reduction
Section 3.3	Cofactor expansion (general)
Section 3.3	Cramer's Rule
Section 8.3	Linear Regression (line of best fit)
Section 8.10	Approximate solutions for inconsistent     systems (least-squares)
Section 9.1	Partial pivoting
Section 9.1	Jacobi method
Section 9.1	Gauss-Seidel method
Section 9.2	LDU decomposition
Section 9.3	Power Method for finding eigenvalues

 Return to Elementary Linear Algebra Page