Skip to main content

Section WILA What is Linear Algebra?

We begin our study of linear algebra with an introduction and a motivational example.

Subsection LA Linear and Algebra

The subject of linear algebra can be partially explained by the meaning of the two terms comprising the title. “Linear” is a term you will appreciate better at the end of this course, and indeed, attaining this appreciation could be taken as one of the primary goals of this course. However for now, you can understand it to mean anything that is “straight” or “flat.” For example in the \(xy\)-plane you might be accustomed to describing straight lines (is there any other kind?) as the set of solutions to an equation of the form \(y = mx + b\text{,}\) where the slope \(m\) and the \(y\)-intercept \(b\) are constants that together describe the line. If you have studied multivariate calculus, then you will have encountered planes. Living in three dimensions, with coordinates described by triples \((x,\,y,\,z)\text{,}\) they can be described as the set of solutions to equations of the form \(ax+by+cz=d\text{,}\) where \(a,\,b,\,c,\,d\) are constants that together determine the plane. While we might describe planes as “flat,” lines in three dimensions might be described as “straight.” From a multivariate calculus course you will recall that lines are sets of points described by equations such as \(x=3t-4\text{,}\) \(y=-7t+2\text{,}\) \(z=9t\text{,}\) where \(t\) is a parameter that can take on any value.

Another view of this notion of “flatness” is to recognize that the sets of points just described are solutions to equations of a relatively simple form. These equations involve addition and multiplication only. We will have a need for subtraction, and occasionally we will divide, but mostly you can describe “linear” equations as involving only addition and multiplication. Here are some examples of typical equations we will see in the next few sections.

\begin{align*} 2x+3y-4z&=13 & 4x_1+5x_2-x_3+x_4+x_5&=0 & 9a-2b+7c+2d&=-7 \end{align*}

What we will not see are equations such as

\begin{align*} xy + 5yz&=13 & x_1 + x_2^3/x_4 - x_3x_4x_5^2&=0 & \tan(ab)+\log(c-d)&=-7\text{.} \end{align*}

The exception will be that we will on occasion need to take a square root.

You have probably heard the word “algebra” frequently in your mathematical preparation for this course. Most likely, you have spent a good ten to fifteen years learning the algebra of the real numbers, along with some introduction to the very similar algebra of complex numbers (see Section CNO). However, there are many new algebras to learn and use, and likely linear algebra will be your second algebra. Like learning a second language, the necessary adjustments can be challenging at times, but the rewards are many. And it will make learning your third and fourth algebras even easier. Perhaps you have heard of “groups” and “rings” (or maybe you have studied them already), which are excellent examples of other algebras with very interesting properties and applications. In any event, prepare yourself to learn a new algebra and realize that some of the old rules you used for the real numbers may no longer apply to this new algebra you will be learning!

The brief discussion above about lines and planes suggests that linear algebra has an inherently geometric nature, and this is true. Examples in two and three dimensions can be used to provide valuable insight into important concepts of this course. However, much of the power of linear algebra will be the ability to work with “flat” or “straight” objects in higher dimensions, without concerning ourselves with visualizing the situation. While much of our intuition will come from examples in two and three dimensions, we will maintain an algebraic approach to the subject, with the geometry being secondary. Others may wish to switch this emphasis around, and that can lead to a very fruitful and beneficial course, but here and now we are laying our bias bare.

Subsection AA An Application

We conclude this section with a rather involved example that will highlight some of the power and techniques of linear algebra. Work through all of the details with pencil and paper, until you believe all the assertions made. However, in this introductory example, do not concern yourself with how some of the results are obtained or how you might be expected to solve a similar problem. We will come back to this example later and expose some of the techniques used and properties exploited. For now, use your background in mathematics to convince yourself that everything said here really is correct.

Suppose you are the production manager at a food-packaging plant and one of your product lines is trail mix, a healthy snack popular with hikers and backpackers, containing raisins, peanuts and hard-shelled chocolate pieces. By adjusting the mix of these three ingredients, you are able to sell three varieties of this item. The fancy version is sold in half-kilogram packages at outdoor supply stores and has more chocolate and fewer raisins, thus commanding a higher price. The standard version is sold in one kilogram packages in grocery stores and gas station mini-markets. Since the standard version has roughly equal amounts of each ingredient, it is not as expensive as the fancy version. Finally, a bulk version is sold in bins at grocery stores for consumers to load into plastic bags in amounts of their choosing. To appeal to the shoppers that like bulk items for their economy and healthfulness, this mix has many more raisins (at the expense of chocolate) and therefore sells for less.

Your production facilities have limited storage space and early each morning you are able to receive and store 380 kilograms of raisins, 500 kilograms of peanuts and 620 kilograms of chocolate pieces. As production manager, one of your most important duties is to decide how much of each version of trail mix to make every day. Clearly, you can have up to 1500 kilograms of raw ingredients available each day, so to be the most productive you will likely produce 1500 kilograms of trail mix each day. Also, you would prefer not to have any ingredients leftover each day, so that your final product is as fresh as possible and so that you can receive the maximum delivery the next morning. But how should these ingredients be allocated to the mixing of the bulk, standard and fancy versions? First, we need a little more information about the mixes. Workers mix the ingredients in 15 kilogram batches, and each row of the table below gives a recipe for a 15 kilogram batch. There is some additional information on the costs of the ingredients and the price the manufacturer can charge for the different versions of the trail mix.

Raisins Peanuts Chocolate Cost Sale Price
kg/batch kg/batch kg/batch $/kg $/kg
Bulk 7 6 2 3.69 4.99
Standard 6 4 5 3.86 5.50
Fancy 2 5 8 4.45 6.50
Storage (kg) 380 500 620
Cost ($/kg) 2.55 4.65 4.80

As production manager, it is important to realize that you only have three decisions to make — the amount of bulk mix to make, the amount of standard mix to make and the amount of fancy mix to make. Everything else is beyond your control or is handled by another department within the company. Principally, you are also limited by the amount of raw ingredients you can store each day. Let us denote the amount of each mix to produce each day, measured in kilograms, by the variable quantities \(b\text{,}\) \(s\) and \(f\text{.}\) Your production schedule can be described as values of \(b\text{,}\) \(s\) and \(f\) that do several things. First, we cannot make negative quantities of each mix, so

\begin{align*} b&\geq 0 & s&\geq 0 & f&\geq 0\text{.} \end{align*}

Second, if we want to consume all of our ingredients each day, the storage capacities lead to three (linear) equations, one for each ingredient.

\begin{align*} \frac{7}{15}b+\frac{6}{15}s+\frac{2}{15}f&=380&&\text{(raisins)}\\ \frac{6}{15}b+\frac{4}{15}s+\frac{5}{15}f&=500&&\text{(peanuts)}\\ \frac{2}{15}b+\frac{5}{15}s+\frac{8}{15}f&=620&&\text{(chocolate)} \end{align*}

It happens that this system of three equations has just one solution. In other words, as production manager, your job is easy, since there is but one way to use up all of your raw ingredients making trail mix. This single solution is

\begin{align*} b&=300\text{ kg} & s&=300\text{ kg} & f&=900\text{ kg}\text{.} \end{align*}

We do not yet have the tools to explain why this solution is the only one, but it should be simple for you to verify that this is indeed a solution. (Go ahead, we will wait.) Determining solutions such as this, and establishing that they are unique, will be the main motivation for our initial study of linear algebra.

So we have solved the problem of making sure that we make the best use of our limited storage space, and each day use up all of the raw ingredients that are shipped to us. Additionally, as production manager, you must report weekly to the CEO of the company, and you know he will be more interested in the profit derived from your decisions than in the actual production levels. So you compute

\begin{align*} 300(4.99-3.69)+300(5.50-3.86)+900(6.50-4.45)&=2727.00 \end{align*}

for a daily profit of $2,727 from this production schedule. The computation of the daily profit is also beyond our control, though it is definitely of interest, and it too looks like a “linear” computation.

As often happens, things do not stay the same for long, and now the marketing department has suggested that your company's trail mix products standardize on every mix being one-third peanuts. Adjusting the peanut portion of each recipe by also adjusting the chocolate portion leads to revised recipes, and slightly different costs for the bulk and standard mixes, as given in the following table.

Raisins Peanuts Chocolate Cost Sale Price
kg/batch kg/batch kg/batch $/kg $/kg
Bulk 7 5 3 3.70 4.99
Standard 6 5 4 3.85 5.50
Fancy 2 5 8 4.45 6.50
Storage (kg) 380 500 620
Cost ($/kg) 2.55 4.65 4.80

In a similar fashion as before, we desire values of \(b\text{,}\) \(s\) and \(f\) satisfy the following.

\begin{align*} b&\geq 0 & s&\geq 0 & f&\geq 0 \end{align*}
\begin{align*} \frac{7}{15}b+\frac{6}{15}s+\frac{2}{15}f&=380&&\text{(raisins)}\\ \frac{5}{15}b+\frac{5}{15}s+\frac{5}{15}f&=500&&\text{(peanuts)}\\ \frac{3}{15}b+\frac{4}{15}s+\frac{8}{15}f&=620&&\text{(chocolate)} \end{align*}

It now happens that this system of equations has infinitely many solutions, as we will now demonstrate. Let \(f\) remain a variable quantity. Then suppose we make \(f\) kilograms of the fancy mix, \(b = 4f-3300\) kilograms of the bulk mix, and \(s = -5f+4800\) kilograms of the standard mix. We now show that these choices, for any value of \(f\text{,}\) will yield a production schedule that exhausts all of the day's supply of raw ingredients. (We will very soon learn how to solve systems of equations with infinitely many solutions and then determine expressions like these for \(b\) and \(s\)). Grab your pencil and paper and play along by substituting these choices for the production schedule into the storage limits for each raw ingredient and simpliying the algebra.

\begin{align*} \frac{7}{15}(4f-3300)+\frac{6}{15}(-5f+4800)+\frac{2}{15}f&=0f+\frac{5700}{15}=380\\ \frac{5}{15}(4f-3300)+\frac{5}{15}(-5f+4800)+\frac{5}{15}f&=0f+\frac{7500}{15}=500\\ \frac{3}{15}(4f-3300)+\frac{4}{15}(-5f+4800)+\frac{8}{15}f&=0f+\frac{9300}{15}=620 \end{align*}

Convince yourself that these expressions for \(b\) and \(s\) allow us to vary \(f\) and obtain an infinite number of possibilities for solutions to the three equations that describe our storage capacities. As a practical matter, there really are not an infinite number of solutions, since we are unlikely to want to end the day with a fractional number of bags of fancy mix, so our allowable values of \(f\) should probably be integers. More importantly, we need to remember that we cannot make negative amounts of each mix! Where does this lead us? Positive quantities of the bulk mix requires that

\begin{align*} b\geq 0 &\quad\Rightarrow\quad 4f-3300\geq 0 \quad\Rightarrow\quad f\geq 825\text{.} \end{align*}

Similarly for the standard mix,

\begin{align*} s\geq 0 &\quad\Rightarrow\quad -5f+4800\geq 0 \quad\Rightarrow\quad f\leq 960\text{.} \end{align*}

So, as production manager, you really have to choose a value of \(f\) from the finite set

\begin{gather*} \set{825,\,826,\,\ldots,\,960} \end{gather*}

leaving you with 136 choices, each of which will exhaust the day's supply of raw ingredients. Pause now and think about which you would choose.

Recalling your weekly meeting with the CEO suggests that you might want to choose a production schedule that yields the biggest possible profit for the company. So you compute an expression for the profit based on your as yet undetermined decision for the value of \(f\text{,}\)

\begin{align*} &(4f-3300)(4.99-3.70)+(-5f+4800)(5.50-3.85)+(f)(6.50-4.45)\\ &\quad\quad=-1.04f + 3663\text{.} \end{align*}

Since \(f\) has a negative coefficient it would appear that mixing fancy mix is detrimental to your profit and should be avoided. So you will make the decision to set daily fancy mix production at \(f=825\text{.}\) This has the effect of setting \(b=4(825)-3300=0\) and we stop producing bulk mix entirely. So the remainder of your daily production is standard mix at the level of \(s=-5(825)+4800=675\) kilograms and the resulting daily profit is \((-1.04)(825)+3663=2805\text{.}\) It is a pleasant surprise that daily profit has risen to $2,805, but this is not the most important part of the story. What is important here is that there are a large number of ways to produce trail mix that use all of the day's worth of raw ingredients and you were able to easily choose the one that netted the largest profit. Notice too how all of the above computations look “linear.”

In the food industry, things do not stay the same for long, and now the sales department says that increased competition has led to the decision to stay competitive and charge just $5.25 for a kilogram of the standard mix, rather than the previous $5.50 per kilogram. This decision has no effect on the possibilities for the production schedule, but will affect the decision based on profit considerations. So you revisit just the profit computation, suitably adjusted for the new selling price of standard mix,

\begin{align*} &(4f-3300)(4.99-3.70)+(-5f+4800)(5.25-3.85)+(f)(6.50-4.45)\\ &\quad\quad=0.21f+2463\text{.} \end{align*}

Now it would appear that fancy mix is beneficial to the company's profit since the value of \(f\) has a positive coefficient. So you take the decision to make as much fancy mix as possible, setting \(f=960\text{.}\) This leads to \(s=-5(960)+4800=0\) and the increased competition has driven you out of the standard mix market all together. The remainder of production is therefore bulk mix at a daily level of \(b=4(960)-3300=540\) kilograms and the resulting daily profit is \(0.21(960)+2463=2664.60\text{.}\) A daily profit of $2,664.60 is less than it used to be, but as production manager, you have made the best of a difficult situation and shown the sales department that the best course is to pull out of the highly competitive standard mix market completely.

This example is taken from a field of mathematics variously known by names such as operations research, systems science, or management science. More specifically, this is a prototypical example of problems that are solved by the techniques of linear programming.

There is a lot going on under the hood in this example. The heart of the matter is the solution to systems of linear equations, which is the topic of the next few sections, and a recurrent theme throughout this course. We will return to this example on several occasions to reveal some of the reasons for its behavior.

Reading Questions WILA Reading Questions

1.

Is the equation \(x^2 + xy +\tan(y^3)=0\) linear or not? Why or why not?

2.

Find all solutions to the system of two linear equations \(2x+3y=-8\text{,}\) \(x-y=6\text{.}\)

3.

Describe how the production manager might explain the importance of the procedures described in the trail mix application (Subsection WILA.AA).

Exercises WILA Exercises

C10.

In Example TMP the first table lists the cost (per kilogram) to manufacture each of the three varieties of trail mix (bulk, standard, fancy). For example, it costs $3.69 to make one kilogram of the bulk variety. Re-compute each of these three costs and notice that the computations are linear in character.

M70.

In Example TMP two different prices were considered for marketing standard mix with the revised recipes (one-third peanuts in each recipe). Selling standard mix at $5.50 resulted in selling the minimum amount of the fancy mix and no bulk mix. At $5.25 it was best for profits to sell the maximum amount of fancy mix and then sell no standard mix. Determine a selling price for standard mix that allows for maximum profits while still selling some of each type of mix.

Solution

If the price of standard mix is set at $5.292, then the profit function has a zero coefficient on the variable quantity \(f\text{.}\) So, we can set \(f\) to be any integer quantity in \(\set{825,\,826,\,\ldots,\,960}\text{.}\) All but the extreme values (\(f=825\text{,}\) \(f=960\)) will result in production levels where some of every mix is manufactured. No matter what value of \(f\) is chosen, the resulting profit will be the same, at $2,664.60.