Game Theory — Nash Equilibrium, Strategies & Payoff Matrix

Game theory studies decisions where your best move depends on what other people do. A payoff matrix shows the outcome for each combination of choices, and a Nash equilibrium is a set of choices where no player can do better by changing alone.

Those three ideas, strategy, payoff, and equilibrium, are the core of most introductory game theory problems. Once they click, many textbook examples become much easier to read.

Game theory definition: what question is it asking?

In an ordinary optimization problem, you choose the best option in a fixed situation. In game theory, the situation can change because other players are choosing too, either at the same time or in response to you.

So the question changes from "What is my best move?" to "What is my best move given what others might do?" That shift is the main idea behind strategic interaction.

Strategies and payoffs in plain language

A strategy is a player's available choice or rule for acting in the game. In a simple one-shot game, a strategy may be just one action, such as cooperate or defect.

A payoff is the result a player gets from a particular combination of choices. It might represent money, points, utility, or any ranking where a larger number means a better outcome for that player.

In a two-player game, those outcomes are often organized in a payoff matrix. Each cell matches one strategy from Player A with one strategy from Player B.

How to read a payoff matrix

Here is a standard Prisoner's Dilemma style payoff matrix. The first number in each cell is Player A's payoff, and the second is Player B's payoff.

$$\begin{array}{c|cc} & \text{B: Cooperate} & \text{B: Defect} \\ \hline \text{A: Cooperate} & (3,3) & (0,5) \\ \text{A: Defect} & (5,0) & (1,1) \end{array}$$

Read each cell as one complete outcome:

• If both cooperate, each gets $3$.
• If one defects while the other cooperates, the defector gets $5$ and the cooperator gets $0$.
• If both defect, each gets $1$.

The exact numbers are not a law of game theory. They are just one payoff pattern. What matters is the incentive structure: each player is tempted to defect, even though both would rather end up at mutual cooperation than mutual defection.

Nash equilibrium: the stable outcome

A Nash equilibrium is a set of strategies where no player can improve their own payoff by switching alone, while the other players keep their strategies unchanged.

Another way to say it is that each player's choice is a best response to the others' choices.

That does not mean the outcome is best for everyone. It only means no one has a one-sided incentive to move away from it.

Worked example: finding Nash equilibrium

Use the matrix above.

If Player B cooperates, Player A compares cooperate for $3$ with defect for $5$. Defect is better.

If Player B defects, Player A compares cooperate for $0$ with defect for $1$. Defect is still better.

So for Player A, defect is the best response in either case. By symmetry, the same is true for Player B.

That means $(\text{Defect}, \text{Defect})$ is a Nash equilibrium. Once both players are there, neither one can improve by changing alone.

But it is not the best joint outcome. The total payoff at $(\text{Cooperate}, \text{Cooperate})$ is $3+3=6$, while the total payoff at $(\text{Defect}, \text{Defect})$ is only $1+1=2$.

This is the key insight: a Nash equilibrium can be stable without being collectively best.

Common mistakes students make

One common mistake is thinking Nash equilibrium means the best possible outcome for everyone. It does not. It only means no player benefits from changing alone.

Another mistake is reading the payoff matrix from only one player's perspective. Every cell has to be checked from each player's point of view.

Students also sometimes forget that the model depends on the payoff structure. If the payoffs change, the best responses and equilibrium can change too.

When game theory is used

Game theory is used in economics, auctions, pricing, negotiation, voting, network design, and evolutionary biology. The details differ by field, but the same core question keeps returning: how should one agent act when others are choosing too?

In more advanced settings, game theory also studies mixed strategies, repeated games, and games with more than two players. For a first pass, though, pure strategies and a payoff matrix are enough to build the main intuition.

Try a similar problem

Try your own version by changing one payoff in the matrix and recomputing the best responses. For example, ask what happens if mutual cooperation pays $(4,4)$ or if mutual defection pays $(2,2)$. That is one of the fastest ways to see that equilibrium depends on incentives, not on the labels attached to the strategies.

If you want to go one step further, compare this setup with a coordination game, where players benefit from matching each other's choices. Seeing both cases side by side makes Nash equilibrium much easier to recognize.