Answer
See below
Work Step by Step
The given system can be written as $x′ = Ax$,
where $A=\begin{bmatrix}
6 & -1\\
-5 & 2
\end{bmatrix}$
The transformed system is $y′ = (S−1AS)y$ where $x = Sy$
To determine S, we need the eigenvalues and eigenvectors of A. The characteristic polynomial of A is
$p(λ) = \begin{bmatrix}
6-\lambda & -1\\
-5 & 2-\lambda
\end{bmatrix}=(6-\lambda)(2-\lambda)-5=(\lambda-1)(\lambda-7)$
Hence, A is nondefective by Corollary 7.2.10. The eigenvectors are easily computed:
$\lambda_1=1: v=r(1,5)\\
\lambda_2=7: v=s(1,-1)$
Set $S = \begin{bmatrix}
1 & 1\\
-1 & 5
\end{bmatrix}$
then from Theorem 7.3.4, $S−1AS = diag(7, 1)$
so that the system becomes:
$\begin{bmatrix}
y_1'\\
y_2'
\end{bmatrix}=\begin{bmatrix}
7 & 0\\
0 & 1
\end{bmatrix}\begin{bmatrix}
y_1\\
y_2
\end{bmatrix}$
Hence, $y_1'=7y_1\\
y_2'=y_2$
Both of these equations can be integrated to obtain $y_1(t)=C_1e^{7t}\\
y_2(t)=C_2e^{t}$
Return to the original variables, we have:
$x=Sy=\begin{bmatrix}
1 & 1\\
-1 & 5
\end{bmatrix}\begin{bmatrix}
C_1e^{7t}\\
C_2e^{t}
\end{bmatrix} \\
\rightarrow x_1(t)=C_1e^{7t}+C_2e^{t}\\
x_2(t)-=C_1e^{7t}+C_2e^{t}$
