Given that \( B \) is the matrix of cofactors of \( A \), we use the relationship:
\[ AB = \det(A) \cdot I_3, \] where \( I_3 \) is the \( 3 \times 3 \) identity matrix. Therefore: \[ \det(AB) = \det(A)^3. \]
We know: \[ (2\alpha^2 - 3\alpha) = \alpha. \]
Rearranging: \[ 2\alpha^2 - 3\alpha - \alpha = 0 \implies 2\alpha^2 - 4\alpha = 0. \]
Since \( \alpha \neq 0 \), we get: \[ \alpha = 2. \]
Using the relation: \[ 2\alpha^2 - \alpha\beta = 3\alpha, \] substitute \( \alpha = 2 \): \[ 2 \cdot 2^2 - 2\beta = 3 \cdot 2 \implies 8 - 2\beta = 6 \implies 2\beta = 2 \implies \beta = 1. \]
Substitute \( \alpha = 2 \) and \( \beta = 1 \) into matrix \( A \): \[ A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 2 & 1 \\ -1 & 2 & 4 \end{bmatrix}. \]
The determinant of \( A \) is: \[ \det(A) = 1 \cdot \begin{vmatrix} 2 & 1 \\ 2 & 4 \end{vmatrix} - 2 \cdot \begin{vmatrix} 2 & 1 \\ -1 & 4 \end{vmatrix} + 3 \cdot \begin{vmatrix} 2 & 2 \\ -1 & 2 \end{vmatrix}. \]
Calculating each minor: \[ \begin{vmatrix} 2 & 1 \\ 2 & 4 \end{vmatrix} = 2 \cdot 4 - 1 \cdot 2 = 6, \] \[ \begin{vmatrix} 2 & 1 \\ -1 & 4 \end{vmatrix} = 2 \cdot 4 - 1 \cdot (-1) = 9, \] \[ \begin{vmatrix} 2 & 2 \\ -1 & 2 \end{vmatrix} = 2 \cdot 2 - 2 \cdot (-1) = 6. \]
Thus: \[ \det(A) = 1 \cdot 6 - 2 \cdot 9 + 3 \cdot 6 = 6 - 18 + 18 = 6. \]
Since: \[ \det(AB) = \det(A)^3 = 6^3 = 216. \]
Therefore, the correct answer is Option (4).