Step 1: Simplify the integrand.
Given,
\[
f(x)=\int x^2\cos^2x\left(2x\tan^2x-2x-6\tan x\right)\,dx
\]
Using
\[
\tan x=\frac{\sin x}{\cos x}
\]
and
\[
\tan^2x=\frac{\sin^2x}{\cos^2x},
\]
we get
\[
x^2\cos^2x\left(2x\tan^2x-2x-6\tan x\right)
\]
\[
=
x^2\cos^2x
\left(
2x\frac{\sin^2x}{\cos^2x}
-2x
-6\frac{\sin x}{\cos x}
\right)
\]
\[
=
x^2\left(2x\sin^2x-2x\cos^2x-6\sin x\cos x\right)
\]
Step 2: Use trigonometric identities.
We know that
\[
\cos 2x=\cos^2x-\sin^2x
\]
So,
\[
\sin^2x-\cos^2x=-\cos 2x
\]
Also,
\[
2\sin x\cos x=\sin 2x
\]
Therefore,
\[
2x\sin^2x-2x\cos^2x
=
2x(\sin^2x-\cos^2x)
\]
\[
=
-2x\cos 2x
\]
and
\[
-6\sin x\cos x=-3\sin 2x
\]
Hence, the integrand becomes
\[
x^2\left(-2x\cos 2x-3\sin 2x\right)
\]
\[
=
-2x^3\cos 2x-3x^2\sin 2x
\]
Step 3: Identify the antiderivative.
Now observe that
\[
\frac{d}{dx}\left(-x^3\sin 2x\right)
\]
Using product rule,
\[
=
-\left(3x^2\sin 2x+x^3\cdot 2\cos 2x\right)
\]
\[
=
-3x^2\sin 2x-2x^3\cos 2x
\]
This is exactly the simplified integrand.
Therefore,
\[
f(x)=-x^3\sin 2x+C
\]
Step 4: Use the given condition \(f(0)=\pi\).
Substitute \(x=0\):
\[
f(0)=-(0)^3\sin 0+C
\]
\[
f(0)=C
\]
Given that
\[
f(0)=\pi
\]
So,
\[
C=\pi
\]
Therefore,
\[
f(x)=-x^3\sin 2x+\pi
\]
Step 5: Final conclusion.
Hence,
\[
\boxed{-x^3\sin 2x+\pi}
\]