\displaystyle u=x^2,\;dv=\cos(nx)\,dx\implies v=\frac{\sin(nx)}{n},\;du=2x\,dx.\int_{0}^{\pi}x^2\cos(nx)\,dx =\left.\frac{x^2\sin(nx)}{n}\right|_{0}^{\pi} -\frac{2}{n}\int_{0}^{\pi}x\sin(nx)\,dx.O termo de fronteira some pois \sin(n\pi)=0.
No segundo integral, faça \displaystyle u=x,\;dv=\sin(nx)\,dx\implies v=-\frac{\cos(nx)}{n},\;du=dx.\int_{0}^{\pi}x\sin(nx)\,dx =\left.-\frac{x\cos(nx)}{n}\right|_{0}^{\pi} +\frac{1}{n}\int_{0}^{\pi}\cos(nx)\,dx =-\frac{\pi(-1)^n}{n}.