Answer
$C= x^2y+x\cos y-y^2$
Work Step by Step
We are given:
$$(2xy+\cos y)dx+y^{-1}(x^2-x\sin y-2y)dy=0 (1)$$
Here,
$M(x,y)=\frac{\partial}{\partial y}(2xy+\cos y)=2x-\sin y$
$N(x,y)=\frac{\partial}{\partial x}(x^2-x\sin y-2y)=2x-\sin y$
$M_y=2x-\sin y \;\;\;\;,\;\;\;\;N_x=2x-\sin y$
$M_y=N_x$
$\Rightarrow$ (1) is exact differential equation
Therefore there exists a potential function $\phi$ such that
$\frac{\partial\phi}{\partial x}=2xy+\cos y$ ___(2)
Integrating (2) with respect to $x$ holding $y$ fixed
$\phi =x^2y+x \cos y+f(y)$ ___(3)
Where $f(y)$ is arbitrary function of $y$
Hence we get:
$\frac{\partial\phi}{\partial y}=x^2-x\sin y+f'(y)=x^2-x\sin y-2y$ (4)
We have
$f'(y)=-2y\rightarrow f(y)=-y^2+C$
$C$ is constant
The solution is:
$\phi(x,y)= x^2y+x\cos y-y^2+C$
and $\phi(x,y)=0$
so:
$C= x^2y+x\cos y-y^2$