Why might the lord’s wife have had such power over Gawain?

She is beautiful and tries hard to seduce him. She makes the seduction very difficult to resist; we also find later that she is simply a "pawn" in the scheme cooked up by the old woman who is really Morgan le Faye, a witch and the half sister of King Arthur. Thus magic is also involved.