Are you using aux mag only (Mag Usage: AuxOnly) (that works best).
What is your AuxMagSettings.BoardRotation / Attitude->Magnetometer->AuxiliaryMagnetometerOrientation
for OP GPSv9 and DJI/Naza GPS/mag it should be 0,0,0
for PixHawk and APM GPS/mag if should be 0,180,0 (or 180,0,180 which is the same thing)
Are you using INS13 (Attitude->Settings->AttitudeEstimationAlgorithm).