[jdl]

I think I've encountered a serious problem with INS13 in current next.589. I currently have 3 flyable UAVs with Revo boards (a wing and two copters) running this next and they all suffer the same in different extent. I have checked in the old telemetry records and I'm sure that the wing and the copter did not have such a problem with 16.09 firmware.

Today I was tuning and testing a new 250 quadcopter with Revo (Next 589 firmware) and encountered a constant slow yaw rotation when the UAV is motionless on the ground. All sensors are calibrated, MAG is green. Horizon in PFD is correct, almost no movement, but the compass ring is rotating all the time.

This is when INS13 is selected as Fusion Algorithm. Looking in the scopes, AttitudeState.Yaw and GyroState.Z are insane (attached image BAD & UNFLYABLE_ZMR250_UAV2_INS13). AccelState and MagState are almost rock solid meanwhile.

I did not initially noticed this strange behaviour and did take off in Attitude mode. Copter got highly "unsteerable" and I crashed after 10-15 seconds, thankfully managed to drive it to a soft spot and low altutude before I cut the thrust. No damages.

I investigated the telemetry record, spotted the problem and made additional testing: switched first to Basic(Complementary), then to INS13+CF. In both cases there was not even a sign of that problem. AttitudeState and GyroState scopes show fine and plausible values (attached image OK_ZMR250_UAV2_INS13+CF).

Tested autonomous modes: PositionHold, RTB, Autoland, also basic Attitude, all they performed fine in INS13+CF.

Further, checked telemetry records from the two other UAVs I'm currently flying with next.589 and found they also have problems with YAW estimation in INS13 - not constant rotation but oscillations. (attached images BAD & FLYABLE_Z84_INS13 and BAD & FLYABLE_ZMR250_UAV1_INS13)

These oscillations are not too big and as I fly primarily in Rate, I've not noticed till now there is a problem with the INS13 in current next.589 ... But I checked older records and there were not such yaw oscillations with these UAVs when they were with 16.09 firmware.

Here are attached configuration files for all three UAVs and a short .opl log from today. Copter is on the ground, INS13 initially, then after some time I changed Fusion Algorithm to INS13+CF and Attitide.YAW and GyroState.Z immediately settled down.

I hope this information should be sufficient to reproduce the problem.

Btw, would it be safe enough to switch my UAVs to INS13+CF? I tested Complementary + MAG + GPSOutdoor and the short test flight was ok. But maybe INS13+CF is better choice?! At least until INS13 is fixed.

As I recall, Complementary + MAG + GPSOutdoor may not be a good option for a plane. But what about INS13+CF?

I've almost finished a long-range-intended Mini Talon build, with Revo as FC and OpLink for control but now I  hesitate to maiden it with next.589 and that yaw issue with INS13 I've encountered as it affects not only autonomous modes but basic level (Attitude) flight also.

[f5soh]

EKFSettings changed a lot between 16.09 and r589, here is current 'next' defaults to the left and your settings to the right:



Try importing the UAV file attached for new default EKFSettings.

Same for AttitudeSettings -> MagKp and MagKi, your settings refers to old settings and you will expect unsafe results using Complementary+Mag+GPS even if seems ok currently.

Here is the new values:

[TheOtherCliff]

Trying to parse to a simplified version of what this says.

From what I read, INS13 does not work well in next.589, but INS13+CF does work well.  I recall seeing that the mag has a new setting for 3D vs. 2D and the default is 2D.  Maybe all you need to run INS13 bare is to set this back to 3D so it acts like it did before.  Setting is in EKFConfiguration.

shared/uavobjectdefinition/ekfconfiguration.xml:
   <field name="MapMagnetometerToHorizontalPlane" type="enum" units="bool" elements="1"
      options="False,True" defaultvalue="True"
      description="Set to True to suppress effect of magnetometers on Roll+Pitch State estimate" />

One last thing is that mag must be well calibrated and correctly aligned on model or you get EKF oscillation and divergence.

Let us know if this setting helps your issue so I can add it to my list. 

If this is the cause, then it sounds like INS13 bare needs to automatically set this to 3D.

[jdl]

Thanks for replies.

@f5soh: I was unaware that EKF & Attitude defaults have been changed though I should have checked these defaults when updating to next.589. Sorry. Thanks for pointing me that!
I applied new (correct for next.589) defaults to all three UAVs. Cannot test wing right now but unfortunately, nothing changed for the two quads, they exhibit the same bad behaviour - light yaw oscillations for UAV1 quad (flyable) and constant yaw rotation for UAV2, being stationary on the ground. This is when INS13 is used.
If I switch them to INS13+CF, all seems perfect... Here are screenshots I made just now for UAV2 with new defaults.

As I said before, there were no such yaw oscillations for UAV1 when is was loaded with 16.09, same hardware and settings!? UAV2 was never flown with 16.09, I've finelly built it days few ago...
UAV2 flyes beautifully with INS13+CF, do you consider this algorithm safe enough or there are known dangerous issues with it?

@TheOtherCliff: I'm confident all calibrations are done thoroughly and mags are correctly aligned. Funny looking angles int "virtual rotate" you may have seen in the uav files reflect the real orientation of MAG boards. Especially for UAV2, when checking in Configutarion / Attitude / Magnitometer - all indicator bars were almost all the time at "0" position, rarely -1, and both mags are green while 3D rotating the frame in hand.

I've tested with "MapMagnetometerToHorizontalPlane = False". Nothing changed for good for UAV2, in fact I suppose that exact change made the copter unsteerable to extent that made me crash. Test flight with default "MapMagnetometerToHorizontalPlane = True" in Attitide mode at least let me land safely. So "MapMagnetometerToHorizontalPlane = True" is good, I'll continue to use it.

My previous tests with UAV1 and next.589 showed that it flies relatively well no matter of that mapping of mag to 2D, but I left it set to default "true". Yes, the yaw oscillations are there with UAV1, though they are light and practically not noticeable in Attitude mode.
 
The wing flies pretty ok with INS13 and "MapMagnetometerToHorizontalPlane = True". Yaw oscillations with it also does not seem to affect noticeably the flight.

[TheOtherCliff]

Just a point of reference:  It looks like I had INS13 next.r550 (December) on a quad for testing the fix for Altitude Hold.  It worked fine.

My EKF was 10,10,10,etc not the default of 25,25,25,etc.  I didn't do that on purpose, I hand converted the 16.09 UAV file to next.r550 and didn't look at the EKF values.

[TheOtherCliff]

I don't know if it is the technically correct thing to do, but I do fly autonomous fixed wing (4 channel), and for that I always set my yaw/rudder to manual.

[jdl]

I've also rebuilt Next.589 Revo firmware with the AltitudeHold Fix you published, also fixes for  ADC on output channels , tested it with UAV1 (my older ZMR250 quad), worked fine. I've flown with UAV1 and Next.589 at least several hous flight time total till now without noticing these yaw oscillations issues, though telemetry records clearly show them present before take-offs. I fly rate and only use sometimes PositionHold before landing in LOS if the terrain is difficult. Also I've engaged RTB several times when FS checking.

But with newly built ZMR250 frame (UAV2) with Next.589 and INS13 yaw is not just unstable, it rotates... It is completely unflyable in Attitide mode, PositionHold is unusable too.
Testing Basic(Complementary), Complementary+Mag+GPSOutdoor, INS13+CF - all they give very stable attitide and predictable handling. This makes me confident that the hardware of that particular revo board, AuxMag and GPS are ok.
If INS13+CF is considered safe to use, without known hidden issues, I'd like to switch all my UAVs to it.

My wing has no yaw controls, it is two elevons type (3 channel), so yaw oscillations in attitude estimation should not affect it, I guess...

[TheOtherCliff]

I see that you are using an I2C aux mag which is normally configured with 180,0,180 or the equivalent 0,180,0

I see that you have:  <field name="BoardRotation" values="180,0,93"/>

Typically, this would be reasonable if your GPS/mag was rotated 90 degrees so that the cable came out the side and not the front/back.  I am guessing that you added 3 degrees for better alignment.  My things to check include whether you do have the GPS mounted so that the GPS cable comes out the side (not front/back) of the GPS, and whether the 93 matches the side (left/right) it actually comes out.  E.g. does it need to be -87 or 273 or ? because it comes out the other side.

[f5soh]

Can you try to recalibrate the mags ?
Currently the AuxMag scales are something like 0.8,0.82 and 1.00 so pretty asymmetric.



- Start calibration,
- Press the "Save position" button five times and move/rotate the quad around all axis to cover all the 3D space,
- Press the "Save position" button for the last step and save settings.

[TheOtherCliff]

One last thing is that mag must be well calibrated and correctly aligned on model or you get EKF oscillation and divergence.

@TheOtherCliff: I'm confident all calibrations are done thoroughly and mags are correctly aligned.

[jdl]

I'm currently away from home for a few days... Will go on with tests when I'm back.

....
I see that you have:  <field name="BoardRotation" values="180,0,93"/>
....
it need to be -87 or 273 or ? because it comes out the other side.

UAV2 is built with BN-220 GPS only and HMC5883L AuxMag on standalone PCB (CJMCU-49). It is mounted between the lower carbon plates of the frame. All power lines are carefully twisted. Last flight tests show this AuxMag position is good and quite well protected from the parasitic magnetic fields.

I think the alignment is correct, I've checked it more than once. Moreover, the three bars showing differences between internal and AuxMag in GCS/Configuration/Attitude/Mags stay almost all the time at 0 - 0 - 0 for the three axles after mags are calibrated.

Here are photos of UAV1 & UAV2. UAV1 uses AuxMag from the BN-880 GPS.

It has many flawless flights with 16.09 (INS13) and after upgrading to Next.589 with the same calibrations also flies well, but there are now these slight yaw oscillations in PFD and scopes while still on the ground.

@f5soh: I've done the mags calibration using just the exact procedure you pointed. This is in the open field, away from power lines, underground communications, even away from the road. Away from any metallic objects as car, chair, laptop, phone or watch I've done it more than once for UAV2 and I'm confident it is done well. Asymmetric AuxMag scales are due its placement in the tiny space of the ZMR250 frame with all its surroundings, I suppose.
Same asymmetry, even worse, presents in UAV1 but it flew reliably without issues with 16.09. UAV1 flies well now with Next.589 too, but now these slight yaw oscillations are present, while they didn't existed with 16.09.

I'll make another test when back to home. I'll flash UAV2 with 16.09 with the same settings and calibrations (with 16.09 EKF defaults) and will test if the yaw rotation is present there.

[TheOtherCliff]

...the three bars showing differences between internal and AuxMag in GCS/Configuration/Attitude/Mags stay almost all the time at 0 - 0 - 0 for the three axles after mags are calibrated.

If it stays close to 0,0,0 no matter what bank angle the quad has then the aux mag board rotation is correct.

Did you try replacing the whole Settings.EKFConfiguration with the old version?

[jdl]

Yes, it was old 16.09 settings initially, I just imported old 16.09 UAV1 config file after update & erase for UAV2 Revo FC with next.589 and then checked and made manual corrections where necessary. Calibrations are done for the new Revo and Mag in UAV2, of course.

Later I followed the f5soh advice and imported new EKF defaults for 589.

Yaw rotation persists with both configurations.

I'd like to see how this build will behave when I revert it back to 16.09. I'll report back results asap.

Btw, UAV1 (older ZMR250) flies well in autonomous modes and Attitude with next.589 and EKF defaults from 16.09. Only concern I had was the drop when engaging AltitudeHold /RTB /PositionHold, but after applying your fix it works as in 16.09.
Also I noticed these light slow yaw oscillations with UAV1 stationary after updating to next.589. I do not see such ones in telemetry records from UAV1 while it was running 16.09.

[TheOtherCliff]

Just some thoughts.

I have seen PFD oscillations like that in two cases.  Mag needing recalibration, and trying to use NMEA GPS protocol instead of Ublox protocol.  It wouldn't hurt to try recalibrating mags as all three of us discussed before.  There was a problem with mag calibration that it did not zero the bias and one the gains correctly.  That has been fixed, but if it were me, I would set all the mag calibrations to default and try mag cal again.

Are you getting any alarms in your telemetry?

Magstate showing steady but compass rotating sounds like an EKF divergence.  That can happen if the vehicle sits motionless for a while.  If all calibrations are good, it may never diverge.  With lesser calibrations it may diverge after just a couple minutes.

Mag calibration difference with onboard mag is close to 0,0,0 for all bank angles / orientations?  Say nothing bigger than a 2 or 3?

Maybe something like accel calibration was bad, or gyro calibration needs to be done again.

Have you done thermal calibration?  Gyro calibration needs to be done again after thermal calibration.

[f5soh]

Found the issue.
https://bitbucket.org/librepilot/librepilot/pull-requests/504

Can you test the revolution firmware attached? (based on latest Next)