So to be clear, all GPS modes and any non-GPS mode that uses CruiseControl use both CruiseControl and baro all the time. Of course CruiseControl adds zero thrust when the sticks are centered.
Any mode that uses the baro and has either a loss or gain in altitude as long as you are traveling forward and recovers when you stop either doesn't have enough power to travel constantly at that angle, has bad altitude PIDs, or (usual case) has a baro that is getting blown on. Higher pressure makes it thing it is lower and so it gains altitude. Lower pressure (Bernoulli effect or big hole in back) makes it think it is higher and so it looses altitude.
Also, there are more PIDs in VtolPathFollower. Some for altitude, some for horizontal, and one especially for VR mode horizontal.