Does it only happen far away?
How are your RC and FC failsafe set?
What protocol does your receiver use?
If it sometimes happens close by, then it could be a bad connection. It happened to me.
Do you have to arm it after it falls? Or just give it throttle? If you must rearm then the FC may have reset (bad FC?).
Are you using unflashed ESCs? You must set the LVC to nicad or this will happen. You have to time your flight or use a lipo alarm to avoid damaging your batteries by over discharging them.
Best way to know for sure is to log a flight with telemetry and a laptop GCS. The reason will be there after some digging. Did it go into failsafe? Did the RC command it to fall? Which signal wire came loose?
Another way is to assume that it is the FC failsafe. The following is dangerous in that your quad could fly away if you don't adjust it correctly (so that it descends even with a fully charged battery). You could set failsafe to an autoleveling flight mode and have it force throttle to be a little below hover on a fully charged battery. If your falls change into float downs then you know the FC is going into failsafe and that is usually caused by loss of signal from the RC receiver.