"GPS Only" is just that. It works if the GPS works and still works even if the MCU (CPU) is not working.
"GPS+MCU" has the same GPS data as "GPS Only" plus mag data. It only works if the MCU is working. This is the connector you want to connect to.
Do not use the SWD debug port or you may damage something since it's power pinout is different.
That is not a tiny battery. It is a large capacitor which acts like a battery and keeps recent gps almanac data for several hours. You should really let the GPS gather almanac data for 12 minutes before the first flight of the day, but you can fly more quickly (say a couple minutes) the rest of the day. Do as I say, not as I do.