I am not the most knowledgeable on OSD, but here are some thoughts.
Are you using the configurator that HolyBro recommends? Not some other configurator that you downloaded.
Does Librepilot Mavlink implementation lacks of RSSI value?
If configured to display RSSI, I expect that the OSD should display the RSSI icon/text whether or not there is RSSI info coming in. Further, if you power on the OSD but not the FC, the OSD will have no idea what FC or protocol is being sent and can't "hide the RSSI because of protocol" if that is even a possible thing. To be clear, it is still possible for it to "hide RSSI until it gets valid RSSI data" if it is coded that way. I have no idea how it is coded.
Thus is seems more likely that it is not capable of RSSI or that RSSI has not been turned on correctly with the correct configurator?
There is a special compile option for Mavlink OSD (Scarab version) to have LibrePilot compatibility. I believe that without that, it will mostly work, but there may be some small things that don't. I don't know about the HolyBro version.
MW_OSD/Config.h://#define LIBREPILOT // Uncomment this if you are using the latest LibrePilot MSP Module
MW_OSD/Def.h:#ifdef LIBREPILOT
Also, you could look for updated HolyBro firmware/configurator for the OSD.
Finally, if HolyBro is compatible hardware, you could put other firmware on it or try to get the HolyBro source code. These are advanced procedures. Save/backup/download the current firmware before changing it in case you need to go back.