I moved from telegram to signal to hosting my own matrix instance and connecting through a VPN. My assumption is that any remote server is compromised

Then why don’t they do end-to-end encryption by default instead of making it a dark pattern where you have to specifically opt-in to it as a separate chat group?