20 minutes is not enough, but I think there is a better approach to this problem, one used in many other multiplayer games. Each player would have a certain value assigned, so called reputation, with some reasonable minimal and maximal values. The player's reputation increases with each adventure they complete without disconnecting, while disconnecting from a running game would decrease it; disconnecting from the lobby before the game is started would hurt the reputation even more. Then, when the matchmaker selects players for an adventure, it attempts to make match for players with the same reputation levels. The reputation can optionally slowly increase over time.
TL;DR: This system ensures that if a player keeps disconnecting from adventures, he will eventually be put into games only with players that do the same. Similarly, players that finish their games will have matches mostly with other responsible players.
I would actually believe the devs have something similar on their plans, it is quite a common practice.