The mushroom people are located in the Evil Forest and can only be freed after getting BMO as he's needed to unlock one of the gates. The first set is up just after where you free BMO and the second set is down next to the cage which previously held Princess Bubblegum.