Here is the
Curing Eldergrove thread. It doesn't have the tiny bit about the refugees, but gives the overall story. The final part is buried in the
New Paphos (The Base) thread somewhere.
The short version:
There was an ancient war that made it to Eldergrove. Eldergrove was powered by a magical mystical plant based intelligence guided by a council of elders. The elders screwed up and the intelligence panicked. The result was that all of the citizens of the city were infected by and taken over by plants controlled by the intelligence. A group of adventurers found the core of the intelligence and convinced it to release the citizens. One of the adventurers even stayed behind to help guide the intelligence.
Most of the citizens were not able to survive the removal of the plants and died. However, there were many that had originally been young enough to be viable without the plant infestation. The were moved to The Base/New Paphos who both didn't want them and couldn't handle them. Someone came up with the idea of moving them to the Manor, which is where they ended up.
That all said, from the survivor's perspective, they were taken over by their guardian plant intelligence, then they were waking up and getting shipped off to New Paphos, then to the Manor. There is no reason to believe that everyone was completely briefed on the actual story. So, quite frankly, your explanation is pretty much on-point from the character's perspective.