Riding base town suggestions in SW Ontario

Hi. I'm going to be doing some riding with friends this season and we are looking for a town where we can do a few loops (day trips) from over the course of 2-4 days.
Not looking for crazy hills or technical single track. Just some mid-grade riding levels that everyone can do without getting discouraged. Trails or low use roads preferred. Rural settings.
We're from the Windsor area and we would be alright with driving 4 hours or so to get there. Can't go to the US unfortunately because some people can't cross the border.
Any suggestions are welcome.
Thanks.