Here's an interesting quote I came across from Calvin on whether to keep going in existing non-ideal churches or start your own. It's worth pointing out that this doesn't address the issue of starting new churches for missional reasons, or congregations leaving an existing denomination because they persecute true believers.
Dreadful are those descriptions in which Isaiah, Jeremiah, Joel, Habakkuk, and others, deplore the disorders of the Church of Jerusalem. There was such general and extreme corruption in the people, in the magistrates, and in the priests, that Isaiah does not hesitate to compare Jerusalem to Sodom and Gomorrah. Religion was partly despised, partly corrupted. Their manners were generally disgraced by thefts, robberies, treacheries, murders, and similar crimes. Nevertheless, the prophets on this account neither raised themselves new churches, nor built new altars for the oblation of separate sacrifices; but whatever were the characters of the people, yet because they considered that God had deposited his word among that nation, and instituted the ceremonies in which he was there worshipped, they lifted up pure hands to him even in the congregation of the impious. If they had thought that they contracted any contagion from these services, surely they would have suffered a hundred deaths rather than have permitted themselves to be dragged to them. There was nothing therefore to prevent their departure from them, but the desire of preserving the unity of the Church. But if the holy prophets were restrained by a sense of duty from forsaking the Church on account of the numerous and enormous crimes which were practised, not by a few individuals, but almost by the whole nation,--it is extreme arrogance in us, if we presume immediately to withdraw from the communion of a Church where the conduct of all the members is not compatible either with our judgment, or even with the Christian profession.
John Calvin, Institutes 4.1.18
HT to CQOD.