You know, PR really nailed canon. From the standpoint of this mod, it's easy to see why factions used most of the ships they do.
My tip: Upgrade the ISDs. They become awesomely tough and versatile. Like a miniature fleet in and of itself. There's a reason they stay in service for decades and the Empire built 25,000 of them. This is the ship I rely on and few planets can stand one or two of these in 1.1.
Dreadnaughts are a bit weak in ship to ship combat, but few other ships that size have heavy turbos, so it's often got an advantage in range. It's also extremely well-protected against fighters and bombers. Also, it's slow, so if it gets in trouble it's probably not coming back.
The Avenger and Defender don't really compete well against each other. The former is a mid-high-end dedicated anti-fighter platform, while the latter is a very high-end fighter-bomber. The Defender can hold it's own simply because it's so advanced, but pit them against each other and the Avenger will come out on top. The Defender holds the clear advantage against larger ships like the Recusant though. So Avenger for the little stuff, Defenders for the bigger, escorted stuff.
My opinion of them is that they're effective, but just too expensive to leave strikecraft coverage to them. Their also awfully fragile, just like all frigates, and often times heavy fighter-bombers can just turn their launchers on the Lancer and turn the tables real quick. (Frigates are really in one of the worst positions in the game. Big enough to get hit by large weapons, small enough to have weak hull/shields.)
For larger groups, you need small wolfpacks of Lancers to tear through them fast enough, but either way, it requires a lot of micromanagement to use them to their full potential. Don't underestimate the damage output of those quad lasers, either. In a real pinch they can wear down cruisers surprisingly fast as support for your heavy hitters.
That's it in a nutshell, though if I'm going to micro, I'll just use the Acclamators. heavier hull/shields, and the quad turbos put out enough flak to apply an old machine gunner's term: Accuracy by volume. Makes me chuckle to watch two of them flank a big dogfight and just rake the area, watch little explosions start cookin off all over.
And after that, just roll right up nose to nose with whatever you want vaporized and let em do their thing. Work VERY well on space stations, and a pair can even overwhelm an early ISD. But it only works if you can distract the ISD BEFORE the Acclamators get in range. If she has the full run to pound on em, you may still get her, but one Acclamator is definately dead, and the other is gonna be in a world of hurt.
As far as the fighter cover, Defenders or Avengers seem to work very well with the flare systems to draw off the potentially devastating concussion missiles, and heavy enough shielding to grind down most opposition. You are correct about the cost however. Two wings of either and I could have my pick of light frigates or 4 Bayonets, which I really like for cheap stand-off power. They work surprisingly well in dissuading enemy corvettes from joining the strikecraft dogfights. If you really want a couple wings of them to round out the strikecraft however, you can get one of each deployed from the Super Trans VII Escort carrier. cost about on par with 3 of them and you get an Avenger, Defender, Bomber, and 3 plain ole TIEs. oh, and the ability to repair your other ships mid battle. Now THAT'S a deal.
And Yes, the ISD-4s are just this side of terrifying as far as firepower, they're actually what I was referring to as being able to comfortably take a Praetor 2-1 while the rest of the fleet engages the Praetor's support. I don't know that the Rebels have a ship capable of the same, but will find out. they still don't do so hot if you can get your bombers in on them, but that's what _your_ support craft are for, right?
As you may have noticed, certain ships lend themselves to how I fight, Dreadnaughts/Carracks engage the enemy at long range to soften up fortification and draw the enemy fleet out, with Acclamators waiting to establish rapid dominance in the resulting skirmish. Later in the game, the ships change with ISDs/Strike Cruisers hitting at range (and increasing firepower as the enemy attempts to close range) and Carracks or ATR-6s to nail the coffin shut. But if you really want one absolutely awesome slugging match between your fleet and the AI, choose someplace like Empress Teta or Corellia and raid the planet out from under the AI. When your fleet arrives to clear the skies the enemy fleet will blitz your position. I've done that a couple times when I didn't care about losses (older ships) or just wanted to be able to sit back and watch the light show.