AMD developers, etc. have mentioned on twitter/facebook and so on that tessellation is one area that will see a substantial performance increase... so I don't think its so nuts to suggest it - even if it doesn't make sense given the useage that doesn't mean AMD won't do it.
Got any links to statements? Not questioning you, just wondering if they don't mean generally in the next year/two, or maybe drivers, mostly guys who do twitter type stuff for AMD are driver guys. I've said before though, their tesselator being a single block is pretty easy to massively increase the power for a very small die size increase, still think its nuts to waste a 15% die size bump on tesselation when 28nm is next year and frankly will offer, what, similar transistor count to now at maybe 30-35% of the current size, IE 28nm is a two process jump forward, and to glofo, to a significantly higher quality fab. Few people would upgrade to a 6870 if it was 5% faster in general but 50% faster in tesselation, in 3 games. While lots would upgrade if it was 35-40% faster, but didn't have great tesselation performance.
You say double the size, but your numbers are suggesting a 4x increase in size. A 225mm2 core, if it's a square will be 15mmx15mm. To double the size of the core, it'd be 450mm2 which would be 21mm x 21mm. Such an increase in size wouldn't take 100 yeilds per wafer down to 25, but rather more around the 70 per wafer region.
I have no idea why you think the physical length of each side not being double doesn't take up twice the space. Its a wafer, its not a long thin piece of silicon and you can increase the width as much as you want. Double the size and you'll more than half the amount of cores you get off a wafer as yield goes down at a non linear rate as core size goes up. If there were 100 potential cores at 225mm2, then at 450mm2 you'd have probably 44-47potential cores, a silicon wafer is circular, meaning squares don't fit in perfectly, the bigger the square the more space lost on the edge the bigger the cores get, in general. But thats before yields go down, if on the 225mm2, 100 potential cores, you only got 80 working cores, or a 80% yield, that yield would be substantially down if you doubled the core size, its all theoretical with the numbers I'm using because they arent' real, but the yield's will go down exponentially and at one point just become almost completely unworkable.
At 500mm2 (a bit more, 523 iirc) the GF100 is for all intents and purposes, unmanufacturable, it can't be made at a profit because yields are so painfully low. If AMD have gone from a 330-340mm2 core to a 400mm2, and Nvidia dropped down to a little under 400mm2, you can bet that above 400mm2 is where the yields start to drop very dramatically.
15mm side's up to a 21mm side, doesn't mean a 1/4 drop in cores per wafer, it will mean a noticeably over 50% drop in cores per wafer.
Draw a square, divide it into 15mm sided smaller squares, then divide it into 21mm sided squares, see how big the difference is
The same will happen the other way, drop the size of a 5870 by half, a 5770, and you'll get more than double the cores per wafer, because then out of 200 potential cores, the yield will be higher again, call it 90%, so you'd get 180 cores, vs 80 5870 cores, more than double.
I'll hold my hands up and admit that previously I've stated I wouldn't be surprised if the new cards offered 70% better performance. I base that on the transition from the 3870 to the 4870 which was on the same manufacturing process but looking back I think power and die size will be a limit the 6000 series gains.
The 5870 was designed for the 40nm process from the ground up, on an efficient process, the 3870 wasn't, its a cut down design from a 65nm part that was then changed into a 80nm part, it was on a brand new 55nm process, literally the first card on it, and it wasn't even close to optimised, they stripped out about 30% of the core before making the 4870, it had a 512bit internal ringbus which took up 20-25% of the die space. Basically the same core with a crossbar memory controller, designed for 55nm and coming out 6 months later would have been half the size of a 4870. Theres very little to strip out this time around.
The 5870 is a really quite ridiculously good design. The 480gtx is what 60% larger, and offers 10-20% performance improvement, the 5870 is already pretty damn efficient. The 3870, was not, for its core size it was quite horrible, while the 4870 was truly fantastic in terms of efficiency, 4870-5870 lost some of the die size/performance because they increased the size 10-15% to account for some, "issues", with the TSMC process.