Honestly I don't think you're looking at the 'big picture'. And in no way should you be paying legal fees, doing any of the fence work and they must cover the cost of any damage etc done to your garden during the changes.
Ignore being neighbours etc.
You are giving up some of your land, now it doesn't sound like much but it is part of the value of your property if you ever were to sell it (hence the mortgage revalue). It will devalue your house value selling it, not to mention the 'extension' will likely make your house less desirable if it is overlooked or blocks out the light etc.
Then you've got noise issues which to be fair you'd have whether he was coming onto your land or not but there are regulations they need to follow for this too
I don't know what they house prices are in derby but round this way £1500 wouldn't get much. I also know of someone who got 10k just letting someone put the scaffolding on their side of boundary because the neighbour didn't want the extension.
Look at it more objectively and you'll likely agree that you are undervaluing the piece of land the neighbour wants, like several of us have said earlier, he could have easily kept on his side of the boundary but decided he wants some of YOUR land to increase the value of his home while devaluing your property.
If you didn't sell it to them (still finding it hard to believe that planning got passed without actually owning said land...) they would have to go through planning permissions, architect etc again. Those of us saying 5-10k are not being greedy, we're looking at the larger long term impact on your propery.