I speak regarding pennies mainly, but Gouby published his book (The British Bronze Penny from 1860 to 1970) back in 1986 which did contain a lot of varieties anyway, but more and more come to light and seem to pick up more popularity when someone publishes another book with a mention to it. He did another study which looked mainly at bronze between 1860 to 1869 where he would have added previously unrecorded varieties and then an even more comprehensive version entitled "The British Bronze Penny (Victoria 1860 to 1901) Specialised edition" which many other varieties and minor attributes where mentioned. Then you have Michael Freeman's book which followed the same sort of path with more up to date versions being released, I think the last being undertaken by Spink but they only updated the prices (I think). Then the even more granular book by David Groom but this concerns 20th Century pieces, however it goes into some great detail about varieties and minor changes to designs that even the former 2 missed. You also have places like the "unconfirmed varieties" threads on this forum, someone recently had their work recognised in their area of collecting as a previously unrecorded variety and is awaiting publishing. To answer your first question, I think that it's more of a thing now because as people near completion of their collection they want to be able to create more boxes to tick and to carry on the collecting. Varieties give you the chance to do that, and as numismatics gains more popularity and more coins are being hoarded, people are starting to see more and more differences in design and in turn, open up more varieties for the collectors.