I care for 46 of these! - Don't worry only about 4/5 live in the house. The rest have come to me as either "feral" or "used to living outside". The vast majority of the "outside" cats actually live in the stable blocks or in the hay/straw stores - where as you can imagine they have ideal environment.
The cats have come from a variety of sources, the RSPCA have homed many with me, as have the Celia Hammond Trust. I tend to get the cats that are difficult to re-home, blind cats are almost impossible to home. Deaf cats are not much easier & the number of older cats that get thrown out is appalling. Also cats with disabilities like Chloe on the right are hard to home and many are out to sleep simply because they are unlucky enough to be on the "too hard to re-home" list.
Some of the little chaps are in a terrible state when they arrive and it can take a very long time for them to be healthy enough to be released down the stables to join the main "outside cat" gang