Thirty one per cent of people paying a mortgage or rent spend more than a third of their disposable income each month doing so, a survey for BBC Panorama suggests.
Housing charity Shelter said that makes mortgage or rent costs 'unaffordable'. It said family budgets are being put under 'enormous pressure' because of a 'shortage of affordable homes'.
The survey of 1,003 people also suggests 46% of people think property prices are too high in their area.
According to Shelter and the Joseph Rowntree Foundation, spending more than a third of your disposable income on rent or a mortgage means you may not be able to afford other basic needs.
Shelter chief executive Campbell Robb said: 'The widely accepted test of affordability is that housing costs should take up no more than a third of your income.
But in reality, many families don't have any option but to pay out much more,' he said. 'This sees some faced with impossible choices every day - including between putting enough food on the table or paying for the roof over their head.