Urban renewal programmes that build mixed-tenure communities are vital to help deliver the new homes that Britain needs. However, our recent visit to two estates in west London underscored the need for these developments to consult and engage with current residents, as well as add to the stock of social rented and affordable homes.

Britain has failed to build enough homes for its people for nearly four decades. The consequences of this failure are acutely felt in London, where house prices are now around one-third higher than they were in 2008, at their pre-financial crisis peak. Rising numbers of Londoners live in expensive, often poor quality and overcrowded privately rented accommodation. How to solve the city's housing crisis will be the major question facing candidates at the forthcoming mayoral elections in 2016.

In the postwar period, local authorities were responsible for building the bulk of the new homes that the country needed. They lost that role in the 1980s, and housebuilding rates have never recovered. Today, local authorities are once again building social homes for rent, but borrowing constraints limit the number of homes they can supply. In London and elsewhere, they have to look to better use of their own land, agreements with developers and estate regeneration programmes to contribute significant volumes of new housebuilding – including homes for social rent.

In that context, IPPR recently published a volume of essays, edited by our chair Andrew Adonis and research fellow Bill Davies, on the concept of 'city villages' – major urban renewal programmes that utilise local authority land to build new, mixed-tenure communities and contribute to increased rates of social and affordable housebuilding. The report contained a number of different essays from contributors including renowned academics such as the late Peter Hall, the architect and urban theorist Richard Rogers, and local authority leaders, developers and housing experts.

Among the case studies cited in the report was one – the Earls Court development – that has proved controversial and deeply contested. Consequently, local residents and community organisers fighting against the development asked Andrew Adonis and myself to visit the area, which includes the Earls Court exhibition centres, major Transport for London works facilities, and two housing estates – West Kensington and Gibbs Green. This we did on 21 May.

The estates were built in the 1960s and 1970s (though they include a number of homes that are between 15 and 20 years old), and contain a mix of houses and flats. Some are housing association rented and some freehold owner-occupied, but the bulk of them are local authority homes for social rent. The estates are not architecturally distinctive, but the flats and houses, and wider public realm, are in good order, and a long way from the popular stereotype of brutalist, Clockwork Orange-style housing. They are scheduled for demolition as part of the overall master plan for development. Residents are to be rehoused, though they currently have no certainty about where or when. The council has entered into an options agreement with the developers, Capco, granting them an option to purchase the land. The newly elected Labour local authority is currently in negotiations with the developer about the scheme, which the local party opposed in its manifesto; it is therefore currently restrained legally in terms of what it can say or do.

The residents and their representatives told us that they oppose the development on a number of grounds. First and foremost, they want to protect the homes and the community they live in. The estates are not dilapidated or crime-ridden, so there is not the same support for redevelopment that we have sometimes found elsewhere. They argue that housing to meet local needs – not property development, much of which will be bought by foreign buyers – should come first. They suggest that infill could supply more housing, particularly family homes, within the existing estates. Their goal is to transfer the estates to community ownership, and they are mounting a case to that effect.

There are important lessons here that reinforce important aspects of the city villages concept. The first is that residents should be fully consulted and balloted on any proposals. Without such rights, people are effectively at the mercy of the decisions of others – whether local authorities or developers. The second is that city villages must always add to the stock of social and affordable homes – densification should not come at the expense of mixed communities or the rights of existing residents to be rehoused in suitable homes. Third, policymakers should further explore the potential of community ownership (and the adjacent concept of land trusts) to give tenants greater control and a real stake in their communities. The potential for local authorities to use their land and assets to drive a significant expansion in housebuilding and community renewal is clear, but it will only work with the support and consent of existing residents, not if it is against their wishes.