The former Labour cabinet minister Charles Clarke argues that the Labour party needs to be more open to new ideas and diverse opinions if it is to revive its fortunes.

IPPR has done well to mark the 20th anniversary of the final edition of Marxism Today with a special edition of its quarterly journal PPR. Marxism Today, founded in 1978, played an important role in the evolution of centre-left thinking through the 1980s until 1991, when it gave up the ghost – significantly, two years after the fall of the Berlin Wall and the type of Soviet communism which had constructed it.

Marxism Today promoted diversity of thought. It resisted unequivocally the centralised and conservative thinking which had dominated much of the left both in the Communist party, which was Marxism Today's main concern, and in the wider labour movement, notably the trade unions and the Labour party.

This diversity was ideological in that it challenged the narrow and inflexible analyses of economic class which had tended to dominate the thinking of many 'left' groups. It acknowledged that there were large parts of the population which could not be easily reduced into a crude class analysis, but which had legitimate hope and aspirations which progressive politics ought to be supporting and encouraging. Of these, probably the most important group was women, who were very much underrepresented in traditional left thinking, but also left out were a wide range of different minority ethnic groups, small business people, faiths, communities and sexual orientations.

Each of these had their different forms of organisation, and Marxism Today sought to give expression to them and to promote discussion of the ways in which the diversity of the population might be reflected in political action which could build a more tolerant and cohesive society.

Marxism Today also recognised the importance of politics in culture, ranging from film, theatre and books to fashion, music and cooking. Here again it recognised the importance of diversity.

The common ground with 'Thatcherism', which it sought to identify and analyse, was the recognition that the individual, with individual economic and social characteristics, was becoming increasingly important, alongside the decline of the traditional collective, whether in economic production, cultural experience, political activity or even church worship.

This was in fact a tough argument to make in labour movement circles, bringing as it did the harsh light of reality into many comfortable, cosy and mistaken corners which dominated left politics.

Their most important target was the world communist movement, riven as it was between the 'Euro-communism' of Italy and Spain and the hard-line Soviet version which controlled Russia and eastern Europe until the rise of Mikhail Gorbachev and the collapse of the Soviet empire. Ronald Reagan and Margaret Thatcher always tried to give their hawkish militarism the credit for that largely peaceful transformation, whereas I always felt that more credit should have gone to the d?(C)tente of the Helsinki process, the Ostpolitik represented most of all by West German chancellor Willy Brandt, and the internal ideological challenge from Marxism Today and others.

But there was a massive impact upon Labour too. Following the 1979 general election defeat, the Labour party was full of 'Labour sectarians' who, in their different way, saw economic class as the main political identifier and eschewed the need for wider political alliances and activity. (To give a flavour: in that era we had a serious argument about whether a market stallholder on the Ridley Road in Hackney was entitled to join the Labour party, since he had no employer and so was not a trade union member.)

Notable among the sectarians were the vituperative Bennite shock troops, who in the early 1980s sought by a variety of means to intimidate their opponents, and the Trotskyite Militant tendency, which was ruthless in its exploitation of any foothold it could get on power. Very many Labour activists were highly unsympathetic to the type of diverse and modern politics which Marxism Today was trying to foster, though parts of Tribune and the newly forming Labour Co-ordinating Committee were trying to promote those values and took strength from Marxism Today analyses.

Throughout the 1983 Labour leadership election, Neil Kinnock was clear that his leadership had to embrace diversity. He therefore spoke, before he was formally elected leader, together with Eric Hobsbawm at a Labour party conference fringe meeting entitled 'The Forward March of Labour', reflecting Hobsbawm's important article in Marxism Today a couple of years earlier. An interesting illustration of putting this approach into practice was the political means used to weaken, isolate and then defeat the Militant tendency in Liverpool, where the Labour leadership sought to promote and support the range of groups which reflected this diversity, such as housing cooperatives, ethnic minority organisations, women's campaign groups and the churches. This was the backdrop to Neil Kinnock's famous attack on Militant at the 1985 Labour party conference.

In fact, the dominating political issues of the era, notably the 1984/85 miners' strike, which polarised Labour opinion deeply, and then the imminent general election, made it politically very difficult to open up Labour thinking more quickly after 1983. It is no coincidence that IPPR, itself a major means of promoting more diverse thinking, was only formed in 1988, after the 1987 general election defeat, and even then was very strongly opposed by influential old Labour diehards like John Prescott.

The last issue of Marxism Today was published six years before New Labour came to power. Its intervention helped to create the more fluid and flexible political climate in which Labour could change itself and build the alliances through which electoral victory came.

Something similar is needed now. Recently in Liverpool I reflected on the similarity between the now emergent 'Blue Labour' ideology and that of the Militant tendency. Though Blue Labour thankfully lacks the Trotskyism and the nastiness, it shares a reductionist view which sees social class as the only important indicator, is thoroughly romantic in its political vision, and utterly reactionary in its policy prescriptions.

Labour needs a lot better than that.

We need to start with a serious assessment of the experience of the Labour government of 1997–2010. What did we do right – what did we do wrong? To what extent were our failures brought about because we had to deal with unpredicted events, or because our prescriptions, policies and implementation were flawed or even plain wrong? I think that the Labour record in all fields will stand up to such scrutiny better than many (including in the Labour leadership) think, but we need to carry out the analysis.

On that basis we need to understand and then explain why Labour lost the 2010 general election. Certainly we needed to give a better reason for voting Labour than that we were not the Conservatives.

These conclusions should lead us to a clearer view of how Labour should oppose now. It is not enough, when confronted with George Osborne's disastrous autumn statement or David Cameron's suicidal European veto, simply to say that we are against. We have to say what we would have done were we in power. Not necessarily in minute detail but certainly in broad brush strokes. It is wrong to believe that simple oppositionism will bring Labour back to power – the notion of an inevitable electoral pendulum is a dangerous delusion.

And finally we have to set out our own view of what a social democratic government really would look like in the modern era, within the real constraints of modern times. This is not at all easy, with the world changing so very quickly and the global environment so difficult to influence.

Labour is understandably terrified of division. But if there is no division, there will be no argument, no discussion, no thought. And that would be truly fatal.

The issue is how discussion is conducted – and this is where Marxism Today provides a template. We need something like it now to allow us to address these challenges in an open way which strengthens Labour and its electoral prospects.

Charles Clarke was MP for Norwich South from 1997 until 2010 and served as home secretary from December 2004 to May 2006.