The enemies of liberalism are on the march. The challenge is not just to reinvigorate the Liberal Democrat party, says David Hall-Matthews, but for liberals of all parties to rediscover the core values of pluralism and decentralisation. They must cooperate to fight the oncoming battles and to promote active citizenship and political engagement.

In his resignation speech, Nick Clegg set out a stark warning for liberals – not just those in the UK, but across Europe and beyond. Liberalism is under threat, he argued, losing arguments and elections just at the time when it is most needed. The forces of small-mindedness, petty nationalism, populism, authoritarianism, violence, irrationality, apathy and the politics of 'fear and grievance' are all on the march, to varying degrees, around the world. Clegg ascribed this in part to economic uncertainty – both the long aftermath of the financial crash, and the longer-term impact of globalisation on people's sense of community.

This implies a threat to liberalism itself, at the philosophical level. Liberalism does not offer certainty – in many ways it celebrates uncertainty, messiness, diversity of thought and aspiration. Has that now been rejected? UK voters may have yearned for the illusory clarity of single-party government, despite many commentators conceding that the 2010–2015 Tory–Lib Dem coalition was more stable than any had predicted it would be. Or could it be worse than that? The electorate may have spurned the very idea of elite liberal rationality: of weighing up the evidence, compromising, trying to do the decent thing.

Is liberalism under threat?

Politicians of all major parties must hope that liberalism does not face an existential threat. But do they believe it does? The rise of Ukip, the populism of which rests on a cavalier disregard for facts (as though 'facts' are a weapon of the establishment), is unmistakeably a challenge to the liberal order, although it already appears to be on the wane. Globally, Putin's neo-dictatorship and ISIS terror are fundamentally illiberal – but they are no more significant than recent liberal turns in international relations, such as the increasing economic strength and political integration of the BRICs.

In the UK context, is the astonishing success of the Scottish Nationalist party (SNP), with its broadly social-democratic approach, really a threat to liberal values? For Clegg, having fought a centrist, makeweight campaign, all radicals are a threat. He went as far as to cite 'unity' in his speech as a fundamental liberal value, though it could be argued to be the opposite of liberal respect for difference. Ed Miliband, too, found himself forced to decry the SNP as a nationalist danger, primarily for tactical reasons. Both ultimately found it difficult to convince floating voters that their differences from the SNP were greater than their common values.

This points to a broader philosophical challenge for liberalism. It is too easy to define it negatively – as the opposite of all the threats listed above. How should 21st-century liberalism be defined positively? Do its multiple strands cohere around a philosophical outlook, or do divided interpretations present a significant hurdle? Within the Liberal Democrats – as well as among those Labour, Conservative, Green and, yes, SNP MPs who see themselves as liberally minded – there are clear divisions. Different groups of liberals are keen on different issues: economic liberalism, social liberalism, civil liberties, community politics, political reform and the dilution and devolution of power – not to mention longstanding liberal causes such as internationalism and environmentalism.

After the general election debacle, and with a Lib Dem leadership campaign underway, there is an opportunity, as well as a necessity, to set out a clear, positive liberal vision for society. For Tim Farron and Norman Lamb, the priorities will be, first, to reinvigorate disconsolate members and activists, and second, to inspire new supporters to rally to the cause. With a 25 per cent increase in party membership in the first week after polling day, there may be an opportunity, based on a sense of injustice (or perhaps regret) at the election result, to catch the tide and build renewed momentum. Ironically, after such a crushing defeat, nothing would suit the Lib Dems better than a quick by-election in a Tory-facing seat – such as the one that would follow were Zac Goldsmith to stick to his promise to resign in Richmond Park if the government announces the expansion of Heathrow.

However, there is also a much harder challenge to face. No amount of by-election victories will disguise the overwhelming loss of voice and influence that Liberal Democrats have suffered in the House of Commons – and, by extension, in the media. What positive vision of liberalism will inspire people, in all parties and in none, to win liberal battles in the coming years?

What binds liberals together?

What do liberals hold most dear? Which core philosophies do they all share, and which of their beliefs and preferences conflict with each other? Which can be traded in order to persuade other parties to adopt others? Indeed, is the spirit of deal-making compromise and moderation a liberal philosophy in itself? Many Lib Dem activists – particularly those who see the party as naturally left-of-centre – take great exception to the notion of liberalism as a split-the-difference philosophy with the aim of offending no one. They reject the tactical logic behind the party's election campaign, with its appeal to the reasonable centre ground and its manifesto designed as a template for possible coalition negotiations, because the number of voters likely to vote actively for compromise was never large. More importantly, they reject the principle of centrism as weak – as allowing the party to be defined only in relation to the changing position of others. However, such antagonism towards compromise is ironic for a party that believes in political pluralism. Proportional representation – which is supported by almost all liberals – would necessitate government by negotiation. (Whether one necessarily has to occupy the centre ground to negotiate effectively is a question for another day – or perhaps for another party).

This internal division within liberal philosophy is telling. First, it suggests that a battle for the soul of liberalism (as opposed to Liberal Democrat strategy) cannot be focused on the question of where it fits on a left–right spectrum. Second, it reveals that liberals do not feel the need to agree with each other all the time. Just as they prefer a model of government in which different parties battle for influence but then come together to thrash out effective policies, liberals bring individual ideas forward for debate. They actively want to argue before they agree. The preamble to the Lib Dem constitution declares that no one should be 'enslaved by " conformity'. This does not make them easy to manage (Paddy Ashdown was once applauded by a conference for telling activists that leading them was 'like herding cats'), and also explains why so many Liberal Democrats were willing to go into coalition with the Conservatives but were dismayed that anyone (inside or outside the party) should think that this implied they agreed with any Conservative policies at all.

Asking liberals what they do stand for invariably produces a range of responses. A single liberal will say that they stand for several things, as some recent, high-profile examples illustrate. Nick Clegg's last conference speech, in Liverpool in March 2015, set out a liberal vision (notably at the expense of campaign messaging), based on the four strands of freedom, tolerance, democracy and internationalism; in his resignation speech, he declared liberalism to be all about opportunity, liberty, fairness and sustainability. Tim Farron's leadership campaign is centred on 'optimism', while Norman Lamb's is focused on tackling discrimination.

These concepts overlap and interlink, of course, but they can also point in different directions depending on which takes precedence, or on how they are applied to real-world situations. The theme that most clearly links disparate liberals is freedom, which can be associated, equally, with policies to enable individual empowerment; checks on concentrations of power (including, but not limited to, that of the state); and the removal of restrictions on the movement of people or goods. Similarly, most liberals share a sense of decency, based on belief in the essentially good nature of humankind, but also on the responsibility of the state to protect the vulnerable from discrimination and abuse. For some this is restricted to promoting freedom of conscience and choice in personal matters, such as sexuality; for others it involves the positive promotion of human rights and active citizenship.

Fairness is a third binding liberal principle, but is perhaps the one open to the greatest variety of interpretation. For social liberals, following in the tradition of William Beveridge, the raison d'etre of the state is to address the 'five giants' (want, squalor, idleness, disease and ignorance) that prevent people from making their own way in life. But for others, fairness could merely refer to fair-mindedness and respect for others' opinions, or, more narrowly, to due process – including fairness in the electoral system.

What does liberalism offer?

The diversity of approaches that are possible under the banner of liberalism has always been both a strength and a weakness. Liberals have always eschewed identity politics, that of class as much as that of nationality. Refusing to build and promote the interests of specific groups has meant that liberal support is inherently transitory. (Before the Lib Dems in 2010–2015, has a party of government ever adopted policies to upset so many groups, such as students and public sector workers, inclined to vote en masse for it?) However, it has also enabled liberalism to appeal to voters on multiple fronts. It could be argued that, once in national coalition, the risks of taking different lines in different regions have come home to roost. Yet the fundamental flexibility of its philosophy suggests that liberalism may again now best survive by applying itself in different ways to different issues.

Liberalism as a philosophy starts with individuals and how they work together, and with a mistrust of the centralisation of power (whether in the hands of the state, or, for example, of service providers). Its strongest move now will therefore be to address the concerns of those who feel that they have lost a sense of their local community. While 'hope' versus 'fear' is a crude expression of liberal values, liberalism can make a distinct, decent offer to those who have been tempted by Ukip, or who have not voted in recent years. Moreover, this approach has been tried and tested, starting at the last time that liberalism was at a low ebb as a national force in the early 1970s. Community politics was not conceived to promote liberalism per se, still less the Liberal party. Nor was it about the redistribution of power – which, perhaps ironically, can only occur as a top-down process (unless it's going in the wrong direction). Rather, it was the practical implementation of the liberal principle of starting from the bottom and letting people take control of their own lives.

This may undoubtedly help liberal politics in other ways, by generating new voters, activists and candidates, although the original instigators of community politics saw that as a by-product at best. On the one hand, people are unlikely to feel empowered by anything that looks like a recruitment drive; on the other, there is no guarantee that newly inspired community activists will share other aspects of liberal philosophy. Indeed, it is both possible and desirable for other parties to re-engage communities in this way too, especially in areas that feel disenfranchised (for example, by virtue of being safe seats) or that have experienced significant population turnover.

Relatedly, liberalism has a great deal to offer through direct political engagement and active citizenship. These efforts not only act as a prophylactic against being taken for granted or ignored but, more importantly, are means of helping individuals and communities to have a say over their own living spaces, workplaces and local environments. Once again, democratic re-engagement – and electoral reform – will not be achieved in the near future as a direct result of Liberal Democrat influence, except at grassroots levels. Nonetheless, liberals in all parties have an opportunity to capitalise on the manifest iniquities of first-past-the-post. It has not escaped even the right-wing press that the biggest beneficiaries (in terms of seats per votes cast) have been the SNP and DUP, with the Greens and Ukip its biggest losers.

The cause of proportional representation will be helped by the fact that it is no longer being promoted in the perceived self-interest of one party, and that it better reflects the new reality of multiparty politics. For liberals, proportionality is also the answer to the West Lothian question that the Tories have unwisely reopened. If there is to be any tinkering with which MPs have jurisdiction over which areas, it is logical to extend that to regional devolution within England, given that there is now likely to be further devolution within Scotland and that there is no public appetite for an English parliament.

How can liberalism be influential?

In all of the above areas, liberalism can provide positive answers to populism, apathy and narrow sectionalism. Liberal Democrats will articulate the case for them, and potentially stand to benefit from doing so, but they can do neither alone. Liberalism can only continue to thrive, in the face of both uncertainty and the exaggerated certainties of the right, through cross-party alliances. Whatever one's views on the Lib Dems' success in maintaining a stable coalition government, they manifestly failed to change the adversarial nature of politics. There was no attempt to govern by consensus, least of all by demonstrating the kind of respect for opposition parties (especially minor ones) that they will now rely on.

Nonetheless, there are liberals within all parties who share both a desire for greater mutual respect and concerns about the politics of fear and exclusion. It is hard to see them coalescing in practical terms around positive agendas of change in the foreseeable future – not least because all other parties also contain distinctly illiberal elements, as well as because the desire for post-tribal politics appears limited. It will also be tough to interest the media in alternatives to populism, and in localism tougher still. However, there are big opportunities for liberalism as negatively defined. It is probably no coincidence that the current surge in Liberal Democrat membership followed Clegg's warnings of multiple threats to liberalism. Whether the danger is real or imagined, it can certainly be turned to liberalism's advantage.

If the Conservatives stick to their manifesto, liberals will be at the forefront of many coming debates – from state surveillance to membership of the European Union. While some Lib Dems may struggle to argue that they now oppose more cuts (except those they can demonstrate that they stopped when in government), social liberals (and social democrats) across all parties can unite to oppose further attacks on welfare – and, concomitantly, to oppose wasting billions on replacing Trident.

Importantly, such a cross-party coalition on liberal issues will have to include the SNP. With some exceptions (and a lot of unknown individuals newly elected), the SNP has been on the right side of many of these arguments. But how can the building of such an alliance be reconciled with Lib Dem and Labour antipathy towards independence? Can petty nationalists be liberals, when petty nationalism is one of the stated threats to liberalism? Perhaps petty nationalism is not as big a problem as partisan rhetoric claims it is – Ukip support, for example, comes as much from disenfranchised communities responding to a populist critique of elite power as it does from an appeal to patriotism. There is much more to the SNP than independence, too – their manifesto had a lot to say to liberals (evidenced by the fact that many English people would have voted for them if they could, even though they did not want to see the break-up of the UK). Liberalism has a long history of supporting geographical communities that feel distant from political power – including in Scotland. It remains to be seen whether Liberal Democrats can reconcile themselves to that aspect of the nascent liberal resurgence, or whether they will let it pass them by.

David Hall-Matthews is a governance consultant, campaigner and analyst. He is a former senior policy adviser to the Liberal Democrats, and was chair of the Social Liberal Forum between 2009 and 2012.

This article appears in edition 22.1 of Juncture, IPPR's quarterly journal of politics and ideas, published by Wiley.