Only a few months ago, in the spring, there was a sense of giddy possibility about the Greens. Zack Polanski’s radical new leadership felt fresh and dynamic, especially compared with those of the other parties. Former Labour supporters, in particular, were switching to the Greens in huge numbers, Green candidates were winning across the country, and the party was second behind Reform UK in some national polls.

Like Reform, the Green party presented itself as a revolt against a failed status quo. Unlike Reform, the Greens’ urgent stance on the climate crisis gave them a potent moral authority: even many people who did not support them believed that the Greens were right about the most universal issue.

After an unprecedentedly hot summer, some of that authority and momentum remains. Last weekend, at the party’s bustling conference in Brighton, impressive-sounding statistics about the party’s growth this year – along with modern icons of Green success, such as Hannah Spencer, victor of February’s Gorton and Denton byelection – were presented to determinedly cheering delegates in crammed halls.

This week, in the high-profile London constituency of Holborn and St Pancras, where Polanski hopes to succeed Keir Starmer in Thursday’s byelection, the usual cheery squads of Green activists have been striding back and forth between wealthy streets and council estates, distributing thick bundles of a party newspaper with the front-page headline “Greens close in!”. Given the party’s recent election record, Polanski’s populist appeal and the possibility that some of the many local leftwing voters may see the contest as a low-risk way to warn Andy Burnham not to follow Starmer’s premiership rightwards, a Green win is not impossible.

But, even if it happens, the problems the party has accumulated since the spring will not have gone away. For some voters, the novelty of Polanski’s leadership has faded. For others his style, more radical and provocative than previous Green leaders, has allowed the party to shift to extreme positions, particularly on Israel, with the Brighton conference passing a motion (which the Jewish Polanski neither supported nor opposed) that “Zionism is racism”.

Meanwhile, Burnham’s confident, sometimes iconoclastic start as prime minister has removed Polanski’s unique selling point as a good communicator who does not accept the usually conservative terms of British political debate. In his conference speech, Polanski admitted: “Now I get asked, isn’t your job done?”The idea that one of the Greens’ main purposes is to shift Labour leftwards and towards bolder climate policies, and that that approach is yielding results, was everywhere at Brighton, from mutually encouraging chats between activists in the queues to upbeat contributions from the main stage.

It’s not a role that should be dismissed as minor. The Greens’ influence over Labour – both undermining Starmer in record time by taking so many votes from his party and creating political space for Burnham’s coup and governmental refresh – has arguably been a sign of Britain belatedly catching up with other European democracies, such as Spain, where centre-left parties have long been forced to be bolder by more leftwing ones. After decades of Labour paying most attention to forces to its right, the Greens’ important place in Labour’s calculations makes a refreshing change.

And yet, being an influence rather than a definite national rival – the Greens are now about 20 points behind Labour in some polls – is undeniably a comedown. During Polanski’s first months as leader, he regularly said he wanted the Greens to “replace” Labour. Partly because of that apparent possibility, the Greens attracted respected leftwing strategists such as Michael Chessum and James Meadway, who had worked under Jeremy Corbyn’s leadership and became disillusioned by Labour under Starmer.

At some leftwing gatherings last spring, those who hadn’t switched to the Greens could feel a bit over-cautious and old-fashioned. It’s possible that the rightwing media even encouraged this shift, giving Polanski plenty of coverage, in order to divide the left and prevent Labour from ever holding sole power again.

With Labour’s electoral future now looking more promising, one of the Greens’ leaflets in Holborn and St Pancras modestly presents Polanski as “a strong voice … to keep Labour in check”. In his Brighton speech, Polanski said: “Vote Green and watch Labour move.”It’s a typically pithy and clever line: explaining the mechanics of politics to voters, and treating them like grownups, in ways that party leaders rarely do.

But it may not be an inspiring line for more tribal Greens who decided decades ago that Labour, and alliances with Labour, were not for them. Another Green conference theme was “Real hope. Real change.”It sounded defensive, and too similar to Labour’s conference theme the previous week: “Hope again”.

The Greens do still hold the initiative on some issues: advocating rent controls, wealth taxes, an end to fiscal conservatism, and more radical climate policies, with Labour expected soon to allow a limited expansion of oil and gas drilling in the North Sea. There are also signs of the Greens positioning themselves as the party of the most disadvantaged, such as asylum seekers, benefits claimants and transgender people – groups that Burnham’s less judgmental government may treat better than Starmer’s did, yet probably within limits set by Labour’s considerable number of socially conservative supporters.

“We will always stand with … everyone who is under attack,” said Polanski in Brighton, a statement that could be seen as too vague, impractical and idealistic. Yet if this country is to become a more truly representative democracy – a possible upside of the current political fragmentation – why shouldn’t the most vulnerable effectively have their own party? The most privileged have had the Tories for centuries.

The astute political analyst Rob Ford suggests that the Greens’ poll decline “may not matter much” if the party still campaigns effectively and attracts strong support from the dozens of relatively youthful, highly educated and diverse urban constituencies. Thursday’s byelection will test that theory.

Yet either way, there is something disturbing about a party with such a crucial cause having to face a reduction in status. The Greens’ drift back towards the political margins reflects not just their own mistakes, but the reluctance of many Britons to face up to climate realities. As with Extinction Rebellion, it’s more comfortable to call them extremists and ignore what they say.

Andy Beckett is a Guardian columnist Do you have an opinion on the issues raised in this article? If you would like to submit a response of up to 300 words by email to be considered for publication in our letters section, please click here.