Over the Rainbow — studio executives wanted to cut Judy Garland’s number from The Wizard of Oz

A cry of despair from the forgotten, rural heartland of America, beaten down by years of economic depression. A deep yearning for a technicolour fantasy land where “troubles melt like lemon drops”. The journey towards a strange old man in a tower who pretends to be more than he is . . .

The year was 1938, and MGM was making a movie of L Frank Baum’s 1900 book The Wizard of Oz. America needed cheering up as it struggled to haul itself from the Great Depression. War loomed in Europe.

Two Jewish American songwriters — both sons of immigrants — were commissioned by the studio to knock out a number for 16-year-old Judy Garland. Harold Arlen and Edgar “Yip” Harburg had already collaborated on hits such as “It’s Only a Paper Moon” and “Lydia the Tattooed Lady”. Harburg was best known for the song “Brother Can You Spare a Dime?”, which brilliantly captured the human cost of the Depression.

“I grew up in the slums,” Harburg once said. “I know what it is to have your father come home from the sweatshop after working 12 hours a day.” So he didn’t want to write “a maudlin lyric of a guy begging. I made it into a commentary. It was about the fellow who works, the fellow who builds, who makes railroads and houses — and he’s left empty-handed. This is a man proud of what he has done but bewildered that his country with its dream could do this to him.”

Harburg saw the story of The Wizard of Oz as a parable about what might be achieved under President Franklin D Roosevelt, if his countrymen rediscovered their hearts, brains and courage and worked together to build a brighter future. He had a passionate belief in the power of music to win hearts and minds. “Words make you think a thought,” he said. “Music makes you feel a feeling. A song makes you feel a thought.”

Arlen dreamt up the melody of “Over the Rainbow” — opening with a bold leap straight up an octave from Middle C — while parked outside Hollywood’s famous Schwab’s Pharmacy. Harburg initially thought the tune too schmaltzy. But he took it to his old classmate, Ira Gershwin, who thought it would work if they picked up the tempo. The sweetness was offset by darker underlying chords.

Garland delivered the song five minutes into the film, yearning to escape the monochrome of Kansas into the vibrant colours of Oz. Producer Louis B Mayer initially wanted to cut it because it slowed the action and his star sang it “in a barnyard”.

There were also concerns that they wouldn’t shift sheet music because that opening octave hop would put the song beyond the range of the average Joe. He was talked around. The song won an Oscar, ultimately voted the best ever song in a movie and the greatest song of the century by the Recording Industry Association of America in 2001. It became Garland’s signature song, its optimism increasingly painful as her life imploded and her voice cracked. She gave a hollow laugh as she sang it at her final concert in Copenhagen in 1969, three months before her fatal overdose aged just 47.

Often described as “America’s Song”, if not its anthem, “Over the Rainbow” has been covered by every jazz singer going since 1939 — Gene Wilder recently died listening to Ella Fitzgerald’s version — as well as pop, rock, classical, folk and dance acts. David Bowie reworked it as the chorus of 1972’s “Starman”, yearning to escape the sexual restrictions of Kansas-grey England. Jazz pianists Keith Jarrett and Dave Brubeck have both recorded classy versions.

More recently, Hawaiian singer Israel Kamakawiwoʻole had a hit with his 1993 ukulele version, medleyed with “What a Wonderful World” and capturing the song’s naivety and melancholy with a lightness of touch that breezed from a morbidly obese frame. Complications from Kamakawiwoʻole’s weight would leave him dead, four years later, at just 38, but his version continued to chart across the world — in 2010 it reached number one in Germany.

In the 1980s the elderly Harburg conceded: “My generation unfortunately never succeeded in creating that rainbow world, so we can’t hand it down to you. But we could hand down our songs which still hang on to hope and laughter, in times of confusion like these.”