Larry Hagman's 'Dallas' Costars Recall His 'Silent Sundays' When He'd 'Whistle' to Communicate (Exclusive)

When the cast of Dallas, including Patrick Duffy, Linda Gray, Joan Van Ark, Charlene Tilton, Audrey Landers, and Steve Kanaly, reunited this week at Oscar's cabaret venue in Palm Springs, one person was not there — and yet his spirit loomed large.

Larry Hagman, who played J.R. Ewing on Dallas for 14 seasons before he died in 2012 at the age of 81, was on the collective mind of the cast.

"Oh, his character, J.R. was such an asshole," Gray, who played Hagman's wife Sue Ellen, exclusively tells PEOPLE. "But Larry was wonderful."

"That's what made Dallas a great show," says Tilton, who played Hagman's niece, Lucy. "Dallas was, to the best of my knowledge, really the first show that gave you characters that were all truly flawed, But they were loved by the audience."

“Oh he’s here,” says Duffy, who played Hagman’s brother Bobby on the series. Duffy, 74, says when he first met Hagman ahead of filming the pilot episode in 1978, they shook hands — but it didn't feel like it was the first time. 

"I went home and told my wife. I said, ‘I think I just met my best friend.’ Oh, that was at the table reading of Dallas. It’s the same way I felt when I met my wife. And same way I felt when I met the lady I'm with now. There was an unspoken history," he says. “So who am I to say when that started? So I don't care. I participate in the moment. So when I shook Larry's hand, it was like, ‘Oh s---, this is not the first time.'"

"I'm still connected to Haggy,” Duffy adds.  “Absolutely. I think of him and I start to giggle. Haggy was my best friend — probably many times."

Chiming in, Van Ark, 79, says: “He did several of the Knots Landing episodes too. He brought all that energy and that passion that he has and the jokester, all the levels Larry brought, oh my God. The jokes with him on a set... I think that's the happiest place in the world. And the silent Sundays where he wouldn't talk. Oh boy.”

“He wouldn't talk to anybody on Sundays,” Duffy remembers. 

“He would whistle!” Gray, 82, says. “To get your your attention, he would whistle.”

Gray had to deal with this eccentricity over the years, as they were close friends off set and traveled together with their spouses. 

“We would travel to Europe and every Sunday he wouldn't speak. He would whistle. Whistle. His wife would show the menu and point and all that. And he would whistle what he wanted," she recalls.

Was this vocal rest?

“No, it's bullshit. Bullshit!” Gray says with a laugh. “He did this for years and years. He was like a little boy. But it was also control. We’d be like, ‘Can we order this for you?’ And one day, we were in Germany, I said, ‘I'm done. I'm not not playing this anymore.’ And he was like a four year old. He was just like, ‘Oh, somebody caught me.’” 

Landers, who played Hagman's lover Afton, remembers a different, but still playful side of Hagman.

“For my first episode, I was coming in for only two episodes... at Lucy's wedding,” she says, referring to Tilton’s character. “And the first scene was in bed with J.R. So I walk out on the set and I have this slip-on, covering my body, my shoulders exposed. Larry climbs in wearing jeans, cowboy boots.”

She pauses as her former castmates laugh. “But yeah, shirtless," she says.

“So we’re under the covers," Landers continues. "There’s a champagne bucket next to the bed. I had never met anyone, even Larry. The director says, 'Action!' Larry reached over, grabbed a handful of ice from the bucket, and puts it down my back. The camera is rolling. And I'm there. I was taught never to stop until the director says, ‘cut’ and never break concentration. And I am like, ‘Oh, do I say my first line? Oh my God.’ And I'm trying not to scream. And there's like, this ice dripping down. And then, of course, everybody was in on the joke. So the whole crew starts laughing and the director says, ‘cut.’"

Landers concludes. “And then Larry says, ‘See , that's how we break the ice in Dallas.’”

Dallas ran for 14 seasons from 1978 to 1991, earning four Emmys along the way. The show followed the wealthy Ewing family and the drama that surrounded their booming oil business and their entangled personal lives.

In 2012, the show inspired a revival on TNT that ran for three seasons. Fan-favorite characters returned, including Duffy's Bobby, Gray's Sue Ellen and Hagman's J.R. Sadly, after season 1 aired, Hagman died at age 81 that November.

“Larry Hagman was my best friend for 35 years. He was the Pied Piper of life and brought joy to everyone he knew,” Gray, 82, said in a statement released to PEOPLE at the time. “He was creative, generous, funny, loving and talented and I will miss him enormously. He was an original and lived life to the fullest. The world was a brighter place because of Larry.”

Much of the cast — including Hagman, Principal, Duffy, Gray, Kanaly and Tilton — reunited back in 2004 for a TV special titled Dallas Reunion: The Return to Southfork, referring to the fictional Texas land where the primetime soap opera took place. Their latest reunion took place at Oscar's in Palm Springs, California.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

All 13 original seasons of Dallas can be streamed on Amazon Freevee.