Meet Milt Larsen, The Man Behind Hollywood's Magic Castle

It’s 10:00 a.m. on a Wednesday, a very ordinary time to be anywhere. I’m standing in the middle of the Magic Castle, a very specific, bizarre place. It’s empty, covered in very ordinary, unflattering light with an exquisite stillness. No one is around, except for two carpenters in the dining hall. I let myself in through the back door, which was open, in hopes of finding Milt Larsen, the guy who created the Castle. But he’s nowhere in sight.

It takes me no less than fifteen minutes to discover I’m in the wrong building entirely. I follow a serpentine path out one building and into another next to the hotel (which, I later learn, is only related to the Castle in name), everything illuminated in that very ordinary light. When I finally find Milt Larsen, he’s sitting at a desk, in front of a green screen. Opposite his desk are three ventriloquist dummies, sitting there in frightening stillness the way dummies do.

I keep wanting to call Milt “Walt”, because he reminds me of Walt Disney were he were still alive. He’s got that cheeky mustache and that classic showman disposition we remember in Disney. He admits he’s always been a fan of Disney’s work, especially his ability to entertain with G-rated material. At 85, Larsen is rather spry and seems always up to sling one of his one-liners (“I was born in 1931; you can do your own math.”)

I sit between Milt and the dummies and can’t stop feeling that someone is watching me or that a secret hatch may open up beneath me, swallowing me into the haunted castle’s innards.

I’m here, ostensibly, to talk about the new theater in Santa Barbara opening under the Magic Castle umbrella. But I’m really here to get the whole story of Milt and his Hollywood landmark from his own mouth.

A Pasadena native, Larsen was raised in a family of magicians. During the Depression, his father was a successful attorney but decided to give that up to pursue magic. His mother and father founded the magic publication Genii “The Conjurer’s Magazine”, which is still going strong today. There were always magicians around in young Milt’s life, and if there was a magician performing in Southern California, he’d go see them with his family. They were fanatics.

When I ask him who his favorite magician no longer alive is, he explains, “When I was kid there was a guy called Dante the Magician. He was better known in Europe than he was here. He had a great stage show with scenery. He looked like a magician, with the white goatee. He was a super showman. At that point, Blackstone was probably better known, but Dante was king in my opinion. He was a good friend of my dad’s and had a ranch down in the San Fernando Valley by the Van Nuys airport. He bought two city blocks in the ‘20s. In his later years, he did OK,” he chortles.

When I ask him who favorite living magician is, he tells me he was a big fan of Siegried & Roy before their accident and “David Copperfield, of course, is a dear friend and a wonderful magician. My favorite was Lance Burton, as a person. When he’d perform, it was like the first time he’d even done it with his enthusiasm and exuberance, ‘Hey, I’m doing magic!’ David kind of yawns his way through these days.” Larsen also has little enthusiasm for video-driven magicians like David Blaine or others that rely on cheap editing or VFX tricks.

In the ‘50s, Milt and his brother Bill Jr. got jobs in the booming television industry. Milt, a songwriter and joke writer, wrote on the NBC show Truth and Consequences starring Bob Barker for 18 years. Bill Jr. was working at CBS. It was at the Truth and Consequences offices, where the Hollywood & Highland Center now sits, that Milt would look out at the building that would become the Magic Castle.

“It looked like a haunted house,” he explains, “like something out of the Charles Addams family or something. Obviously, it had to be haunted. I kept daydreaming about it. My dad had had the idea of doing some sort of club for magicians. But he died very early. Nothing ever happened with that. I found the house, talked to the owner. He had this old house and didn’t know exactly what to do with it. He bought it to get the hill. I did a deal with the property owner to lease the house and turn it into the Magic Castle. The Castle itself is the original mansion. People wonder why it can be so big on the inside but not on the outside. It’s a magic illusion.” The Castle opened in 1963. Since then, the Castle has had long patches where staying solvent was difficult. But since 2014, following a brief conflict over profit participation, their financials seem strong. Or, at the very least, stable enough for expansion.

Larsen tells me this is in part because magic is the hottest it’s been in a long time right now. I ask him why that is and he claims, “one of the reasons for the super-popularity now is simply the world we live in. We live in a fantasy world.” These are frightening times, and a bit of escapism here and there can be a powerful antidote. I also note that the magic world seems more accepting of women and minorities than it ever has - even though there’s by no means equal representation. Larsen tells me, “my mother was the first woman on television. At the San Francisco Exposition, the World’s Fair in 1939. She did her act on television. As far as we know, that’s the first lady to ever be on television. Then she had her own show, The Magic Lady on KTLA back in the ‘40s. 1948 I believe. That was a syndicated show. She was The Magic Lady in the hoop skirt with the fairy princess, you know?”

He tells me that the magic community recognizes the historic white maleness of the industry and is working on making things better. “Today, there are a lot of ladies. There are a lot of ladies out there doing it. Juliana Chen is great. At one time, it was strictly a man’s thing. At that time, magicians wore tails, top hat, and took a rabbit out of their hats,” he remembers.

If you’ve never visited the Castle, it’s a pleasantly claustrophobic space that operates on Alice In Wonderland logic. There’s a ghost piano (and a real pianist sitting somewhere out of sight), amongst many other fairly well-known bits of lore (again, think Disney levels of detail). There are endless other easter eggs lurking about, many of which we’ll probably never know.

The Castle has been host to many celebrities throughout the years. Milt tells me Bob Barker (94) still comes often for Sunday brunch. Others like Madame Wu, Johnny Carson, Orson Welles, and Cary Grant have been regulars over the decades. “Today we have members like Nic Cage and Jason Alexander. They like to come because they can hide here. They’re not going to be in the National Enquirer. Although one of the National Enquirer guys is a member, but he knows not to write about it,” Larsen chuckles.

Then he shows me an old, yellowing photo of Milt, Dick Sherman, and Cary Grant. Dick Sherman and Robert Sherman wrote musicals like Bedknobs and Broomsticks (which Milt makes a cameo in as well). Sherman and Milt still make parodic music to this day.

“That’s me with my hoaky mustache from the ‘70s,” Larsen laughs. “Cary was actually a very good magician. He started on vaudeville as a stilt-walker and doing magic. He loved magic. He just enjoyed it, you know. When I was just out of high school, I worked at a magic shop on Sunset Blvd. One day he came in and practically bought out the store. Little things, diamond petty tricks...kid stuff. He spent like $300 or $400 worth of stuff, which is a lot of money back then. He said, ‘I’ll send a car to come pick up the stuff.’ The manager for the store had gone out to lunch. He came back, I told him I sold a bunch of stuff to Cary Grant. He said, ‘YOU DID WHAT? Why didn’t you call me?’ I said, ‘How am I supposed to know where you’re eating lunch.’ He was pissed off at me.”

Larsen has volumes of anecdotes and jokes. He’s published a couple-of-books-worth, in fact. And he writes a weekly blog on his Friday commute from L.A. to Santa Barbara, where he and his wife live and where his new project is taking shape.

“For the new place we’re doing, we thought it would be nice to do something in the Santa Barbara area,” where Larsen has lived the last twenty-five years. “The Magic Castle is 25,000 square feet of bars, restaurant, theaters, the whole shot. So, for Santa Barbara, it’s 4,000 square feet. It’s tiny by comparison. It’s just going to be this cabaret theater. Basically we want to make a club for close-up magic. Not the big illusions. Just a little fifty-seat theater. The whole MC is built on close-up magicians. You know, intimate magic.” Renovations have just begun, and they hope to open up before year’s end.

Otherwise, Milt doesn’t perform very often these days, but he was never a big performer to begin with. He was more of a variety show performer (“I wish they would return!”) and comedian than a full-time magician. His last larger stint as a performer was opening for the Amazing Johnathan in Vegas. “The first time I performed with him was 9/11,” he recalls “That was our opening day in Vegas in 2001. That was the weirdest day. They shut down the strip. And, at that time, we were at the Golden Nugget downtown. And we figured if the terrorists are gonna bomb any place, they’re going to bomb the strip. Not the Golden Nugget.”

On my way out, I tell Larsen about the time I saw a young kid propose to his girlfriend at the Castle during dinner. He says that’s common. “One of our magicians is a pastor. I got married later in life. It was both of our first marriage. We were around 60. We worked together for a long time. She was the costume designer on Truth and Consequences. She did all my stage shows. We finally got married. We knew we were gonna get married. We didn’t want a big church wedding. So I decided I’d surprise my wife. We threw a party at my home,” under some false pretenses. But it hit a snag. Her family wanted to cancel (because he hadn’t told them what the real intention of the party was). So he had to reveal his ploy and spoil the surprise. His trick, as it turned out, was a dud.

But it all worked out in the end. “We’re happily married,” he beams. He was, after all, always a better storyteller than a magician. “I’m surprised we didn’t get married and divorced on the same day.”