Octavia Spencer Says She's 'Felt More Racism' in L.A. Than Back Home in Alabama: 'I Was an Anomaly'

Octavia Spencer said that she "felt more racism when I first moved" to Los Angeles than she ever did in her southern hometown of Montgomery, Alabama, as she appeared on an episode of the WTF with Marc Maron podcast released last week.

"I think everywhere is heavy. Everywhere has its history," explained the Oscar winner, 52. "I think everywhere has problems."

She continued: "You can't deny that Southern history is intense. It is. But what's beautiful for me is that stuff preceded me. I was a child of the '70s … As you grow older and the things that you can remember, that wasn't a part of my history. I learned about it. It's not anything that I experienced."

Spencer said that when she first moved to L.A. in the '90s, she expected the city to be a "free and liberal-thinking place," but during a shopping trip on Rodeo Drive, people made her feel like she "was an anomaly."

"It is so funny. It's right out of Pretty Woman," she said. "One of the first things that you do when you move to [L.A.], or at least that I did, is you wanna go to those historical places ... all of those landmark places.

"I remember going into a shop and being followed. At first ... I was just like so excited, like just walking around, and then I realized that I was being followed. It was just kind of strange in that way," Spencer added.

The Truth Be Told star said that "the funny thing is, after getting over that initial [culture shock], I've not really necessarily experienced" it again, adding: "But it was glaringly obvious."

RELATED VIDEO: Allison Janney and Octavia Spencer Went from Crushing on the 'Same Guy' to BFFs: 'We Liked Each Other Better!'

Spencer said she has "a home there now" in Alabama, where she spends "weeks at a time," noting the importance of "spending quality time with family."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

After buying a Spanish-style house in the Toluca Lake neighborhood of L.A. in 2013, she claimed to Ellen DeGeneres that the home is haunted by a ghost whom she calls her "protector," adding: "I love him, because he kind of sorts out the bad people that shouldn't be there. He runs them out."