NEW CANAAN — There’s a story Richard and Sandra Bergmann like to tell as they walk visitors from the door, past a fireplace, to a back sitting room in their Park Street home — the former residence of Maxwell Perkins, illustrious editor of F. Scott Fitzgerald, Ernest Hemingway, Marjorie Kinnan Rawlings and the famously long-winded Thomas Wolfe.

According to legend, the Bergmanns said, Perkins would sit beside that fireplace in the winter, reading a voluminous Wolfe manuscript and tossing superfluous pages into the flames as he deemed fit.

Seventy years, nearly to the day, after Perkins’ death — June 17, 1947 — the fireplace, the Perkins’ original bookshelves, the original moldings and some other features remain from the time the Scribner’s editor lived at the home. But in many other ways, the home has changed.

“In a sense, we gutted the whole house completely,” said Richard, a career architect who oversaw the renovation when he and his wife bought the home in 1973.

The pair replastered the walls, repainted, redecorated and altered the floor plan drastically. Most notably, the Bergmanns removed a portion of the second floor, creating a subsurface office for Richard, forming a loft-like effect and open floor plan on the entry level.

“When I first looked at the house, I didn’t want to come near it because I knew it was going to take a lot of work,” Richard said. “But one day I came back here and found the door open. I walked in and it dawned on me what I could do by taking out the floor and doing things like that.”

Since the Greek revival house was built in 1836 by a pair of bachelor brothers, Theophilus and Philo Fitch, the Perkins House has experienced a variety of tenants and undergone considerable changes.

By 1865, the home had doubled in size and became a boarding house. In 1891, Eleazer Fancher purchased the residence and built a shoe factory on the property, which at that time extended to the end of the block, where Citibank is now located. Helen Rogers, a neighbor across the way, whose home was where the Park Street Lot now stands, annoyed by the smell of tanning leather wafting up the hill from the shoe factory, bought the home and its property in 1902, moving the factory further away on what is now Elm Street and renting the house to a New York lawyer until 1919. At that point, the New Canaan Country School occupied the home for five years, until Perkins, his wife Louise, and their five children moved in in 1924.

Over the course of its more than 150 years, the house expanded, with one addition taking on a Victorian style. In what was once the New Canaan Country School playground on the south side of the house, Perkins’ wife planted large hemlocks, the beginning of a garden and viewing area for one of Richard’s large sculptures.

According to Sandra, the garden is a place often stumbled-upon by strangers, who assume it’s part of a public park. Through the glass windows in Richard’s office, out of which his architectural firm, Richard Bergmann Architects is run, the visitors can be seen sitting and waiting for a Metro North train or their tour of the Glass House to begin or, in once instance, arguing.

“We’ve had a couple come in and argue out their marital problems sitting on our bench,” Richard recalled.

“I used to think I had to go out and introduce myself, but they would run away. So now I don’t go out. I like it, it’s our gift to the street,” said Sandra, who claimed not to be startled by the unannounced guests. “You don’t have to worry about anybody sneaking up on you because of the sound of the gravel crunching.”

But, after more than 40 years living in the home, the Bergmanns are again looking to make a change.

“I want to be able to lease out these lower levels that have been our office to someone else. I want to retire from practice. We’re either going to sell or rent the upstairs to someone else or stay here while we’re making the transition. That’s not clear yet,” Richard explained.

In order to do so, the Bergmanns have requested a variance from the Zoning Board of Appeals that would allow them to run a first-floor business without residing in the home and for as many as eight employees to work out of the home. The request is still being considered.

The Bergmanns have lived in New Canaan in since 1961. Richard is a native of Wisconsin, while Sandra hails from Oklahoma. The couple said a move away from the northeast is likely, should they be able to sell the home, which has been on the market for more than two years.

“Sandra’s a Southern girl. She claims that ever since I dragged her North she hasn’t been warm,” Richard said. “But it’s been a fun house to live in.”

justin.papp@scni.com; @justinjapp1