Capt. Russ Davies, a spokesman for the Anne Arundel County Fire Department, said that, following interviews with the family, it is thought that the occupants were last seen inside the home.
"That will be the primary focus of our investigation," Davies told FOX 5's Jennifer Davis Tuesday morning. "However it's still an active investigation, so we'll follow any lead in attempt to locate the occupants."
Davies also said a search for the missing family members will begin when crews stabilize the wreckage and pump out tens of thousands of gallons of water from the basement - likely Wednesday.
"We're going to be talking about days not hours to get this done," Davies said.The massive fire gutted a 16,000 square-foot mansion just west of downtown Annapolis early Monday, and six people who may have been inside are still unaccounted for.
The blaze was first called in to firefighters around 3:30 a.m. Monday morning, and it quickly grew to four alarms. Skyfox captured the devastation from above the 900 block of Childs Point Road. More than nine hours after it started, crews were still tackling hot spots Monday night.
Some neighbors had said the owners of the seven-bedroom house were supposed to head out of town Tuesday.
Neighbors say some of those who are missing could be children, as the couple who lives in the house was due to leave Monday on a trip with some of their grandchildren. The husband and wife, Don and Sandy Pyle, are still missing. Mr. Pyle, 55, is the COO of Virginia-based IT firm Science Logic.
The Pyles built the home in the early 2000s. Fire officials say the mansion did not have a sprinkler system because it was built before statutes were put in place requiring them.
Much of the house collapsed, and given the size of the structure, the ATF was brought in to help with the investigation. The ATF's specialized team arrives Tuesday, and they will begin searching for remains on Wednesday morning.
There are no initial signs of arson or anything suspicious.
There are no hydrants near the home, which made attacking the fire that much more difficult.
Real estate websites put the value of the home at around $6 million.
The cause of the fire has not yet been determined.
The Associated Press contributed to this report.