Palestinians fear the razing of buildings near the fence will set a precedent for other towns along the route of the barrier, which runs for hundreds of kilometers around and through the Israeli-occupied West Bank.
The demolition is the latest round of protracted wrangling over the future of al-Quds, home to more than 300,000 Palestinians.
Israeli forces cut through a wire section of the barrier in Sur Baher under cover of darkness early on Monday, and began clearing residents, Reuters reported.
Bulldozers and mechanical diggers began tearing down homes on both sides of the barrier as security forces prepared a partly constructed nine-story building for demolition.
“They have been evacuating people from their homes by force and they have started planting explosives in the homes they want to destroy,” said Hamada Hamada, a community leader.
An Israeli court ruled in June that the structures violated a construction ban. The deadline for residents to remove the affected buildings, or parts of them, was Friday.
But Palestinian owners said their buildings lay within areas run by the Palestinian Authority.
“I built this house stone by stone. It was my dream to live in this house. Now I am losing everything,” said Fadi al-Wahash, 37, his voice breaking as a bulldozer destroyed his unfinished three-floor house.
“I had a permit to build from the Palestinian Authority. I thought I was doing the right thing,” he said.
Israeli Internal Security Minister Gilad Erdan said 700 police and 200 soldiers were involved.
Palestinian Prime Minister Mohammad Shtayyeh said the Palestinians would complain to the International Criminal Court about the demolitions in Sur Baher.
Jamie McGoldrick, the United Nations humanitarian coordinator, and other UN officials called on the Israeli authorities last week to halt the demolition plans. They said 17 Palestinians faced displacement from the plans to level 10 buildings, including dozens of apartments.
The International Court of Justice in The Hague issued an advisory opinion in 2004 that building the barrier on occupied territory was “contrary to international law”.