Canada's three opposition parties united against his Liberal Party, which has been mired in a corruption scandal.
Mr Martin will seek the dissolution of parliament on Tuesday, and a date for a general election - thought likely to be 16 or 23 January.
Recent polls suggest Mr Martin's Liberals will take most votes in the election - but will not win a majority.
In parliament on Monday evening the opposition voted by 171-133 to bring down Mr Martin's government.
Mr Martin said he would visit Governor General Michaelle Jean on Tuesday to ask her to dissolve parliament, after which the election date would be announced.
Lost support
His government has only been in power for 17 months, but has failed to shake off a scandal dating from a previous Liberal administration.
In the late 1990s C$100m ($85m; £50m) of public money was paid by the Liberal government to advertising agencies, for little or no work in return.
It has been alleged that Liberal officials demanded kickbacks for awarding the contracts.
Mr Martin is not implicated in the scandal, but the opposition says he has lost all moral authority.
The prime minister fought off a no-confidence motion in May by a single vote, but lost the support of the New Democrat party earlier this month following a row over private health care spending.
The New Democrats joined the Conservatives and the Bloc Quebecois in bringing the latest motion after Mr Martin rejected their ultimatum for an early election in February.
Rhetoric
The vote will now take place a month earlier, in the depth of the bitterly cold Canadian winter, meaning the election campaign will run over the Christmas and New Year holiday period.
It will be the first winter campaign in 26 years.
Mr Martin and his government will stay in office until the election is held.
Some pundits say the campaign is likely to be bitterly fought, on the evidence of the heated rhetoric flying around in recent weeks.
Conservative leader Stephen Harper accused the Liberals in Quebec of links with organised crime.
The Liberals denounced such "false smears" and threatened to sue.
In the most recent polls the Liberals score about 36%, against 31% for the Conservatives, suggesting the Liberals - in power since1993 - will have to govern with the help of opposition parties.
The Liberals say they will campaign on their economic record, having wiped out a huge budget deficit, and seen the economy boom and unemployment fall to a 30-year low.
