Ranil Wickremesinghe said he had a majority in 225-member legislature and his sacking was illegal.
"As far as the prime ministership is concerned, the person who has the majority support in parliament has to be the prime minister, and I have that majority of support", he said.
Speaking to jubilant supporters outside his Colombo home, Rajapakse called on Wickremesinghe to step down.
He said that the issue should be resolved peacefully and there should not be a crisis in the country.
The swearing-in ceremony, broadcast live, was a moment of high political drama for Sri Lanka, with Rajapaksa grinning as he shook Sirisena's hand.
Sacked Sri Lankan prime minister Ranil Wickremesinghe on Saturday demanded the parliament speaker call an emergency session so he can prove his majority, officials said.
Since his rushed swearing in, Rajapakse is yet to announce the formation of a Cabinet - which under the Constitution is automatically dissolved when a prime minister is removed.
Since failing to win a third term as president and being defeated by his erstwhile party secretary Maithripala Sirisena in January 2015, Rajapakse 72, has attempted to stage a comeback.
Colombo was on the verge of facing economic sanctions from Western nations over Rajapakse's human rights record before his government lost office.
A private television network loyal to Rajapakse televised him being sworn in as the new prime minister, replacing Wickremesinghe. Two other constituent party leaders from minority Tamil and Muslim parties, Mano Ganesan, Palani Digambaram and Rishath Bathiyutheen also said they would be firmly in support of Wickremesinghe and called Sirisena's move to sack Wickremesinghe as illegal and unconstitutional.
It is a constitutional coup because the serving prime minister has not legally ceased to function in office before a new prime minister has been appointed.
Mangala Samaraweera, the country's finance and media minister, also criticised the move saying it was "unconstitutional".
The dramatic developments came after deepening disagreements between Sirisena and Wickremesinghe over economic policy and day-to-day administration of the government.
His decade in power saw Colombo crush a decades-long Tamil Tiger uprising.
Last week, it was reported that Sirisena accused his senior coalition partner the UNP of not taking seriously an alleged conspiracy to assassinate him and Gotabaya Rajapaksa, the former top defence ministry bureaucrat and brother of ex-president Mahinda Rajapaksa.
But Sirisena has reneged on a pledge not to run for re-election, sparking tensions with Wickremesinghe who is believed to have his own presidential ambitions.