The ministry on Saturday, in a strongly worded statement, accused China of threatening aviation safety and waging psychological warfare on the island's people with the balloons, days before key Taiwanese elections.
China's defence ministry, which last month declined to comment on the balloons, did not immediately respond to a request for comment.
The potential for China to use balloons for spying became a global issue last February when the United States shot down what it said was a Chinese surveillance balloon. China said the balloon was a civilian craft that accidentally drifted astray.
Taiwan is on high alert for Chinese military and political activity ahead of this Saturday's presidential and parliamentary elections. It says China is exerting military and economic pressure in an attempt to interfere in the elections.
China views the island as its own territory, a claim Taiwan's government rejects.
Since last month Taiwan's defence ministry has reported several instances of Chinese balloons flying over the Taiwan Strait. It has said over the past week some balloons had flown over Taiwan island near major air bases.
In the latest incident, revealed by the ministry on Monday in its daily report on Chinese military activities over the past 24 hours, it said three balloons had flown over the strait's sensitive median line on Sunday.
However, only one crossed Taiwan island, right at its southern tip, according to a map the ministry provided.
The balloons all headed east before vanishing, it added.
The Taiwan Strait's median line previously served as an unofficial barrier between Taiwan and China, but Chinese fighter jets, drones and now balloons regularly fly over it.
China's Taiwan Affairs Office reiterated last week that the strait's median line "does not exist" and that Taiwan's ruling Democratic Progressive Party (DPP) is "hyping up the threat from the mainland as the election approaches" and inciting confrontation.
