Police later stormed the supermarket in the small town of Trebes and the attacker was killed, Interior Minister Gerard Collomb told reporters on site.
The attacker was known to authorities for petty crimes but was not considered an Islamist threat, Collomb said, adding that he was believed to have acted alone.
More than 240 people have been killed in France in attacks since 2015 by assailants who pledged allegiance to Islamic State or were inspired by the group. Islamic State claimed responsibility for Friday's attack but offered no details.
"He was known for petty crimes. We had monitored him and thought there was no radicalisation," Collomb said, before adding: "He was known for possession of drugs, we couldn't have said that he was a radical that would carry out an attack."
Collomb said three people were wounded.
He said that before the attacker seized hostages he killed one person while stealing a car with a bullet in the head in the nearby historic town of Carcassonne. He then shot at police officers before taking hostages at the supermarket.
President Emmanuel Macron said the incident appeared to be a terrorist attack and security forces were securing the area.
Trebes Mayor Eric Menassi told LCI TV that the man had entered the Super-U supermarket screaming "Allahu Akbar, (God is greatest) I'll kill you all".
BFM TV reported that he claimed allegiance to Islamic State Collomb said the man had demanded the release of Salah Abdeslam - the prime surviving suspect in the Islamic State attacks that killed 130 people in Paris in 2015.
Abdeslam, a French citizen born and raised in Brussels, went on trial in Belgium last month. He is accused of "attempted murder in a terrorist context" over a Brussels shootout in March 2016, four months after he fled Paris on the night of the carnage during which his brother was among the suicide bombers.
COLD ROOM
The gunman at around 11 a.m. (1000 GMT) first held up a car, killing one person and wounding another. Then he fired on police officers in Carcassonne, a town that is a U.N. World Heritage site and a major French tourist attraction. He wounded an officer in the shoulder before heading to Trebes about 8km (5 miles) to the east, where two more died.
Carole, who was shopping at the supermarket, described how people had taken refuge in a cold room.
"A man shouted and fired several times. I saw a cold room door, I asked people to come and take shelter," she told Franceinfo radio. "We were ten, and we stayed an hour. There were more gunshots and we went out the back door."
A 45-year-old lieutenant-colonel swapped himself in exchange for one of the hostages, Collomb said.
France is part of a U.S.-led coalition fighting Islamic State in Syria and Iraq and has thousands of soldiers in West Africa fighting al Qaeda-linked militants,
In February, Collomb said French security forces had foiled two planned attacks so far this year as Islamic State militants set their sights on domestic targets in response to the group's military setbacks in Iraq and Syria.
