Soldiers opened fire after the man jumped into the Imjin river and ignored repeated warnings to stop.
The North and South remain technically at war after the 1950-53 conflict ended in an armistice and not a peace treaty.
There are thousands of North Korean defectors living in South Korea, but few attempt to make the journey the other way.
Instances of trying to cross the heavily-militarised border in broad daylight are extremely rare.
The incident comes as South Korean workers returned to the Kaesong industrial park in North Korea, five months after work was halted amid high political tension.
The man, who has not yet been identified, was spotted by guards near the western border in Paju, north of Seoul, at around 14:23 local time (05:53GMT).
Soldiers fired off warning shots and told him to return to the South. When he disobeyed the order and jumped into the river, he was shot dead, military officials told Yonhap news agency.
A military spokesman also told the Agence France-Presse news agency that the man’s body had been retrieved.
The shooting comes amid a thawing of ties between the North and the South as more than 800 South Koreans poured into the jointly-run Kaesong industrial complex.