It links Ethiopia's capital, Addis Ababa, to the Red Sea port of Djibouti - a stretch of more than 750km (466 miles).
Travelling at 120km/h, the new service cuts the journey time down from three days by road to about 12 hours.
The $3.4bn (£2.7bn) project was built with the help of funding from a Chinese bank and will have Chinese staff.
The track runs parallel to the abandoned Ethio-Djibouti railway, built more than 100 years ago.
'It will transform my business'
Getachew Betru, chief executive of Ethiopia Railways, says it will be much cheaper and more reliable than travelling to Djibouti by road.
"In Ethiopia currently if you want to bring your container from Hong Kong to Djibouti it will take you maybe two, three weeks. But it will take you more than that to take it from Djibouti to Addis Ababa. It will now take us one day or more," he said.