Specifically, it was the Surrey Iron Railway, which opened in 1803. It was a horse-drawn railway that transported iron from the Surrey Ironworks to the Thames River.

While other early railways existed, this is widely considered to be the first public railway.