Richardson was born in Five Islands Village, Antigua. He began his career with the Leewards Islands in 1982 as an opener.

After his second season he was called up by the West Indies to tour India in the 1983–84 season. Richardson joined a successful West Indies Test team captained by Clive Lloyd batting in the middle order. His first tour started inauspiciously when Richardson lost his luggage and was left with few clothes. Veteran fast bowler Andy Roberts felt that Richardson was not getting enough practice as in the nets even bowlers were given a chance to bat ahead of him and by the time Richardson had an opportunity the main bowlers had finished. Roberts went out of his way to bowl at Richardson during the tour to make sure he had some preparation.

On 24 November 1983, Richardson debuted in the fourth match of the six-Test series, at which point the West Indies had a 2–0 lead, replacing Gus Logie who had bagged a pair in the previous Test. In his first innings Richardson too failed to score a run when was the victim of a poor umpiring decision. He was given out leg before wicket off the bowling of off-spinner Shivlal Yadav though he had hit the ball. He was more successful in the second innings, making 26 before he was bowled, and the match ended in a draw.

1985 World Cup

Australia hosted the World Championship of Cricket in February and March 1985 to commemorate the founding of Victoria. During the group stages the West Indies faced Sri Lanka at the Melbourne Cricket Ground on a pitch with uneven bounce. Though the West Indies won the match, a delivery from Ashantha de Mel reared and hit Richardson in the face; with Larry Gomes, he was one of two West Indian batsmen to retire hurt during the game.

Captaincy

Late in 1991, West Indies captain Viv Richards informed the West Indies Cricket Board of his intention to relinquish the Test captaincy and retire after the 1992 World Cup. Though Richards had publicly picked Desmond Haynes as his successor, the board chose Richardson to take over the captaincy and Richards was dropped from the team. Richardson supported the board dropping his predecessor, which led to ill-feeling towards him in Antigua, the home of both men. The West Indies never lost a series under Richards' leadership, so there was a great deal of pressure on Richardson. Under his captaincy, Curtly Ambrose and Courtney Walsh led the bowling attack and Brian Lara emerged as a world-class batsman. In the 4 years of his captaincy, the West Indies only lost one series – versus Australia in 1995 which was the West Indies' first series defeat since 1980.

Trivia

Richardson played 86 Test matches until 1995 scoring 5,949 runs and 16 centuries. He was very successful against Australia, hitting 9 centuries against them, and scored his highest score of 194 against India in Guyana in 1989. He also played 224 One Day Internationals including 3 World Cups.