The colors of the Olympic rings

Is there a hidden significance?  Officially, the answer is no...

The following is quoted directly from the IOC: "The five rings represent the five continents. They are interlaced to show the universality of Olympism and the meeting of the athletes of the whole world during the Olympic Games. On the Olympic flag, the rings appear on a white background. This flag translates the idea of the universality of the Olympic Movement. At least one of the colors of the rings, including the white background, can be found on the flag of every nation in the world."... Baron Pierre de Coubertin conceived both the symbol and the flag. Not coincidentally, Coubertin was the founder of the modern Olympic Movement. 

But... note this in the Wikipedia entry:

Prior to 1951, the official handbook stated that each colour corresponded to a particular continent: blue for Europe, yellow for Asia, black for Africa, green for Australia and Oceania and red for America (North and South considered as a single continent); this was removed because there was no evidence that Coubertin had intended it...

Here's the cited 1951 reference*:

We probably will never know for certain whether Coubertin's committee intentionally chose "yellow for Asia, black for Africa" (and red for the Americas) and that position was later "walked back" when such stereotyping became less popular, or whether the attributions of ring color to regions were just a creation of some enterprising writer or reporter.

*"Decision adopted by the Executive Committee". Bulletin du Comité International Olympique ( Olympic Review ) (Lausanne: IOC) (25): poo. January 1951.