Rising population was thought to be bad for our standard of living, with classic books such as Limits to Growth and Population Bomb projecting fewer resources per person and environmental collapse. A new book, Empty Planet, makes the forecast that world population will hit a peak and then decline. The authors take this as positive in general, with some challenges along the way. I want to focus on one downside of less population: fewer one-in-a-million people.

The old pessimists made very poor forecasts. Famine did not kill millions of people. The air in developed countries got cleaner, not dirtier. The world produced more food, not less. The prices of oil and other commodities did not soar persistently. The pessimists missed how technological improvement would enable the world to grow more food, use fewer finite resources per unit of production, and improve environmental quality all at the same time. Those technological improvements didn’t just happen, but were the result of people finding better ways to do things. The green revolution in agriculture is a good example, and owes much to one great scientist, Norman Borlaug. Behind many of the great innovations of the past century lie one person with brains, vision and drive.

There are many smart people in the world. Some, but not all, are also creative. And some of these, but not all, are highly energetic and ambitious in their goals. At the extreme, we have superstars in science, engineering, business and the arts who are one-in-a-million in developing valuable innovations. These people produce new ideas that millions of other people can use. This is a huge point: the benefits of the creative geniuses are enjoyed by the rest of us.

Nobel laureate William Nordhaus looked at business innovators and concluded that they were able to capture only two percent of the total social benefits of their innovations. The other 98 percent of the benefit was spread across consumers and workers. The whole world benefits from the great innovators.

At first glance, more population seems good, because with a larger population we’ll have more one-in-a-million people. But some of our one-in-a-millions never achieve their potential.

Consider a smart, creative, persistent person born in China in 1950. She was most likely born in a poor, rural province (where a majority of China’s population lived at that time). She had limited educational opportunities. She had to survive the famine caused by the Great Leap Forward and the displacements of the Cultural Revolution. She probably could not fulfill her potential. And the same goes for people in many other less developed countries. Much of the world denied opportunities to women, and various minorities were discriminated against in different countries. Add in challenges faced by gays and others who didn’t match dominant stereotypes, and the world clearly missed many of the potential gains possible from one-in-a-million people.

The economic progress of poorer countries, combined with liberalization of social attitudes in many countries, enables more superstars to achieve their potential. That is now helping the world progress. In this way, economic progress starts a virtuous circle in which those with superstar potential have a better opportunity to succeed, leading to even more economic progress. With economic progress usually comes educational progress.

Education is part of the broadening of opportunity that helps the superstars achieve their potential. Education also has a very large impact on population growth: More educated women prefer to have fewer children. One of the Empty Planet authors was interviewed by Wired and said, “We polled 26 countries asking women how many kids they want, and no matter where you go the answer tends to be around two.” They cite formal education as a big factor, and also the informal education that comes from better communication through smartphones and Internet access.

With lower birth rates, we’ll produce fewer one-in-a-millions, but we’ll nurture them better in terms of nutrition, education and job opportunities. Opportunity will spread outward from males of the majority ethnicity to all (or at least most) of the world’s population.

The long-term growth of the world’s standard of living depends on the interplay of these two forces. The lower population growth that seems likely is a negative when viewed as a simple numbers game, but not so bad when nurture is taken into account. We’ll know definitively in 50 or 100 years.