Abstract: Using the impact of the Soviet Union’s collapse on the performance of theoretical mathematicians as a natural experiment, we attempt to resolve the controversy in prior research on whether specialists or generalists have superior creative performance. While many have highlighted generalists’ advantage due to access to a wider set of knowledge components, others have underlined the benefits that specialists can derive from their deep expertise. We argue that this disagreement might be partly driven by the fact that the pace of change in a knowledge domain shapes the relative return from being a specialist or a generalist. We show that generalist scientists performed best when the pace of change was slower and their ability to draw from diverse knowledge domains was an advantage in the field, but specialists gained advantage when the pace of change increased and their deeper expertise allowed them to use new knowledge created at the knowledge frontier. We discuss and test the roles of cognitive mechanisms and of competition for scarce resources. Specifically, we show that specialists became more desirable collaborators when the pace of change was faster, but when the pace of change was slower, generalists were more sought after as collaborators. Overall, our results highlight trade-offs associated with specialization for creative performance.
Previous version published in Academy of Management Proceedings (2017) under the title of “Can Specialization Foster Creativity? Mathematics and the Collapse of the Soviet Union”
Keywords: knowledge specialization and creativity, change and creativity, knowledge frontier, Soviet collapse and productivity of mathematicians
Here is a good summary of the paper in HBR: When Generalists Are Better Than Specialists, and Vice Versa