Newer upstarts and legacy companies have spent recent years competing intensely for top agents. But recent data suggests the years of fast growth may be ending