The influential National Association of Realtors and several brokerages were ordered to pay damages to home sellers who said they were forced to pay excessive fees to real estate agents.