When a Challenge starts, only the first 20 guild members whose stars are added to the Guild total contribute stars.

 

If you want to have your stars counted and your Guild has people come in and out during the Challenge, be sure to start contributing from the beginning.