You can use it at any time on your village. It is effectively equivalent to a flag, in the sense that if you had a recruitment or a population flag equipped, and you finished an upgrade on your stables or farm whilst that flag was equipped, your recruitment speed or max farm capacity, respectively, would scale with the upgraded building.