(e) The Secretary shall issue a decision as promptly as possible, affirming, modifying, or setting aside, in whole or in part, the decision under review, and shall set forth a brief statement of reasons therefor. Such decision by the Secretary shall be the final agency action within the meaning of
5 U.S.C. 704.