We study the optimization and sample complexity of gradient based training of a student neural network in the high-dimensional regime, in both regression and classification settings. We provide a sharp analysis of the SGD dynamics, and derive scaling laws for the risk that highlight the power-law dependences on the optimization time, sample size, and model width.