I think you were asked this because the concern you mention suggests that you use p-values without understanding what they are. P-values already fold in information about sample size, i.e. they tell you whether a result is significant in light of the sample size, effect size, and noise observed.