That's a common excuse and one I personally used a lot. The reality is you don't eat enough although you think you do, you compare how much you eat vs others who eat less yet gain more weight but the factors as to why that is is due to them being older/shorter/less active and so much more. Eat MORE, simple as that, the only limiting factor is money and how much you're willing to keep eating despite being on the verge of throwing up. In order to gain weight I used to eat 4000 calories daily which is 2x the average daily intake for men and I heavily doubt you eat anything close to that

I'm a powerlifter that started off from 50kg and bulked up to 74kg within a year.  You need to eat more, if you're scared of being fat, you'll forever be skinny.