The gene studied is believed to act as a growth suppressor, reducing birthweight.
But the UK-based researchers found a particular variant passed down from the mother can add 93g (3.3oz) to the birthweight, or 155g if passed down from the maternal grandmother.
Details are published in the American Journal of Human Genetics.
Professor Gudrun Moore of University College London and colleagues looked at a gene called PHLDA2 in nearly 9,500 DNA samples taken from mothers and their babies, collected in three separate studies.
They found a gene variant called RS1 appeared to change the way in which the gene functioned, leading to higher birthweights.
"The gene is already known to have a profound effect on birthweight by acting as a growth suppressor," Prof Moore told BBC News.
"We have found a genetic variant of PHLDA2 that when inherited from the mother, causes the baby to be 93g bigger on average, or even 155g bigger on average, if inherited successively from the mother's mother."
The RS1 variation was found in around 13% of the individuals studied, with 87% possessing the RS2 variation.
"We suggest that the more common RS2 gene variation, which is only found in humans, has evolved to produce a smaller baby as a protective effect to enhance the mother's survival during childbirth," said Prof Moore.
"Dad's lack of involvement in evolutionary terms may stem from his own survival not being at stake and he can continue to reproduce with other females."
The PHLDA2 gene is unusual in that only the copy inherited from the mother is active, while the copy inherited from the father is "silenced". This silencing of the paternal gene results from molecular processes around the DNA known as epigenetics.
Scientists do not know why, but have speculated that it is to ensure birthweight is reduced to ensure the mother survives childbirth.
Dr Caroline Relton of Newcastle University said: "Although this study looks only at birthweight as an outcome, it is possible that this genetic variant may have longer-term health consequences.
"Indeed the long-term health consequences associated with extremes of birthweight might be due in part to this and other contributory genetic factors."