This is essentially the nature vs nurture question, in which case the answer is always yes and no.
For example, if you grow up throwing a baseball, your humerus (upper arm) and shoulder joint will literally morph into a differently shaped bone than if you don't grow up throwing. This is called humeral retroversion.
It enables the shoulder to "lay back" like this: 
We know this is an adaptation because we'll see the pitcher's non-throwing arm not have that ability.
It's also where the expression "throw like a girl" came from. Because so many girls used to grow up not throwing, when they would try to throw something in adulthood, they'd throw it like a shot put. They'd push the ball since their arm couldn't move like above.
There are all kinds of adaptations like this. The Sports Gene does a phenomenal job looking at a lot of this, even citing research showing growing up playing tennis can make your dominant arm a little longer than the other!
I've only cited bone so far, but it's a big one. For many sports, if you can't get in the sports' positions, everything else doesn't even matter.
But there are all kinds of other things to entertain. Mitochondrial density, whether a gene is turned on / off (and whether the ability to change that goes away once we hit a certain age).
Depending on the context, yes, what you do growing up can dictate what you can do as an adult.
However, as the great sprint coach Charlie Francis said, "You can't turn a donkey into a thoroughbred."
For instance, something you do see in the pitcher's non throwing arm is an unusual ability to easily dislocate the joint. Something called a sulcus sign. 61% of professional pitcher's have it, with 89% having it in both arms. (More info and videos.) That is, they tend to have an unusual level of mobility to begin with.
Many, like insane parents, take an example of what-you-do-while-younger-impacting-adulthood to think they can just practice their child into athletic stardom. It doesn't work that way either.
Note: I used sports because it's often the easiest to relate to, but it doesn't stop there. We're learning all kinds of things about metabolism too. If a person was obese at one time, that can seriously and, as far as we can tell right now, forever, change their metabolism, making them always more likely to become obese again in the future.
Lastly, while most remedial it is not the least important: habits die hard. If you get into a habit of running or being active young, that's only going to help you look more like a runner when you're older. After all, if nearly 3/4 of the people around you are overweight or obese and you're not in part because you're active, well, you sure look like a runner than most! (Even if you don't run at all!)