The actor says when he and his fellow action movie veterans meet up, they quickly realise their hectic schedules aren't the sole reason why their reunions are so rare.
He told the Sunday Express newspaper: "People think Arnie and Bruce and me hang out all the time but because of our work projects we may not talk for a year.
"Then, all of a sudden, we get together and realise there is a reason we don't get together; too many egos in the room! So we say, 'See you in a year, have to go regroup.' "
Although the blockbuster trio might not train together as much as people may expect, Sylvester admitted he has had to teach himself how to "work out economically" to get in the best shape for his film roles.
He admitted: "Over the past 30 years I've learned how to work out economically.
"I would use the home gym and I would focus on what I call the movie body parts; instead of doing the whole body I would focus on what the cameras needed.
"No one cares what your ankles look like, right?"