Originally answered in a post here.
Q. If the entire universe is expanding and all known galaxies are moving away from each other, shouldn't there be a point that is absolutely stationary?
A.The stationary point you imagine would have been the centre. As it turns out, the universe is believed to have none.
Further details at Where is the centre of the universe?.
To summarize:
The fact that the universe is expanding uniformly would not rule out the possibility that there is some denser, hotter place that might be called the centre, but careful studies of the distribution and motion of galaxies confirm that it is homogeneous on the largest scales we can see, with no sign of a special point to call the centre.
As for the balloon analogy:
A good way to help visualize the expanding universe is to compare space with the surface of an expanding balloon.
When thinking about the balloon analogy you must remember that. . .
The 2-dimensional surface of the balloon is analogous to the 3 dimensions of space.
The 3-dimensional space in which the balloon is embedded is not analogous to any higher dimensional physical space.
The centre of the balloon does not correspond to anything physical.
The universe may be finite in size and growing like the surface of an expanding balloon, but it could also be infinite.
Galaxies move apart like points on the expanding balloon, but the galaxies themselves do not expand because they are gravitationally bound.
Cover image Illustration by Rauner Special Collections Library