Comets have elliptical orbits, meaning they are oval-shaped. The Sun is located at one of the foci of the ellipse, not the center.
Here's a simple analogy: Imagine you have two thumbtacks on a piece of paper. If you tie a string to both thumbtacks and then use a pencil to trace the path of the string as you pull it taut, you'll draw an ellipse. The thumbtacks represent the foci of the ellipse.
In the case of a comet, the Sun is one of the foci, and the comet travels along the elliptical path, getting closer to the Sun at its perihelion (closest point) and farther away at its aphelion (farthest point).