Ok do jazz musician typically improv during comps or do they stick to the progression? If so how is that possible to just be able to improv new chords on the spot while staying in key?
Yes, they improvise, but also stick to the progression (more or less). The improvisation is done by changing chord voicings and rhythms, adding extensions to chords (9ths, 11ths, 13ths etc.), using chromaticism between chords, playing bass lines or harmonies to the bass part, and substituting chords (tritone subs for example.)

Many rhythm players will imitate and play off of what the soloist is doing rhythmically and harmonically. Really good players can even serve as a catalyst for the soloist's ideas.

not a very technical answer but I hope it helps
Most of the time, yes. The original changes are only serving as a template, and you work of that. You might want to change the progression to a new set of changes, re-harmonizing the tune. You may want to add substitute chords to the progression. And as said, combine different aspects of comping together (basslines, full chords, broken up chords, lines, octaves etc). Most of the reasons they can do it is because they pay attention to leading tones and chord functions, two different chords may go to the same place, and how you voice your chords will allow you to get away with some stuff.
Fusion and jazz musician, a fan of most music.

Quote by Guthrie Govan
“If you steal from one person it's theft, and if you steal from lots of people it's research”

Quote by Chick Corea
"Only play what you hear. If you don't hear anything, don't play anything."

In an ideal world, you are improvising a counter-melody in 3 or 4-part harmony to go with the melody/solo that's already being played.

It's more difficult and more rewarding than a "solo" solo IMO.
"There are two styles of music. Good music and bad music." -Duke Ellington

"If you really think about it, the guitar is a pointless instrument." - Robert Fripp