Does Advantage get rid of worms?

No it does not. It just kills the fleas that are the primary transmitters of tape worms. Your cats just haven't been shedding worm segments lately. Now that the fleas are gone they need to be wormed to get rid of the worms as well