In the past you may have been told that you need to remove all of the caps on the plastic bottles you plan on recycling, but the past version of you was wrong. In today's terms, with today's recycling technology, you should always leave the caps on.
Why? Well today's recycling machines sort recyclables by size. And when caps are separated from bottles, often they are deemed as unrecyclable material and sent to the landfills, which is the last place you probably mean for them to end up. Leaving the cap on makes it more likely for the entire bottles to be sorted properly and shipped off to be turned in to your next plastic bottle, or toy car, or coffee lid.
It should be noted that you also shouldn't crush your plastic bottles. Crushing bottles sometimes tricks the sorting machines into thinking they're dealing with paper or another material, causing it to send it to the wrong place. So the bottom line, just be lazy and toss it in the recycling as you finish and everything will be fine.