We show that qubits traveling along closed timelike curves are a resource that a party can exploit to distinguish perfectly any set of quantum states. As a result, an adversary with access to closed timelike curves can break any prepare-and-measure quantum key distribution protocol. Our result also implies that a party with access to closed timelike curves can violate the Holevo bound.