‘Copper’ is a British colloquialism for policemen, but ‘copper’ is also a somewhat archaic synonym for ‘penny’, which gives the link to the saying: “turning up like a bad penny”.
Hence also the old joke: ‘What do you call a policeman’s night shift pay?’ — ‘Copper nitrate’.