Having had their minds transferred into the other person's body, the Colonel, with the mind of the Prisoner, is taken back to London so to wake up in No.1 Buckingham Place. All unpleasant memories of the Village have been wiped from the Prisoner's memory.
Now in the Colonel's body the Prisoner's mind controls his movement, and it is the Prisoner's mind that thinks. The television viewer knows this because we hear his thoughts in the voice of the Prisoner. What's more the Prisoner's handwriting is the same! So working with that hypothesis, would not the voice we hear also be that of the Prisoner's and not that of the Colonel's?
Admittedly, the cadance would be that of the Colonel's, because the Prisoner's mind would be speaking through the Colonel's vocal cords, so the level would be the same. But the Colonel would speak in the manner of the Prisoner. As we hear ,his whole manner of speaking changes from the way he speaks when he thinks, to the way he actually speaks. Although the Prisoner does not have a marked accent, he does speak differently to the Colonel, and that is the way the Colonel should have spoken.
At the end of the episode, when we hear the Colonel speaking through the Seltzman body, Hugo Schuster should have actually spoken with the same English accent as the Colonel. And Seltzman's mind, now in the body of the Colonel, would have gone out into the world to continue his experiments in peace, but speaking with an Austrian accent.
Be seeing you