If No.6 is the alter ego of No.1, then when the various No.2's who speak to No.1 on the telephone, must surely have recognised his voice, being that of No.6! Well they would, if that was the case, only it wasn't. Because at the time, no-one knew who No.1 was during much of the production of the Prisoner, not right up until the very end with Fall out in fact. The production crew thought No.1 might turn out to be the Butler, or that No.1 would be Lew Grade, who was head of ITC at the time. But in the end Patrick McGoohan made himself No.1, as he once said "Who else could No.1 have been?"
So all through the 16 episodes of the Prisoner, No.1 wasn't anyone at all, simply a supposed voice on the other end of a telephone! It was once written that No.2 is the only person to have spoken to No.1, but that is not the case. The Supervisor-No.28 once spoke to No.1 in the episode It's Your Funeral!
Be seeing you