Kiddushin, Chapter 4, Mishnah 13. This mishnah begins a discussion of various occupations in which there is a danger of excessive closeness between men and women, out of concern that it may lead to sin.
The text of the mishnah:
"Lo yilmad adam ravak sofrim" - a bachelor, meaning a man who is not married, and usually the reference is to a young man who has never been married, may not teach young children. The reason is that it was a common practice in their day for the mothers to come and remain on the premises, so that the teacher would spend too much time in their company, and there is a concern that this might lead him to sin.
"V'lo telamed ishah sofrim" - similarly, a woman should not teach young children, because the fathers and the brothers of the children come to the place, and they may come to a situation of yichud, being alone together in a private place.
The opinion of Rabbi Eliezer:
Rabbi Eliezer adds to the words of the first tanna and says: "Af mi she'ein ishto imo lo yilmad sofrim" - not only a bachelor who is not married at all, but even a married man whose wife is not with him in the city at that time. Even though he has a wife, since she is not present with him, he is not fit to occupy himself with teaching young children.
In summary: the mishnah establishes two restrictions out of concern that they may lead to sin: a bachelor does not teach young children because of the mothers who come to the place, and a woman does not teach young children because of the fathers and brothers who come there, lest they come to a situation of yichud. Rabbi Eliezer extends the restriction even to a man whose wife is not with him.