Nedarim, Chapter 11, Mishnah 3. This mishnah continues with Rabbi Yosi's view on the question of which vows are considered vows of self-affliction, which the husband may annul, and which are not.
"Konam she'eini nehenis livriyos":
The wife vows that she will not derive benefit from any human being, and nevertheless the mishnah rules: "eino yachol l'hafer" - the husband cannot annul this vow for her, because it is not a vow of self-affliction. The reason is that according to Rabbi Yosi, she is still able to derive benefit from her husband.
The Gemara explains that the husband is not included at all in the category of "human beings"; rather, he is in a unique category of his own. Therefore, even though she is forbidden to benefit from everyone else, since she is able to benefit from her husband and has no need for others, there is no self-affliction here.
The mishnah further rules: "vichola hi lehanos b'leket shich'chah u'pe'ah" - three of the gifts to the poor that are left in the field:
Leket - the stalks that fall during the harvest.
Shich'chah - the items forgotten in the field.
Pe'ah - the corner of the field that the farmer must leave over.
Benefiting from these gifts is not considered benefiting from human beings, and therefore the vow involves no self-affliction, since other avenues of benefit remain open to her: from her husband or from the gifts to the poor.
A vow that withholds benefit from kohanim and Leviyim:
One who vows "Konam kohanim u'leviyim nehenin li", thereby forbidding the kohanim and Leviyim to derive benefit from him - regarding the Terumah and maaser that are given to them, the mishnah rules "yitlu al korcho". The kohanim and Leviyim take them against his will, because he has no power to make a vow that withholds the Terumah and maaser from them; these gifts are due to them from the outset and are not under his control. True, ordinarily the owner has the right to choose which kohen and which Levi he gives them to, but once he has forbidden all the kohanim and all the Leviyim, that is not a practical choice, and therefore they take them against his will.
In contrast, one who says "kohanim eilu u'leviyim eilu nehenin li" - "yitlu acheirim". Since he forbade only certain kohanim and certain Leviyim, the vow takes effect and is valid, and therefore it is other kohanim and other Leviyim who take the gifts.
In summary: In this mishnah we learned that according to Rabbi Yosi the husband does not annul a vow of "Konam she'eini nehenis livriyos", since the husband is not included in "human beings" and benefit from leket, shich'chah and pe'ah remains open to her, so there is no self-affliction here. We also learned that one who forbids all the kohanim and Leviyim from benefiting from him - they take the Terumah and maaser against his will, whereas one who forbids only certain kohanim and Leviyim - his vow takes effect, and others take in their place.