Gotta love rabbis. Ynet reports:
Rabbi Yoav Malka deals with the instruction in Pirkei Avot to keep conversation with the woman to a minimum. Malka explains that a man must be careful not to be overly-influenced by his wife: "Women's verbal abilities are higher. Their proficiency and fluency might make the man feel inferior and damage his self-esteem and sexual identity."
Rabbi Malka is National Religious, not haredi.
Personally, I'm attracted to smart, eloquent women. But, then again, I'm a heretic.