Companion, I refuse you.

Eh idk, a certain amount of companions staying around no matter what makes a little bit of sense. Can you imagine even the Inquisitor being able to expunge Cassandra from Skyhold when she is basically a founding member and senior member that is in good with all three of the advisors?
Not to mention that if you could expel everyone, you'll probably end up short a mage or something and find yourself unable to access certain areas or unable to complete certain quests. People were displeased when Solas leaves at the end of the game for this reason. Unless you have Viv, Dorian or a mage Inquisitor in your roster you essentially blocked yourself from completing areas and quests that requires a mage.

But I do expect companions to do more than just take it and accept your abuse. I expect at least some of them to fight back. You think Vivienne would just leave quietly after she got slapped and not plot retribution? I think not. She's probably going to try and denounce me at court--make an operation pop up on my war table as she tries to exact retribution from afar.

Yes, exactly (though I detest the Solas punch scene). It makes me furious that they don't get to punch back, both Solas and Dorian. I think it's unfair and feels like abuse.

Or maybe back to the Origins way of things, I'll cut ya, don't think I won't do it!

 "I hope you succeed, Warden. But I hope it kills you!" 

When i'm playing in hardcore mode
(Nightmare+solo) i don't recruit anyone and i use only the PC.

This statement doesn't make much sense.

^^This is incredibly annoying in DAI.

I don't think I've ever refused a companion in DA, well maybe Fenris on one DA2 playthrough.

My completionist instinct makes me disinclined to refuse the content a companion brings.

It likely has to do with maturity - the companions don't swing back, because they're smarter than that and know that it won't bring them anything other than petty satisfaction...

As for me, I usually recruit everyone, no matter of their approval level with the character I play later (... though usually I manage to befriend everyone ). The only time I didn't recruit Sera was the first play-through, when I missed the item in initial quest and it got locked out after I got to Skyhold.

To add a little more to that, if I may. It actually makes perfect sense for Cassandra, Varric and Solas to stay with you no matter what, even if no one else does; even if, in Solas' case at least, you get to punch him too. All three of them are feeling guilty because they believe what's happening in Thedas presently is their fault. The Templars and the Seekers of Truth, the two Orders that Cassandra always considered just and under which she had served for years respectively, have gone rogue and are doing now more harm than good - in her heart, she believes she could have done something about that. Varric believes that it's his fault the Red Lyrium was set loose on the surface. And Solas, well, the Breach was caused by his Orb. They want to make amends - even if it means being at the side of a person who's horrible in their eyes. 

Which is kind of sad when you see what's happening to Cassie (for example) if her approval of you reaches rock bottom - she resorts to drinking, because that's the only way she can cope. Forget punching, that scene is in many ways worse, for me at least. 

Wait, she can be punched? I thought only Dorian and Solas could be punched.

No, I meant that she wasn't punched. Bun Inquisitor can threat to kill her if she won't leave Skyhold fast enough - in which case she does the sensible thing and flees.