Derpykat5

It's just an analogy. Here; let me try one more time.

If you're playing a horde shooter and your friend reveals they can just spawn a boss on top of you at any time, it kind of kills your desire to keep playing - at least with them.

No offense, but you seem overly fixated on all the wrong things.

Whether it ends in a TPK isn't relevant. If you're playing capture the flag and your opponent reveals they can just teleport your flag to their base it'll have roughly the same effect. If the GM can just say "you lose now" it'll seriously demotivate anyone who is trying to enjoy the game, for whatever reason.

Overall, the difference between having an in-character "please stop being murderhobos" moment and having an out-of-character "please stop being murderhobos" moment comes down to how likely it is for the players to take the message to heart. If it's just some dude that's telling them to stop being murderhobos and is an unwinnable fight if the players refuse, that sets a distinctly different tone than the GM pausing things for a moment to explain the current situation to the players.

Both can work, but keeping it as a narrative element has a higher chance of failure, since it's possible the players could interpret this as just another NPC encounter instead of the GM's thinly veiled wishes for the future of the table.

Overall, the only people who care about the story are the people at the table, and having a moment of jarring change in the characters to set the narrative back on track is fine. You'd probably want to do something like that anyway to paper over the past behavior, otherwise the players could listen to you and be understanding of what you want, and still get punished for the stuff they've already done.

That's better communicated through... communication.

I don't know about you, but if I were playing a game to win and my "opponent" reveals that they can just cheat and instakill me whenever they feel like, I'm more likely to just stop playing the game than to try to play it for fun. Even if I did try to play it for fun, it would be hard to really enjoy it when I know that any encounter can just be a big middle finger.

If you don't explicitly tell people what they're doing wrong and how to fix it, it's unlikely that they'll figure it out on their own.

You check the label and realize it actually says "Tenser's floating Dikc", but the salesman is already gone.

Arcane Dye - for when you don't want to worry about mixing your whites with your reds.

Healing Ward - blocks all heal effects on the target.

Heap Metal - for when you don't feel like paying a hireling to collect the gear of that bandit camp you just slaughtered.

Antilife Spell - wait, I guess that's just Power Word Kill...

Burning Lands - when Fireball just isn't Fireball-y enough for you.

Chromatic Orc - summons an orc ally. A different color each time you cast the spell.

Control Hater - a more powerful version of Command that only works on things that are currently hostile to you.

Fire Stork - that wizard must have really hated storks. Incinerates the nearest stork.

Fig Cloud - an alternative to Hero's Feast for vegan parties.

Glyph of Barding - acts as armor for your horses.

Prismatic Ball - hold on, I've been notified this was just a mistranslation of Chromatic Orb. Never mind.

Rope Brick - this is what you get when you let the Barbarian make spells.

Sheep - conjures a sheep.

Stonespin - does what it says on the tin.

Runbeam - force the target to move their full speed every turn.

Rime Stop - I was kicked from a table after bringing this into a Rime of the Frostmaiden campaign.

Wash - why would you spend a 9th level spell on Prestidigitation?

In my eyes, the Rule of Cool is best used as the opposite of the Air Bud Clause. (For those who don't know; the "Air Bud Clause" refers to a rule in basketball that basically says "it's not allowed just because there's no rule against it".) TTRPGs are imperfect systems, and you are going to run into a scenario that isn't covered in the rules. Rule of Cool is best used here, rather than to bypass rules that do exist.

But also; some systems can be really crunchy, and a lot of the time it can be more fun for everyone involved if you just say "you know what, that's cool, let's do it" than to pause for five minutes to leaf through some rulebook (because seriously; you can't always know the entire rulebook by heart) trying to determine if and why they can't.

Of course, doing this too much is dangerous. Hence "in moderation".