I played Arena, then Daggerfall, then Morrowind.. loved them all... but then came Oblivion, and I literally at one point threw the box across the room. I didn't come back to it without a serious set of mods applied, and I still haven't gotten very far into the game before getting irritated.

Examples?
  • Bandits with glass armor in a dungeon on the same freaking isle as the Imperial City.
  • Guards who don't care if you open and ransack every container out in the open, even when there are four of them around you, but by the Nine don't you dare eat a piece of meat from the table in your room at the Inn!
  • No more levitation, because the level designers were too lazy to think of ways to prevent the bypass of content and so invented some random bullcrap lore about it being illegal now.
  • Mini-game after mini-game after mini-game... thanks, but I don't really want to play spin-the-wheel with every NPC in the game, and lockpicking for someone with decent hearing and reflexes is a joke.
  • Horrible voice scripting, such that a noble woman can go through four different accents in the same conversation.
  • HVS continued... I don't want to hear a conversation between two NPCs about how such-and-such has such fantastic plate armor in stock when the speaker is, in fact, a *BEGGAR*.
  • Oh, and the Redguard/Nord/Breton NPCs that in the caverns shouldn't be able to see their hand in front of their face, yet they can see flawlessly enough to put arrows into your stealthed black-clad Khajiit arse from farther-than-max-clip-distance away.
  • Oh, and Khajiit and Argonians that aren't digitigrade, because they couldn't figure out how to animate them. Lo and behold "A different *race* of ..." even though they managed to do it for the Scamp and Clannfear... ::Sighs::
That being said? Waiting on ES5.