I'll take on the second question about how quests might be more diverse and/or emotionally engaging. These objectives might be helped if there were more reputation dependent quests. Not just the sort of quests where you open up new areas and quest chains, but quests where there is significance to your choice of deciding to ally yourself with one faction over another. WoW has the structure for this in place with the Aldors and Scryers (and to a less developed extent with Booty Bay and the Bloodsail Buccaneers), but the background lore diminishes any conflict between the two and any significance to the choice other than the tactical concerns of which faction's loot will better suit your objectives.

I think this starts with the stated objective(s) for the encounter, be it quest or boss encounter. If a major objective is to emotionally engage the player then the encounter developer will focus on making decisions that work towards engagement. Keeping with the WoW examples - Nefarian is a major character the player can make an emotional connection with. From his encounter in Black Rock Spire to the corruption of Vaelastrasz, Nefarian is actively trying to halt the player’s progress, which leads the player to develop feelings towards the character. This type of activity could be enhanced through the frequency of encounters with the NPC and through building up characters like Vaelastrasz.

I think the keys to develop emotional interaction are, first it must be an objective, early and/or often contact with the NPC, visual stimulation - we need to see the NPC doing good/bad things to remember them, and lastly the actions the NPC takes has to be sufficient to cause the desired emotional response - don't expect me to care if the NPC just takes one sheep from a farmer that has 100 sheep. Biggest problem: no matter the quest, the repetition kills any attempt at emotion. When you turn anything into a mechanical process, any 'magic', feelings, emotion, etc., disappears. Any decent single player (even "ride the rails") adventure game has you do important things once, somehow ties it into a narrative, and allows you to become attached to your character in the process.

Key: a game makes you feel a part of the story. I currently see NO way of doing that in any current MMORPG. Go back to an old style MUD with active moderators who create and develop story arcs and allow player participation, and maybe something could happen. The closest I saw of late was online D&D using an actual dungeon master. If you can't have a lasting effect on the real world, you will never be tied into the narrative. Emotional ties cannot develop as a result of the game world. Because of guild activities and relationships? Sure. But not due to the game. I think the repeatability of MMOGs also contributes to the lack of emotional engagement with bosses and storylines.

First, there's no structure to the plot (intro, climax, conclusion, ending). For example Agent missions in Eve seem to tell parts of a story, but it's like reading chapter 5, then chapter 2, then chapter 5 again, etc. As described in this Wiki Article, MMO stories seem to lack a great deal. Second, the players are supposed to be characters in the story, but we can't really interact with the NPC's. In other words, there are no consequences for our actions as described suggestions about good fictional characters you'll see almost zero resemblance to MMO storylines. There's more, but those are my three big ones. So the question remains: How could you incorporate any of those elements into our MMO experience, and do we really want to? That’s strange.

Many MUD quests achieved this sort of emotional engagement. The barriers these days have to with a bunch of complicated factors, but they include (IMHO)

It is this type of narrative structure that might be impossible in a MMO. Imagine a quest that didn't give a promised reward. Player would send bug reports. How would the game dev respond? Can the devs say, 'We want the players to develop negative emotions to this NPC to enhance later quests? Therefore, we (the NPC) lied to you.' It would be exciting to see a MMO that controlled a story line like the series Lost or Heroes. But because of the issue of replayability, I'm not holding out.

Even given the limitations that Raph describes, however, I think there are more tricks like this that would help. Another, hinted at in some of the above replies, would be quest NPCs who recur constantly in the progressive life of the character. How about for every WoW character above level 20, you get an "archenemy", an NPC who appears every once in a while to try and mess with you. Rather like ambushes in COH. Or have more characters like Hesingway who pop up at three or four points in the character's questing life, and remember what's gone before. Part of the reason that the NPCs are just mechanical XP-granting machines is that you know you will NEVER see them again UNTIL you have another character that is progressing past that point.

Great idea. Or one who ganks you four or five times while you're in his zone or area. Then imagine that ten levels later, he's the boss at the end of an instance. You'd be extra motivated to kill him (at least the first time). A little bit off topic, but IMHO the real issue here is that players have no effect on their worlds. You can visit any of the shards in any of the popular MMO's and they are essentially the same, you can go talk to NPC_01 on any of them and he will ask you for the exact same 10 kills/items/etc. If you play a class on one shard your skills and abilities are exactly the same as the guy next to you playing that same class and very close to the guy on the other side playing a similar class.

Now if that NPC decided that well the 23815798289 bear hides that he had in his warehouse was enough, and decided that perhaps the dear population was now getting out of hand, or that he needed more firewood to cure the hides, or more stone to build a larger storehouse, or the local goblins took issue with his call to eradicate the bears then while it was really all similar quests, it would at least feel like a player's actions had an effect. If the footings of a new storehouse showed up, new fires with skins tanning over them were erected and goblins started to raid and destroy these new structures, the player would feel more engaged.