My tease on Friday night was actually the product of a slightly heated conversation on metrics and the like. I wasn't actually planning on writing anything but I've mulled over it slightly today.
"The data does not seem to support that polished quality sells better than new features."
"Popular opinion supports that polished quality doesn't need to sell better simply because it encourages existing customers to remain engaged for longer."
Larkonis Trassler 2011
The word metrics has been thrown around a great deal with reference to CCP over the past few months. Simply put, whether rightly or wrongly, some people are implying that CCP are relying more and more on relatively poorly sourced and hazily interpreted data to make their business decisions. That's CCP's prerogative really. CCP is, after all, a business and if they want to head in a direction that they think will provide them with more profits then they'd be a fool not to take it. However, right now I believe that they're travelling in the wrong direction following a half finished map.
An anecdote, if you'll allow me. During the CSM 3 summit we had a meeting with the GM department. Not just one or two but all the lead GMs and then some, headed up by GM Grimmi and CCP Arkanon (head of IA). Now, the GMs do great work. I don't think there's many people who'd have the patience to put up with doing the job they do. But there were problems and there are still problems with regard to policies and game knowledge or communication skills among some GMs (this occurred once when I received a petition response from a GM who's first language was obviously not English, it was eventually solved but was no doubt as frustrating for him as it was for me). There were lots of nice blue graphs, the one that showed a 79% satisfaction rating with concluded petitions really piqued our interest. The GMs seemed quite chuffed at this. We weren't. We had a hard time convincing them of our opinion that their 'metric' was skewed. The vast majority of satisfactory outcomes will have come from easy to fix issues like billing and whatnot. Again there was no way for CCP to measure this. If you're anything like me, by the time you've bounced your petition around you just rage close it and don't rate the thing (not to mention I wasn't aware of the damn feature until maybe 3 months prior to the summit). Now, this is something players have themselves to blame for partially but if GMs had spent more time communicating and interacting with the players they could have gained more accurate metrics (regular devblogs on how the GM department works and how to file and rate a petition) and also prevented a lot of resentment (clear policies on things like reimbursement and the like, they are there but they're hard to find). At the time however if they'd looked at the forums (and I realise they are sometimes ruled by a very vocal minority) they would have realised the volume of dissatisfaction that the players were displaying despite their :metrics:.
Metrics are only one piece of the puzzle. Time spent on reconnaissance is never wasted. In my experience it's far better to get 'eyes on' a problem and analyse it up close with your own eyes and ears than by viewing data presented on a spreadsheet. As a soldier I wouldn't be comfortable planning an attack on a position with an incomplete map copied from a hazy airfot, if I could I'd try and have a look at it myself. As an engineer if I'm given a problem to solve I'll get a look at the equipment and talk to the operators, not thrash out a solution based on reports and drawings alone. Likewise the problems that CCP face shouldn't be dealt with relying on second hand data interpreted by people with no connection to the issue (I'm looking at you PI). It might require a bit more of a time investment and maybe a bit more risk from the people on the coal face, they may risk having a few naughty words uttered in their direction or may even let slip the wrong thing in a discussion, but you'll end up with a much better end result. The CSM was supposed to bridge this gap, and to an extent it has, they've managed to start the ball rolling, there are more Dev blogs, more forum poasts and we even had a live event last night. Let's see a bit more. I recall Torfi being interviewed for Eve TV and saying he occassionally cruised round watching people in local. Don't just watch for God's sake man! Get amongst it, get some real opinions from real people on real problems.
Horizontal expansion at work: The future of Eve. This is no good at all.
Building on core design and iteration. See how much better this is?
Oveur's quote came to light during the first CSM 5 summit during the infamous :18 Months: debacle. We'd heard similar things during the CSM 3 summit, Quantum Rise was held up as example of an expansion that was full of fixes, clearing a lot of the 'technical debt' but slim on features and it was regarded as being not very successful at all in terms of boosting subscriber numbers (I'd argue that this poor uptake was probably partly due to the nano nerf and missile changes, but then, I really loved my nanos). I'd like to have seen the metrics on the post expansion droop in subscriptions that inevitably occurs once the hype has died down and the bitter vets realise that nothing has really changed. How long can you keep expanding outwards, continually adding new features and ignoring the core of the game? By concentrating on returning to 'old' content and building upon it you may forgo the initial surge of new subs and resubs that you get when a new feature is released (this is disputable though, it may draw back players who felt upset about the lack of content in that area), but how many of those really stick? Fixes and iteration of features are an incredible aid to player retention FACT. I don't have any metrics to back this up unfortunately, because, well, CCP don't have or don't release them, I do have access to a great many almost delirously happy forum posts, blogs and opinions passed by word of mouth about how such and such an upcoming fix will make their lives so much better. Even smaller fixes which don't include any content of note are enormously well recieved (as seen with Team BFF's efforts with Incursion 1.2, I do hope they are allowed to continue what they're doing). But, I fear these are like plugging individual holes on a colander. How much 'old' content and how many incomplete features have been ignored time and time again come patch or expansion day? Incarna and Dust are, unfortunately, a rapidly approaching reality. Once they're done, if it's not too late by then, it will be time to slam the anchors on and begin to shore up the base and weld that colander into a shiny pot.
No comments:
Post a Comment