Green Monster Games (Curt Schilling)

[Zehn - Vhex]

Originally Posted by Ngruk

I've often wondered what 'you' mean, as players, when you say/scream/whine/yell/cry/legitimately complain about companies 'not listening' or being 'open' to player input/feedback during an alpha or beta.

When testing and I report a bug, I know that it may not get acknowledged. I figure I'm not the only one who's probably reported it and the team may have gotten sick of saying "Thanks" the 372nd time someone reports that they can't stand back up after sitting on a bench (that was a fun one). We also can recognize that it can take time to fix some things and they may be low priority.

When you report a mining node being underground, twice, and 4 years later it's still not fixed. Or when it takes THREE YEARS to fix judgements not always landing. Not missing, being dodged or parrying. Just when you judge, you lose mana and nothing happens. Or how 4 years later they still haven't fixed the combat processing problem where your heal lands, you lose mana, but the tank dies anyway.

It starts to make you wonder if they're even listening.

You also have to remember -alot- of us are bitter because of EverQuest and the god awful communication involved there. Alchemy is working as intended indeed.

Anyways...

As for feedback/input and feeling like you have a hand in that, that's more or less a mixed bag. I know I probably wasn't the originator of the idea's and I'm sure I'm not the only one who suggested them, but seeing EverQuest2 implement like, 30 of the things I suggested in the beta over the past couple years just feels kinda neat.

Blizzard has actually gotten really good at that lately. With Bluetracker and whatnot, you see GC and others posting on an almost daily basis commenting on community idea's. I may not always agree with their idea's from time to time (Tigole being against Horde getting Paladins and saying that he likes a lot of trash in raid zones because it's a good pacing mechanic for example), but they've been phenominal about communicating.

There have been other companies however, where the development team seems to have no mouthpiece. Turbine was pretty terrible at times. NCsoft has had issues with that. Sony was downright terrible at communication during EQ.

You obviously can't respond to every post. But you should at least make an effort. These people are an asset for you to use. Make use of it and more importantly, let them know they're helping more then just a once a month PR spin by some face without a personality.

Anyways...

Originally Posted by Draax

I have tested ( not open beta test ) 3 games so far and I don't get why some people feel they have a reason to want/need to change the aspect of the game with things such as combat or whatever in beta let alone alpha. That should be up to the developers if they feel their system isn't what they like, and its not like your paying for the game when testing (maybe wasting your time). At least you get to know not to buy this game once its released based on the core fundamentals of it.

Developers miss things. We have the luxury of getting to nitpick while they have to look at the grand scope of things. MMO's have the advantage of being able to change things. The earlier they start listening to the players finding these things, the better.

Typically this is more when it comes to matters of convenience. Remember having to click each time you wanted to combine something in WoW? Originally there was no "Combine all" functionality. If you had 120 pieces of linen cloth and wanted to make linen bolts, you had to click combine 60 times.

Remember the days before linked flightpaths? I actually suggested it in beta, many people did.

Developers miss this shit all the time.

And even more then that, you gotta remember, they've been at this for 3+ years at this point, sometimes even closer to 6+. It's become a labor of love so some grand feature they think kicks ass may be a piece of complete shit to somebody who's only experiencing it for the first time. I would sacrifice a virgin goat if it meant I could go back in time, find the Mass Effect team and tell someone that the Mako fucking sucks and that they need to give it a "hover over fucking mountains" ability.

Anyways...

We also have a stake in the product as well. We may enjoy 70% of what a game has to offer. It's our duties to ourselves to try to make it 100%. Granted everyone has different ideas of what would make it perfect, but that's why there's a development team that can tell you "No we're not going to give rogues deathtouch. We'll just give that to DK's instead LAWL"

You're doing yourself a disservice if you keep your mouth shut if there's an aspect of the game that pisses you off. You can't just say, "Well..I only like 70% of what your game has to offer so I'm only going to pay for 70% of it." It doesn't work like that. If you're going to pay full price for a game, you damn well get to say what the fuck annoys you about the game. They may not do it, but at least you get to say it.

[CylusSoulreaver]

I've yet to participate in a Beta where it was "easy" to report a a bug.

I should just be able to press a shortcut on my keyboard and create a bug that will fill in the "necessary" information: offset in the world, the region (dungeon, city, etc...) that I'm in, a capture of my screen, my tty and chat log, whatever quests/bitflags/etc that I have active, my buffs/debuffs, my level/class/spec, etc etc... All of those packets and data are available so not sure why it couldn't be used to make reporting bugs easier. If folks in Betas could simple press a key (spam and data requirements aside) and give you most of the info that you need, the team would get much more feedback (assuming QA/GMs/Producers aren't being too heavy handed when filtering it).

Get rid of the /bug or menu->bug report UI buttons and make it really simple to report something. If people abuse it, kick their stupid asses from the closed beta and find people that actually will contribute. Easier that it is to contribute, the more people you'll have doing it *shrugs*

Am sure that GMC has some good internal QA tools; just need to extend that to the smaller betas and make it easier for them to report stuff

And Zehn, who's to say that your flightpath suggestion reached deaf ears? There's always a schedule to keep and things that are higher priority

[Zehn - Vhex]

I'm not saying it was on deaf ears. It was just more of a "oh hey, neat" kinda thing. I think every player feels that when something they've been wanting to see for a long time finally gets put in.

[Flight]

There are two issues being debated. To underline them :


1. Betas, historically, have been massively counter productive due to the mentality, expectations and application of the testers.

2. QA process development - including community involvement - in the industry has been extremely poor. Management of those processes has also been poor. This has been convoluted by the various degrees to which companies have committed to using their community testers in their QA processes and/or using the 'beta' phases as PR.


While we are discussing them as two points, they are not disparate - 2) is causal to 1).



If the process development, communication and methodology involving the 'community' is poor then it's almost certain that the company will have the same problems in house. It's the culture that is inadequate. For example, lack of in depth technical knowledge or coordination/communication by line management leads to uncontrolled and inefficient use of dev time. eg devs just turn up at work, takes a look at the bug log, picks whatever strikes their fancy, works on it for 2 days when its fixed after 1, to get it 'just right' and enhance it.

You can't fix this from the top down. It has to be right from the foundations up. It is yet another (series of) example(s) of the culture I've been banging on about for years. Note, due to my 'mancrush' on Brett (cheers for that one, Steve ) and confidence in 38S, I'm certain this won't apply to Copernicus - which is why I was willing to put money on them not having the kind of 'betas' we are used to.



On Zehn's point, about feedback to the testers, I agree. I'm in a high profile beta at the moment. I get into the game and it's unplayable due to a graphic glitch. I've bugged it and started a thread on the relevant part of the beta forum, asking if it's a known bug or should I keep trying to fix it my end. It's been quite a while and I've had no response. They've lost a couple of weeks of my time due to lack of someone their end coordinating the bug dB with tester communication. Feedback does not need to be given on minor bug reports, but there certainly needs to be coordination and communication, if the process is going to be in any way efficient.

[Draegan]

I hate beta hippies too Zehn.. Oh I hate them so much.

WAR had a pretty good beta process. Decent bug reports. Great communication in the forums. But the execution of what they deemed a problem or what was reported as an issue was retarded.

[darksensei]

Originally Posted by Ngruk

I've often wondered what 'you' mean, as players, when you say/scream/whine/yell/cry/legitimately complain about companies 'not listening' or being 'open' to player input/feedback during an alpha or beta.

TLDR: Don't ask for player feedback on an issue and then do nothing. Other than that, players shouldn't expect anything other than legitimate bugs being fixed.

I agree that beta testers should be there primarily to report bugs, and they should not be designing the game. So, here is the best example I can think of where it was frustrating to be part of the beta process. On the WoTLK beta boards they posted threads for players to give suggestions for ret talents, holy talents, and prot talents. So players proceeded to give 20+ pages of feedback on each of the specs, and many of the responses were quite detailed because they asked which areas in the trees seemed dense or sparse, or which talents felt mandatory or useless. So over the course of the beta we see some decent changes to the retribution and protection tree, but all the Holy feedback yielded zero results. Then, perhaps in October, too close to release to change much, they create a post saying they think Holy Paladins are "amazing" healers.

Now were they OP in WoTLK launch? Once geared, no question. But it's not because the talent design was solid. So it felt extremely frustrating to have them directly ask for feedback, see positive results for the other two trees, and see nothing done to the holy tree (sacred cleansing is still in the game).

[Draegan]

The last beta I was in was WAR. There were plenty of people talking about how slow and unresponsive combat was. How world design was bad. How the leveling curve was good! but they nerfed it right before release.

They basically had all the tools they needed but didn't heed any of them.

Most devs shouldn't listen to the playerbase most of the time.

[tad10]

Originally Posted by Ngruk

I've often wondered what 'you' mean, as players, when you say/scream/whine/yell/cry/legitimately complain about companies 'not listening' or being 'open' to player input/feedback during an alpha or beta.

What's the expectation? This thread could go wildly askew if people give asshat answers

Nice way to sidestop folks asking for more info about your game. Kudos.

* * *

Fruitless line of inquiry. Devs are always going to think they're smarter/know more than the playerbase, and (depending on the playerbase) they may be right. Exceptional beta testers are going to get lost in the crowd.

The information flow is something like beta tester -> bug report form -> internal QA guy -> a different bug report form/tracking list -> internal tester -> a confirmed bug report form (prioritized) -> development manager -> a bug tracking list (prioritized) -> development guy (maybe bug gets fixed). There's so many ways this can get screwed up, it's amazing any bugs reported by outside beta testers ever get fixed -- if they do get fixed it's probably because an internal tester/developer reported and prioritized it. The exception is for those few bugs reported by many, many people.

That's why Zehn's underground node has never been fixed -- maybe a Blizz dev will read this thread and fix it now.

[Ninen]

Its great seeing publicized bug tracker style info.

Its one thing to submit something you don't agree with and have it just disappear into the void. (And really, a significant portion of what we feel are bugs are this. The legitimate BSD or underground mining node or whatever is a rarity.)

Its another to submit a bug, see that it has been automatically added to a list by the end of the day, and be able to look it up on BugTrax or whatever and see a comment as small as "not a bug" from the devs. Your issue will never be resolved (unless the masses, and therefor the suits, agree with you); but you at least have closure.

Depending on the setup, you can sometimes see how big a priority or where on the list the dev's are at, and all that.

And in the end; all those "its not a bug, we want it this way" listings become the seeds of the FAQ for your mouthpieces and public.

[Ngruk]

Originally Posted by tad10

Fruitless line of inquiry. Devs are always going to think they're smarter/know more than the playerbase, and (depending on the playerbase) they may be right. Exceptional beta testers are going to get lost in the crowd.

Hard thing is to understand, for the most part, they do. They can't NOT be smarter than you and know more than you about the game you're testing, and they're making.

That's not to sound pompous or arrogant by any means, it's just true. I've often watched players explode on boards when they feel they aren't being listened to or aren't making an impact, even though they are clearly providing invaluable and important feedback.

However the value and importance to people is often one of perspective.

Is that node bug important? Sure is. Is it more important than the exploit? likely not. Nothing happens in a vacuum and often times those bug lists run in the thousands, and it becomes very much a matter of priority in getting bugs fixed and builds completed prior to patching.

The other thing to think about is that your node bug? Sure it may not get fixed on the pass submitted, but that doesn't mean it moves up in priority during the next build or fix or patch. You should have a game that sees fewer bugs per patch for sure, but that node bug still falls in line and priority the same way sometimes, through many patches.

It is a frustrating experience to be a beta tester, imo, ONLY when there is a lack of, or absence of respectful TWO WAY communication. Unfortunately there aren't enough QA managers, or community folks, to respond to every tester or every post.

[Ninen]

Part of the rage at stuff like the node bug is that we know how tiny and silly it is, and think we know how easy it would be to fix. Loading up the zone file and fixing a z-coordinate shouldn't take up an afternoon.

Now maybe something is fucked (be it internal process, be it your toolset, be it spaghetti code, be it whatever) and doing that *does* take up more than 30sec of work. That's its own problem.

And if these tiny things always get pushed off until the higher priority things are fixed... they'll never get fixed. There will always be bigger fires.

Now maybe you setup something were a certain volume of tiny stupid things get an aggregate listing and higher priory in the queue. We've all seen that x.x.1 patch whose patch notes are 8 pages long but only 400k download.

Perhaps its simply a coder who decides he's tired of looking at this on his list and/or explaining why its not fixed yet on the forums/to the forum guys and thus, works through his morning coffee break to get'er'done. We've seen that too.

Unfortunatly, other than simply treating each of us like GOD, there's no right way of doing it.

Communication, hopefully by folks who haven't proven themselves to be worthless asshats (yet), is the key to making the bitter pills go down easier.

[Jovec]

Originally Posted by Ngruk

Hard thing is to understand, for the most part, they do. They can't NOT be smarter than you and know more than you about the game you're testing, and they're making.

That's not to sound pompous or arrogant by any means, it's just true. I've often watched players explode on boards when they feel they aren't being listened to or aren't making an impact, even though they are clearly providing invaluable and important feedback.

Your points on communication and priorities are spot on, but I want to comment on the quoted text.

Players do often know better than devs. Although they might not know why an aspect of gameplay works or does not, they know because they experience the end result of it with relatively little bias. They aren't privy to the original design ideas and concepts and don't have any personal or time/budgetary attachment to them either. Perhaps this is more to do with focus testing than beta testing, but being open to feedback that may run counter to months or years (should focus test sooner) of work can really be insightful. Bug fixing is also separate and is more about proper allocation of resources by scheduling enough time and money to fix them - bug fixing and code cleanup milestones need to exist and cannot be an afterthought or last minute thing.

The biggest and most recent example I could think off is WoW's dual-spec implementation. It completely missed the mark with the original proposed level 80 limit, and it mostly still does as the players who can't afford the 1,000 gold in their 40s are the ones who need the benefits of dual spec the most - in general they don't have the same access to guild, gear, and power-leveling resources that make specs more on a non-issue as they are leveling. Blizzard simply could not fathom abandoning their original 1-spec system that far to make it both very affordable and available to even lower levels to promote grouping. Just like their original meeting stones.

[Lonin]

Originally Posted by Ngruk

Unfortunately there aren't enough QA managers, or community folks, to respond to every tester or every post.

Is that the case, or is it that most companies aren't willing to spend the money to hire enough community folks? I imagine it can be difficult getting execs to sign off on more staff when the community section often doesn't produce easily tangible results.

If you really can't find enough community people, my schedule is open.

[Ngruk]

Originally Posted by Lonin

Is that the case, or is it that most companies aren't willing to spend the money to hire enough community folks? I imagine it can be difficult getting execs to sign off on more staff when the community section often doesn't produce easily tangible results.

If you really can't find enough community people, my schedule is open.

It's NEVER about hiring 'enough', that's never going to be a problem. It's about hiring enough of the RIGHT people.

Your community folks, especially in this space, are the end all be all of your company. Once your game hits the market your community MUST be as good as it ever will be and IN PLACE running SEAMLESSLY.

There is no second chance in this industry and that's especially true in this arena.

One of the hurdles we've had to overcome is this industry telling me "we'll handle your CS, and QA...."

Uh, no thanks. That would be like Disney outsourcing their park guides. If your CS and QA folks aren't as committed and invested as everyone else, there's trouble to be had.

One of the ironies I have found in the past here is the willingness to create 'entry level' positions in what I think is arguably the most crucial area of a companies business, CS!

I disagree with the tangible results because, and I think I said this earlier, I am one of those customers with a lot of discretionary income, and I will absolutely be loyal, AND pay more for service that makes an experience enjoyable for my family and I. I will ALWAYS fill up my car at one of three stations, regardless of price of gas, simply because all three places have people that make me feel like I'm the only customer, and are always respectful and kind.

Same goes for my gaming. I've been a HUGE fan and loyal supporter of Voodoo for many many years. I was because Rahul and the gang (Pre buyout) we're open and personal as any company I'd ever met.

[Gnome Eater]

Originally Posted by Ngruk

Hard thing is to understand, for the most part, they do. They can't NOT be smarter than you and know more than you about the game you're testing, and they're making.

I am sorry, but while it might be nice to think this, this is probably incorrect.

During the StarCraft beta, several players were much better and knew a lot more about StarCraft than developers. Blacklizard was the first player who famously used to crush blizzard developers at their own game, while it was in beta.

Developers might have the benefit of a more holistic vision, but just a quick browsing of EJ will show you that there are people who can write several page essays about incredibly technical details of how certain spells work.

Manly for example figured out an explot on how to abuse rolling ignites after they were fixed by using a very specific amount of haste and and abusing the travel time of certain spells. The problem is that developers lack any ability a priori to figure out which players are actually worthy of paying attention too and which ones are just idiots who happen to write in a non-retarded manner.

In the end, it is just not worth it to write novellas when they will most likely get lost in the signal-to-noise ratio. The most easy way to gain direct access to developers is either by playing alongside them or by having a fan-site with a huge following - for example, several fansites have direct access to developers even though the people writing for those fansites are awful theorycrafters.

edit:
I agree with you entirely about CS. Blizzard CS is downright amazing, professional, and usually curteous. They also have fairly strict guidelines, but if you ask for something politely and are fairly reasonable about it, they tend to err on the side of helping you out (for example, refunding items a few hours after the theoretical deadline)