I’m not sure what to think about Sony’s forthcoming reality show “The Tester“.
On one hand, they’re mocking quality assurance by turning it into a competition.
On the other hand they’ll hopefully get people to talk about testing issues and this is a rare public look into what is usually a secretive business.
Bugzilla, why can’t I select from a list of the most common people to assign a bug to?
Instead of that, I have to use my powers as a bugzilla admin to look through the user database and find the e-mail address of the person I want, since otherwise I might use the wrong one.
A god-damned e-mail address! What is this, 1998? Am I on a fucking NeXT here?
Give me a god damned list with AJAX and offer me a drop-down autocompleted list of people when I start typing their name.
There has been some really interesting discussion at two very different places recently regarding the quality of life issues for developers. I just wanted to point them out, because I think they’re important not just for developers, but for Quality Assurance personnel as well. I don’t think it is even possible to get a job in the United States as a regular QA contractor without agreeing to work whatever hours you are given and on whatever schedule you are given.
There is also something very interesting in how Epic’s Mike Capps treats his employees, versus what he has said in the past when running for the igda board.
My one hope is that some day the United States comes around with this Working Time Directive style of doing business and eliminate the ability for any non-essential-social-services people to be worked beyond 50 or so hours.
This has been given a much better treatment in the light of day elsewhere.
Full video of Mike Capps dishing out his bizarre treatise on why it is OK to overwork people after the break, the warning here is that it is long, but well worth watching. Please let me know what you think in the comments.
Overall I want a lot from Bugzilla that it doesn’t deliver yet. Most of all, I want Google Gears support.
Why?
Because then I can edit/add to bugs when I’m offline and come back later and have it actually do the changes.
It’ll make it more likely that I work through my bug lists instead of ignoring them.
What do you want to see from Bugzilla?
One of the major downsides to quality assurance is the lack of a career advancement path.
On a project nowadays, you may have forty to sixty temporary testers, after which maybe two or three are brought on as full-time testers. In a department of forty to sixty full-time testers, you may have maybe five leads that cycle between products or ten leads that vacillate between leads and individual contributors. Finally, you’ve usually only got a single QA Manager, although some organizations are splitting it up so that there is a manager per fifty testers or so.
In short, you have a major funnel from temp to tester to lead to manager and if any level is filled, your opportunities for advancement are severely restricted.
The question is…why should the funnel exist? There are many ways that organizations can allow testers to grow without forcing them through the funnel. These are a few of the possible ways that your organization can allow growth within QA without “breaking” the funnel.
My people have a saying: “Cate bordeie, atatea obiceie”; the rough translation of that would be that each household has its habits. I have witnessed the truth of that in my career as a game tester – truly each and every company I have worked for or with had its own approach to testing and quality assurance.
The first game company that employed me as a tester was a large European developer with thousands of employees across the world. They’ve been around for a while, and as such had well-established procedures and standards. That was rather fortunate for me, for it set my expectations for solid documentation and clear processes. It was also one of the few companies I have seen to use negative testing, although that was rather the testers’ own initiative rather than any official procedure. Another cookie to them for using test coverage assessments in functional tests, something I have yet to see in the other game developer companies I’ve worked with.
Having such a massive testing department, the focus was less on the quality of testers and more on reporting: there were very strict rules on how to submit bug reports, down to capitalization and punctuation. We had to write daily reports detailing the progress of testing, in a very specific format. However, during training we were never taught *how* to test. We were drilled into the use of the bug reporting system, the Great Tome of Wise Policies and Ergonomic Rules, we were shown how to check the Java files against the technical documentation, but when it came to actual testing, “Just play the game and find bugs” could sum up our training.
I find that this is an oversight that quite a few game developers are guilty of. They assume that gamers know how games should work in general, so they don’t need to be instructed on what it means to be a tester.
My current company, an independent middle-sized MMO developer, has initially made the same mistake. but since I am now the one writing the training materials, I am trying to correct that, teaching new testers how to… well, test. Still, we are recruiting testers from the ranks of our players, in the (not entirely mistaken) assumption that they are better suited to test the product. However, I find it that often it is the testers who have no previous knowledge of games who find the most glaring design and usability problems. They are not grown used to ignoring them by their years of gaming experience, where the same mistakes are being made over and over until they become almost features.
Thus, I am ever appreciative of the fresh perspective of my Indian team, which are testing for us as outsourced partners. I have heard saying that working with Asian testers is usually a pain, they have no initiative and no imagination. True, there are huge cultural differences between us, but what has been branded as a lack of initiative I have found to be rather a … shyness. Almost one year of working together has eroded that shyness and they are more efficient than ever. I can rely on their thoroughness to spot every little typographic error, every glitch and code error. I can rely on their implacable logic to spot design inconsistencies that are otherwise overlooked. They take such an immense pride in doing their job well – their team motto is “Let no bug go free”. And from what I hear these are common traits in Indian testing companies – this aspiration to excellence, the rigorous organization and amazing spreadsheet-fu.
Alas, one thing I still find difficult to teach them is negative testing. In fact, I believe it is a hard thing to learn for anyone, as I found precious few game testers who knew how to thoroughly break games. In a recent conversation with a tester for a relatively well-known North American game developer, I asked if they ever did negative testing and his reaction was rather bemused: “Err, no, we only tested to see if it works”. On the other hand, their company’s test process seemed more organized than our own, with prodigious checklists and test scripts, so I suppose a more formal testing leaves little room for the exploratory “let’s see how we can break this” method.
That being said though, I am very curious if any game developer out there uses formal test design techniques, like, say, orthogonal arrays or decision tables or equivalence class partitioning, since all that I know of only use exploratory testing and sometimes use-cases. One of or other test teams uses test driven development methods for their unit tests, but they are mostly programmers and don’t use any formal design techniques that I know of. This has been a subject that’s been haunting me for a while now – how can game developers adapt these techniques, many of them quite useful outside our domain, to our rather unique flavour of testing? But I guess that is the subject of another rambling story.
After the break, because it hasn’t been displayed enough times already.
Read the rest of this entry »
We’ve all had them…those fellow employees who just suck the life out of any team, department or company that they are associated with.
However, it can be difficult as a lead or manager to figure out whois the poison pill in a team because most of the time, employees will openly bitch about other employees only within their circle and they rarely include management in their bitchfests for fear of being seen as complainers. Because of that, we see the morale problem, but we have problems isolating the cause.
So how do we detect the problem people and once we do, how to we correct the problem?
Back in 1940, Chevrolet released a management training video called “Hired.” You can view the second half at the beginning of the “Manos: The Hands of Fate” episode of Mystery Science Theater 3000, or you can watch the entire video over that the Internet Archive. If you are having problems getting your testers to perform, stop what you are doing and watch this video. If you can’t watch it, I’ll sum up the points here with the terminology properly changed. Read the rest of this entry »