-------------------------------------------------- A Proposal for Quality Assurance at CARE by Michal Wallace -------------------------------------------------- Contents: Introduction QAA's Required Skills QAA's Goals QAA's Role in the SDLC Design Development Deployment Evaluation Summary Introduction All too often, Quality Assurance comes too late in the Development Life Cycle. It is almost as if the QA Analyst is given the role of a book critic: able to point out what works and what doesn't, point out flaws in style and structure, but with the work already done, powerless to do much aboutit. Ideally, the QA Analyst takes on the role of editor. An editor does everything from inserting a comma to working with the author to re- arrange the whole story. Their interest is split between helping authors create the best product they can create, and demanding that the audience gets what they pay for. CARE has been using the critic style of post-hoc QA for too long. We need a better process - one that starts in the design phase and carries through until after the product is deployed - and Quality Assurance Analyst to implement it. So what makes for a good Quality Assurance Analyst? And what would this process be like? This proposal attempts to answer those two questions, and to argue that the role is much like the story editor's afterall. * QAA's Required Skills Software QA Analysts need a variety of skills to perform their jobs. The two foremost are a strong computer science background, and the ability to understand and model business processes. Specifically, they must be able to: - Analyze database design. - Design SQL queries, either by hand or using visual tools such as Access. - Write simple computer programs (for automated testing) - Understand and discuss more complex source code. - Analyze software usability ("How does the UI help or hinder business?") - Write quickly, clearly, and well, with proper grammar and attention to detail. - Work with the BSA's to model exactly which processes occur in the existing system (or lack thereof), and which must occur in the new system. - Design test scenarios (stories) that model real-world usage of the software. - Proofread and/or edit system documentation. - Interview users and support group to gather feedback once the system is deployed. QAA's also require several "soft" skills. They must be able to: - Ask specific questions to gather details about reported issues. - Keep track of issues related to many different products. - Mismatch, and listen for what's missing in a solution. ("But what about...?") - Think like a power user or a complete novice. * QAA's Goals A QA Analyst's main goal is to make better software. He or she will also: - Design, maintain, and implement quality assurance tools (such as test scripts and bug tracking databases) and written policies. - Aim to improve documentation and training procedures. - Organize an ISD test group to help with multi-user and stress testing. - Work with the ISD steering committee as a technical resource familliar with all systems, to ensure that ISD is serving CARE in the best possible fashion. * QAA's Role in the SDLC * Design The QAA should participate in on all major design meetings, and SRS/FRS walkthroughs. In this phase, the QAA has four major roles: - To make sure all requirements are being addressed in the best possible way. - To construct business scenarios (stories) that explain how the system will be used. These stories include interaction between people and events outside of CARE, the actions (and mistakes) of CARE employees, and the software. - To collect sample data for each possible scenario. This is important, as it helps to insure that the application is robust enough to handle the many exception conditions that turn up. - To evaluate and critique functional and database design, and ensure that the design supports all test data and scenarios. * Development In the development phase, the QAA takes the test data and scenarios and begins using them to test the system. The general duties include: - Take each build and manually evaluate all changes and additions the developers have made. - Report bugs, enhancements, and suggestions to the developers via a bug tracking tool. (When necessary, the QAA will organize formal or informal meetings to discuss particular items) - Collect bug reports from developers, BSA's, users, etc. The QAA will make sure that each bug is adequetly described so as not to waste the developers' time, and act as the clearning house for assigning them to the developers. (ie, all bugs come through QA.) - Take the user interface shell to the users (or some similar group) and ask them to act out the test scenarios. By watching how well the users interact with the interface, the QAA can uncover and correct usability flaws. * Alpha Testing As the interface stabilizes, testing becomes more redundant. Testers must check to make sure none of the fixes re-open old bugs. This sort of regression testing is best automated by having a testing program act out the scenarios. In this phase, the QAA will: - Build automated test scripts (using a tool such as Microsoft Visual Test) to act out the test scenarios. - In very rare cases, the QAA may be unable to test all variables in a particularly complex scenario. When this happens, the QAA may opt to manually review a section of source code and validate the programmer's logic. - Test system and installation against a minimally-equipped user PC (whatever the current standard is), and "clean" this machine between tests. - Give copies of the test scenarios to the Technical Writer, who can then use them to construct a tutorial for the users. * Beta Testing - Call on actual users (or other ISD members acting like users) to physically use the system. First use the same test scripts that the automated system uses. This allows users to try their own unique style in using the program. Users should make notes of bugs and anything confusing, or needlessly complex. The QAA will evaluate these responses and decide whether it is a bug, a training issue, or a usability problem. - Write multi-user and stress-test scripts, and coordinate their use with the ISD test group. - Test for things that the user might do. these may not be supported features, but they shouldn't break the system. (ie, accidentally importing the wrong file or type of file) * Deployment In the deployment phase, the QAA should participate in training and coordinate with the help desk to track incoming complaints. Specifically: - Review training plans (with the technical writer) to make sure all topics are covered. - Offer critique of trainer's presentation style to ensure that users get the best training possible. - Take note of questions asked during user trainings. - Once the system is deployed, check in with the help desk (or monitor Track-It, etc) for responses, questions, etc. These will be addressed on an issue-by-issue basis. * Evaluation Finally, the QAA will be respsonsible for eliciting and evaluating user feedback for future development. Specifically, the QAA will determine: - What parts of the process are still being done manually, and why. This may be a training issue, or it may uncover additional required functionality in a future version of the product. - Which parts of the process take the most time, and wether they can be further optimized. - In web applications, how widely the system is used, (by tracking usage logs) * Summary The QA Analyst must demonstrate a variety of computer science, business, and communications skills through all phases of the software development life cycle. Essentially, the QAA's role is that of storyteller and editor, developing scenarios to explain and test software's role in business operations. By following the suggestions in this proposal, the QAA will create a robust Quality Assurance system within the CARE ISD.