Introduction
Overall Objective for Software Test Activity
- Validation of requirements as detailed in Software Requirements Specification
- Verification of functionality as detailed in Concept of Operations
- Detection/Isolation of functional issues
- Detection/Isolation of exceptions
- Determine status of current project revision
- Pre-Alpha: Pre-testing stage
- Alpha: Internal module testing and functionality testing phase
- Beta: Software deemed feature complete, with bugs
- Release Candidate: Candidate for final product
- Release: Production release
Reference Documents
- Concept of Operations
- Project Management Plan
- Software Requirements Specification
Description of Test Environment
Configuration
- Operation System: All
- Desktop: Google Chrome v37+, Firefox v30+, Internet Explorer
- Mobile: Android, iOS
- Active network connection
The test environment will be the same as the user environment and will have the configuration outlined above.
Issue Reporting/Tracking
In the event of a test discovering one or multiple error(s) during its automated run, the developer running the test must log relevant test information such as test type/purpose, where/how it failed, and if possible, provide insight as to the potential cause of the error or solution that will prevent the test from failing in the future.
After the test failure has been logged, the developer may only continue testing if error implication has been assessed, or if he thinks he can fix the problem himself, he may transition into the code, attempting to debug the code.
At any point in which a developer believes he has fixed an issue that has been logged by a failed test case, that developer should re-run the tests that caused the failed entry, and if the test passes, the developer should mark the associated failed test case as fixed for all other members to see, with a brief summary of what was done to fix the problem.
Stopping Criteria
Errors discovered during the testing procedure may or may not render other further test cases obsolete. For example, if test case for verifying database connection were to fail, it would not be practical to test cases which involve database operations; therefore, the test procedure will stop. On the other hand, failure of the test case checking if events are viewable to other users does not constitute stopping test procedures checking if a user is able to create an event. Once an error has been identified, its implications must be assessed before a decision to halt testing can be made.
Unit/Integration Testing and System Testing procedures of the V-Model can be administered in two stages for a comprehensive testing procedure. The first stage is the functionality test which will use the set of “expected inputs/actions” as the inputs provided to the unit and/or system, whose response will be checked against the “expected response” to decide if the test case have passed or failed. Failure of a System Test will entail verification of System Design, while Unit/Integration Test failure entail verification of Program Design.
If no test cases fail, we can further execute a second stage of exception testing by providing “unexpected or unlikely inputs/actions” as inputs to the unit and/or system. If an expected response to such an input has been established (exception handling implemented), it can be used determine a failure or pass of the test case; otherwise, the response is logged and reviewed for determining potential vulnerabilities (security) or flaws that can be addressed to improve stability and reliability of the application.
Tests should be functionally comprehensive, meaning they should conclusively determine the functional state of the project. In the case that no tests fail during a comprehensive test of the system, the developer should feel confident that the program is fully functional. Though this is a positive result, it does not imply that the work is done. A functional system test can not go through the application and look for design “bugs” or implementation failures. Following a successful run of tests, the developer should continue to look through the application for any cosmetic issues. If a design issue is found, it should be documented the same as other failures of the system.
Any live or production version of the software should comply with the following standards:
- All core features from the “Must Have” section of “The Proposed System: Operational Features” must completely pass all tests and be fully functional. [see Concepts of Operations]
- All core “Must Have” features should have a user friendly interface and easily accessible by the implemented interface of the system.
- Any “Would Like to Have” features from the “Concepts of Operations” document must either be implemented fully and available to the user, or not included in the production version until they are complete, tested, and fully functioning.
- Known cosmetic and design flaws may be included in live versions of the application if they do not interfere with ANY feature of that production version. Also, design flaws should only be left in cases where they are trivial by nature and unnoticeable to the average user. Any known design implementation failure which is deemed obviously incorrect should be corrected before moving to the software’s production version.
If at any point, an unknown bug is found on the production version, the site should be updated at the next available designated maintenance time. If a severe bug is found on the production version of the software, the site should be replaced with the last known running version and the found bug should be fixed and the changes should be updated to production after extensive testing at the next available designated maintenance time.
Description of Individual Test Cases
| TEST NO. |
1 |
| OBJECTIVE |
Verify new user data is being sent, accepted and added to database |
| ACTIONS |
1. Enter new user’s name, email, desired username, and password.2. Click “create New Account” button3. Check the database in the table where the user information is stored and make sure that the information entered has been transferred to the appropriate spot in the database |
| CONDITIONS |
Start at login page of website |
| EXPECTATIONS |
Database table for users should contain the account information for the new user according to the given input. Additionally, a unique user ID should be created as a key for the user, and user variable should be initialized to the default settings. |
| TEST NO. |
2 |
| OBJECTIVE |
Verify existing user’s are able to login with their account info. |
| ACTIONS |
1. Enter existing user’s email/username, and password.2. Click “Login” button |
| CONDITIONS |
Start at login page of website |
| EXPECTATIONS |
User should be logged in and redirected to home page of site. There should be some section of the page that says your are logged in with the appropriate user name. This will demonstrate that the database is successfully getting and posting the correct information about the user. |
| TEST NO. |
3 |
| OBJECTIVE |
Verify event creation information is properly stored in the database |
| ACTIONS |
1. Click on a button for “Setup a game”2. Select what type of game 3. Input event location and time4. Click “Launch game” |
| CONDITIONS |
Start at home page of website, logged in as a user |
| EXPECTATIONS |
After clicking on “Setup a game”, the page should redirect to the game creation page. Once “Launch game” is selected, the page redirect back home. The table dedicated to game events in the database should show the newly created game’s provided information, as well as a unique game ID, and other game variables(such as how many people are going, and game “heat”) initialized to their default values. |
| TEST NO. |
4 |
| OBJECTIVE |
Verify existing game event details are viewable to users |
| ACTIONS |
1. Search for an event previously entered into the database.2. Click on the event from a list of returned events |
| CONDITIONS |
Start at home page of website, logged in as a user. Test 3 must have been run successfully before attempting this test. |
| EXPECTATIONS |
User should be redirected to the existing game page. All information regarding that game should be displayed on the page (including: date, time, location, how many have committed to going, how much “heat” the game has, and comments and additional details specified by the game master.) |
| TEST NO. |
5 |
| OBJECTIVE |
Verify users are able to commit to playing at an event |
| ACTIONS |
1. Filter/Search for a game that is in the database2. Click on the game3. Click the button called “Commit to play” |
| CONDITIONS |
User should be logged in and on the home page. Tests 3 & 4 must run successfully before attempting this test. |
| EXPECTATIONS |
When the user clicks on the game he/she should be redirected to the existing game page. When “Commit to play” is clicked, the number of people displayed in the “Number committed” section should be increased by one. The user table in the database should now have information that links the user to the game he/she has committed to. |
| TEST NO. |
6 |
| OBJECTIVE |
Verify users are able to un-commit to playing at an event |
| ACTIONS |
1. Click “Uncommit” |
| CONDITIONS |
User should be committed to play a game. User must on the page of the game he/she is committed to. |
| EXPECTATIONS |
When the user clicks on “Uncommit” on the game page, the “Number committed” should decrease by one. The link between this user’s entry in the users table and the entry for this event in the game table should be removed. |
| TEST NO. |
7 |
| OBJECTIVE |
Verify events are properly filtered by the filter feature on the home/browse page. |
| ACTIONS |
1. Click “filter” 2. Select an event from a list of game types3. Repeat 1&2 (at least once), selecting a different game type |
| CONDITIONS |
User should be logged in. There should be multiple game events in the system. There should be more than one type of game in the system. |
| EXPECTATIONS |
Each time the available events are filtered, the list of events should show only the events where that event’s game type matches the filter selection. All events of that game type should be returned if the date and time of that game has not passed. If a filter is selected in which no events’ game type matches that filter, a message should display informing the user that there are currently no scheduled games available for the selection he/she has made. |
| TEST NO. |
8 |
| OBJECTIVE |
Verify user verification of an event is stored and displayed properly |
| ACTIONS |
1. Browse upcoming events to find the game the user has previously committed to2. Click on that game event to show details3. Click “Verify this game” |
| CONDITIONS |
User should be logged in and on the home/browse page. A game must exist in the system schedule for some date and time in the future. |
| EXPECTATIONS |
When verify this game is clicked, the attribute in the database inside the game event table which specifies how many people have verified this event should increment by one. The user’s view on the site should indicate that he/she has verified the event, and the counter displayed for how many people have verified this event should display the new value from the database above. |
| TEST NO. |
9 |
| OBJECTIVE |
Verify that user profile/information pages are viewable to the user and stored in the database |
| ACTIONS |
1. Click user name at the top2. View account and user details |
| CONDITIONS |
User should be logged in. Tests 1 & 2 must have been run successfully before attempting this test. |
| EXPECTATIONS |
When username is clicked, user should be redirected to his/her own profile page. Account details should be visible to the user (including: name, name, email, rep points, upcoming games committed to play in, and past games the user has created/verified” |
| TEST NO. |
10 |
| OBJECTIVE |
Verify that user rep points are added when the user verifies an event |
| ACTIONS |
1. Browse upcoming events to find the game the user has previously committed to2. Click on that game event to show details3. Click “Verify this game” 4. Click user name at the top5. View number of reputation points this user has accumulated |
| CONDITIONS |
User should be logged in and on the home/browse page. A game must exist in the system schedule for some date and time in the future. Test 9 must have been run successfully before attempting this test. |
| EXPECTATIONS |
When verify this game is clicked, the attribute in the database inside the users table specifying the number of reputation points that user should increment by a set amount. When the user click his/her username, the user should be redirected to the user’s profile page. This page should indicate that the user’s reputation points have increased by the set amount for verifying an event. |
| TEST NO. |
11 |
| OBJECTIVE |
Verify that user rep points are added when the user creates a new game event |
| ACTIONS |
1. Click on a button for “Setup a game”2. Select a game type from the dropdown or input your own game type3. Input event location and time4. Click “Launch game”5. Click user name at the top6. View number of reputation points this user has accumulated |
| CONDITIONS |
User should be logged in and on the home/browse page. Test 9 must have been run successfully before attempting this test. |
| EXPECTATIONS |
When “Launch game” is clicked, the attribute in the database inside the users table specifying the number of reputation points that user should increment by a set amount. When the user click his/her username, the user should be redirected to the user’s profile page. This page should indicate that the user’s reputation points have increased by the set amount for launching a new an event. |
| TEST NO. |
12 |
| OBJECTIVE |
Verify that event locations are correctly pinned on map of UCF campus |
| ACTIONS |
1. Browse/Filter through events to find an upcoming game2. Click on the game for additional details |
| CONDITIONS |
User should be logged in and on the home/browse page. Test 3 must have been run successfully before attempting this test. |
| EXPECTATIONS |
When the game is clicked on, the user should be redirected to the game event page. The map of UCF should have a pin designating where on (or near) campus the game is being held. The location of the pin should match the text location input by the user and displayed in event details. |