COS80022 Software Quality and Testing Assignment
Assignment 2 (P&C) –
This part of Assignment 2 is mandatory for all students targeting at grades of Pass, Credit, Distinction and High Distinction.
Testing
Suppose that a teacher maintains a file ‘list.csv’, which contains the information of students in an academic unit. The information for each student (i.e., each row of the csv file) includes the student’s ID, surname, given name, and the individual marks for weekly tasks, assignment 1 & assignment 2. The teacher has written a python program, namely record.py, which undertakes the following four major operations on the ‘list.csv’ file:
- “Add” function. By running the command “python record.py add <student ID> <surname> <given name> <mark for weekly tasks> <mark for assignment 1> <mark for assignment 2>”, the teacher can add the information of a new student into the csv file.
- “Delete” function. By running the command “python record.py delete <student ID>”, the teacher can delete the information of a student with the given ID.
- “Update” function. By running the command “python record.py update <student ID> <surname>
<given name> <mark for weekly tasks> <mark for assignment 1> <mark for assignment 2>”, the teacher can update the information of a student with the given ID.
- “Report” function. By running the command “python record.py report <student ID>”, the teacher can calculate and report the full mark and final grade of a student with the given ID.
The detailed description for these functions of the program record.py is given in Appendix A. You are required to play the role as a software tester, undertaking the following tasks.
Task 1: Traceability matrix (3 points)
Traceability matrix is a useful tool supporting a good test planning as well as efficient test management and control. You are required to create and maintain a traceability matrix (in a tabular format) to map and trace requirements with test assets (including test conditions, test cases, and execution status) created in the whole testing process.
Hint: A sample traceability matrix will be provided in a template (‘SQT_Assignment2_template.doc’). Relevant information about traceability matrix can be found in lectures (e.g., slides 73-75 in Week 5’s lecture and slides 16, 22&41 in Week 6’s lecture).
Task 2: Functional testing (19 points)
You are required to undertake all activities for the dynamic testing of all four major functions described in the beginning of this document and detailed in Appendix A. Specifically, you should complete the following three sub-tasks:
Sub-task 2.1: Test analysis (4 points)
You are required to analyse the requirements given in Appendix A on all four major functions of record.py, and define corresponding test conditions, which are normally elaborated into high-level test cases. Note that each defined test condition should be associated with a professional description (in one or two sentences) about “what to test”. All test conditions should also be put into the traceability matrix created in Task 1 to map with requirements.
Hint: Relevant information and examples about test analysis and test conditions can be found in lectures (e.g., slides 15-25 in Week 6’s lecture).
Sub-task 2.2: Test design (10 points)
You are required to design concrete test cases based on the test conditions defined in sub-task 2.1. Note that:
- Each test case should contain concrete values for input data as well as expected result such that they can be directly executed.
- One test condition may be associated with one or more concrete test cases.
- You should select and apply appropriate techniques (e.g., black-box and white-box) when designing test cases for each particular function (“add”, “delete”, “update”, or “report” in record.py).
- All test cases should also be put into the traceability matrix created in Task 1 to map with test conditions as well as requirements.
Hint: Four sample test cases are provided in a template pytest script ‘test_assignment2_xxxxxx.py’ (further described in Appendix B) and briefly described in the sample document ‘SQT_Assignment2_template.doc’. Relevant information and examples about test design, test cases, and concrete testing techniques can be found in lectures (e.g., slides 68-69 in Week 5’s lecture, slides 26-30 in Week 6’s lecture, and Week 8&9’s lecture content). You have also practiced different techniques for designing test cases in our labs (e.g., lab work in Week 6, 8&9).
Sub-task 2.3: Test implementation, execution and reporting (5 points)
You are required to schedule and execute all test cases designed in sub-task 2.2. You should also analyse and report the results of test execution. Note that:
- Some test cases may have dependency relationships and different orders among them may affect the test results. For example, you may have a test case TC#1 that adds a student with ID ‘123456’ and another test case TC#2 that deletes the student with ID ‘123456’. In this case, it is better to schedule the execution of TC#1 before TC#2. In a word, as an important activity for test implementation, you should schedule the execution order of test cases in a smart and efficient way.
- Although a template pytest script (‘test_assignment2_xxxxxx.py’) has been provided, it is NOT mandatory for you to execute test cases using pytest. You can choose whatever tool(s) you prefer to undertake the test execution activity. Manual execution without the support of any automation tool is even allowed.
- All test execution results (pass and/or fail) should be analysed, summarised and reported. They should also be recorded in the traceability matrix created in Task 1.
Hint: Relevant information and examples about test implementation, execution and reporting can be found in lectures (e.g., slides 31-41 in Week 6’s lecture).
Task 3: Non-functional review (3 points)
You are required to undertake some static testing activities, especially review, for the non-functional characteristics of the program record.py. You should identify some potential issues of the program with respect to one or more quality characteristics, e.g., performance efficiency and security. You can also suggest some solutions to address these issues such that the program can be improved.
Hint: Relevant information about non-functional characteristics and static testing can be found in lectures (e.g., Week 3, 10&11’s lecture content).
Submission
This is an individual assignment, which totals 25 points, 25% of the whole assessment of this unit. This assignment is mandatory to all students. You are required to complete all three tasks and compose your answers into one single document in .doc, .docx, or .pdf file.
Every student should submit his/her own work by 23.59pm Sunday the 28th of May 2023. The assignment should be submitted via the assessment submission system in Canvas which integrates with the Turnitin plagiarism checking service. Also provided in the Canvas’ submission system is a rubric detailing the assessment criteria for each of the above tasks.
You will be penalised 10% of the assessment’s worth for each calendar day the task is late, up to a maximum of 5 days. After 5 calendar days, a zero result will be recorded. Students with special circumstances (acute illness, loss or bereavement, hardship or trauma) may apply for an extension up to five days.
Appendix A: Description of program record.py
This description can be treated as an informal yet detailed requirements for each major function of the program record.py.
Functional hierarchy
The overall functional hierarchy of the program record.py is shown in Figure 1.
Figure 1 Functional hierarchy of record.py
The program has four major functions “add”, “delete”, “update”, and “report”, which are implemented by modules ‘addStudent’, ‘deleteStudent’, ‘updateStudent’, and ‘reportStudent’, respectively. All these four modules will make use of a submodule ‘checkExist’ for checking whether a given student ID exists in the file “list.csv”. The modules ‘addStudent’ and ‘updateStudent’ will further utilise another submodule ‘checkValidity’ for checking whether the input student information is valid or not. The required format for each student’s information is given as follows.
Required format for student information
Each row of the file “list.csv” represents the information of a student, containing six items, which are the student’s ID, surname, given name, and marks for weekly tasks, assignment 1, and assignment 2. Table 1 gives the required format and/or value range for each item.
Table 1 Required format for student information
| Item | Required format/value range |
| Student ID | Must be 6 digits, no letter allowed, e.g., 123456 |
| Surname | Must contain letters only, length between 2 and 35 (inclusive) |
| Given name | Must contain letters only, length between 2 and 35 (inclusive) |
| Mark for weekly tasks | Must be an integer, value between 0 and 30 (inclusive) |
| Mark for assignment 1 | Must be an integer, value between 0 and 30 (inclusive) |
| Mark for assignment 2 | Must be an integer, value between 0 and 40 (inclusive) |
“Add” function
The module ‘addStudent’ is executed as follows:
- Check whether the provided student information satisfies the required format/value range given in Table 1. If not, the student information cannot be added and the module will return a string “not added”. If all the information is valid, go to the next step.
- Check whether the provided student ID exists in the file “list.csv”. If it already exists, the student information cannot be added and the module will return a string “not added”. If the ID is new (that is, it does not exist in the file), go to the next step.
- Add the student information as a new row in the file “list.csv” and return a string “added”.
“Delete” function
The module ‘deleteStudent’ is executed as follows:
- Check whether the provided student ID exists in the file “list.csv”. If the ID does not exist in the file, the module will return a string “not exist”. If the ID exists in the file, go to the next step.
- Delete the row related to the ID from the file “list.csv” and return a string “deleted”.
“Update” function
The module ‘updateStudent’ is executed as follows:
- Check whether the provided student information satisfies the required format/value range given in Table 1. If not, the student information cannot be added and the module will return a string “not updated”. If all the information is valid, go to the next step.
- Check whether the provided student ID exists in the file “list.csv”. If the ID does not exist in the file, the module will return a string “not updated”. If the ID exists in the file, go to the next step.
- Update the row related to the ID in the file “list.csv” with the provided new information, and return a string “updated”.
“Report” function
The module ‘reportStudent’ is executed as follows:
- Check whether the provided student ID exists in the file “list.csv”. If the ID does not exist in the file, the module will return a string “not exist”. If the ID exists in the file, go to the next step.
- Calculate the final mark using the following formula:
Final mark = mark for weekly tasks + mark for assignment 1 + mark for assignment 2
- Determine the grade based on the following scheme:
| Final mark | 80-100 | 70-79 | 60-69 | 50-59 | < 50 |
| Grade | HD | D | C | P | N |
- The module will return both the final mark (as an integer) and the grade (as a string).
Execution scenarios of record.py
The following shows some typical scenarios when running the program record.py.
Normal cases
- For correctly running “Add” function, seven parameters must be provided in the command line, including the function name ‘add’ and the six items in Table 1. For example,
- For correctly running “Update” function, seven parameters must be provided in the command line, including the function name ‘update’ and the six items in Table 1. For example,
- For correctly running “Report” function, two parameters must be provided in the command line, including the function name ‘report’ and the student ID. For example,
- For correctly running “Delete” function, two parameters must be provided in the command line, including the function name ‘delete’ and the student ID. For example,
Exceptional cases
- If no parameter is provided when running the program, it will exit with a warning message. For example,
- The program will also exit with a warning message if the number of provided parameters is not correct. For example,
- The program will also exit with a warning message if the function name is not correct. For example,
- If invalid information is provided for “add” and “update” functions, the information cannot be added and updated, respectively. For example,
- For the “add” function, if the given ID already exists, the information cannot be added. For example,
- For the “delete”, “update”, and “report” function, if the given ID does not exist, the corresponding operation cannot be undertaken. For example,
Appendix B: Provided python/pytest code
You are given a file “assignment2pack.zip” for this assignment.
After unzipping the provided “assignment2pack.zip”, you will find the following files:
- A template pytest script ‘test_assignment2_xxxxxx.py’, which contains four sample test cases. If you want to generate your own script based on this template, remember to replace “xxxxxx” in the file name by your student ID.
Note the sequence of these four test cases, which is “test_add”, “test_update”, “test_report”, and finally “test_delete”. Such a sequence was deliberately organised (as a test implementation activity) to guarantee the satisfaction of pre-conditions of some test cases. For example, only after adding the student information with ID ‘123456’ via the test case “test_add”, can we successfully update its information via the test case “test_update”.
- The “list.csv” file, which should be used as part of the input for testing, especially when you run the pytest script ‘test_assignment2_xxxxxx.py’.
- The source code of the program ‘record.py’ in the folder “studentList”.
- The “list.csv” file (exactly same as the other one) in the folder “studentList”. You may use this file when you want to directly run the ‘record.py’ program in the folder “studentList”.