Software Engineering

Course Projects Grading Scheme

Quality and quantity of student project deliverables will be evaluated based on two aspects:

1. Quality: Knowledge and mastery of course materials, demonstrated through proper use of class concepts, clear communication of ideas and solutions, and professional preparation of project deliverables
2. Quantity: Estimated project size, measured by software metrics (use case points and cyclomatic complexity) calculated by the student team and verified by the grader

Contents:

1. Project Quality Evaluation
2. Project Quantity Evaluation
3. Combined Grade Assignment
4. Public Review

Aspects of quality evaluation include:

• All important concepts should be defined in a glossary
• All diagrams should be explained in text
• The project report should avoid duplication:  if a use case looks very similar to an existing one, omit it from the report and mention briefly what was omitted and why
• The report should minimize “communication noise,” such as grammatical errors, non-uniform typesetting, and incorrectly labeled or referenced figures and tables
• References to credible sources, such as published papers or books and online websites, must be provided for all ideas and claims that are not common part of average person’s knowledge

For example, assume that there are 10 project groups with six members each, except for Groups 4 and 6 that have only five members.  Groups 1 and 6 said that “all members contributed equally,” and other groups reported different percentage contribution for different team members. Table I shows the grade distribution after the reports were graded for quality and the points were assigned based on the reported contribution breakdown. The second row (highlighted in yellow) shows the total number of points assigned for the given deliverable as a whole. Each team can earn a maximum of 100 points on different aspects of the project deliverable. Earned points for individuals may be different from their reported contribution breakdown because their deliverable may after quality grading be assigned less than 100 points.

To assign the grades in a meaningful and fair way, we need to consider additionally each project size. It is possible to have a neat report for a small and simple project, or to have a poorly written report for an ambitious project. For this reason, we also consider the quantity (size) of each project.

2.1  Use case points

For Report #1, estimate the size of your project in terms of use case points only for the use cases elaborated in this report.
Revise the size estimate from Report #1 in Report #2 to include only the use cases for which interaction (sequence) diagrams are provided and which will be demostrated in Demo #1.

Use case points are described in Section 4.2.1 of the lecture notes and in the slides (also check the Wikipedia page).
At the end of Report #1, show the detailed work for estimating the size of your project in use case points, based on your detailed use cases as well as the domain model.
Students must show step-by-step the process of calculating their project size. Every element estimate must be justified. For example, the number of points assigned to each actor or to a scenario of each use case must be explained.

The “Complex internal processing” technical complexity factor (TCF) should be guessed based on your Mathematical Model (if any is included as part of Domain Analysis in Report #1).
Other TCFs should be calculated only if such features will be implemented in your system. For example, “Performance objectives,” “Reusable design,” “Portability,” etc., should be assigned a non-zero Perceived Complexity only if you can demonstrate that you are actually working on supporting such features in your system.

We assume that all students will work under the same “environmental factors.” Therefore, do not calculate Environmental Complexity Factor (ECF) for your project; i.e., calculate only Unadjusted Use Case Points (UUCP) and Technical Complexity Factor (TCF).
To calculate your Use Case Points, use ECF = 1.

Continuing with the example from Table I, assume that student groups reported their project size in Use Case Points as shown in Table II.  The last row of Table II (highlighted) shows each project size normalized to the largest project, which is the project from Group 1 that has UCP = 116.

The self-estimated size may be revised by the grader if found to inadequately represent the design described in the report. Use cases that are found to be a slightly retouched carbon copy of existing use cases will be discounted at a 50% rate. (Recall that redundancy/duplication is discouraged!)
If the project size estimate is not provided in a timely manner, the grader will assign it the size 10% lower than any of the reported sizes for other projects in the class.

2.2  Cyclomatic complexity

At the end of Report #2, revise the earlier project size estimate from Report #1, this time to include only the use cases for which interaction (sequence) diagrams are provided and which will be demostrated in Demo #1. In addition, this time your size estimate should use cyclomatic complexity metric applied to your sequence diagrams. The cyclomatic complexity estimate should be used instead of the “Complex internal processing” technical complexity factor (TCF) that was guessed in Report #1 for computing the use case points (as described in the previous section above).  Show the detailed cyclomatic complexity calculation for your project.
Cyclomatic complexity is described in Section 4.2.1 of the lecture notes and in the slides (also check the Wikipedia page).

The quality score (range: 0 – 100) will be multiplied by the estimated project size, which will be normalized to the range [0, 1]. The resulting combined score for each project represents its quality score adjusted by its size.

Multiplying the earned points of each student (Table I) with the normalized size of their project (Table II, last row) will yield the final individual grades as shown in Table III.

For easier understanding, we could normalize the final individual grades to the range  0 – 100,  based on the highest score (Student 2in Group 3), as shown in Table IV.

A remaining issue is how to encourage teamwork and at the same time avoid punishing hard-working members of teams with slackers:

• Teams that reported “equal contribution” (such as Group 1 and Group 6 in the Table I example) should be rewarded for teamwork, because an important course objective is to learn how to work well within a team.
• Students in teams with slackers who invested extra effort (e.g., the Stakhanovite Student 2 from Group 3) should not be penalized, because it may not be their fault that some students just do not care to contribute. They, however, should not be excessively rewarded for individualism either, because they should be encouraged to try to get along with their teammates and get them to contribute more.

Because project size estimate plays a critical role in grade assignment, in an effort to reduce errors and ensure uniform and fair approach, we will solicit peer reviews of each project’s size estimation.  Software review is a well-known and well-respected practice in the industry, see Wikipedia article.
After each full project report due date, the reports will be made available in a public folder on the Sakai course site.  All students in the class are invited (optional, not required) to review the use-case-point calculation by other teams and notify the instructor and graders about potential errors or ambiguities.
Students are also welcome to make corrections to their own use case calculations if they discover better approaches in the reports by other teams.  In this case, the corrected estimate should include the explanation of what exactly was corrected and in which report this practice was observed.  Corrected calculation without such explanations will not be accepted.
Deadline for comments or corrected estimates:  Five (5) days after the full report due date.

BACK TO:

• Course home page
• Course syllabus page