A checklist for grading the first software engineering report

Points will be deducted from the maximum possible score if the following issues are found in the report.

Note that these items are about minimizing the “noise” that can impact clear and effective communication between the project team and the grader. This is not about nitpicking! Effective communication in the form of a professionally prepared project report is critical for meaningful and fair grading.

Project Management—typical issues: (maximum possible score: 10 points)

project was apparently poorly managed, because the report is of substandard quality
the perceived novelty of the project is low
unfocused project with too many (unrelated) features that are haphazardly conceived and superficially designed
poor and inconsistent writing, different styles, difficult to read and understand
diagrams unclear, unintelligible, UML diagrams created using different tools
non-UML symbols are used without defining their meaning
lack of consistency and traceability between the requirements, use cases, user interface, and the domain model
report is incomplete, not self-contained and no references are provided for non-obvious claims
information or diagrams used from existing sources are not properly credited
missing page labeling (pagination), section headings, section numbering is messed up
figures and tables are not labeled, missing captions, or not described and referenced in the text
fonts and line width are different for different parts of the report
the report is not following the prescribed format
Other: ______________________________

Section 1: Customer Problem Statement—typical issues: (max score: 9 points)

the team obviously exhibits the lack of knowledge and understanding of their problem domain—desired system features are envisioned naively and superficially
a similar project was implemented previously, and it is unclear what is new about this project
explicitly list the novel features of your project and for your other functions that existed in the past projects, indicate if your idea/design/implementation will be different in any way
(when describing the differences, refer explicitly to the specific past project)
the proposed system will require large amounts of data (e.g., for statistical analysis or machine learning) but the report does not describe how this data will be acquired
the proposed system will require special hardware or sensors (beyond what a standard laptop or smartphone contains) but the report does not describe how these will be acquired
problem statement is poorly written and difficult to understand
problem statement is written from programmer’s, not customer’s viewpoint: it describes the software-to-be rather than the problem that needs to be solved and the business goals that will be achieved
glossary inadequate, missing terms, unclear definitions
Other: ______________________________

Section 2: System Requirements—typical issues: (max score: 6 points)

requirements statements (or user stories) are not enumerated
incomplete—more features or functions are mentioned in the problem statement than covered in the requirements and no explanation is provided for why they are missing
overcomplete—some functional requirements appear from nowhere, are not derived from the problem statement or do not accurately capture the customer business needs
improperly formulated: your requirements should define the problem to be solved by the software, not the software that solves it!
the requirements are so abstract or generic that they do not define any concrete problem at all
in other words, the requirements statements are not testable (unclear how to write acceptance tests to verify that the given requirement is satisfied)
some of the requirements are very complex (unclear how they can be tested) and should be split into several simpler requirements
business policies are required to resolve ambiguous scenarios, but are not mentioned
non-functional requirements are not enumerated
non-functional requirements are fuzzy and ambiguous—unclear how they can be tested
missing priorities for requirements (or initial rank-ordering for user stories)
the assigned priority weights are misplaced and will not make this project competitive with the rest of the class (e.g., too much attention paid to login and user registration, too little attention on the problem domain)
a high or maximum priority is assigned to more than half of all requirements, which renders the prioritization effort useless
Other: ______________________________

Section 3: Functional Requirements—typical issues: (max score: 30 points)

not all stakeholders are humans, or stakeholders are not identified at all
actor definitions unclear; the difference between actors unclear—their roles appear to be same or similar—clarify their differences
actors poorly named, such as “Yahoo!”, which happens to be your current provider—you should instead assign a conceptual name, derived from the service that this actor provides to make it substitutable with a similar system
missing actors that inexplicably appear in later parts of the report
UC’s are named not as verb phrases
UC numbering is not consistent throughout the report
casual UC’s do not list the associated actors; actor relationships to UC’s are not shown
use cases appear from nowhere—not derived from the system requirements
confusion between <<include>> (or <<uses>>) and <<extend>> relationships between the use cases
too many use cases are “elaborated,” but only superficially and none of them passes a reality check—none of them would be able to perform an actual task in a real system
use case duplication: the “elaborated” use cases essentailly are a slightly retouched carbon copy of each other—either the use cases are poorly identified or they are redundant;
even if there is a need for several similar use cases, their differences should be highlighted and their existence justified—can these use cases be replaced with a single use case with subroutine use cases («include» or «extend»)?
elaborated UC’s are not well formated—unclear what schema is used to describe the detailed use cases
scenario steps are very coarse, fuzzy, simplistic, or apparently missing—it is impossible to assess whether the listed steps would allow the user to successfully accomplish the task, or whether the relevant requirements are satisfied
no alternate scenarios considered (only the main success scenario shown); or alternative scenarios are very naive and simplistic;
alternate scenarios should be broken into individual steps (instead of a one-sentence mention) and properly explained in the text narrative
unclear where alternate scenarios branch away from the main scenario, or whether and how they merge back
formatting of elaborated UC’s is inconsistent; e.g., switching between bullets, numbers, and prose
use cases describe what happens inside the system, instead of focusing on what happens on the boundary of the system (how the system behaves relative to the actors)
terms specific to the problem domain are used in use cases but not defined in the glossary
evident contradictions of use cases with the requirements
entry/exit conditions (or preconditions, postconditions) not stated or contradictory (e.g., being logged-in is a precondition, but then the login appears as a step in the event flow)
the focus of use cases is misplaced—instead of focusing on the most important functions of the system, they are focused on supporting/clerical activities, such as login, user registration, account management, viewing different tables, etc.
UC diagram uses non-UML notation, without explaining the meaning of the non-standard symbols
use case diagram shows more or less UC’s than mentioned in the text or UC’s are not shown by names in the diagram
UC diagram is not described in narrative
UC diagram is inconsistent with the use case descriptions
UC diagram is missing labels for UC relationships
traceability matrix missing or inaccurate or no description provided (some use cases appeared out of nowhere and are not responding to the system requirements)
not clearly stated which system sequence diagram corresponds to which use case and scenario (main success scenario or alternative scenarios or both)
system sequence diagrams inconsistent with the corresponding scenarios in use case description
Other: ______________________________

Section 4: User Interface—typical issues: (max score: 15 points)

no sketches of visual appearance are provided for different screens
some mockup interfaces unexplained or inadequately described
the navigation path through the screens is unclear or not described
not described how each component of the user interface addresses the specific requirements (referred by their labels);
difficult to see how the user interface will support the execution of your use cases
user effort estimation does not appear to represent a typical usage scenario
worst-case scenarios for estimating the user effort are not considered
all of the estimations are vague and shown just as “variable keystrokes”—you should instead analyze the user effort with some sample typical input sequence
terms specific to the problem domain are used in describing the user interface but not defined in the glossary
Other: ______________________________

Section 5: Domain Model—typical issues: (max score: 25 points)

the report shows a lack of understanding of what the domain model is and how it is to be derived
poor modular design: there are fewer (or much fewer!) domain concepts than use cases or system requirements—this implies that concepts are too coarse grained and are assigned too many responsibilities per concept
unclear or not explained how the concepts/attributes/associations were derived
poor choice of concepts and concept names, such as “login”, “buy”, “sell”, etc. (concept name should be a noun-phrase, such that it represents the role for a given concept)
methods/functions shown as concepts or elements of concepts—although functions are “modules,” in object-oriented domain modeling we focus on identifying objects, not their methods or stand-alone functions
if you are using functional programming for your modular design, this fact must be explicitly stated
some concepts have unclear/undefined responsibility, or a disproportionally large responsibility, or too many responsibilities, resulting in a poor modular design
missing specification of the protocol for communicating with other systems (used by your system but developed by 3rd party)
traceability matrix missing or inaccurate, or no description provided; some concepts appeared out of nowhere and are not responding to the use cases and elements of the user interface
system operation contracts are mostly trivial (no effort was made to identify significant potential issues that need to be enforced by a contract)
contracts are missing for some preconditions/postconditions identified in detailed use cases that evidently must be enforced by contracts
mathematical model missing, but apparently should be present
terms specific to the problem domain are used in the domain model, but they are not defined in the glossary
Other: ______________________________

Section 6: Plan of Work—typical issues: (max score: 5 points)

no timeline diagram, or the diagram is cluttered and difficult to read
breakdown of responsibilities missing, or uneven (some students assumed disproportionate responsibilities), or fuzzy
missing text description for the plan of work
Other: ______________________________

Section 7: References—typical issues: (up to 5 negative points)

references missing or incomplete
only URL without a title shown
some parts of text or figures or design ideas appear to be adopted from elsewhere, but sources are not credited and citations are missing
(for anything that you did not invent and is not part of general knowledge, a reference should be cited)
references not cited or otherwise mentioned in the main document, so it is unclear which reference is related to which fact
Other: ______________________________

Modified: Wed Jan 18 16:05:29 EST 2012
Maintained by: Ivan Marsic