Identifying what research methodologies to implement at certain points of the project at hand can be challenging. In order to understand what methods to use, you should be familiar with what evaluative research is. The 2 types of evaluative research fall under either a Formative or Summative Evaluation. You can apply qualitative, quantitative or a combination of methods for both types. Just make sure you have a test plan in mind to keep the project on task, on budget and visible for team members to collaborate.
Formative Research
Formative evaluations are more frequent and are typically done during:
- Discovery Research
- Design Phases
- Onset of a development cycle
During these evaluations you are able to garner insights on features, designs or your product in general. You are able to identify pain points in your user experience and improve the interface before production begins. Post-Production design changes become more costly and time consuming, therefore pushing project timelines out. Formative evaluations focus on determining which aspects of the design work well or not, and why.
Formative Evaluations tells you what is or isn’t usable in your UX and why
Formative Evaluations employ mostly qualitative methods:
- Heuristic Evaluations or UX Audits
- Usability Testing with small sample sizes
- In Depth User Interviews
You can start as early as you like to start testing design concepts, the goal is to be as iterative as possible to get as many insights as possible. If you already have rough concepts during a discovery interview, you can screenshare designs at the end to get initial candid feedback that the design is in the right direction. Discovery Research is a great time to conduct a UX Audit or Heuristic Evaluation as well, allowing the team to provide their expertise knowledge for recommendations and to validate through testing. Then during your iterative design testing sessions you can start to create a prototype and continue to test and discover insights to shape the design direction.
Summative Research
Summative evaluations are less frequent and are typically done during:
- Discovery Research
- Onset of a Development cycle
- Post Development Cycle
During these evaluations you are able to understand how well a design or product is performing. Summative evaluations are effective only when you have a relatively complete or fully functioning prototype since it involves quantitative judging metrics and comparing against competitive products. You will be able to determine time on task, completion rates and success rates. Conduct this when you’re further in the project phase, typically during the development phase and or post development phase. You can also use this to compare against the previous or older version test results to determine how usable is my product and did it improve.
Summative research is a process of evaluating the final or near-final state of a feature, a website, or an application at the end of a project or development cycle. It helps evaluate whether a design, feature, or application meets the user goals. If a project or development cycle started with formative research, the results of summative research can be compared to those of the formative research in order to measure success or progress. This is referred to as “benchmarking” so that you can determine whether you’re improving your own designs over time or beating out a competitor.
Summative Evaluations employ mostly quantitative methods with larger sets of participants:
- Competitive Usability Testing
- Benchmark Usability Testing
- Competitive Heuristic Evaluations or UX Audits
Summative Evaluations tells you how well the UX is performing. The goal is to get the big picture and assess the overall experience of a finished product.
