VIBHUTI
BHATT
Overview
This project is an experiment on menu redesign with ephemeral effects with onset delays and predicted option display based on previous choices made by users.
​
We referred to a research paper "Ephemeral Adaptation: The Use of Gradual Onset to Improve Menu Selection Performance" and modified our hypothesis along with the testing conditions to estimate the results if ephemeral menus are adaptable in general web interfaces.
The Conditions
In this experiment, we followed the paper and its experiment with the new functionality of data prediction. Static menus where all the options are stays where they are without any background highlight or different color but our conditions to check time efficiency and error rate comparison in both the types of menus (ephemeral and static menus).
​
There is one nested condition within predicted ephemeral menus, onset delays with a few milliseconds.
​
Based on these conditions data was collected from multiple people and churned in Minitab with a general linear model to analyze affecting factors. The result of which particular menus work better to reduce time and error rate was derived by me and my team.
Spring 2021 || 3.5 Months
Qualitative UX Research
Team Project || 3 People
Methods
Quantitative Data Analysis, General Linear Regression, Data collection, Python Programming
Tools
Github (Python 4.0 platform), Excel sheet, Minitab, Google Drive, Zoom
Role
User Research, Data Analyst, Designer of experiment
Interface Design & Menu Data
A proIntel(R) Core(TM) i7 - 2.50GHz Processor with 8 GB of RAM and Microsoft Windows 10 Home was used for developing our experiment, which contains a menu interface and different condition functions. The Interface was created using a Python thinker and the backend algorithm was supported by Python version 3.8. The system recorded all the response times and error rates.
​
Data of menu were dishes names classified by the cuisine of respective countries. There were three menu tabs in the interface menu 1 had the starter dishes, menu 2 had main course dishes, and menu 3 had beverages in the list. Based on participants' choice static and dynamic menu will work.
In an ephemeral adaptive menu, predicted dishes that are suitable to the previously chosen dish will gradually come closer to the top.
Demo Video of Experiment
Participants
No. of participants: 11
​
Age group: 20 - 40
​
Recruitment process: Online flyer distribution, Survey, and asking friends for volunteer participation.
​
Requirement: All participants needed to be computer literate and must be using the Python programming language or have any platform that can run a Python program. Most of the participants were running it on Spyder Anaconda for running the experiment.
How to perform an experiment?
Total time duration of the experiment: 45 minutes (including post-interview)
​
-
Participants were first given primary instructions and a sample video to give a demonstration of how to interact with the interface.
-
Each participant had to complete both the conditions, each condition had 3 trials and each trial had 15 sequences of items to be selected from the menus. Totally 90 data was collected from one participant.
​
Attention Please
No brief instruction of predicted and static behavior of the menu was given because we wanted to record spontaneous responses of the users. Participants had to find the displayed item in the respective menu.
How the menus works?
Primary three conditions within Static and adaptive menus
-
Controlled functionalities
-
Short-onset (250ms delay)
-
Long-onset (500ms delay)
The interface is designed with 3 pull-down menus with 16 items in each menu. Items were separated into groups of 4 different cuisines(Indian, Chinese, Maxican, American). The adaptive conditions were identical in the static conditions apart from predicted items’ position within the menu; delay for the short and long onset time are the same.
Adaptive menu conditions used the same adaptive algorithm to predict a set of 3 items that were likely to be in the same food ordering for the users. In the static menu, all the items are placed in a fixed position.
Hypothesis Formation
Hypothesis Statement 1
​
" Is response time with the predictive menu is shorter than the static menu?"
​
- What if we can improve the efficacy of clicking on menu options by changing the position of the potential predicted options, locating them closer to the top.
Hypothesis Statement 2
​
" Does Short-onset delay or Long-onset delay improvise the reaction time with the menu options in compare to static menus?"
​
- Different delays of the menu options can be noticed by eyes and increase the interaction speed with the tabs.
What are our conditions to Test the Hypothesis ?
The two nested menu types we tested were:
​
Predicted Menu
-
Control: Menu with predicted items closer to top without delay.
-
Short-Onset: Ephemeral adaptive menu, where predicted items are closer to top. All items will appear over a 250ms delay.
-
Long-Onset: Ephemeral adaptive menu, where predicted items are closer to top. All items will appear over a 500ms delay.
​
Static Menu
-
Control: Traditional static menu with items scattered in fixed position all over the 16 row.
-
Short-Onset: Static menu, where all items are in a fixed position. And all items will appear over a 250ms delay.
-
Long-Onset: Static menu, where all items are in a fixed position. And all items will appear over a 500ms delay.
How to Measure & Collect Data?
The response time for selecting the item from the menus was recorded by subtracting two click events time by datetime.now() in Python. The system will determine which item for selection and the current item to select is displayed above the trial buttons. The response time is recorded until the correct item is selected. If participants click a wrong item from the respective menu; it will be calculated as an error.
Another criterion of measurement is the response time difference for predicted and static menus. The difference it makes in item selection is if predicted items are closer to the top. Some primary questions had been asked after the completion of the experiment.
Our collected data is here.
Statistical Tests in Minitab
Table 1: ANOVA table for Hypothesis 1
.png)
Table 2 : ANOVA table for Hypothesis 2
Table 3 : No. of calculated factors for nested condition of Hypothesis 2
To test both hypotheses, we have used randomized complete block design(RCBD) on the collected data. First, five data points were not included in the test as it was sample data points.
Hypothesis 1 is focused only on predicted condition and response time. Table 1 shows ANOVA stats for Hypothesis 1 and P-value is 0(<0.05) with 95% confidence interval.
Hypothesis 2 has a nested condition; hence, three levels of onset time are nested within the predicted and static menu. Table 2 shows interaction factors that are taken into account for the calculation; respected ANOVA stats are represented in Table 3.
Final Results
We observed that the predicted menu’s performance is better than static menu and it affects significantly on response time. Besides, in predicted and static menu conditions, we found that both short-onset and long-onset response time are faster than control which has no delay. Thus, hypothesis 1 and hypothesis 2 are proved and supported by our experiments.
Detailed Report is here!