The risk of unconscious bias
Unconscious bias is an issue that can affect the design of any product, including many of the ones we use daily. For example, social media platforms are used by billions of users, however the way they function is often dictated by only a small number of software developers. When the infamous ‘like’ feature was introduced to Facebook in 2009, the developers behind it viewed the feature as simply an innovative way to determine the interests of different users, in order to better tailor the content on their newsfeed. With the designers being predominantly young, male engineers, they did not predict the immense social and mental impacts this small feature would go on to have for the millions of users. It was not long before social media appeared to correlate with depression and anxiety among young people, with many users being negatively impacted in the event they receive especially few ‘likes’ on a post.
The issue of unconscious bias can occur in medical device development as well, without appropriate oversight. For example, designers might introduce their own unconscious biases in the physical design of a device, by not sufficiently testing it on a wide enough variety of users. Such design bias has recently been brought to light in the U.S. for example, with senators calling on the FDA to launch a post-market study of pulse oximeters and skin colour, following reports that these devices have a lower accuracy for patients with darker skin. Though more information is needed to confirm both the accuracy and potential cause of these reports, issues such as these likely point to a lack of appropriate testing or consideration for different user groups during the device development.