Modular oversight methodology: a framework to aid ethical alignment of algorithmic creations

• DOI: 10.1017/dsj.2024.23
• Publication Date: 2024-11-18T10:18:52Z
• Authors: Kyriakos Kyriakou; Jahna Otterbacher
• Keywords: Drawing. Design. Illustration; Black-box systems; Software engineering; Engineering design; NC1-1940; TA174; Human oversight; Algorithmic auditing; Quality assurance

Abstract Evaluating the algorithmic behavior of interactive systems is complex and time-consuming. Developers increasingly recognize importance accountability for their creations’ unanticipated resulting implications. To mitigate this phenomenon, developers not only need to concentrate on observable inaccuracies that can be measured quantitatively but also more subjective outcomes perpetuate social bias, which are challenging identify. We require a new approach involves humans in scrutinizing behavior. It leverages combination quantitative qualitative methods support an ethical, value-aligned design system’s lifecycle, informed by users’ perception values. date, literature lacks agreed-upon framework such approach. Consequently, we propose oversight framework, Modular Oversight Methodology (MOM) , aids assessing involving carefully crowdsourced society-in-the-loop. The facilitates development execution process tweaked according domain application use. Through process, assess human under inspection, extract valuable insights will aid its present MOM as first step toward tailoring robust, domain-specific solutions exercise over algorithms, means software keep generated output fair trustworthy.