Model AlGovernanceFramework forGenerative AlFostering a Trusted Ecosystem0Published 30 May 2024INFOCOMMMEDIADEVELOPMENTAUTHORITYCONTENTSExecutive Summary3Accountability6DataTrusted Development92and DeploymentIncident Reporting16Testing and Assurance19Security21Content Provenance23Safety and Alignment R&D26Al for Public Good28Conclusion31Acknowledgements32Further Development34MODEL AI GOVERNANCE FRAMEWORK FOR GENERATIVE AIEXECUTIVE SUMMARYGenerative Al has captured the world's imagination.While it holds significanttransformative potential,it also comes with risks.Building a trusted ecosystem istherefore critical-it helps people embrace Al with confidence,gives maximal spacefor innovation,and serves as a core foundation to harnessing Al for the Public Good.Al,as a whole,is a technology that has been developing over the years.Priordevelopment and deployment is sometimes termed traditional Al.To lay thegroundwork to promote the responsible use of traditional Al,Singapore releasedthe first version of the Model Al Governance Framework in 2019,and updated itsubsequently in 2020.2 The recent advent of generative Al3 has reinforced someof the same Al risks (e.g.,bias,misuse,lack of explainability),and introduced newones (e.g,hallucination,copyright infringement,value alignment).These concernswere highlighted in our earlier Discussion Paper on Generative Al:Implications forTrust and Governance,4 issued in June 2023.The discussions and feedback havebeen instructive.Existing governance frameworks need to be reviewed to foster a broader trustedecosystem.A careful balance needs to be struck between protecting users anddriving innovation.There have also been various international discussions pullingin the related and pertinent topics of accountability,copyright and misinformation,among others.These issues are interconnected and need to be viewed in a practicaland holistic manner.No single intervention will be a silver bullet.This Model Al Governance Framework for Generative Al therefore seeks to setforth a systematic and balanced approach to address generative Al concernswhile continuing to facilitate innovation.It requires all key stakeholders,includingpolicymakers,industry,the research community and the broader public,to collectivelydo their part.There are nine dimensions which the Framework proposes to be lookedat in totality,to foster a trusted ecosystem.a)Accountability -Accountability is a key consideration to incentivise playersalong the Al development chain to be responsible to end-users.In doing so,werecognise that generative Al,like most software development,involves multiplelayers in the tech stack,and hence the allocation of responsibility may not beimmediately clear.While generative Al development has unique characteristics,useful parallels can still be drawn with today's cloud and software developmentstacks,and initial practical steps can be taken.Traditional Al refers to Al models that make predictions by leveraging insights derived from historical data.Typical traditional Al models indludelogistic regression,decision trees and conditional random fields Other terms used to describe this include"discriminative Ar.2 The focus of the Model Al Governance Framework is to set out best practices for the developmentand deployment of traditional Al solutions.Thishas been incorporated into and expanded under the Trusted Development and Deployment dimension of the Model AI Governance Frameworkfor Generative ALGenerative Alare Al models capableof generating text,imagesor other media types.They leam the pattems and structure of their input training dataand generate new data with similar characteristics Advances in transformer-based deep neural networks enable generative Al to accept naturallanguage prompts as input,including large language models(LLM)suchas GPT-4,Gemini Claude and LLaMAMODEL AI GOVERNANCE FRAMEWORK FOR GENERATIVE AIb)Data-Data is a core element of model development.It significantly impactsthe quality of the model output Hence,what is fed to the model is importantand there is a need to ensure data quality,such as through the use of trusteddata sources.In cases where the use of data for model training is potentiallycontentious,such as personal data and copyright material,it is also importantto give business clarity,ensure fair treatment,and to do so in a pragmatic wayc)Trusted Development and Deployment-Model development,and the applicationdeployment on top of it,are at the core of Al-driven innovation.Notwithstandingthe limited visibility that end-users may have,meaningful transparency aroundthe baseline safety and hygiene measures undertaken is key.This involvesindustry adopting best practices in development evaluation,and thereafter"food label"-type transparency and disclosure.This can enhance broaderawareness and safety over time.d)Incident Reporting -Even with the most robust development processes andsafequards,no software we use today is completely foolproof.The sameapplies to Al.Incident reporting is an established practice,and allows for timelynotification and remediation.Establishing structures and processes to enableincident monitoring and reporting is therefore key.This also supports continuousimprovement of Al systems.e)Testing and Assurance -For a trusted ecosystem,third-party testing andassurance plays a complementary role.We do this today in many domains,such as finance and healthcare,to enable independent verification.AlthoughAl testing is an emerging field,it is valuable for companies to adopt third-partytesting and assurance to demonstrate trust with their end-users.It is alsoimportant to develop common standards around Al testing to ensure qualityand consistency.f)Security-Generative Al introduces the potential for new threat vectors againstthe models themselves.This goes beyond security risks inherent in any softwarestack.While this is a nascent area,existing frameworks for information securityneed to be adapted and new testing tools developed to address these risks.g)Content Provenance-Al-generated content,because of the ease with whichit can be created,can exacerbate misinformation.Transparency about whereand how content is generated enables end-users to determine how to consumeonline content in an informed manner.Governments are looking to technicalsolutions like digital watermarking and cryptographic provenance.Thesetechnologies need to be used in the right context.h)Safety and Alignment Research Development(R&D)-The state-of-the-science today for model safety does not fully cover all risks.Acceleratedinvestment in R&D is required to improve model alignment with human intentionand values.Global cooperation among Al safety R&D institutes will be critical tooptimise limited resources for maximum impact,and keep pace with commerciallydriven growth in model capabilities.i)Al for Public Good -Responsible Al goes beyond risk mitigation.Itis also aboutuplifting and empowering our people and businesses to thrive in an Al-enabledfuture.Democratising Al access,improving public sector Al adoption,upskillingworkers and developing Al systems sustainably will support efforts to steer Altowards the Public Good.AMODEL AI GOVERNANCE FRAMEWORK FOR GENERATIVE AIFostering a Trusted Al Ecosystem1.AccountabilityPutting in place the right incentive structure for different players in theAl system development life cycle to be responsible to end-users2.Data3.Trusted4.Incident5.Testing andEnsuring data qualityDevelopment andReportingAssuranceand addressingDeploymentImplementing anProviding externalpotentially contentiousEnhancingincident managementvalidation andtraining data in atransparency aroundsystem for timelyadded trust throughpragmatic way,asbaseline safety andnotification,third-party testing,data is core to modelhygiene measuresremediationand developingdevelopmentbased on industryand continuouscommon Al testingbest practicesimprovements,as nostandards forin developmentAl system is foolproofconsistencyevaluation anddisclosure6.Security7.Content ProvenanceAddressing new threat vectors that ariseTransparency about where content comesthrough generative Al modelsfrom as useful signals for end-users8.Safety and Alignment R&DAccelerating R&D through global cooperation among Al Safety Institutes toimprove model alignment with human intention and values4Responsible Al includes harnessing Al to benefit the public by democratising access,improving public sector adoption,upskilling workers and developing Al systems sustainablyThis Framework builds on the policy ideas highlighted in our Discussion Paper on Generative Al and drawsfrom insights and discussions with key jurisdictions,international organisations,research communitiesand leading Al organisations.The Framework will evolve as technology and policy discussions develop.