In this lively debate you will hear from Calligo’s Practice Leads as they discuss their key takeaways from 2023 and their data predictions for 2024 and beyond.
Topics discussed include:
Regulation of AI including the EU AI act
AI hallucinations & AI bias
Data governance and data fines
Dashboard fatigue
Data ROI
Artificial Intelligence (AI) and machine learning have revolutionized numerous industries, offering automation and efficiency. However, achieving the optimal balance between human input and machine automation in AI model development is crucial but often overlooked. In our recent Beyond Data podcast, hosts Tessa Jones and Peter Matson were joined for a compelling discussion with the co-founder of Trubrics, Joel Hodgson, where the importance of AI explainability, trust, user feedback, and ongoing monitoring were explored.
Joel highlighted the challenge of model adoption, a common issue in the data science landscape. Organizations invest significant time and resources in developing AI models, only to face skepticism and underutilization from non-technical stakeholders. This hesitation often arises from a lack of trust and understanding. Education and transparency are vital tools to address this challenge.
Another significant hurdle is the gap in effective communication between business professionals and data scientists. Bridging this divide is essential to incorporate valuable domain knowledge into the model development process. The solution lies in creating feedback loops that enable collaboration between domain experts, business users, and data scientists throughout the model’s lifecycle. These feedback loops are crucial for gathering user insights, improving model performance, and building trust.
Trubrics’ approach of “machine learning monitoring from the users’ point of view” shifts the focus from traditional machine learning metrics to user perception. Evaluating AI models based on their impact and utility to users, rather than just accuracy, is essential. Users’ experiences, trust, and satisfaction play a pivotal role in determining the effectiveness of AI models. Monitoring should identify issues impacting the user experience and ensure AI models align with user expectations.
Trust emerged as a cornerstone in AI adoption. Trust is not limited to data scientists but extends to end-users, employees, and the entire organization. It involves transparent communication, feedback loops, and alignment between different groups. Over time, as individuals become more familiar with AI in the business world, this trust can be built and weaved into organizations’ culture, just as our trust in everyday technology has.
Monitoring AI models’ performance is essential. Technical monitoring involves tracking various model characteristics, while business-facing monitoring assesses alignment with expectations and business impact. These two facets of monitoring are crucial in ensuring AI models continue to meet user needs and business objectives and therefore must be aligned when identifying the reasoning and desired outcomes from such models.
Measuring and evaluating the Return on Investment (ROI) for AI models presents considerable challenges, especially when examining their performance over extended periods. Striking a balance between the continual expenses associated with model maintenance and the value it delivers requires a nuanced approach. Organizations need to account for both the initial and ongoing financial ROI assessment, recognizing that it can become less clear-cut.
According to a recent research report conducted by Calligo, in collaboration with the Global CIO Institute, “36% of business leaders measure the success of an ML project in financial terms, while 11% either have no way to gauge success or go by gut feeling“. This suggests that determining the ROI for ML and AI initiatives isn’t solely tied to financial gains; it also involves a significant degree of uncertainty when the desired ROI isn’t well-defined at the project’s outset.
In conclusion, AI explainability and the balance between human input and machine automation are crucial in AI model development. Education, transparency, effective communication, user-centric monitoring, and trust-building are essential elements in this endeavor. As AI continues to shape our world, achieving these elements will be pivotal to ensure responsible and ethical AI development and its successful integration into our lives. Organizations like Trubrics are at the forefront of this mission, working towards making AI a valuable and trusted tool in our increasingly automated world.
Listen on Spotify or watch below
The term ‘ESG’ (Environmental, Social and Governance) is everywhere. In its own right, the potential impact is important enough, but it can so often be viewed as a standalone initiative. At its worst it becomes a tick box exercise, when in fact its real benefit is in informing and driving fundamental changes in your organization’s wider actions and endeavors.
In January 2023, the EU’s Corporate Sustainability Reporting Directive came into effect. Under its terms, all large companies and all listed companies (except micro-enterprises) must disclose information on the risks and opportunities arising from social and environmental issues, and their impact on people and the environment.
Set against this we have an AI revolution taking place – witness the activity on LinkedIn, with almost every other post lauding the benefits of some ChatGPT derivative or similar, leading to something of an AI feeding frenzy.
Looking through an ESG lens, the environmental impact of AI is huge. According to calculations by the specialist in sustainable data science, Kasper Groes Albin Ludvigsen, published in Medium at the end of 2022, ChatGPT could have consumed as much electricity as 175,000 people in the month of January 2023 alone. Equally, there are numerous articles that reference AI’s huge water impact.
One thing is clear. Whilst there can be many positive outcomes and by products from AI on ESG, the true end-to-end cost of this next wave of Digital Transformation is not yet well understood.
Given we are still trying to get to grips with the effects of the Industrial Revolution from an environmental perspective, how good is humankind’s track record of not repeating the mistakes of the past? How can we exploit opportunity without understanding the true cost and impact?
Developing an ESG strategy that is in harmony with your Digital Transformation yields multiple advantages. And whilst ESG reporting is now mandatory for corporations in the EU, doing so helps quantify the benefits that exist for every party:
Detractors of ESG will point to the irony that a robust ESG process itself has an environmental impact: data centers in the EU consume more than 2.7% of the bloc’s electricity. And the Ukraine war has highlighted that the geopolitics of power supply will increasingly affect decisions on data processes and sovereignty – when Cloud storage and transference requires so many terawatts of electricity, securing a good price must be balanced against political and geographic risk.
Digital transformation is, by its very definition, a process of huge change. Done right it unlocks competitive advantage, delivers cost savings, drives productivity, opens up new opportunities and delivers compliance with ESG obligations. But done half-heartedly or implemented sporadically it will almost certainly be a huge waste of time, effort and resources.
Deloitte calculates that digital transformation could unlock as much as US$1.25 trillion in additional market capitalization across all Fortune 500 companies. However, done incorrectly, market value could actually be eroded, putting more than US$1.5 trillion at risk.
When it comes down to it, successful digital transformation requires only three things:
And whilst that sounds simple, it involves significant planning and project management resources. It’s not possible to retro-fix a digital solution onto your existing processes – a successful digital transformation requires a center-out approach, incorporating data privacy and protection and considering ESG objectives at the very heart of policy and technology.
When digital transformation is done correctly, “it’s like a caterpillar turning into a butterfly,” but when done wrong, “all you have is a really fast caterpillar.”
MIT Sloan Professor George Westerman
The comprehensive and insightful data analysis and management required to power your digital transformation needs a huge team of business experts, platform designers and technology specialists, all following a clear process:
Simply ‘ESG washing’ operations with fancy reports is both ineffective and expensive. That’s why Calligo ensures that every digital transformation we drive is engineered with careful attention to its environmental impact. Future-proofing your data use in a way that protects everyone’s future.
To help you navigate the expansive topic of digital transformation, we’ve put together a comprehensive eBook, outlining all the key considerations for your organization. And if all this sounds daunting, don’t worry – we’ve seen plenty of similar challenges. Data privacy, for example. Once seen as a vague afterthought or something for someone else, today it takes center stage – the concept of Privacy by Design even has its own ISO standard (31700). Understanding the end-to-end ESG impact of Digital Transformation is heading the same way.
If you want to learn some more, or if you want specific advice, consultancy support or technical implementation, why not talk to our experts, who can get your digital transformation journey underway?
