Legalweek recap: 2024, the year of practical experimentation

In March 2023, the buzz at Legalweek speculated on, “Where will generative AI take the legal industry?” — masking the real fear, “Can gen AI replace knowledge professionals?” At the time, I would have named the answers discussed as bullish. But after a year of conversations, experimentation, and failures amongst customers and vendors alike, I would change my judgment to "hungry for evidence." And based on that experience and what I heard at Legalweek this January, I think in 2024 we’ll see the industry make big demands for said evidence.

Innovators must leverage data

After decades of precise data management and optimization, the switch to experimentation and testing around generative AI represents a substantial shift for legal tech. Even brave early adopters in 2023 only took single steps forward and two steps back. In this year’s sessions, I heard firms and vendors begging for evidence and data to evaluate emerging technologies. In particular, attendees and speakers called for which data lawyers are putting into these systems, and for transparent results data on the accuracy of these gen AI tools.
 
Without that data, an organization can't know what to change in people, processes, and their tech stack — or where to integrate generative AI effectively. What's more, experimentation is hard, mentally, after decades of precision. For instance: how do you drive innovation without a clear understanding of the business impact of new technologies to existing legal processes? Statistically speaking, the more tools you buy, the less adoption you get. So for technology buyers, how do you place your bets? Without that data, that evidence, firms will be stuck grappling with this dilemma. 

Staying competitive

And yet, the belief that successfully applying generative AI will be critical to staying competitive was overwhelming. Speakers emphasized that starting from real problems, not just applying gen AI to any process or situation, will be critical to success. Firms must lead with real-world use cases and prioritize them by business value versus effort and cost to solve.
 
For one US firm to build just one model in 2023, it took hundreds of hours of partners’ time, with many millions invested, and not just in technology. By contrast, another firm spoke of prioritizing data-driven decisions more than five years ago. They invested in data architecture, hygiene, and process improvements for their lawyers and support staff; which resulted in a cultural shift among their practice areas. They were one of the few firms who reported testing gen AI successfully, well ahead of their peers.

We heard from multiple industry experts that for organizations to adopt generative AI practically and cost-effectively, it will be imperative that they revisit their knowledge management fundamentals. Gen AI must be grounded in real firm knowledge, and the most centralized location of that optimal knowledge is in a firm's data centers and knowledge documents. Better yet, committed to information architecture and codified in a knowledge library. This puts many European, Canadian, and Australian organizations at an advantage.

Making it stick

We heard that in 2023, finally, we in legal tech are dealing with too much interest instead of not enough. ChatGPT was great for piquing that interest, especially for knowledge management (KM) functions. Educating firm leaders, clients, IT, partners, and attorneys was critical last year and remains critical moving forward. But to actually change the practice of law requires change management — and that is hard.
 
Just as the advent of the internet prompted a paradigm shift, the legal industry must navigate these complexities and more, this year and beyond. The internet wasn’t a silver bullet, and neither is generative AI. Novel technology demands that an organization reconsider institutionalization and service delivery. Only then will gen AI improve measurable business outcomes and build long-term competitive advantages. As they say, fortune favors the bold.

Join us in New York for ConnectLive 2024, where new and exciting sessions explore the modern workplace, security strategies, and leveraging new technologies like iManage AI. London registration page coming soon!