Representing AI in law
A bonus monographic issue
Hello, dear readers! When I started my PhD, my plan was to work on disciplinary translation: drawing from my background in law and computer science, I wanted to find ways to help scholars (and practitioners) from one side of the divide to talk with their counterparts about shared problems. Ultimately, I decided to follow a different path, and as a result my thesis is mostly about matters of law and regulation (even if informed by technical disciplines) than a truly interdisciplinary work. Today’s post is an excerpt from my work in that direction.
Until very recently, I planned to write a thesis on the methodology of AI regulation scholarship. I wanted to propose some methods that legal scholars could use to deal with technical questions about AI without becoming experts. The without becoming experts part is fundamental: I think that the idea that all lawyers will need to become lawyers is misguided for quite a few reasons. First, because it understates the opportunity costs: taking one or two courses in programming is not enough to provide any meaningful insight into complex software systems. Second, because computer science—just like the law—is a discipline, in the sense that it teaches you to view the world in a certain way, and one that does not necessarily align with the perspective you need to address legal issues (see Malcolm 1993). Third, because not all questions about AI—and surely not all legal questions—are questions about technology, but rather about how societal dynamics are altered by sociotechnical change. We need technical expertise to address the challenges from AI, but this does not and should not mean automatic acceptance of the idea that these challenges have a correct technical solution.
What I wanted to do is to find a way to help lawyers frame the technical issues that are relevant to them. Whenever we think about AI technologies, we rely—explicitly or not—on models of what these technologies are, how they are arranged and what they can do. This insight is not novel: Michael Birnhack, for one, had proposed that every form of regulation embeds certain assumptions about technology, even if they are constructed in technological terms. My proposal was that we could make these assumptions explicit through the use of representational models. Lawyers can draw from a variety of methods (such as interpretive techniques for legal text and socio-technical approaches to institutional practices) to make explicit those implicit models.
These models, in turn, can play a variety of roles. They might allow lawyers to follow Birnhack’s call and reverse engineer the assumptions about technology that are embedded in seemingly technology-neutral laws. Models might help lawyers by allowing them to express points of uncertainty about technical knowledge and how they relate to technical discourse. Finally, the use of shared models might help lawyers communicate with technical experts in order to address these uncertainties. So, modelling AI technologies from a legal perspective could provide scholars (and practitioners) with various tools for action.
I am still quite fond of this idea, but ultimately decided against making it the core of my thesis. As my hands-on experience with AI technologies fades into the background—it has been now more than five years since I earned a paycheck for my data science skills—I no longer have any particular advantage in carrying out this line of research. And my research interests, as I mentioned in other issues, are becoming more focused on the legal issues than on the legal ramifications of technology. Nonetheless, I think there is something to this approach, which is why I decided to share it here instead of leaving it to languish in a cloud folder. Hopefully this will be of interest for some of you.
What is a model?
The word “model” appears with various meanings in the context of technology. Legal scholars and policymakers are likely acquainted with thinking of models as templates: for example, scholars and organizations often propose model rules as suggestions for legislative reform or guidelines for elaborating international treaties. AI designers, contrastingly, speak of models as procedures to generate outputs, such as the large language models used to generate and process written text. To end this non-exhaustive list of meanings of “model”, researchers in the natural and social sciences often speak of models as maps to the phenomena they study. In the following paragraphs, I want to elaborate a bit on this latter meaning, which I believe can be useful for tech law scholarship.
A (representational) model stands for an object. The late philosopher of science Ronald Giere gave us the example of a city map, which offers a representation of that city, which can be used to guide a tourist as they go from one place to another. A map can be valuable in itself: the Galleria degli Uffizi here in Florence has a room, reopened a few years ago, in which it showcases gorgeous maps that stand as works of art of their own. But usually we care about maps because of their instrumental value: they are expected to help us make sense of the territory they refer to. Giere argues that the same is true of a scientific model, which derives its value from three properties—partiality, context-dependence, and similarity.
The first helpful property of a model is that it provides a partial representation of the territory it covers: for the sake of delivering a legible picture of some features, it removes other aspects of the territory and even distorts some of what remains. Coming back to Giere’s example, a subway map shows the connections between stations and lines, but the distance separating two consecutive stations usually has nothing to do with the physical distance between the buildings. The same logic can be applied to more abstract models. Whenever a socio-legal scholar deploys the attitudinal model to study judicial decision-making, they are not (necessarily) making a judgment about the judge as a human being; instead, they affirm that the judge’s policy preferences are the key factor to understanding judicial decisions in the long run, trumping any transient influences in aggregate. A model’s partiality thus works as a mechanism for directing a attention, allowing the model’s users to cast aside some factors that they deem irrelevant for their purposes.
Since relevance is connected to the purpose for which a model is used, the value of any particular model is interest-relative. A model that is helpful in a particular context might be useless or even misleading if used for another task. If you are waiting for a late train in a subway station, you might be in for a surprise (and perhaps some leg ache) if you decide to walk to the next station on the grounds that it appears to be close to your position in the subway map. Likewise, the individual attitudes of judges might be a useful guide for more explicitly politicized bodies such as constitutional courts and not so much for lower courts more tightly bound by precedent and other institutional constraints. Assessments of a model’s suitability for a given purpose should therefore evaluate whether the aspects emphasized by a model’s partial construction are relevant for representing the represented object in that particular scenario.
Finally, one must consider what makes a model acceptable as a representation in the first place. The intuitive formulation here is that a model is relevant because it is somehow similar to the object in a meaningful way. The subway map is similar to the subway network in the sense that the connections between stations in the map correspond to connections in the real world, while the attitudinal model is similar to actual judicial behaviour to the extent that judges decide as if nothing but policy preferences mattered. But, as the previous example shows, these similarities often break down, not just because of context shifts as the ones described in the previous paragraph but also because of the omission of elements that are actually relevant for the purposes of the model or even because of a misrepresentation of the elements deemed relevant. Whenever the claims of similarity break down, any decision based on the model’s properties is unlikely to produce the expected results when applied to the real object.
Representing artificial intelligence technologies
Representational models can come in a myriad of forms. As the previous section shows, some models, like maps, are physical objects. Others, like the attitudinal model of judicial decisions, are conceptual objects that are nonetheless similar to the object they represent from a certain point of view. Models can also take mathematical forms, such as those provided by formal logic or by statistical methods. The format and structure of these models may differ considerably, but they all use one object (a map, a concept, an equation) as a stand-in for the object of interest (a city, a judge, a phenomenon). In this section, I provide some examples of technological objects that policymakers and scholars might want to model when discussing or designing AI regulation.
A model might, for example, represent the technology targeted by regulation. For example, discourse about the impacts of AI in society often focuses on the harms of algorithms. As used in computer science and related disciplines, the term “algorithm” refers to a formalized description of a procedure for producing outputs from a given input. However, the widespread adoption of AI technologies and other means for automation has led to a metonymic expansion of the term: in public discourse and the social sciences, an algorithm is not just an output-generating procedure but the overall decision logic powered by that procedure. In this latter sense, slogans such as “ditch the algorithm” (and its less polite variants) have entered popular discourse. And, in this broader sense, an algorithm can be seen as a representation of the decision logic it powers—that is, as a model of that logic.
Taken as a model, the broader sense of algorithm can be assessed in light of the three factors identified above. An algorithm is a partial view of the socio-technical system that produces decisions, since it focuses on the abstract decision logic that produces outcomes and downplays the role of other elements, such as the data used to feed the decision procedure, the programming tasks carried out to transform that procedure into executable computer code, the reliability of the computer hardware used to carry out the algorithm, or the environmental impact of training and using an AI system. The algorithm is similar to the system in that the system outputs are produced by the decision logic referred to as an “algorithm”. Because of that similarity, modelling a system as an algorithm makes sense if one’s interest is to assess—or regulate—the outcomes and how they are produced. If that is the case, policymakers can direct regulation towards the algorithm’s properties, while scholars might assess what kinds of issues are caused by these algorithms and how regulation addresses them.
Framing AI as an algorithm might be inadequate in some cases. One potential risk is algorithm fetishism, that is, ascribing to the algorithm specific properties that are produced by a broader network of technical and social practices in which the algorithm plays a role. Modelling AI as an algorithm might also lead to ineffective measures if the issues that regulation is meant to address are caused by the factors that such a model abstracts away. While changes to the decision logic sometimes impact factors such as the environmental impact of an AI system, these effects might not be sufficient to achieve regulatory goals or be accompanied by undesirable side effects. In cases like those, other models of AI would better serve policymakers and scholars.
Recent approaches to AI regulation, such as the Brazilian AI Bill or the EU AI Act, are usually framed not in terms of AI algorithms but of the systems that implement said algorithms. By modelling an AI system as a computer system that implements an algorithm, said provisions engage with computation as a material practice that happens in the world: it requires time to be executed, demands energy and hardware, and so on. This frame, in turn, allows regulators to adopt various measures directed at the material elements of an AI system, such as the specification of requirements for data management and system design, even if they entail no changes to the decision logic said system implements.
This instrumental malleability comes at the expense of abstraction, as the model must now consider elements that are simply overlooked when one frames AI as an algorithm. In addition, modelling AI as a computer system might also prompt regulators and regulated actors towards technical solutionism: the mere fact that an issue is created (or amplified) by technical means does not mean it can or should be addressed through technical fixes. Modelling AI as an algorithm is not inherently better or worse than a model of AI as a system, so the choice between these approaches will depend on policy goals and the trade-offs involved in the choice of what details can be swept under the rug.
Modelling socio-technical processes
In addition to technical artefacts, models can also stand in for socio-technical processes. Consider how various provisions, such as the risk management framework from Article 9 of the EU AI Act, set up obligations that must be observed throughout the entire life cycle of an AI system. The concept of a life cycle is not defined in the AI Act proposal, but an analogy with human life cycles suggests that a life cycle covers the period between the start of software development for an AI system and the end of its operations. Within that period, a system is created, used, and changed through various technical processes. These processes, in turn, can be represented by life cycle models.
As with the AI objects discussed above, the life cycle of AI admits different models, each directed towards particular purposes. For example, the software engineering scholar Ralf Kneuper divides a machine learning system’s life cycle into four phases. The first phase—requirements analysis—encompasses tasks related to the identification of the goals and conditions that drive software development. The second phase of this life cycle consists of the design processes for deciding the system’s general contours, such as choosing the algorithms the system uses to learn and the configurations of these learning algorithms. The third phase of this life cycle, which is unique to machine learning systems, is the training phase, in which the system builds its predictive model by applying its learning algorithm to a set of training data. Finally, the validation phase experiments with the system’s input data to see how the system behaves when exposed to new contexts. At each phase, software developers have different goals, which must be completed to have a finished machine learning system that can be used.
Let us contrast that example with another life cycle model, drawn from one of my favourite law review articles. Legal scholars David Lehr and Paul Ohm, who have a computer science background, proposed in 2017 an overview of the technical workflows involved in the construction of a machine learning system. While their proposal is heavily informed by technical factors, their ambitions are ultimately legal: they aimed to highlight aspects of machine learning that, in their view, require further attention from legal scholars. In particular, Lehr and Ohm are interested in how the decisions made during the construction of machine learning system—during the workflow they call playing with the data—can be relevant for understanding the sources of potential harm stemming from the use of these systems, as well as the proper legal response to the undesirable consequences of this usage.
The “playing with the data” workflow presented by Lehr and Ohm roughly overlaps with the software development life cycle model presented above, as both models look at the technical practices that happen before a machine learning system can be deployed in the real world. However, this workflow provides quite a different image of the development process than the one offered by the development life cycle model presented above:
As the table above summarizes, each life cycle model emphasizes different aspects of the life cycle of a machine learning system. These differences reflect the different aims of each model: Kneuper’s model has a prescriptive goal, as it is meant to provide a reference point for coordinating the various actors involved in software development processes. Lehr and Ohm, contrastingly, propose their model as a descriptive account of a machine learning system’s life cycle, which helps lawyers understand the impact of software development practices on the system’s outcomes. Each model calls attention to different practices, which in turn lend themselves to different assessments of a system.
Just like the models of an AI system, these two models are not the only possible frames for the life cycle of an AI system. Furthermore, the characteristics that make them suitable for their intended purposes might hinder other kinds of assessment. The model on the left-hand side in Table 1 unites all programming tasks into a single design phase, which might not convey much information if one is looking into issues such as the integration of the system with existing programs. In contrast, the detailed breakdown provided by the model on the right-hand side has many phases that might be unwieldy in practice, especially if one is not focusing on the data treatment stage.
The examples presented above do not exhaust the variety of possible models in the context of AI regulation. One might obtain a different model by changing the target of the representation, which can be an object like an AI system or a process like a life cycle. The same target may be represented in different ways, depending on the purpose one wants to build a model for, the aspects deemed important for representation, and the similarity between the model and the represented target. But, in the context of technology regulation, useful models all share one property: they allow the users of that model—which might be policymakers, scholars, or others—to abstract away from some pieces of technical knowledge while addressing AI-specific issues.
Sometimes, abstracting some aspects of a process or object is a deliberate decision. Removing some detail might allow an observer to build theories that make general claims from individualized observations. Abstraction might also direct a programmer’s efforts, saving energy that might otherwise be spent on factors that pose little risk to one’s goals. Whenever a model is deliberately built, its constructors can weigh the various trade-offs above while choosing how to represent the object. And, once these choices are made, their outcomes can be scrutinized and critiqued. An explicit model is, therefore, another locus that may be used to evaluate AI regulation (or of regulatory scholarship).
But, as the algorithm and AI system examples suggest, some models are never explicitly formulated. They appear instead as the result of the accumulation of various assumptions about the represented object or process, which are used to make sense of the representation target or to supply gaps in our knowledge about it. Consider, for instance, how the model of AI system used by the EU AI Act takes for granted that the provider of a high-risk AI system will have the means needed to conduct extensive technical changes to that system, and it creates obligations that ensure that the system will have certain technical properties. These assumptions, even if implicit, portray the represented object and, in doing so, guide the actions of those who rely on them; for example, by suggesting potential targets for regulatory intervention. But any assessment of the ensuing model—even if to see if the existing assumptions are coherent in the first place—will require an explicit formulation of what is implicit in regulation (or scholarship). The following section proposes why carrying out such a formulation might be a worthwhile endeavour in the context of AI regulation.
Happy holidays!
Thanks for reading! If you enjoyed this, please subscribe and receive future issues in your inbox:
