When algorithms are conceptualized as black boxes, they are simultaneously rendered a problem of the unknown.3 As unknowns, algorithms do not simply signify lack of knowledge or information. The black box notion points to a more specific type of unknown. What the pervasive discourses on transparency and accountability surrounding algorithms and trade secrecy suggest is that algorithms are considered knowable known unknowns (Roberts, 2012)—that is, something that, given the right resources, might be knowable in principle. All that is needed, according to popular discourse, is to find a way of opening up the black box. Indeed, a key mantra in science and technology studies, “opening up the black box,” implies disentangling the complexities and work that goes into making a technical device appear stable and singular.4 The impetus for opening up the black box can also be seen in calls for greater transparency and accountability characteristic of the “audit society” (Power, 1999). In a climate of auditing, organizations are increasingly asked to be transparent about their dealings and ways of operating. Universities, for example, are asked to produce more and more paper trails, including assessment records, numbers of research outputs, and lists of funding received. As Marilyn Strathern (2000a) puts it, axiomatic value is given to increased information. Today, scholars have extended the notion of auditing to the field of algorithms, arguing for the need to conduct audit studies of algorithms in order to detect and combat forms of algorithmic discrimination (Sandvig et al., 2014). While such eforts are certainly admirable, what I want to examine critically as part of this chapter is precisely the hope of intelligibility atached to calls for more transparency. There are many ways of knowing algorithms (broadly understood) besides opening the black box and reading the exact coded instructions that tell the machine what to do. Indeed, while some things are “fundamentally not discoverable” (von Hilgers, 2011: 42), the widespread notion of algorithms as black boxes constitutes something of a red herring— that is, a piece of information that distracts from other (perhaps, more pressing) questions and issues to be addressed. The metaphor of the black box is often too readily used as a way of critiquing algorithms without critically scrutinizing the metaphor itself. What is gained and what is lost when we draw on the metaphor of the black box to describe algorithms? To what extent does the metaphor work at all?
Neither Black nor Box
43