Neither Black nor Box
(Un)knowing Algorithms
The power of algorithms can be felt in all spheres of society. Algorithms do not merely filter news or arrange people’s social media feeds, they also are routinely used to help law enforcement, financial institutions, and the health sector make decisions that may have a profound impact on individual lives (Pasquale, 2015). Yet algorithms are not the kinds of objective and neutral decision makers that they are frequently held out to be. Examples of bias in algorithmic decision-making now abound, and scholars and policymakers are trying to figure out the best ways to govern and make these systems more accountable. A major obstacle to holding algorithms accountable, however, is to know them in the first place. Many of the algorithms that we are most concerned with and interested in—for example, those that underlie popular media platforms—are often governed by trade-secret protection. The algorithms running Facebook and Google are the “secret sauces” that give shape to the information and data flowing online. Algorithms are made to be opaque—not just to protect businesses but due to the technical necessity of handling the complexity of the system. Knowing algorithms, then, is severely burdened by their often “black-box” nature. As Frank Pasquale writes, knowing what algorithms do maters because authority is increasingly expressed through them: “Decisions that used to be based on human reflection are now made automatically” by encoded rules that are “hidden within black boxes” (2015: 8). Algorithmic decision-making is often described as impenetrable and secretive, concealed behind a “veil of code” and trade law (Perel & Elkin-Koren, 2017). The story often goes that, if only we could make algorithms more transparent, we would stand a beter chance of governing the big corporations that make these automatic decisions on our behalf.
While “transparency is not just an end in itself,” transparency is still seen as a necessary condition for greater intelligibility (Pasquale, 2015: 8). As Barack Obama (2009) famously stated in the opening of a memorandum of the Freedom of Information Act: “A democracy requires accountability, and accountability requires transparency.” The question remains as to what exactly it is that should be made transparent and what transparency is believed to help reveal. In her review of
41