The Virtues of Deliberative Policymaking:
In "Public Participation in ICANN," Palfrey, Chen, Hwang, and Eisenkraft have made a valuable contribution to the growing literature analyzing ICANN. They have wrestled to the ground a very large amount of data, and attempted to draw from it a set of conclusions about the effectiveness of ICANN as a model of public participation.
The study's presentation and analysis of data contain much of interest, and much that could assist ICANN (and other policy-making bodies) in improving its use and management of online public forums. But the study's value is diminished by two rather fundamental shortcomings: (1) its misapprehension of both the theory and the practice of ICANN's policy-development process, and (2) the sizeable gap between the broad scope of the study's conclusions and the very narrow -- indeed, myopic -- focus of the analysis from which they are derived. Simply put, the study scrutinizes a small and misleading corner of ICANN (namely, its online public comment forums) and leaps to a sweeping (and, in my view, unwarranted) conclusion.
At its heart, the study appears to have been built around a basic misunderstanding of ICANN's policy-development mechanics. ICANN is an experiment in deliberative, rather than "democratic" or "representative," policymaking. As a thinly-staffed, non-profit coordinator of technical resources, ICANN cannot reasonably be expected to erect a global democracy, nor can it hope to achieve a Board or community of participants that is mathematically representative of the world's population of Internet users. Given the primarily (though, to be sure, not exclusively) technical nature of its responsibilities, the ICANN model instead seeks to foster a reasonable and legitimate policymaking process that is open, transparent, and available to all, but structured to achieve consensus through dialogue and deliberation among informed stakeholders. The ICANN Board acts not as a legislative body that cooks up policies on its own initiative, but as the overseer of this deliberative, bottom-up, consensus-based policy-development process that takes place primarily in the Supporting Organizations. (Of course, the Board also does other things besides policy-setting, such as the implementation of policy, oversight of the staff and finances, and ministerial corporate activities.) Accountability to the Internet community is achieved through a variety of mechanisms, including the wide diversity of channels by which ICANN Board members are selected, the Memorandum of Understanding with the United States government, the various registry and registrar agreements, the Memorandum of Understanding with the Internet Engineering Task Force, and its own Bylaws, which calls for an Ombudsman (not yet hired), and procedures for reconsideration (already in place) and independent review (not yet implemented).
When the ICANN process is functioning well (and I freely acknowledge that it has on occasion functioned poorly; hence, the comprehensive reform and restructuring process of 2002), the role of the ICANN Board with respect to policy development is to ratify the policies that emerge from the consensus-oriented policy development process that takes place primarily in the Supporting Organizations, which are designed to foster informed dialogue and deliberation on proposed policy changes and are charged with actively soliciting public input. In addition to brushing past the central role of the Supporting Organizations, the study fails to distinguish between ICANN Board decisions that establish new policies and those that apply existing policy to make zero-sum choices among contending applications or proposals. This is a significant lacuna in the study's methodology. Three of the four test cases (the selection of new top-level domain (TLD) registries, the selection of a new registry operator for .org, and the approval of the proposed Waiting List Service for .com and .net) involved Board decisions on proposals or applications in which policy was applied, not created; in those three cases, the vast majority of comments posted to the online public comment forums were, predictably enough, from obviously biased partisans. Counting those online comments and comparing the tallies to the Board's ultimate decisions is, in my view, an unenlightening methodology that fails to account for the substantive weight, informativeness, or persuasiveness of a given comment. Rather than analyze the Board's selection of new TLD registry operators (something of a zero-sum game of winners and losers), the study might more fruitfully have examined the deliberative, consensus-based process by which the new TLD policy itself was developed, revised, and ratified. More recently, the study might have looked at the role of public input in the development of ICANN's policies on inter-registrar transfers or the handling of expired domain name registrations. Especially enlightening would be a look at a policy (such as the raising of a fee to fund ICANN activity of general benefit) that required the Board and the ICANN community to balance interests -- looking out for all parts of the Internet, not only the noisy, interested partisans.
Laboring under these misapprehensions about the structure and operation of the ICANN policy process, it is not surprising that the study jumps from data analysis to what are, in my view, some unsupported and erroneous conclusions. For example, though limiting its scope to only one mechanism of input to the ICANN process (the online public forum), the study concludes that ICANN has failed to "attract and incorporat[e] 'representative' input from the global Internet user community, at least with respect to the public online forums." Consequently, the study finds, ICANN needs "an overhaul of [its] governance structure away from its semidemocratic past." In a sense, the first sentence is entirely true, but it misses the point: Due to the their obviously flawed nature, the online public forums and mailing lists were never intended or expected to "attract and incorporat[e] 'representative' input from the global Internet user community." The ICANN process is a complex ecosystem of inputs, deliberative bodies, avenues of participation, and mechanisms of accountability. The ICANN model seeks to bring together a broad slice of the global Internet's interested and knowledgeable stakeholders -- excluding no one -- and to facilitate, through a variety of channels, the development of some measure of consensus among them. Whether that model has succeeded or failed (I would argue that, on balance, the successes have outweighed the failures, and that the reformed ICANN 2.0 is even more likely to achieve meaningful consensus) has vanishingly little to do with what percentage of unauthenticated online comments posted in a public forum coincided with the Board's subsequent decision.
What is puzzling about the study's expansive conclusions is that the study itself goes to significant lengths to cabin the reach of its methodology, noting that "the primary limitation of this report ... is its limited scope and lack of inclusion of meaningful public participation data from within the Supporting Organizations and other ICANN committees." That qualification seems exactly right to me. But the study's failure to acknowledge the central role of the Supporting Organizations left the authors free to derive broad conclusions about the success or failure of public participation in ICANN without undertaking any assessment of the principal designated channels of public participation in ICANN's deliberative process.
In short, concluding that the ICANN experiment in public participation has been a failure because online public forums have been a failure is like saying that television has been a failure because Cop Rock was a failure.
CRITIQUE OF THE STUDY
Below, I elaborate on these criticisms in a somewhat more structured fashion, and attempt to draw some alternative conclusions from the study's data and analysis.
To make a judgment about public participation in ICANN, it is essential to understand how ICANN's organizational architecture embodies a deliberative, rather than a "representative," model of policy development.
The study begins with the observation that "ICANN has repeatedly affirmed its commitment to carrying out a decision-making process that is representative, open and bottom-up," citing the recurrence of the terms "representative," "representation," "openness" and "open" in the various Memoranda of Understanding between ICANN and the U.S. government. ICANN is certainly committed to being open, transparent, bottom-up, and, as a key ICANN policy statement puts it, "representative of the functional and geographic diversity of the Internet." But the study appears to misinterpret the notion of "representativeness" in the ICANN context.
ICANN has never attempted to be -- and was never designed to be -- "representative" of the worldwide Internet community in any mathematically precise way. In view of the vast size of the global population of Internet users, and the specialized technical focus of ICANN's policy-making responsibilities, it would be a hopeless task to try to achieve truly representative statistical proportionality among ICANN's participants, committees, task forces, or Board members. Rather, here's how the U.S. government's foundational 1998 DNS policy statement (the "White Paper") described the core principle of "representation":
For ICANN, then, "representativeness" was to be achieved through (a) input from across the broad, worldwide community of Internet users, with special attention to international participation, and (b) a management structure that "reflects" both functional (i.e., ISPs, technical engineers, business and individual users, civil society, governments) and geographic (i.e., U.S. and non-U.S., developed and developing countries, North and South, etc.) diversity. This more general construction of "representation" (seeking to achieve a balance of interests so that ICANN's decisions tend to benefit the Internet as a whole) is rather different from what the study apparently assumed to be ICANN's objective.
Rather than objectively "representative," ICANN's policy-development process is designed to be both inclusive and deliberative in nature. By "deliberative", I mean that ICANN's general architecture of Supporting Organizations, advisory committees, task forces, public meetings, and inter-organizational liaisons is intended to force stakeholders from across the Internet, functionally- and geographically-speaking, to engage each other in informed dialogue about pending policy issues. Proposal for new policies are presented, analyzed, discussed, dissected, and debated in the relevant committees, task forces, and councils, according to a defined policy development process. The objective is for the diverse participating stakeholders to get informed, explore the facts, hear multiple points of view, think critically about the options, and, through a structured process of dialogues among sub-groups and individuals, to move toward a consensus that maximizes accommodation of all interests. Some participants, of course, will seek to vindicate financial or other interests through particular ICANN policies, so the ICANN structure includes a set of organizational and procedural checks and counterbalances to protect the process from being captured by any particular vested interest.
Curiously, the study regards ICANN's Supporting Organizations as just one more among many options for public input into the ICANN Board. In fact, however, the Supporting Organizations lie at the heart of ICANN's deliberative model. The ICANN Bylaws, for example, make clear that the Country-Code Names Supporting Organization (CCNSO) is responsible for "developing and recommending to the Board global policies relating to country-code top-level domains," that the Generic Names Supporting Organization (GNSO) is responsible for "developing and recommending to the ICANN Board substantive policies relating to generic top-level domains," and that the Address Supporting Organization (ASO) is ICANN's policy-development vehicle for "policy issues relating to the operation, assignment, and management of Internet addresses." The GNSO uses a detailed Policy-Development Process that sets forth a detailed sequence of task forces, reports, public comments, council deliberations, and so forth. A similar Policy-Development Procedure exists for the CCNSO. Those documents define a method for achieving consensus among interested and informed stakeholders through deliberation (including public participation and input), within fairly tight time frames. Somewhat differently, the ASO is built around the open policy forums of the four Regional Internet Registries, each of which hosts online discussion lists and regular face-to-face meetings. The ASO and its Address Council are essentially the aggregators of the RIR policy communities on address-related issues of global, rather than regional, scope. As a result of the Supporting Organizations' detailed policy-development procedures, a policy recommendation from any of them to the ICANN Board represents the final consensus result of a long and often contentious process of dialogue and deliberation among various groups, interests, associations, constituencies, and individuals.
In addition, it should be noted that ICANN has recently created an At-Large Advisory Committee (ALAC), which is a new mechanism for the involvement and participation of individuals through local, regional, or other groupings. All of the activities that take place in these Supporting Organizations and the ALAC are discussed and documented online, and are open to public participation, input, and comment.
The study appears to have breezed past the fact that in the ICANN structure, the Supporting Organizations are intended to provide the primary means by which public input feeds into the creation of consensus policies. For example, public input and participation on generic TLD issues -- such as redemption grace periods, inter-registrar transfer policies, new TLDs, and the Waiting List Service -- is to occur through the GNSO, contributing to the creation of a consensus recommendation. The Supporting Organizations and "public participation" are not two different channels to the ICANN Board; the latter is an inherent part of the former.
The study does make a valid criticism that the ICANN process is complex and difficult for new users to penetrate; it is the hope that the newly reformed ICANN, with its revamped and internationalized website, new At Large Advisory Committee, and dedicated Ombudsman, will do a better job of facilitating participation by newcomers.
The ICANN online public forum is one input tool among many, as the study readily concedes. Indeed, it is perhaps the crudest and least reliable gauge of Internet users' views that is available to ICANN. (Others include the ALAC forums and mailing lists, the regional At Large structures, the Supporting Organization forums and mailing lists, the GNSO constituency forums and mailing lists, the various ICANN-related blogs, and so forth). The online public forum is entirely open and lacks any reliable authentication or verification of posters' identities. There is no way to prevent users from posting multiple messages with multiple identities. There is no way to prevent organizations from instructing their members (or companies their employees) to visit forum.icann.org and post scripted messages, to create an appearance of broad support for, or opposition to, a given proposal. Indeed, the study's own data in Section 3.0 demonstrates that small numbers of heavy posters (some using plainly fictional identities) generate vastly disproportionate volumes of messages. And in some instances, the forum.icann.org channels have indeed been subjected to apparently orchestrated, nearly identical, bursts of submissions. In short, although the messages at forum.icann.org do at times contain thoughtful contributions, a count of the messages pro- and con- on any particular issue is entirely unreliable as a measure of the opinions of Internet users in general.
In the ICANN process, input and comments from any source have to be evaluated across several dimensions:
Disappointingly, the study limits its methodology to (a). Counting online forum comments tells you virtually nothing -- and that's particularly true in the context of a deliberative, consensus-based, technically-oriented policy process like ICANN's. Accordingly, an online public forum for a document or issue that came before the ICANN Board cannot be analyzed as though it were a voting booth through which the Internet users of the world registered their preferences. Contrary to the study's treatment of them, the messages posted at forum.icann.org are so plainly unrepresentative that they cannot seriously be equated with "public opinion," "public commentary," or "Internet users' views." And yet, that is how the study treats them.
At least for ICANN Board-related matters, the value of the online public forum lies not in its utility as a ballot box, but in its role as an open and accessible real-time archive of substantive arguments and relevant information submitted by anyone who cares to post.
The study states that it "seeks to quantify and contextualize the extensive data available at forum.icann.org to determine whether or not public participation through these online forums has been taken into account by the ICANN Board." While the study does an excellent job of quantification, it fails to properly contextualize its analysis.
The study chose four topics to analyze in detail. In my judgment, three of these four prove essentially unenlightening, because the nature of the decision led self-interested groups to use the forum.icann.org message boards as a lobbying tool.
Introduction of New TLDs. The study first examines the ICANN Board's selection of new TLD registry operators in 2000. The study states, erroneously, that "[o]ne of the criteria on which decision-making was to be based was public comment..." In fact, the Criteria for Assessing TLD Proposals, posted 15 August 2000, do not include public comment as a criterion for decision. Rather, the ICANN Board's review process provided for publication of all non-confidential application materials, along with two periods for the submission of public comments. The purpose of the comment periods was not to take a poll and see which applications got the most support (on the unverified, unauthenticated, easily flooded forum.icann.org message boards), but rather to allow anyone to offer substantive arguments or new facts about applicants. ICANN Board members were able to comb through the online comments to see if they found any convincing arguments or other information that they might have missed.
Imprudently, in my view, the study appears to treat the forum.icann.org message boards as some kind of reflection of the views and opinions of the Internet community at large. In reality, the message boards attracted what Prof. James Fishkin memorably calls the "astroturf" grassroots. For example, the .coop proposal put forward by the Cooperative League of the USA attracted 305 comments in favor, and one comment against, a vastly greater number of comments than for any other new TLD application. This was obviously not because the global Internet community was clamoring for a .coop TLD, but because the cooperative associations mobilized their members to flood the message boards.
The public comment periods did, however, prove quite useful in allowing members of communities that were intended to be served by several of the proposed sponsored TLDs to create a public record of their opposition to the proposals -- for example, in the case of the application of the Association Monegasque des Banques (.fin). Moreover, they provided a venue for anyone in the world to register his or her views in the open. The online comment forum was a direct line of communication from any member of the public to the individuals making the selections.
The study asserts that: "Input from the public comments does not appear to have played a major role in the final TLD selections made by the ICANN Board.... [C]onsiderations other than public commentary appeared to play a significant role in final selections, given that positive commentary did not correlate strongly with selected applications." This conclusion is misguided: it equates the forum.icann.org message boards with public opinion, and it ignores the much more important substantive contributions (factual and/or persuasive) that individual messages may have made to the process.
Tellingly, the study points to the new TLD process as possible evidence that ICANN "has failed in soliciting meaningful input from the Internet user community at large... and as such has not lived up to its promise of a representative decision-making process." This statement highlights the study's failure to recognize that the the selection of new TLDs registries was very emphatically not a popularity contest. It was a process to implement an existing consensus policy to add a limited number of new TLDs as a proof of concept. The ICANN Board, acting on the recommendations of the then-Domain Name Supporting Organization (DNSO), undertook what was essentially a zero-sum selection process, judging the applications against neutral criteria. To count the pros and cons on the forum.icann.org message boards and then indict the ICANN process for reaching different results is to misunderstand the unreliable and unrepresentative nature of online public comment forums and the fact that the ICANN Board's decision was a mix of hard facts, neutral substantive criteria, and an quantum of discretionary human judgment. By design, it was not to be influenced by astroturf postings.
VGRS Wait-List Proposal. Here again, the study's use of the forum.icann.org message board as a proxy for public opinion seems unwise. In light of the set of financially-interested parties (registrars, mostly) opposed to competition from a registry-level waiting list service (WLS), it is reasonable to assume that those players were motivated to manufacture evidence of public opposition. It is tempting to assume that the overwhelming message board tallies reflect parallel sentiments in the Internet community at large. But it is entirely possible that average Internet users, if they learned the facts, would agree with the Board that a registry-level waiting list service is simpler and more reliable, and therefore preferable to multiple, unreliable registrar-level offerings. As with new TLDs, the Board's eventual decision to approve the WLS was a case of applying existing (contractual) policies governing the introduction of new registry services. (See, for example, the earlier .name registry's proposal to allow direct second-level registrations). The Board sought input widely, commissioning a report by the DNSO Transfers Task Force and the Names Council, and soliciting comments from the public. The study reports the numbers on forum.icann.org, but its methodology does not shed any light on whether Board members gleaned particularly insightful points or influential arguments from the message boards.
It may be that the WLS decision was taken contrary to the broad Internet community's wishes, or it may not -- we lack the data to know. It was certainly a case in which the ICANN Board acted against a recommendation of a Supporting Organization (though it must be noted that the recommendation was far from unambiguous, as it was accompanied by an alternative recommendation that certain conditions should be imposed in the event that Board decided to approve WLS). However, the WLS process took place under the old, pre-reformed ICANN structure. The new, reformed ICANN boasts a number of streamlined organizational features that are intended to prevent that kind of multilayered breakdown in process that arguably distorted the development of the DNSO's advice and/or the decision of the Board.
Reasignment of .org. As with the selection of new TLD registries, the re-assignment of .org was a zero-sum process in which only one applicant would emerge successful. Due to the likely "astroturf" phenomenon, little weight can reasonably be given to the unweighted relative numbers of pro- and con- messages posted on the forum.icann.org message boards -- particularly where the totals were so small.
Implementation of the Lynn Reform Agenda. The study's analysis of the 2002 ICANN reform process is, in my view, particularly weak. During its intense life, the reform process produced a tremendous amount of deliberation and dialogue within the ICANN community (including the Supporting Organizations, the business, noncommercial, ISP, intellectual property, registrar, and registry constituencies, the Government Advisory Committee, the Security Advisory Committee, the regional Internet registries, the IETF, and so forth), little of which was reflected on the forum.icann.org message boards. Drafts were posted and circulated, comments were sent by email, or relayed in person or on teleconferences. The study fails to capture any of that varied activity, instead asserting that, outside of an ALAC report, "there is no evidence that the Board has taken the Internet user community's input into consideration through this ongoing reform process." Again, the study appears to be equating the highly unrepresentative forum.icann.org message boards with "the Internet user community", while entirely ignoring the numerous other -- and more significant -- channels through which the Internet user community participates in ICANN's deliberative policy-development process.
To get a flavor of all the dialogue, discussion, debate, argument, and negotiation that took place around ICANN reform in 2002 and 2003, click and scroll your way through the working papers, reports, assistance efforts, responses to comments, communiques, updates, etc., that are posted on the archived homepage of the ICANN Evolution and Reform Committee, and its collection of documents. To be credible, the study's conclusions about the role of public input must take account of the many ways in which members of Internet communities around the world took part in the ICANN reform process.
Though I agree with much of the fine-grained analysis presented in the study, I find its more sweeping conclusions to be unwarranted and unsupported. A few final points:
First, I do not agree with the claim that "the online public forums did not play a significant role in achieving a representative and open decision-making process at ICANN." It is certainly the case that the forum.icann.org message boards did not make ICANN more "representative" -- though I have already expressed my sense that the authors of the study are pursuing the wrong kind of "representativeness" for ICANN -- but it is undeniable that they helped to make ICANN more open. For any ICANN decision of any significance, there has been (and still is, in the easily-searchable archives) an open online zone where anyone in the world can post facts or opinions that are accessible to all. No intervention by staff is required, and no editorial discretion can block or bury a posted message.
Second, the study analyzes decision-making processes that took place prior to the reformed structure. The study contends that further reforms are needed to "move ICANN away from its semidemocratic past and toward a structure better suited to the entity's narrow technical management purpose." The study would do well to take a close look at the reformed ICANN 2.0 structure (and its revamped website) and assess whether it will improve (or at least clarify) the ways in which individual Internet users can get involved in the ICANN process.
Third, I find confusing the study's final conclusion that "to the extent that we seek new means of governing the technical architecture of the Internet ... we ought to look beyond ICANN, which may never have been the right place for such experimentation given its limited technical mandate." It is unclear what the authors mean by "governing the technical architecture of the Internet," or why they assert that a "technical mandate" is inappropriate to govern "technical architecture." Certainly, for public policy issues such as hate speech, online gambling, and the availability of online pornography, ICANN is not the right place for experimentation: it neither wants nor could assume, as a practical matter, any role in those areas. But in the area of coordinating a chunk of the Internet's technical plumbing (in the form of its systems of unique identifiers), we ought to be looking squarely at ICANN, not beyond it. Anyone interested in the DNS and related technical issues really should participate in ICANN (heck, it's a great organization amid a lively community populated by thousands of interesting and engaged volunteers from around the world!). The world doesn't need a "new means of governing the technical architecture of the Internet"; it needs to understand the profoundly successful technical protocols, operational practices, and policy consensuses that have made the Internet the most powerful, resilient, scalable, and flexible mechanism of communication ever, and to support through active participation the organizations like ICANN that keep the Internet stable and reliable, as it grows and evolves.
In sum, my congratulations to the authors for their impressive quantitative and analytic work. As a friendly critic, my advice is to (a) expand the methodology to address the various mechanisms of public input (e.g., the Supporting Organizations, GNSO constituencies, regional Internet registries, country-code TLD registries, Governmental and Security Advisory Committes, Internet Society chapters, civil society groups, and so forth) that are much more important to the ICANN process than the forum.icann.org message boards, and/or (b) rework the conclusions so that they fit the otherwise excellent data analysis.