Real Time Disclosure, Technically Speaking

2010-01-26

Originally posted on the Sunlight Labs blog.

Last week’s Citizens United v. FEC Supreme Court decision was a game-changer in terms of corporate money in politics. In short, corporations will be allowed to freely spend unlimited amounts of money to support or oppose a candidate, just as long as there is no direct coordination with that candidate’s campaign. Unprecedented amounts of corporate money will now flow into our political process. But here at Sunlight, we’re focusing on the disclosure aspects of the decision. The majority opinion stated:

With the advent of the Internet, prompt disclosure of expenditures can provide shareholders and citizens with the information needed to hold corporations and elected officials accountable for their positions and supporters.

While the decision doesn’t mandate “prompt disclosure”, it does strongly recommend it. So what does disclosure at Internet speed look like?

Examples of Real Time Disclosure

Say you contribute $100 to your congressional campaign of choice. The instant your credit card is approved, or your check is processed, the campaign will send a notification to the FEC of the contribution. The FEC’s servers will verify and record it, and instantly broadcast the details of the contribution to any third party who wants to receive it. It should all take place in a matter of seconds.

Campaign expenditures (and now political expenditures by corporations) would work similarly. For example, when a campaign or corporation decides to spend $25,000 on an ad buy, they would need to disclose that to the FEC within a 24 hour period. Unlike with contributions, disclosing expenditures needs to have a staffer manually enter data, so it would be impractical to demand instant disclosure in this case. But once the FEC receives the data about the expenditure within the 24 hour timeframe, the FEC should instantly broadcast that data to all interested third parties, just like with contributions.

Proposed Solutions

Real time web technologies have come into their own this past year. Heck, there was even a conference about it. And importantly, real time solutions can be created with open web standards and open source software, two things we strongly want to see government embrace.

HTTP

One proposed solution would be based on HTTP, the protocol of the Web: An FEC-powered RESTful API with strong querying capabilities, speaking XML and/or JSON, and made available for read and write access. The past few years have proven that well-documented RESTful APIs are the way to go. Twitter, Google, and Sunlight Labs have all embraced this convention. Contributions and expenditures would be POSTed by registered campaigns and corporations, and the public could read all the data via GET.

One newer real time technology, based on HTTP and RESTful principles, is PubSubHubbub, or PuSH for short. PuSH excels in real-time, event-driven notifications. Usually, when you want to know if a given API has been updated, you need to ask it repeatedly if there’s any new data. This technique, called polling, inefficiently uses up bandwidth because there are a lot of requests made where nothing new is to be found. Instead, PuSH mandates that interested parties subscribe to receive notifications, giving the publishing server a URL to send a notification to, and it’s up to the interested party to take action when they receive that notification at the URL they control. Registering your URL endpoint to receive a notification is known as a Webhook.

XMPP

Unlike HTTP, XMPP (eXtensible Messaging and Presence Protocol) was designed for real time applications and asynchronous communication. You’ve used XMPP if you’ve used Google Talk for chatting, or other Jabber IM services. At the simplest level, XMPP is a way to send instant messages around the Internet, with support for extending the definition of those messages to suit your needs. XMPP also defines a publish-subscribe extension, which would allow third parties to subscribe to a particular set of disclosure data and receive messages from the server instantly.

Note: For simplicity’s sake, the diagram above ignores XMPP’s decentralization features.

Either solution is well-supported by solid open source software. HTTP servers like Apache and nginx are ubiquitous, and frameworks like Django and Rails make creating RESTful APIs a snap. For PuSH, Google has released a reference implementation running on App Engine. The server of choice for XMPP is ejabberd, an Erlang-based project.

Real time disclosure is possible today. The technology is here, and it’s open. Now, we need to muster the political will to see it through.