Infopocalypse: The cost of too much data

Meet big government's next big problem: Digital Overload.
By CHRIS FARAONE  |  December 29, 2010

Our nation is drowning in data. At any given time, federal agencies use more electronic storage units than could fill every NFL stadium from Oakland to Foxboro. At last count, the US government owns or leases at least 2100 data centers, and spends about half of its multi-billion dollar IT budget on digital storage. The United States Census Bureau alone maintains about 2560 terabytes of information — more data than is contained in all the academic libraries in America, and the equivalent of about 50 million four-door filing cabinets of text documents. In addition to the federal deluge, tens of thousands of municipal and state facilities maintain data ranging from driver's-license pics to administrative e-mails — or at least they're required to.

"Going digital" was supposed to be an environmentally conscious way for governments to cut costs while improving efficiency. But it hasn't quite worked out that way yet. Storage capacity is increasing, but the volume of data is also increasing, perhaps just as quickly. Over the next decade, the world will produce the informational equivalent of nearly 100 million Libraries of Congress per year, according to Cisco's Internet Business Solutions Group.

Most of what the government stores are public records, which means that by law they must be made available to anyone who requests them. But while there are ambitious efforts underway to improve storage methods, the sheer bulk of information is alarming.

These quandaries are turning into an increasingly expensive problem, and one that now has its own advisory bureau: the Office of Government Information Services (OGIS) was opened in 2009, solely to serve as an intermediary service between requesters of information and agency gatekeepers. Furthermore, Barack Obama was the first president to appoint a chief information officer, the operational-IT wunderkind Vivek Kundra, who has made it a priority to streamline the government's inefficient, energy-wasteful information infrastructure under a newly enacted Federal Data Center Consolidation Initiative. Kundra claims the US has already reduced its overall IT budget by $3 billion, while increasing functionality in some agencies by more than four times. Still, dire problems persist.

Bogged down
Public data remains, by and large, a disorganized mess. A recent report by the Citizens for Responsibility and Ethics in Washington (CREW) found that despite the administration's policy adjustments, Freedom of Information Act requests are still not being processed much differently than they used to be.

Indeed, from Washington DC to small-town America, governments have quietly begun to acknowledge that the information age has gotten them in way over their heads. Why?

There is too much data. Digital storage is not a natural resource. The amount of information that government agencies may be required to keep — from tweets and e-mails to tax histories — is growing faster than the capacity for storage.

There's not enough manpower to manage all this data. The Obama administration hopes to make more information freely available online. But in the meantime, the old method of requesting data from the government — filing a FOIA request — is bogged down due to an insufficient workforce and long request backlogs.

Private companies are storing public data. This trend in outsourcing, largely the result of too much data and too little manpower, is a potential threat to both access and security, as resources that belong to the people are entrusted to outside vendors, raising new privacy concerns.

1  |  2  |  3  |  4  |  5  |   next >
Related: Slideshow: Boston Tea Party 2009, Does Portland want an elected mayor?, Photos: MassCann's 2010 Freedom Rally, More more >
  Topics: News Features , Citizens for Responsibility and Ethics in Washington, government, Freedom of Information Act,  More more >
| More

Most Popular
Share this entry with Delicious
  •   THE TRIALS OF NADIA NAFFE  |  March 04, 2013
    Young, attractive, ambitious, conservative, and black, Nadia Naffe should have been a right-wing operative’s dream.
  •   HIP-HOP TRIVIA GROWS UP  |  February 26, 2013
    In their fourth year of operation, the Hip-Hop Trivia squad is finally taking the night (somewhat) seriously.
  •   OCCUPY DENIED DAY IN COURT  |  February 22, 2013
    It took more than a year for Suffolk County prosecutors to come to their senses.
  •   CZARFACE SOARS ABOVE THE CLOUDS  |  February 11, 2013
    This week 7LES and Inspectah Deck drop Czarface , a full-length work of adventurous genius revolving around a metal-clad protagonist who feeds on destruction.
  •   THE BPD ADDS INSULT TO INJURY  |  February 05, 2013
    At times, this kind of decision makes you wonder whether the BPD is saving its best awards for officers who've been involved in the death of civilians.

 See all articles by: CHRIS FARAONE