ExecTech

What CIOs say about big data

adrift in a sea of data

The IBM Center for the Business of Government recently interviewed 28 public-sector CIOs at the federal, state and local levels about the challenges of implementing big-data projects. The findings below are excerpted from a larger report, “Realizing the Promise of Big Data,” written by Arizona State University’s Kevin Desouza.

1. Public agencies are in the early days of their big-data efforts.

CIOs overwhelmingly report that they are just getting started with big-data efforts. While they see the value in synthesizing databases and using analytics, the challenge of big data has overwhelmed them.

Many CIOs also lack a tangible framework to guide their big-data efforts. The financial, policy, and pragmatic intricacies required to set up a big-data project are sometimes beyond the time constraints and expertise of many CIOs and their staff. In addition, the distinction between big data and conventional data is often not clearly understood.

All CIOs are aware of the value proposition of data analytics for their agency. While they do not doubt that big data could revolutionize service delivery and streamline business operations, CIOs believe they have “limited bandwidth” to invest in these efforts. CIOs are also aware that at the present time, much of the existing data in their systems goes unanalyzed.

2. Many CIOs fight the perception that big data is a passing fad.

CIOs say they have had to fight the perception that big data is just a buzzword or a fad. All CIOs note they have to tread carefully when discussing big-data efforts, and a large proportion even note that they have avoided using the term “big data” because of its negative connotation in their agency.

CIOs who had launched big-data efforts received support by focusing on the organizational and business value that would be gained from investments in data analytics. Technologies that would be used for data analysis were placed in the background and were never the focus of the conversation.

I do not have a single staff member who knows how to run simple regressions.... We can be the installers of tools rather easily, but then who is going to train employees [outside IT]?

3. Most CIOs are now primarily dealing with the issue of managing large volumes of data, integrating data across database systems and building an analytical capacity to mine data.

All of the CIOs interviewed are now dealing with structured data. None of the CIOs interviewed have begun to explore the intricacies of managing unstructured data.

Even when it comes to structured data, most CIOs admit that they are not dealing with datasets that are truly big. They are experimenting with creating executive dashboards and using information visualization techniques to make the large volumes of information easily accessible and comprehensible.

4. CIOs report that some big-data projects are now focused on streamlining business processes.

Some of the CIOs interviewed report that they are now focusing on identifying opportunities to streamline and re-engineer their business processes through investments in data management. These CIOs are focusing on process management as a vehicle to gain support from stakeholders, both internal and external.

5. CIOs do not anticipate significant investments in new technology for big data.

Eighty percent of CIOs interviewed report that they would not need to make any significant investments into technology, both hardware and software, during the first few years of their big-data programs. Instead, they would need to find more economical and strategic ways of deploying their current information technology assets.

6. CIOs report a need to bolster their human capital, including their analytical capability.

CIOs report that they are in the midst of bolstering their human capital through hiring new staff and training existing staff. In almost all cases, however, stagnant or diminishing budgets have severely impacted the capabilities of most public-sector IT units. CIOs report losing talented individuals to the private sector or early retirements. In addition, the inability to send staff for training and skill development has resulted in playing catch-up when it comes to big-data skills.

We should be in the business of data governance and data policing.... Our business units should be the ones in charge of analytics.

7. CIOs are now exploring approaches to data governance.

The issue of data governance has been the primary concern for CIOs. All CIOs note that most of the data residing in their information systems is not readily suitable for analysis. Most of the data lacks integrity and could not be easily integrated across systems due to a lack of standardization in data definitions, and even if data could be integrated, there are security and privacy considerations that need to be worked through.

CIOs say that poor data governance is the most critical factor holding up agencies in their efforts to pursue big data.

8. CIOs do not recommend IT units as owners of big-data projects.

CIOs caution against IT units being the owners or instigators of big-data projects. Instead, they believe that senior management support is necessary for success. While senior management support may be a requirement for most IT projects, CIOs say the complexity, transformational nature and upfront investments mean active senior management involvement is absolutely essential for big-data projects.

9. CIOs believe that collaborative leadership is crucial for the success of big-data projects.

CIOs report creating interdepartmental or interagency working groups for big-data projects. These working groups brought together key organizational stakeholders to move the project ahead.

CIOs also rely heavily on their professional networks as they traverse the uncharted waters of big data. Most CIOs report checking in with their peers in other agencies to seek out information and insight. Based on our limited sample of interviews, we did find that CIOs who are more connected to their professional communities (e.g., they speak at industry conferences, are named by other interviewees as standout exemplars, etc.) are further along in their big-data efforts.

We continue to build on a fragile foundation.... Data governance is not sexy and no one wants to do it, yet it is our Achilles’ heel.

10. CIOs are becoming champions of analytics and evidence-driven decision-making.

CIOs note that they are becoming the de facto champions for analytics and evidence-driven decision-making within their organizations. This is a role that many of the CIOs did not envision for themselves but one with which they are getting increasingly comfortable. Given that most public agencies have been influenced by the trends of open government, CIOs have had to become stewards and disseminators of data.

2014 Rising Star Awards

Help us find the next generation of leaders in federal IT.

Reader comments

Thu, Feb 27, 2014 Ulf Mattsson United States

Collecting and processing a huge amount of data is now more feasible and cost effective with Big Data technology, and I think that Big Data is changing the way we are dealing with data. We need to learn how to handle this new approach to data. Unfortunately, our IT teams are used to structure the data before loading it into traditional databases, but Big Data is different. It is easy to forget privacy and risk aspects when collecting data that is not yet analyzed and many organizations are rushing into Big Data focused solely on ROI. There is also a shortage in Big Data skills and an industry-wide shortage in data security personnel, so many organizations don’t even know they are doing anything wrong from a security perspective. I think that the CISO/CSO/CPO should also be involved. I think that many organizations shortly will be struggling with a major big data barrier: 1. I think a big data security crisis is likely to occur very soon and few organizations have the ability to deal with it. 2. We have little knowledge about data loss or theft in big data environments. 3. I imagine it is happening today but has not been disclosed to the public. The good news is that some organizations are proactive and successfully using new approaches to address issues with security and privacy in Big Data environments. These new security approaches are required since Big Data is based on new and different architecture. However, some security tools have been released over the last few years, including Kerberos, which provides strong authentication. But Kerberos does little to protect data flowing in and out of Hadoop, or prevent privileged users such as DBA’s or SA’s from abusing the data. While authentication remains an important part of the data security structure in Hadoop, on its own it falls short of adequate data protection. Another development was the addition of coarse-grained volume or disk encryption, usually provided by data security vendors. This solved one problem, protecting data at rest, but considering one of the primary goals behind Hadoop is using the data, one might suggest that it provided little in the grand scheme of Big Data security. Sensitive data in use for analytics, traveling between nodes, sent to other systems, or even just being viewed is subject to full exposure. Big Data technology vendors up until recently have often left data security up to customers to protect their environments, as they too feel the burden of limited options. Today, vendors such as Teradata, Hortonworks, and Cloudera, have partnered with data security vendors to help fill the security gap. What they’re seeking is advanced functionality equal to the task of balancing security and regulatory compliance with data insights and “big answers”. Ulf Mattsson, CTO Protegrity

Wed, Feb 26, 2014 Kevin Desouza

It is good to see the comments on the findings of the report. As the author of the report, I welcome these ideas and reactions. Wanted to respond to the question on ROI raised by Bruce. Yes, calculating ROI is an important issue and one that needs more attention in the public sector. I am working on a project that is going to look at the various metrics that CIOs can use to calculate ROI. You may find my related post of interest - http://www.informationweek.com/government/leadership/government-it-5-ways-to-measure-value/d/d-id/1113943?

Tue, Feb 25, 2014 Shawn McEwen Canada

Definitely agree with #3 - Organizations will need to be strategic in how they plan to extract "load" and transform the data with real-time access in mind to maximize adoption of the end user community.(CXO, LOB, Marketing, Sales -etc) Measuring KPI and gathering data from structured and unstructured targets provide greater visibility to act on information anytime - anywhere. Lastly, careful consideration should be made on TCO and ROI - what does it cost to build, manage and deploy? With emerging technologies - what impact will future integration have on heterogeneous environments?

Tue, Feb 25, 2014

According to IEEE Computy Society Report, "Top Technology Trends for 2014", we have already moved from Big Data to Extreme Data. http://www.computer.org/portal/web/membership/Top-10-Tech-Trends-in-2014 With the inclusion of the mobile web, #1 Top Trend, CIO's will need to embrace commercial cloud computing more in order to keep up with the data inundation, especially for storage and performing detailed analytics at scale cost effectively in order to provide better line of business decision support and citizen services.

Tue, Feb 25, 2014 Al

Calculus was the North Star of my mathematics curriculum as a young man. Statistics was a strange barnacle stuck on the side of some pre-cal class. I think that is the case for almost everyone in the work force today. In hindsight, maybe Statistics would have been a better focus . . .

Show All Comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above