Dirty data is no longer a little secret

Kentucky’s large-scale approach should make information more reliable and easier to share

Dirty data is an unpleasant problem governmentwide, the result of years of well-intentioned but piecemeal automation efforts. Yet few officials are willing to commit to the unglamorous job of cleaning it up.Now technology executives in Kentucky are putting the scrub brush to their data, hoping that their efforts will lead to short- and long-term benefits for the state’s agencies and residents.At the most basic level, redundant and conflicting information leads to multiple records with, for example, one agency listing constituents by last name then first name and another department reversing the order. Factor in shortened versions — Robfor Robert, for example — abbreviations and typos, and there are ample opportunities for inaccuracies that waste employees’ time and lead to poor customer service.In addition to those day-to-day problems, inconsistent data can also undermine longer-term plans to promote cross-departmental information sharing and build new service-oriented architectures that rely on mixing and matching data and applications.The state’s answer is the Kentucky Enterprise Data Architecture (KEDA). Despite its technical-sounding name, it is designed to benefit department heads and program managers by reducing the cost of deploying new applications.Kentucky officials hope to reuse information across multiple departments and applications, a strategy that could make more money available for information technology projects, said Mark Rutledge, commissioner of the Commonwealth Office of Technology.If all goes as planned, KEDA will induce business managers to say, “‘This data project is saving me money, and by the way, it’s shortened the development time so I’ll realize the benefits of a new project sooner than we otherwise would,’” Rutledge said.Kentucky officials expect more reliable data will help them save time and money. KEDA should enable nine Cabinet agencies and other organizations in the executive branch to share information more effectively.In the past, departments stored data in a variety of formats and systems, which suited the needs of individual organizations but made cross-agency sharing difficult, Rutledge said.By contrast, KEDA recognizes data as a state asset, said Neil Downing, director of enterprise information management at IT consultant Keane. Kentucky hired the company to help with the project.The state’s efforts mirror similar attempts at federal agencies to make data sharing easier and more reliable. For example, at the end of August, the federal Office of the Program Manager for the Information Sharing Environment issued the first version of an enterprise architecture framework designed to help federal agencies share and search terrorism information across jurisdictional boundaries.KEDA’s first phase, now under way, entails an inventory of all data repositories, which a task force of IT staffers, departmental representatives and Keane employees are conducting. That effort will identify duplicate data caused by inconsistent formatting.“Within multiple tax codes in the Department of Revenue, I may be Rutledge comma Mark, Rutledge Mark A. or Rutledge A. Mark, depending on the strategy that was used to develop that application,” Rutledge said. “There’s not a common data architecture.”Besides leading to redundancies, inconsistent formats make it difficult for policy-makers to gather complete data for decision-making. For example, as Kentucky works to revise its energy policies, program managers must painstakingly find all the databases at multiple agencies that contain information about the coal industry. Some of that data might reside in Microsoft Access databases, while other records are stored in other relational databases.“We have to make sure that [the multiple data sources] are not duplicated, that they are validated and then map all of them to get a composite view,” Rutledge said. “It’s challenging and it’s time-consuming.”Kentucky’s plan for organizing data, known as a common data framework, will point staff members to sources of record — the data sources that offer the most accurate and up-to-date information.When the framework is in place, “you can draw information from that data and not have to spend all that time and energy trying to map it, massage it or rework it so that it’s usable for the user,” Rutledge said. Wherever data is, he added, “we want to be able to use it. That’s the key objective.”Kentucky enlisted Keane to help the IT department and the interagency task force develop data formats for use statewide.The interagency task force is also working to create guidelines for granting access rights and permissions, while another task force is devising an identity management system. Additionally, Kentucky’s Department of Revenue, the Office of the Governor, and the cabinets for Personnel, Transportation and Economic Development are working on how they will share their respective information.“The role for IT is to ask, ‘What is it that you need and what are some of the risks or concerns that you have so that we can meet the needs and minimize the risks?’” Rutledge said.After the initial phase is completed in about six months, the KEDA team will develop the architectural details of the framework, including formatting standards. It will also develop long-term application plans — including a service-oriented architecture — that will build on the data foundation.Success in each phase of the effort will hinge on the willingness of people outside the IT department to see the value of what looks like an arcane technology problem. “If you just go to them and say, ‘I’ve got this wonderful star schema [for warehousing data],’ people will literally run for the hills,” said John Daly, senior vice president and chief innovation officer at Keane.A better approach is to emphasize the business benefits of KEDA for department leaders, he added.“You come in and say, ‘Your business process is clearly broken because it is pointing at these four systems, and it’s costing you X a year. But if we built it this way, you can see how much easier that would be for you,’” Daly said. “That’s how you build consensus and get people involved.”To promote those kinds of discussions, Keane is organizing workshops with individual departments to determine their business issues. That information will help officials tailor the final framework design.“To provide long-term value through this framework, it’s going to align with their business goals,” Daly said. “We also tell them, ‘Oh, by the way, because we are architecting with a Web services tie, we can quickly move to other information-delivery channels, such as a BlackBerry.’”Rutledge said he’s pleased with the level of support KEDA is receiving from department managers, but he doesn’t want to become complacent. “For any project, the newness is exciting,” he said. “But once you create that energy, you have to sustain it.”One way to do that is to pick projects likely to see quick results. “It re-energizes people if we can say, ‘We accomplished this, and our next target is here,’” Rutledge said. “That’s how you keep the energy going forward.”Kentucky officials want to add business intelligence capabilities to KEDA, which could be a way to demonstrate quick results. Such technology has typically fed historical data to complex analytical systems so experts could identify trends and create performance reports. Newer technologies are more democratic. They emphasize funneling information to people to help them do their jobs more efficiently. Delivery tools include Web portals with summaries of key data important to each job function.“A data architecture framework will mean that, along with real-time business intelligence, we can gain the information we need to make strategic decisions,” Rutledge said.







“If you just go to them and say, ‘I’ve got this wonderful star schema,’ people will literally run for the hills. ” John Daly, Keane









Information assets















“It re-energizes people if we can say, ‘We accomplished this, and our next target is here.’ ”
Mark Rutledge, Kentucky’s Commonwealth Office of Technology



Common formats











Stumbling blocks



















Joch is a business and technology writer based in New England. He can be reached at ajoch@worldpath.com.
X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.