City governments have accumulated enormous amounts of data, and up until the last decade, accessing it was a challenge, even for those working in city governments. With more information going digital, many cities now share datasets publicly. The potential civic benefits are huge: greater government transparency and trust, better-informed decision-making, large-scale efficiencies, and more. Seattle is, as you’d expect, at the front of this movement. A friend and former colleague, Dave Doyle, had the enviable position of Open Data Manager for the City of Seattle’s Open Data Program from 2016 through 2018. I was really pleased to sit down with him to talk about key challenges and successes, surprises, and what’s next in the evolution of open data.
Key Challenges and Successes
One of the main initial goals was very simply to connect users with the data they were seeking. Dave started with the question: who uses the data, and how do we find out what they want to know? Dave and his team started updating the Open Data Portal itself. They had anecdotal feedback that the Portal was hard to use and information difficult to find. Search referrals bore that out: most users came to city datasets through Google rather than the Portal. To improve the Portal’s search experience, the team updated the design to put search front and center. To ensure relevant results, they added and cleaned metadata. And since 25% of users access the site from mobile devices, they made the site mobile-friendly. (If you’re reading this on a phone, take a look at the Open Data Portal mobile experience.) The result was a data-driven overhaul of the site.
Later on, they began to look at other channels like public records requests to identify new datasets for release. Public records requests showed when people couldn’t get information any other way. The team data-mined requests and ran language processing to identify needs via clusters of keywords. This helped identify datasets they could approve without users having to make requests. Dave’s team also found open data champions in different departments who identified datasets they could proactively share.
To identify users, the team grouped requests by email domain, and there were some surprises. The second largest domain was kingcounty.gov, which was in a building right across the street! Another large user group was prison inmates. After initial surprise, it made sense, Dave said – of course an inmate would want police and county records. “That was very eye-opening for me,” he said. “It helped me think about who we wanted to serve.”
The team monitored these improvements via dashboards to track effectiveness. The overall goal was to be more data driven about the information the city shared, connect users with city data, and reduce expensive public records requests. One exemplary success was the Seattle Fire Department’s release of a dataset for their #1 public records request (https://data.seattle.gov/Public-Safety/Underground-Storage-Tank-UST-Records-Residential/xvj2-ai6y). This resulted in a roughly 50% reduction in requests the first year after the dataset was published. It became a great use case for other departments and the public records officers.
The next factor in the Open Data team’s success was communicating with other cities as well as end users. As they made improvements, Dave and his team used Twitter (@SeattleOpenData) to drive awareness of their ongoing efforts. Dave wrote several city Tech Talk blog posts providing updates around issues, and he spoke at national conferences to highlight the team’s progress. City governments have open source communities that share best practices, and the cities of Boston, Chicago, Los Angeles, New York, Philadelphia, and San Francisco shared insights into their metadata standards along with examining Seattle’s updates for lessons. That national recognition reinforced the importance of what Dave and his team were doing internally and created a virtuous cycle.
Seattle also become the first city to do an open data privacy risk assessment. The city’s policy was that data are “open by preference” except when this conflicts with individual privacy, but the city needed a way to deal with these conflicts. The City of Seattle contracted with the non-profit Future of Privacy Forum (FPF) to do the Open Data Risk Assessment, and Dave worked with FPF from the Seattle side. The FPF’s approach was to do a benefit-risk analysis of tool and guideline privacy implications. For example, if Seattle city data were linked with King County, what were the likelihood and risks of someone piecing together information to identify individuals? Would marginalized communities be harmed by data that might make them targets (for redlining, higher insurance rates, etc.)? What strategies could be used for de-identification and risk mitigation? The assessment helped departments determine whether to publish datasets, limit access, or withhold them all together. As a result, departments now have a clear, thoughtful, consistent process for assessing how to share their data, and the city plans to review privacy data governance on an annual basis. Seattle is also one of the few cities with a Chief Privacy Officer and privacy advisory committee, ensuring that the city’s privacy policies continue to evolve and adapt.
All these things have made Seattle one of the most recognized cities in the country when it came to open data management. Other leaders such as San Francisco, New York City, and Chicago began to look at Seattle as an example.
Recommendations for the City of Seattle
Two years are not enough to accomplish everything, so I asked Dave what changes the city still needs to make around open data. His answer was simple: make the Chief Data Officer an executive role rather than having data management roll up to IT.
Seattle is one of the few major cities in the U.S. that doesn’t have a Chief Data Officer (CDO). Having a role completely focused on how the city collects, stores, and shares data is critical for how Seattle moves forward with tools and governance. Departments each have their individual investments in data, but how do you ensure that everyone in the city is collaborating and aligning to the same standards? A Chief Data Officer can do that.
One issue that would benefit from a CDO is how to incorporate the vast quantities of information in legacy databases with those in the cloud. The City has been working on an IT consolidation process for some years now, but retirements can put institutional knowledge at risk. “When the person maintaining a legacy database retires, we’re in trouble,” Dave said dryly. Vendors, of course, are trying to nudge government away from on premises to hosted solutions, but there are many moving parts within city government that need oversight to coordinate them. A CDO could hold those conversations at an executive level and drive data integration strategy across departments. Moving key data to the cloud is inevitable, and as data becomes continuously available, leadership governing it will be key.
The Future of Open Data
“Open Data is coming to the end of a cycle now,” Dave said. The Obama administration kicked off many open data programs and initiatives in 2009 (such as creating https://www.data.gov/). The question now is: how do governments do more with what they have? How do they use it in a way that better benefits municipalities and realizes open data’s full value proposition?
One way cities are doing more with what they have is combining large datasets with predictive analytics to address problems like traffic jams proactively. In one local example, the Seattle Department of Transportation has implemented an adaptive traffic management system (or SCOOT, the Split Cycle Offset Optimization Technique) that detects traffic, predicts flow, and changes lights to flush traffic out quickly around the Mercer area. It has cut time to get out of the Seattle Center after events from 2 hours to 45 minutes. “Smart City” technologies like this also generate more data, which requires local governments consider carefully how to share huge datasets. This means answering questions such as: how to improve data quality at source? How to automate more? How to encourage collaboration across individual departments pushing out datasets in silos? With more data coming online, how to ensure privacy, security, and civil rights?
Through all of this, I was dying to know: What things about government had surprised Dave the most, especially coming from the private tech sector? Dave laughed: “I’ve written an article about that!” (You can read it here: “What government can learn from Microsoft’s cultural transformation.”)
One of the biggest differences Dave found was the length and extent of the budgeting process. The lead time is long (budgets are approved every two years), and any large investment needs City Council approval. In business, a CEO can make a decision and shift everything immediately. City government requires managing up in many different directions and managing the accompanying politics as well. Coming from tech, with its intense pace, city operations can seem stereotypically glacial.
Another odd aspect of city finances is that government requires spending the current budget to justify future allocations. Dave’s natural inclination was to aim for efficiency and savings, but in some cases that was counterproductive. Yet at the same time, he noted, with a public budget you’re very conscious that you’re spending taxpayer dollars. There was a tension between “needing” to spend and wanting to be conscientious that he hadn’t encountered elsewhere.
How about you?
“It was definitely an honor working for the city,” Dave concluded, “and I’m glad I had a chance to serve.” He’d like to see more people go into government for short periods of time. Perhaps government and business could partner around sabbatical-type opportunities for people to take the time to work in government. Many people would love to serve and give back, but the leap in and out isn’t easy. If we could create chances for people with tech skills to step into city government, it would not only benefit the city but would help people better understand how local governments work and set up a positive local exchange. How to get that going is an interesting question that remains to be answered.
How about you – given the chance, would you serve?