"Local Government has been Emasculated" (But so has everything else)

From a systemic perspective, the central-vs-local government clash continues to be pretty bemusing.

On the one hand, central government decides how much localism there should be, and giveths and takeths away from local or super-local (regional/LEPs, etc, whichever the trendier?) all the time.

On the other hand, the hierarchy at all levels is being infiltrated by both the private sector, and the "new", networked form of group organisation which has managed to somehow avoid the "postcode lottery" moniker so far.

Lord Heseltine is advocating even more decentralisation away from central government, and offers some small looks into where local-interest power means that Stuff Gets Done, ie. that the hierarchy needs to get out of the way of people just getting together to achieve something.

A few days ago I started thinking about how a community wifi effort (Google Doc) could benefit and be hampered by a "network" approach vs a "hierarchical" approach. (Hint: there's no straightforward answer.) Richard Veryard suggested that the network/hierarchy split (or spectrum) wasn't too useful, which I'm still unpicking, but it did get me thinking about why we organise into different types of group.

It is perhaps most true to say that democratic models are the most complex they have ever been. This is different to saying that "we need to move to a network democracy" or that "it is local government's time".

It is understanding instead that we have a very real situation of multiple, overlapping, integrating models of democracy organisation/objective/philosophy. There is no "one approach fits all" solution because there never was - we just managed to shoehorn the world into the latest mainstream model through a heavy imbalance of power.

Local Government has been emasculated. But so has central government. And perhaps so have ordinary citizens along the way.

I'm going meta. There is no such thing as democracy, but there is such a thing as a democracy network. Which, confusingly, includes networked democracy - but also centralised democracy, European democracy, local democracy and hyperlocal democracy.

Which means there's no point arguing over which one's best, only over how we make sure they all get along.

Why the Public Sector Needs to Get Stupider

...or "From Intellect to Intelligence".

There are a handful of people whom I've never met in person, but talked to a lot over the years online, and who have influenced my thinking more than I could suspect in that time. Richard Veryard is one of those people. Even back when my blog "Into the Machine" was called "Blunkett is an Arse", Richard prompted me to think heavily about the role of power in society - everything from CCTV through to newspaper headlines and reasons for going to war.

Richard introduced me to Stafford Beer's concept of POSIWID (The Purpose Of the System Is What It Does) which trips off the tongue more easily and addictedly than you'd expect. Recently, he's focused his efforts on Enterprise Architecture ("#entarch") and Organisational Intelligence ("OI"), and I'm very much enjoying shamefully yanking ideas out of his OI Primer e-book.

I don't plug things very often on my blogs (and it's horrifically short notice) but I really do want to mention his 2 courses this week: on Business Architecture and Organisational Intelligence. I think there are still spaces left. And obviously as I've never actually met Richard, I'm not guaranteeing anything. But if you're interested in this post, or this blog, check em out. There's also an Org Intelligence group on LinkedIn for good discussion.

But it's more than just a shameless plug - and more than just a way of paying him back for the insight over the years. Fundamentally, his driving factor (IMHO) is around something which is not just lacking in many groups, but dangerously lacking across society. In fact, my whole last post on the PCC elections, nuclear subs and mythological engineering was about this lack.

In short, we lack systemic intelligence.

What does that really mean though? And more importantly, so what? Many smart people instinctively know that our most common modes of organisation lack an inherent "togetherness". I'm not saying anything new here.

But what we can learn from Richard - and from looking around us, and from taking a moment to think clearly - is how to move from just knowing there is a lack of systems-thinking, to doing something about it. To rejoining the things which are fragmented.

Departments are Inherently Silo Factories

In the public sector, the defragmenting process is bubbling under like the power behind the throne. It's why the Patchwork project makes sense to people. It's why everyone goes on and on about breaking down data silos. It's why joined-up government arose - but ultimately, its fall also highlights not how difficult it is, but how badly prepared we are for it.

Our pin-factory-style, categorical hierarchy mentality likes to divide work up into efficient processes. Many people claim this to be an "efficient" way to focus on a particular labour task - which it can be, to an extent. But this also leads directly to those data silos we mentioned earlier - because it's not the data which is inherently silo'd, but the labour itself. Once a person or a group becomes responsible for an organisational function, both the data and the importance that result from it become an asset - and assets are there to be protected, especially under competitive pressures.

The same effect is at play whether it's one person keeping their spreadsheet private, a department keeping its data in obscure formats, or a company slurping personal data on millions of customers. Data goes hand-in-hand with function, and divided data leads to divided functions.

The Road to Recovery is Balance

The first challenge in reconnecting these functions (and therefore departments, and data) within a system is to get out of the existing model. It's important to remember that an fragmented viewpoint is mutually exclusive to systems thinking. Systems thinking is not about the best way to get two existing units (departments, groups, people) to talk to each other, nor about opening everything up to the glorious light of transparency.

Systems thinking is about how the units depend on each other and what it makes sense for them to share in order to do their (otherwise standalone) job. Object-oriented coders Get This.

In other words, the underlying mental model for a more systemic form of organisation needs to shift to one of multiple contexts - firstly, the overall purpose of the entire organisation, and secondly, how the needs (data, assets, etc) of each unit within that organisation break up and overlap.

That is, there needs to be:

1. Balance between the output of each unit within the organisation, and the organisation as a whole - no one unit should be seen as "more" important, in the same way that the brain is not "more important" than the stomach. The priority is that the organismisation acts as a whole.
   
2. Balance between the "internal" and "external" roles of each unit, so that effort is not wasted on either information over- or under-provision. Each unit should have access to the information it needs to do its job. And yes, this changes over time. Get used to it.

Forwards, stupid workers! 

"Architecting" this is not just a role for a head honcho or a business consultant. Ignore what well-paid people tell you. It's not even a bottom-up process.

The needs of units and the whole organisation change over time - technology relating to certain tasks changes, staff change, external factors mean new pressure, political goals shift. As a result, this process has to be infused into the whole of the organisation. It is a culture and a strategy on one level, but on another level it is also an individual choice. 

The divided labour approach to work - ie. what you have in your head the moment you get to your desk in the morning - intrinsically ties together function with knowledge. "What do you need to be told in order to do you job?" - or from a more negative, jobsworthy viewpoint, "I can't do that because nobody has told me why or how."

Divided labour resists flexibility. And this is why joining everything together takes more than someone banging a gong or drafting a "data sharing" proposal - the rules themselves need to be flexible for flexible working to work.

But so often, the body of working knowledge we hold - and the reasons why we hold it and where we should apply it - are valued. They form a set of ideas centred on our job function that I call "Intellect" - in other words, a specialised definition that takes the immediate environment, and hardens it - formalises it. This means we can know if what we're doing is "right" or "wrong". And we can't be judged for not doing stuff that lies outside of that context. Nobody ever got fired for buying IBM.

Losing this "Intellect", this hard-wired context, is key to making our organisations smarter.

We need to become stupider.

Photo: San Diego Shooter

The Dashboard Wisdom of Stupidity

Being stupid is not a weakness. I heartily wish that I had been more stupid in school, and wasn't so afraid to ask questions when I didn't fully understand something. Because when all the parameters around you are shifting, guess what? Nobody really understands anything. And even more so, nobody can predict what is happening to the people they're connected to.

Asking questions gives us feedback. One of the fascinating areas I looked into after reading Richard's book was the usefulness of dashboards - which are basically designed to answer that perennial question, "What's going on right now?" 

Feedback is only useful if it's fast - fast enough to allow us to relate our actions to their consequences.

And immediate feedback is awesome because it gives us an answer before we've even had time to ask the question. A dashboard, an ever-present source of up-to-date information, assumes we are stupid - or, at least, could turn stupid at any moment. "Hey! Driver! Sooner or later, you're going to need this information, you know?"

And when you're sitting there, protecting your body of knowledge and your data in order to protect your job, hanging on to that "Intellect" you've carefully gathered doesn't make you "Intelligent". It makes you "Wrong".

Your Silo is My Silo

The world is changing at a much faster pace than we're ready for. No, scrap that. The world is change, and we don't like change. Everything we do is about controlling change. Our policies are direct descendants of the 17th century,and hence our mental models are struggling to keep up with the change going on - change that we've created, ironically. (Remember, pin factories didn't always exist.)

"Systems" intelligence and "Organisational" intelligence are misnomers, because people are individuals, not organisations - and organisations only really exist on paper and in law courts, and can't make me a cup of tea in the morning.

But individuals exist within organisations, within systems. There is little to differentiate the two except perhaps where we stand. 

Accepting that we are part of a system is essential to realising that this "systemic" intelligence is also a part of us. It is an aggregative effect - a flow of cause and effect from person to person, and unit to unit. It is not a "culture" or a "programme" or any "Thing", because it is always a dynamicism that acts on every level.

Asking questions is key to opening up that flow - the flow of information and understanding between units, between silos, and between contexts. It breaks down the protection of Intellect. It is a form of empathic efficiency. It is flexibility. 

It is necessary.

Want to know more? Richard's courses on Enterprise Architecture and Organisational Intelligence.

PCCs, Nuclear Subs, and Mythological Engineering

With the low turnout in the PCC elections yesterday,  and this interesting piece on the construction problems with the UK's new nuclear subs, I can't help but think of Barthes' "Mythologies" and the state we're in regarding symbolism vs engineering.

i. asking the right questions

Barthes lays out a sketchy-but-useful/interesting framework for signs and symbols, and in an ever more mediated world (first through broadcast media such as TV, and increasingly through short-attention information-overload) this framework seems more relevant.

The 3 main questions that a mythological viewpoint helps get at are:

1. Is what we're doing actually working?
2. If not, can we find out why not?
3. And if not, are we willing to identify where our own beliefs are causing those reasons?

None of these are significantly mythological - we can fail at something because we lack understanding or experience, for instance. Indeed failing is also learning, if the context and expectations are set out properly.

But the PCC and Nuclear Sub cases, when viewed from a symbolic, mythological viewpoint, become a lot more understandable.

ii. nothing to see here

In the case of the PCC elections, it's telling that many people voted not to improve the operation and efficiency of their Police force, but to keep out people who they think would make their Police force worse. Question #1 has moved from "is it working?" to "is it not breaking?". Or, in other words, from "how can we improve things?" to "how can we stop things from getting worse?"

This shift in rationale is essential to understand. It shows that politics, a symbol in itself, has become a stronger signifier for maintaining the status quo than for setting out sustainable solutions that involve cross-society engagement. We can say that Democracy has become about keeping people out rather than integrating people in.

The PCC elections are clearly to be seen in a light of localism and "democratic accountability". The elections are a symbol for democratic principles, but ones which have left behind everyday operations of how policy becomes policing. The "polis", in every sense, has become an abstract notion, and can only surely be less empowered because of that.

Not only was it seen to be more important to make the system "votable" than have a working system, the idea of the vote itself took precedence over the voting process.

iii. an aside: symbolic actions

Barthes contrasts the object as a "symbol" to the object as part of an "action" - eg. a woodcutter chopping down a tree performs an action on the tree, and so there is a real-world interaction and effect with the tree. However, the image of a woodcutter cutting down a tree, or a generic image of a tree in itself, is "symbolic" in that it has no direct, real-world consequences. A logo of a tree may act as a standard, but is an indirect effect, and can certainly be detached from the actions it inspires. (Many people find it easier to be inspired by a logo than to act based on inspiration.)

iv. unclear power

Nuclear power is another symbol, as any fan of Dr Strangelove will understand. The paradox with nuclear power is that it is "inherently" (or "naturally", as Barthes would put it) powerful and therefore dangerous - we accept this without question. By wielding "Nuclear power" as a symbol, we also wield an intrinsic argument for more control and global sanctions - because power is concerned with potential rather than actual abilities.

This paradox leads to the problems seen under the HMS Astute nuclear programme. Building nuclear power as a symbol is very different to building nuclear power as a "thing" in the real world.

The paradox is this: You cannot design a machine (or an organisation, or a person) based on symbolic or economic principles.Sure, they can feed into the process, but engineering needs to maintain an inherent, internal coherence to provide a "thing" which functions as a whole. Allow a fragmented economic-political process to take priority, and you will end up with a fragmented machine.

Ironically, the nuclear machine does not depend on the nuclear symbol in order to work, but the symbol is impacted by the effectiveness of the machine. In undermining the effectiveness of the machine, the symbol also devalues itself. 

v. tappity-tap

Churchill said that "Scientists should be on tap, but not on top". I don't believe it's a case of scientists (and engineers, and anyone that designs any kind of process) being on top of  politicians, or vice versa though. The two "sides" have very different briefs and contexts that need to work with each other. There is a "what" and a "how", and the twain will always meet somewhere.

Kasabi, Nuclear Power, and the Data Developer Dichotomy

Today's big news is that Talis are shutting down Kasabi, their linked data platform, moving away from semantic data, and losing Leigh Dodds. Talis have probably been the most visible entity in the commercial linked-data realm, and with good reason. They showcased an impressive local data portal for the Open Data Cities Conference a while back in Brighton, and drive various high-profile services like Fix My Street.

But it turns out the demand for the linked data platform just isn't enough to support business. As a data developer, I'm obviously intrigued by this move from Talis.
(Spoiler: There's a lot of background here, and I don't even answer my original question. Feel free to skip to the end.)

How I Learned to Stop Worrying And Love Data

Your front room? [img]

Back at Uni, I read that nuclear power isn't just about physics, technology and war. To wield nukes, a state needs certain organisational structures to support their centralised nature - security structures to protect them, political structures to commission, deploy or maintain them, market structures to permit or prevent sales, and so on. In short, technologies and their usage depend on social-political contexts - and vice versa.

At the other extreme, well away from nuclear power, micro-generation of power is slowly taking a footing. Feeding power into the grid from solar and other electricity generators also requires contexts to be put into place, but the nature and distribution of these are different to those of nuclear power. The consultations yield different emotions, the networks require bi-directional structures, as do tariffs and security measures.

Overall, there is a big difference between the "central" nature of nuclear power, and the "decentralised" nature of micro-generation.This maze of contexts is what Open Data is trying to navigate right now. And Linked Data is part of that, often trying to straddle worlds.

Platform Non-Wars

Once upon a time, XML was the saviour of the data world. Up until then, people had put up with CSVs and related 2x2 grid matrices. Data was a person-to-person communication tool, not a machine-to-machine one. Machines kept their own data, and made it available through their own interface. So passing data around was a presentation job, and people liked doing that in 2D.

XML changed that by structuring data differently - the context of XML was a machine-to-machine medium. Some simple XML can be coded by hand, such as HTML. Other XML (and lots of HTML) is really intended only to be generated and read by machines, which will do the validation, parsing and rendering without the aid of human hands.

XML can afford to be complex then, because the context is strictly defined by standards and parsers. Maybe XML is the nuclear power of the data format world.

Then people discovered JSON, a much more lightweight format which was still supposed to be machine-readable, but which also caters more to a second context - person-readable. Because JSON is so lightweight, it's much easier to debug by hand. And while there are huge industries booming around XML, the lure of JSON appeals to those who don't have rigid structures and definitions in place, who need to do something quickly and often by hand, at least at first.

Like nuclear and micro-generation though, XML and JSON aren't a "versus" or a battle. They're both just about what's appropriate for what context. This is an Important Point.

Open Wide, Please

What could possibly go wrong? [img]

This is where we return to Open Data, and the position of Linked Data within.

Right now, I argue that Open Data is trying too hard to cover everything in one breath. It's doing this because it very rapidly became a cultural symbol rather than an engineering symbol. With the agenda being led by "transparency" and its associated technical underpinnings (yes, the 5-stars of open data are a political quest, not a technical one), we tend to overlook the idea that Data - the other part - is not a standard but something which requires context.

Without context, data is just entropy, and entropy is already handled pretty well by our underlying electronic veins: the binary transistor.

In other words, is "Open Data" an oxymoron? If something is to build on its openness, it needs to flow, and if it is to flow from place to place, it needs to acquire meaning - at which point, is it no longer "Data"? Are we better off talking about "Open Information" - and if so, what does that mean for our tools?

So where does Linked Data and the plight of Kasabi fit into this? From the above, we can see that as "data" moves around the ecosystem, the people using it want different things from it, and will do so using different tools. This is a translation task - to adopt someone else's data, not only do you need to know how the data works in itself, you also need to know how it integrates with your own data. (This is also why Data Engagement is so important.)

The data developer's instinct is to build something generic and beautiful. This is further impounded by a commercial instinct that conforms to economies of both scale and scope.

But in reality data often resists the genericism that technical and economic efficiency loves. A generic data handling system would need to be so generic that it could do anything - at which point, is it any different to any database? Any computer? In her Open Data Cities Conference talk a few months ago, Emer Coleman said that "People are messy." And if people are messy, than people talking to other people is even messier - think Tower of Babel. In the real world, data is equally messier than we think it is.

One Chance Out Between Two Worlds

Not your usual armchair auditor [img]

This is why it's difficult to talk about Open Data and Linked Data in the same sentence, or sell all-encompassing data tools, or come up with "universal" standards that try to transcend contexts. The contexts differ, and translating data from one to another also means translating mindsets, working practices, learning processes, and organisational structures. Data Relativity is still being ignored.

Right now, Linked Data and the Semantic Web are in a funny position. They aim big, and are trying to solve an important problem about data quality. But this big aim means big technology and paradigm shifts - putting Linked Data much more into the realm of an "Enterprise" app in the same way that nuclear power and XML require a certain, quite hefty, amount of planning and structuring to achieve.

A lot of large organisations with "heavy", nuclear-style data have a lot of systems already in place, and a lot of knowledge and resources tied into these - in other words, they have a lot of momentum. This is where Linked Data could make a real difference I think, but the inertia that needs to be overcome is a fundamental issue. Not only that, but because the success of Linked Data is inherently tied to network externalities, that inertia is multiplied up through inter-organisational lines. To be good, many people need to adopt it at the same time.

On the other hand, the economic agenda of Open Data wants armchair auditors and clever freelance developers to innovate quickly for low cost. And the tools, knowledge and skills these kind of people have are geared up not toward "Enterprise-level" data, but toward quick-fire, loosely-knit, human-readable innovation. Find a small problem, and solve it quickly.

These two worlds are currently circling each other. All too often "Open Data" becomes confused with "Big Data" because there's a lot of it. Some of the challenge involves filtering out "Open Data" so much that it's invisible - background noise - leaving just the "Small Data", the useful stuff. Learning a 5-page API reference doesn't help this.

The other challenge is to make the link between "Big Data" and "Small Data" bi-directional. Or omni-directional. There's still massive amounts of work to do around microgeneration of data and feeding user-generated data into "the grid". There's still no real work around standards for data sharing, as far as I know, other than those that exist for content such as RSS, etc.

Right now though, the main challenge is just getting out of the Open Data mindset that data needs to be Big and Open in order to work. It's really important not to get too caught up in the lure of simplicity and easy wins for quick economic and political gain. Ecosystems are not built on top of economies of scale and scope.


Remember Babel.

Dissecting the still-breathing body of "Open Data"

Benjamin Welby has a great 5-part series on Open Data, summarising a lot of the current state in local government, and raising some good questions about how local government can lead on a lot of this. His first article picks up on my "Open Data must die!" post, and got me thinking about the term itself again.

Why is this an interesting thought exercise? Maybe the names don't matter, and function is everything. But at the same time, names crystallise our shared understanding of why we're doing something.

So why do we use the term "Open"?

Partly legacy - open source, open access, open-not-closed. All good stuff.

But in the context of governance, what role does the term "open" play? In fact, if we look at the two aspects of a) service delivery and b) democratic engagement, why is openness important? Answering this question, I believe, is key to moving open data forward - by clarifying new purposes other than openness in its own right. By setting an agenda other than "being open" we can start to look at being smarter and more productive, but within a context of accountability and shared decision-making.


Service Delivery

Let's take a break for a moment and consider the commercial world of "open data" - the Flickrs and the Googles and the like. A lot of the push for open government data came from a shift in the web paradigm from "data-as-a-website" to "data-as-an-API". Suddenly control of the data opened up - an essential inevitability as websites moved to a social model of importing data from users more and more. The website became a store, rather than a presence in itself.

Openness in this commercial world meant (or means) that developers were no longer restricted to the path to access this data that the website builders had pre-determined. You could suddenly write your own slideshow script based on your (or someone else's) Flickr photo stream, for example. Or print it out. Or turn it into text.

To return to government services, we have two types of user though, and two types of data.

First, we need to differentiate between data used by individuals and data used by policy-makers.

Second, we need to compare data about individuals with data about systems. By systems, I mean any organisation or structure that can be seen as a "whole" - such as transport systems, finance systems, demographic systems and geographic areas, etc.

This gives us a basic 2-way matrix on to which we can position "data" (which, after all, is ubiquitous):

Box 3 is greyed out because it doesn't count, imho. Ignore it?

The commercial data mentioned above falls into box 1 - a user accessing their own data. This is largely ignored (or discussed in fairly esoteric circles) under the governmental "Open Data" banner because it's hard. It's probably also the most useful, but maybe one to come back to. People tend to think of health data and benefit information, but one could also include library data (reading histories, current books, etc) and so forth in here.

Box 2 - systemic data used by individuals - is where a lot of people would like to position Open Data, whether it fits or not. The idea of the "individual" accessing data appeals to our consumerist lifestyle, like governments fulfilling their public-good role for the ultimate satisfaction of the private citizen.

(The "Armchair Auditor" idea falls into this precise trap of supposing that individuals will hold Government to account. There is nothing more romantic than a single person overcoming Universal Might.)

Some of this data is actually really, really useful. Transport data, opening times, prices, etc - this is the realm of information that we all use on a day-to-day basis. But that kind of data is "small" information - we consume something very specific in very short bursts, such as "next bus from stop X", or "who is my councillor?". These could almost be considered "facts" about the world.

Other data is also relevant to individuals, but from more of a research perspective. Being able to present data differently, such as with TheyWorkForYou, or aggregating data up to make travel-time-vs-house-price maps, starts to address the notion of how we can interact with the data in a user-experience kind of way. But to engage, users must not only interact with the data, but also with the democratic process (see box 4). The boundary between these two seems relatively unexplored so far.

Box 3 - policy-makers accessing data about individuals. I'm actually going to call this a void box, because I think at this point we transmogrify into anecdotal evidence, rather than data. Policy-makers love using stories about individuals, far more than comparing their stats in a Top-Trumps-esque kind of way. Comment if you feel differently.

Box 4 - systemic data used by policy-makers - really leads us into the idea of "democratic engagement", because it's where the overarching power lies. This is the data/evidence used to make decisions at a higher level. It needs to be aggregated to resolve complexity (and gain understandability), but be reliable and resilient enough for decisions to be made confidently.

This box includes data about government activities, such as spending data, as well as data about geographical areas, such as the IMD and National Indicators and demographics and so on.


Opening up Box 4

Box 4 is the most interesting, because it hasn't really been done before. Or rather, not in an open sense. Plenty of companies use "systemic" data - or "big data" if you like - to explore trends and make decisions. But all that data is valuable and therefore closed.

This is where and why Open Data hits challenges: the systemic data in this box is fundamentally tied to how administrations work and think. That is, it represents a worldview upon which certain types of decision can be made - accountable ones, justifiable ones, trackable ones. Data here, whether it's financial, organisational, demographic statistics or what-have-you, has gone through a long history of hammering out. To be usable, it must be comparable, and to be comparable it must be rigourously defined, in terms of what's collected and what it means.

It's why we have National Indicators and the ESD and consultations on standards within government.

And this leads to a bit of a paradox, which the Open Data world is currently trying to grapple with:

First, the data needs to be understood, because it's so well defined.

Second, the data - even after being understood - is only useful to its original context (the policy-making hierarchy), even after being understood.

Both of these together form an interlocking puzzle: If people don't understand the data, how can they use it? And if not's useful to them in their context, why would they want to understand it?

(Additionally, this paradox is semi-locked inside a Japanese puzzle box. Even if you get both, do you necessarily have the power to influence decisions?)


Democratic Engagement

This gives us effectively a 3-way combo lock on Open Data, which immediately answers our second purpose - that of democratic engagement. There is a philosophical argument for democracy, and indeed a philosophical argument for "openness". But I would say we have that already - people want to engage, even if out of a sense of ownership ("I pay my taxes and vote, therefore I should have a say.").

But people are also busy and/or lazy - there's no philosophical drive to see how government works any more than I have a philosophical drive to see how my computer works while I'm using it. Rather, I look at source code when I want to change it or write my own. The philosophy of "openness" rapidly transforms into a reality of "effect". And it's this sense of being able to "effect" that the idea Open Data comes down to.

Next time you see a dataset being opened up, ask ye 3 questions of it:

1. Can I understand it?
2. Is it useful to me in my life?
3. Who's listening for answers that might lie within it?

Too often "Open" data ignores all 3 of these questions. "Raw data now" ignores these questions. You cannot separate data from policy and expect engagement.

"Open" is a mindset, not a piece of content.

"Open" cannot be an end in its own right. 

Supporting a System of Spaces

I can only describe CityCampBrighton last weekend as an organised maelstrom of ideation. I don't know what that means, but it sounds great, kind of sums up just how much stuff was going on, and at the same avoids any real definition. Bingo.

So there are a few thoughts I'm hoping to blog shortly. But to begin with, why not talk about space?

Stuff Has To Happen Somewhere

There is a notion that says the Universe exists so that everything that happens can happen somewhere. In our virtual, post-Universe model of the world, we often assume that things can happen in a zero-size, flatworld-like dimension which doesn't really exist. From bricks-n-mortar to clicks-n-, um, whatever, the physical idea of "space" is something that has been done. Physicality is expensive, and all that.

But "virtual" or "real", spaces are where things happen. People gather, drink coffee, discuss ideas, make stuff, drink beer, and learn. People dance, have rubber band fights, listen to music, watch their babies, play board games, race cars. Everything happens somewhere but not everything happens everywhere and not everything can be done anywhere. Everything has its own idea of what space it needs.

Stuff happening in space. Source: Hubblesite.org

The CityCamp Space

CityCamp itself follows an open-space meeting style space, with its own rules - namely that a) the sessions are set at the start, and b) you can attend/leave whichever session you want, even halfway through. You are not allowed to be offended if someone walks out on you. (Although I never worked out if I was allowed, theoretically, to walk out of my own session.)

CityCampBrighton obviously discusses a larger "space" - Brighton and Hove - with an aim of setting out the existing "participants" (population) and "rules" (local regulation, etc.) and working out how new ideas can take these forward. In the process, the multitude of "local" spaces (shops, galleries, public facilities, tourist attractions, homes, etc.) come into play. All of these, I believe, are massively important to Things Being Done - to bring people together absolutely requires that idea of a space, set aside from everything else and with its own specific sense of purpose.

Reconstructing these kind of spaces is what geeks like doing. We come up with forums and MUDs and social media so that we can talk to people a) across the world, and/or b) when it's too wet to go out. So there has to be an intrinsic link between "real" space, and these simulations of space. They have to encourage the same kind of interaction and sense of environmental purpose that allows people to talk, and to support Things Happening somewhere.

CityCamp discussion in full swing. Source: Adam Oxford

In her 20 CityCamp Things, Catherine Howe picks this up at point #19:

19. I want to spend a lot more time looking at the intersections of offline and online networks and spaces.

And in his reflections on the day, Toby Blume (one of the judges) also makes the bridge between within-CityCamp space and without-CityCamp space, and the purpose of the former:

Using technology to develop social innovation has huge potential but I still feel it’s an invited space which non-techies are welcomed in to, to a greater or lesser extent. I can’t help feeling there’s more we can do to establish and support social spaces that put technology second to community action

There were so many good ideas at the weekend, but as Toby points out, not all of them were necessarily "right" for the funding prize on offer. Some were fresh ideas and needed more direction. Some were too large. Some might even be too small. In a way, there's perhaps a dilemma over whether CityCamp is there to award funding Dragon's Den style, or to foster networking and new ideas. (Personally, I think this dilemma adds to the creative tension. Don't change it.)

Grow the System

But the whole point of this piece is to highlight that a space isn't just a space - it's one space in an integrated system of spaces. The original purpose of open-space meetings and the Law of Two Feet is that ideas shouldn't be constrained or killed by the space (or conversation) they were born in.

A new idea often requires a new space. Sometimes that's 2 people standing around a different table. Sometimes it's a cafe instead of an office. Sometimes it's a mailing list or a wiki instead of a cafe. Sometimes it's a meeting involving dozens of stakeholders. The idea moves and proliferates from one space to another (or even to many others) depending on who's taking it forward, and what needs to be done about it.

An open space. With an obstacle. Source: Andrew Middleton

What can really kill an idea, then, is throttling this ability to jump from space to space. Either the space that someone would like to use isn't there, isn't quite right for the idea, or the person can't get to it. Taking each of these in turn, how can we support a system of spaces that means any idea has a better chance of survival?

1. Turn private spaces into available spaces. I almost put "make more spaces" here, but I'm not sure that's useful. We have enough space, but more often than not it's parcelled off as "private". The commercial mindset has shifted us to a world in which it's more acceptable to have adverts plastered over everything than it is to have public notices, or even art. This needs changing.

In the physical world, this can mean more negotiation to get more/cheaper/longer access agreements. Or it can mean taking space without permission (think flashmobs, or graffiti).

In the virtual world, gathering has tended towards "private" spaces - i.e. commercially-owned ones - more recently, with the onset of Twitter, Facebook, Google Docs, etc. If the rules bother you, then remember that there is always an option. Learn to code. Use open-source. Host your own service. People do this all the time.

2. Make spaces flexible. Sometimes a space is good, but not quite good enough. It needs tweaking. Physical spaces might need to be rearranged, or items might need to be brought in from outside (projectors, flipboards, power sockets, wifi, kettles). Virtual spaces are actually often harder to change - but should have enough options available to be adapted to the need.

In other words, don't be stingy with what people can do to a space. Too many seminar rooms still have tables all facing the front, and all bolted to the floor. You probably don't know - or shouldn't know, even - what people want to do - trying to predict this can limit the flexibility, or influence the purpose of the space. Be open to change. Ask people to put things back if you like. But don't just put the space before the idea.

3. Allow Ideas to get out, even if People can't (yet). OK, a slightly odd/different one this. But it's fundamentally important to realise the creation and value of an idea, even if nothing can be done about it right now. And to that end, any space that wants to be part of the system of spaces needs to allow/encourage a) somewhere for ideas to be "put on hold" - whether it's someone taking notes for themselves, post-it notes on a window, or an alternative to-do list - and b) a way to connect the people interested in that idea outside of the current space. For instance, attendee lists, Twitter lists, and feedback forms all allow sub-groups to re-connect, form and potentially move elsewhere.


That's 3 starters, anyway. The aim is really just to let ideas - and the people interested in them - to be able to do what they want, where they want to. It's to unlock all that potential currently caged up through middle-scale, short-term management. It's to turn missed opportunities into a framework that can nurture the next generation of ideas, rather than try to control them.

Data Engagement as a Healthy Ecosystem

Writing up the "Data Engagement (Everyday Data)" session from #ukgc12 is tricky, because it opened up a whole set of new questions and thoughts for me. I'm still trying to tie these thoughts together, and might break it down into a series of posts. This one starts out by looking at the many uses and users of data, and how this fits in with the Data Engagement Charter (see Tim's post) that others started to put together in the Saturday session.

Data as an Ecosystem

A year ago, I asked some questions around the relativity of data. It feels like I have some vague answers now, even if they're shifting about still.

The BBC are currently showing an awesome series about the history of plants on Earth, called "How to Grow a Planet" - but it's not just about plants, it's about how plants developed alongside rocks and Sunlight, and then how they developed alongside animals. It's a story of opportunity and of symbiotic relationships - in other words, of how co-dependence can lead to vitality and huge, thriving ecosystems.

The idea of data as an ecosystem isn't a new one. But it is one that's easy to forget, especially when we're staring at technical formats or CSV files, or when we're on a crusade for a much larger philosophy. However, understanding how different data users interact is key to a data itself thriving. During the session, I drew something like this:

At the time, I wanted to capture the important thing - the links from one group to another. In other words, data flows from user to user, from group to group, and gets transformed as it does so. This is the fundamental reason why data is good - not because it is easily automate-able, or easily packageable into binary, but because it can and does mean different things to different people. And yet we are also intrinsically linked through that flow of data at the same time. All different, yet all the same.

Each group has different needs and different backgrounds, and so each link in the diagram, at a data-level, will involve different...

• Access needs
• Numerical understanding
• Contextual, real-world understanding
• Quality needs
• Reliability needs
• etc.

A systemic/network view of data moving around means we can move from a "one size fits all" perspective to a narrower, more specific viewpoint. This means we can ask questions such as:

1. What is our role and our position in the system?
2. Who are our audiences (current and potential) and what do they want?

This gives us a less scary starting point than the idea that every data-handling party needs to cater to every other data-wanting party. Not everyone needs to do everything. We've come to grapple with this the last few years, with the shift from central authorities specifically providing data to an "end user" (if such a user exists) to providing "raw" data for a different audience. But the debate is still murky and needs to be made more explicit.

Data Engagement Charter - A Star Map?

Which is where a Charter can really help, I think. Although not just in terms of giving people an easy-to-follow guide, but also as a way to really map out the different uses and users of data. Si Whitehouse's blogpost on the subject has a 4-way use-case diagram positing 4 different types of people who want to find data:

Can we use this to start working out what the most important needs of each are, and more importantly, who could be filling those needs? And vice versa - if you have an existing audience who are struggling to engage with your data, what can you do to make it meet their needs?

I wanted to use this image to illustrate this post, for some reason:

At some level it represents an opening up of barriers, which can summarise the ongoing efforts to open up data. But at another level it's a great image because it mirrors both the links in the first image above, and the two dimensional axis sitting nonchalantly in the Use Cases diagram. It not-so-neatly sums up the way in which data flow changes according to what the data is being used for right now. Sometimes cars need to flow. Sometimes trains need to flow. With the right barriers and the right flexibility, we can have both. Or all, depending on how many users we're talking about.

So there's no "right" answer to how data should be opened up, any more than there's a "right" answer to which programming language I should use. There is what's most appropriate, perhaps, and certainly what's most fun. Linked data and SPARQL are good in some cases. Excel and Word reports are fine in other cases.

How to Have a Healthy Data Ecosystem

Having said that, it's not quite true. To keep an ecosystem alive means sticking to some rules, otherwise we end up with fragmentation, infestation, and/or the possible collapse of large parts of that ecosystem.

The 2 main components of a healthy system are, for me, openness and feedback.

Openness allows for flexibility and the ability to rapidly make new connections - vital when the environment changes (climate-wise, but also economically, politically, and technologically). Without openness, things can exist for a while, sure, but there is little resilience as time goes on.

Which standard gets used for data doesn't matter as much as whether we can interface with it in the future. We need to avoid data lock-in (or lock-out, depending on where you're standing) - whether that comes from legal, technical or economic aspects. There may be reasons for certain barriers (such as privacy), but these shouldn't translate into a general attitude of making data difficult to obtain. Legal barriers should not prevent technical access from being as easy as possible, for example.

Feedback is also vital, as it provides a way to negotiate the value of data without resorting to a "survival of the fittest" regime. Feedback in a data sense means that the fruits of Data Engagement benefit both parties involved - and that the link between parties gets stronger as a result of it being used.

In other words, we need to consistently and continually make sure that data is useful, otherwise the disadvantages of providing it and acquiring it will outweigh the benefits.

A Data Engagement Charter is an awesome first step to understanding and realising this. Such a Charter, alongside technical prowess and legal openness (another post), can be a very real call-to-arms for this recipe for an ecosystem. To make data successful, we have to understand that we are co-dependent on each other, and that it cannot be a one-way flow, even if we try to make it one.

"Open Data" Needs to Die

Amongst all the UK GovCamp 2012 buzz, point #18 from Tom Sprints' write-up caught me as being one of the more curious:

18. A lot of “open data” sessions just seemed to me to be variations on a theme, and didn’t sell themselves to me at all. I am therefore worried that some of those discussions are either very esoteric, or insufficiently informed by people who understand the issues rather than the tech.

Where have we come from?

As a data geek (I like the word "mechanic" myself), it's been intriguing to see the conversation around "open data" change over successive GovCamps. A few years back, the question was heartily "How can we get hold of data?" - Tim Berners-Lee was starting out on his comeback tour, and mySociety were beginning to show that data could be made useful with some clever tools.

As I remember it (likely in a fairly biased narrative way), the conversation then switched fairly rapidly into "What's the best way to open up data?" - in terms of what data and what platforms were most useful to developers. Suddenly data stores had (experimental) APIs, and the public realm had massive amounts of spending data. There was some loose rhetoric about transparency and accountability, while developers picked things apart with fine Excel toothcombs.

Then things got more interesting, as it turned out everything that had happened so far didn't automagically lead to Amazing Stuff Happening. The question became a necessary "So what?" - as if transparency and accountability weren't enough by themselves! The topic turned to users and reasons and (more often) to interesting examples. Surely, somebody was clamouring for this stuff after all this?

I'm kind of hoping this explains something about why "open data" sessions are a bit fumbly-jumbly now.

Open data got complicated, quickly. Because data is complicated. Jump to the present, and conversations rapidly flit between all of the above either because everybody is involved at the same time, or the people who should be involved, aren't.

"Open Data" is harmful

Or both. The paradox is that it's become difficult to talk about open data firstly because those who were talking about it from one point of view are now talking about it from many points of view. And secondly because those who weren't talking about it before aren't talking about it now. Data silos still exist. Most people still use Excel. Statisticians still output reports.

The term "open data" is meaningless now. Not just meaningless - actively harmful. If you're used to talking about it, then the conversation has begun to fragment and coalesce around more subtle outcrops. And if you're not used to talking about it, then you're put off because nobody can explain what it means - and more importantly, what it means to you. So you carry on as normal.

My session at GovCamp on Data Engagement was, in retrospect, an attempt to get back to the previous question of "So what?". What I really want to do is fence the conversation off from the technical, economic and political aspects of data (although I'm still into all these things) and focus on the why. I desperately tried not to use the term "open data" because I think it would have distracted the discussion. (To be honest, I wanted to find something better than "data engagement" too, hence the phrase "Everyday data".)

And I'm really glad that some of the idea got taken up on day 2 by Tim Davies and others. A "Charter" for engaging with data really starts to delve into how we think about how to make data useful.

I admit I'm a little afraid that the term "Open Data Engagement" just makes the discussion even more vague. What does that mean to you if you have no idea what it is, or what Open Data is supposed to be? Is it all at risk of becoming another buzzword? What about "Data Usability", or "Public Data Engagement"? I'm still aware just how much I hate the terms "Public Understanding of Science" and "Public Engagement with Science". Are we going round in circles?

Should we call a Stats Spade a Stats Spade?

Many people with useful, everyday data and databases really don't think in terms of data. Because the data is about stuff they know, they think of it as "information". Maybe even a "resource". But ask them what "data" they have and they'll probably give you a back-up of their website.

One of the interesting points coming out of the Data Engagement session was that people deal with data all the time - think football, Formula 1, house prices, etc. But do people even refer to this as "data"? Or - more likely - do they call them "stats"? Mention "stats" and people think of tables, averages, and counts.

In a way, "stats" makes sense where "data" doesn't. "Information" makes sense where "data" doesn't. "Data" is tricky because it's all of this and more. It's figures, it's formats, it's visualisations. No wonder even those who understand this get confused when talking to each other. The more you try to take "Data" into the real world, the less the term applies.

Should the "open data" moniker be scrapped instead of more "useful" terms like these? Would this make talking about implementing it more difficult, or easier? After all, any conversation on how to make data useful quickly turns away from talk of even databases and on to other issues (standards, protocols, best practice, comprehension).

Maybe if we talk about our bus times as "public information", and spending figures as "spending figures" then people will be interested in it, and we can stop trying to work out what "open" means.

Maybe.

UKGovCamp 2012 - 5x5 (plus one)

So Friday and Saturday were host to the indispensable UKGovCamp 2012 - a huge gathering of people interested in making public stuff better with technology, roughly speaking. I got along to the Friday day all about talking (rather than Saturday's doing), and gorged myself on thoughtmeat. Seriously, I was feeling dizzy by lunchtime.

Somehow I think I managed to carry on talking sensibly enough to feel useful. I also took notes and recordings of the sessions I was in, but here are 5 points from each that struck some kind of chord with me. They're a mix of things people said, and stuff I thought, but I think I can remember which was which.

I've put up audio from each session here.

Session 1: Data Viz + maps issues + challenges
- Vicky Sargent

This seemed to focus largely on data quality.

• Poor quality data can be exposed through openness.
• Different users/uses want different levels of quality/reliability. 
• Bringing together those who want reliability with those who want usability is hard.
• Getting useful "infoporn" is hard.
• Start by knowing what you're trying to achieve.

Session 2: Open Data as a Business Model
- John Sheridan

This went into how to sustainably fund both open data, and businesses based on it.

• It's not just a choice between "open"/public and "closed"/private.
• Perceptions of data reliability (including being up-to-date) are inherently linked to data management and its economics.
• i.e. Some people think you need a tightly controlled team / contract / business model to maintain data quality... Whereas others think openness is a viable form of reliability. (Cf. Wikipedia "vs" Encyclopedia Brittanica)
• Licensing offers multiple funding models depending on end-user, a la open source software. Chris Taggart doing a lot of this with openlylocal.com
• Does the data business model depend on the size/resources of the dataset/audience?

Session 3: LinkedGov tool to clean up & link data!
- Dan Paul Smith

This was a demonstration of the really impressive work being put together by @LinkedGov (http://www.linkedgov.org).

• Software that extends Google Refine to let you easily link data structures and tidy data.
• If letting people edit data ("cleaning", "linking", etc), you have to be careful not to introduce "new" data such as assumed defaults for new values. 
• Suddenly linked/semantic data is starting to look really powerful. I'm almost converted :-)
• The ability to "modularise" links to other external data lists has huge implications for data as a Distributed Ecosystem.
• Metadata for what's been edited needs to be accessible and clear, to understand who's done what over the lifetime of a dataset.

Session 4: Data Engagement / Everyday Data
-  me

This was an attempt to think about how to get data into something everyday and, not perceived as a "technical" thing. "Slides" and audio available.

• People love data if it's about something they love - e.g. football, F1, sales...
• Language used is massively important - often 2 groups will talk about the same thing, but in totally different ways.
• A range of "necessary" precision was brought up again - how can you transform the complexity of data into simplicity without misleading?
• Does data visualisation have to be 'comprehensively accurate', or can it just be enough to get people to ask more questions?
• Give data context and it's easier to turn data into feedback and so learn from it.

There were some great examples, and some amazing ideas coming out of this for me, which I'll blogpost properly soon.

Session 5: Network society engagement
- Catherine Howe

This wasn't to do with data at all... This was about the advantages offered by moving to a more networked, more engaging approach to decision-making.

• Catherine claims that the current system of engagement/consultation is actually a method to mitigate our own ongoing disappointment in political participation.
• If done better, political participation can be enlightening, rewarding, and fun.
• Bringing people together as part of the consultation process can mean they understand it/others more, and are less disappointed if they don't get what they want.
• Feedback as part of the consultation is vital to success. In effect, consultation moves towards conversation rather than just gathering views. (Does this that changing people's views is an objective of a networked approach? Does this raise questions around the accuracy of the final result, or does a more involved process and more post-process feedback negate this?)
• While I generally agree, I wasn't sure how much success from trials was down to using a networked approach, and how much was down to just using a different approach (i.e. novelty can often be fun in itself) - more consultation iterations needed?

I'm running out of time so won't go over Mike Bracken's speech or the Closing Note. Here are 5 random, general points instead:

• The data landscape is slowly coming together in my head. I know I know something important about it, but I don't know what it is yet. Like Cooper in Twin Peaks. You know, when he has that dream. I need to mull it over and chuck some rocks at a bottle.
• It feels weird not having twitter usernames on name tags.
• GovCampers are a bunch of (mostly beardless) ale-swillers. Much to the surprise of the pub.
• The engineering/development going into the new gov.uk single domain is seriously good.
• The medium T-shirts this year are definitely smaller than the medium T-shirts last year. Or did I get a ladies' one?

Pintless Debate

[In which the debate for/against the regulation of pub companies is ultimately broken down into the futility of arguments.]

The Parliamentary debate on the future of pub licenses has me hooked. Living in Brighton, it's difficult to describe, or even imagine, just what effect local pubs have on every day life - from evening entertainment, to decent food, to convenient meeting and organising places, to Damned Good Beer.

So it was great to see my MP Caroline Lucas weighing in with views from the Landlord of the Greys in Hanover - in fact, this was why I clicked through to the rest of the debate.

Two Pints, Please

In a nutshell, the debate is a classic "is market self-regulation enough?" argument. Most voices in this one argue that large pub companies ("pubco's") have too much power when it comes to setting a) rents for licensees, and b) rules and rates for "guest beers" and other things that help make pubs "interesting" (or affordable).

The motion moves for regulation to free up licensees from this "beer tie" and to review the self-regulatory nature pubs by an independent body.

But as you read through, it becomes clear that the debate is really about:

1. BIS' response to CAMRA's complaint appearing to be taken fairly word-for-word from a BBPA (British Beer & Pub Association) submission without much further input - recently discovered through an FOI request.

2. The Government's apparently "weak" action of apparently rubber-stamping the self-regulatory guidelines as what should constitute the statutory code. (See Adrian Bailey's comment.)

3. What seem to be otherwise fairly "liquid" but one-sided negotiations between tenants/licensees and the pubco's (see here for example).

Brian Binley makes a very interesting point about the unsustainable debt model used by pubco's basically being passed on to landlords - and hence on to consumers, who unsurprisingly either go to a cheaper local pub (if one exists) or the supermarket. Andrew Bridgen goes on to call it "almost feudal".

How [the] debate rages

But over time, the debate threatens to emerge from its pretence of being about the pub model, and into an attack on the political process that is driving it (or being driven by it). At this point, the debate breaks down into 3 types of discourse:

1. Anecdotal/qualitative rhetoric: Stories from constituents, traders, etc. I suggest that the Select Committees' evidence also falls under this as they adopt an "interview" style approach. (Also, here's a good SC report from 2009 on the matter.)

2. Statistical evidence for/against intervention: Ed Davey seems to use stats more than others, for example.

3. Attacks against process and character: With the nature of the BIS response and its apparent "close ties" to the BBPA being thrown open by the FoI request above, this is a third line of argument which seeks to undermine both of the above, on matters of personal principle.

There are also appeals to "external" authority. The OFT, for instance, seem keen not to be involved, which leads some to say they're not relevant, but others to say this merely means regulation has no place in an apparently successful market.

Welcome to politics. What's interesting is how - or if - each of these types of argument "trump" each other. In other words, should we give pubs more choice over beer because 

a) a lot of people say it's a problem?

b) data suggests there is a link between lack of freedom, and pubs closing?

c) the people behind the non-choice have too much economic and political power?

In my mind, this is a bit of a paper-scissors-stone situation. Can any of these really be more important than the others, or do they just lead to a cycle of disagreement? How much do each of these - or all of them combined - duly influence any voting on the matter? And should I really have bought that four-pack of Speckled Hen from Sainsbury's today?

Exit, Stage Left

I also liked the general response to Ed Davey's comment which reads a little like the script for a bad school comedy play:

Brian Binley: Will the hon. Gentleman give way?
Edward Davey: No, I want to make some progress. 
[Hon. Members: “Oh!”]

Also:

Martin Horwood: Will my hon. Friend give way?
Edward Davey: No, but I will in a second.
Brian Binley: Will the Minister give way now?