COVEREDINBEES / A hive of TLDR & villainy https://coveredinbees.org/ A hive of TLDR and villainy quarto-1.8.26 Thu, 05 Mar 2026 00:00:00 GMT How to avoid chasing vampires off cliffs + testing modular research https://coveredinbees.org/posts/chasing_vampires_off_cliffs_plus_testing_modular_research/

I’ll come to the vampire thing in a minute.

This post smushes two things together.

First: I’m testing a prototype writing/output structure that (1) is built to be modular, iterative and as me-friendly as possible, so work cycles are more fluid, but can build to larger outputs; (2) is reproducible/open with each stage permanently stored with its own digital object identifier (DOI via Zenodo); (3) doesn’t silo itself in academia / is open to (and acknowledges) all contributors wherever they are, and however small/large the contibution (including e.g. a useful LinkedIn comment); (4) with tools built in to make sure LLM use in writing and code is clearly delineated just as an academic would with any other source, including automatic inclusion of LLM back and forths.

Here’s a first go. What this looks like:

  • A self-contained project folder with all the code, data and write-ups (this is very rrtools-like) and a landing page that auto-populates from the docs folder (github action here).
  • A docs folder containing discrete ‘chunks’ of work. Here’s the first chunk - more on that below. When each of these is reasonably complete, I can mark it as a release with a DOI number (this 1st chunk is v0.1.1). Each release is archived, and shorter-chunk work has some provenance that can be referred back to.
  • The docs folder can contain anything else for public consumption like data viz, as it does here.
  • There’s also room in there to produce punchier, shorter summary docs targeted at different audiences but building on the same material e.g. slides for policymakers.
  • README and FEEDBACK markdown scripts explaining the project scope and roadmap, and how to comment / feed back / discuss, including instructions for creating a github account, though that’s not necessary. (See here for Claude-Code drafted sources on ways to do better than the author CRediT system / build on github tools.)
  • I’ve also tried to build in tools to make LLM use clearly separable from my own work - as I would with any other work that isn’t mine. See here for an attempt at draft LLM guidelines. Short version: un-cited LLM use isn’t different from plagiarism (though there are arguments to be had - see below again). The project has folders both for LLM-produced summaries/memory docs (marked as non-human-produced) and another for all LLM prompt back-and-forths used in project development (output as human-readable markdown via the export_all_convos.py script; currently only for Claude Code, and I’m working in Visual Studio Code). The first chunk output makes very clear where LLMs were used.

In this first project - RegEconWorks - I’m aiming to gather my work on all things regional-GVA, synthesise thoughts on it, and cycle through any feedback I can pester out of people, in whatever form.

Second…

The actual output

This first chunk is asking:

What difference could adding uncertainty to regional GVA numbers make?

It’s common to acknowledge that ‘spurious accuracy’ is an issue. But lacking any decent uncertainty guesses, we can’t think through the implications very easily. So this makes a data-driven guess at what the uncertainty could be and how that changes GVA at regional level. The ONS rules out error rates for GVA as ‘too complex’ - which it is, within a national accounts framework. But we’re doing some “what ifs” here because I think it’s worth thinking through, with some actual numbers, how much difference it could make.

As it says:

Uncertainty is powerful because it short-circuits many unhelpful ways of thinking about growth and industrial strategy, including spurious rank-building. Error bars direct us back towards our regions, what we concretely know about them, and how that knowledge is built into the choices and structures we make.

Right, the vampires. The chunk starts with a scene from the Lost Boys, mapped to type I and II errors, where the protagonist finds himself racing bloodsuckers through fog towards what transpires to be (spoilers) a deadly cliff edge. This hammers the point home a bit bluntly, but… is head vamp David’s ‘incredible certitude’ (Manski) like using ‘exact’ numbers to steer our economies by?

My point: introducing uncertainty doesn’t mean making things murkier. Quite the reverse - by understanding what’s fog and what’s not, we can extract signal from the mist. That doesn’t just mean ‘better decisions’ - it can change how we think and how we steer. E.g. what does it do to the kind of horse-race analysis that ‘exact’ numbers make so beguiling? (See Prof. Richard Harris’ excellent old piece on what uncertainty in university rankings would mean, for comparison.)

The chunk is written somewhere between academic and informal blogpost style. I have deliberately included “think is what I think and why”, and I’ve ended with some open questions to chew over for the next piece, which I hope will examine the following question:

“If we accept this uncertainty into our regional growth data, what are the decision-making implications?”

So…

Why modular?

Because I want to do work that:

  • Is nimble enough to build up smaller chunks of work sequentially and change direction in the light of new input, can also have a permanent record of progress and contributions, and a DOI number so versions are never lost;
  • Doesn’t get stuck in a silo, allowing me to talk to anyone across academia, policy, community, wherever;
  • Doesn’t force things into the slow, awful academic paper cycle of death, but instead lets me build from smaller chunks to larger pieces - some way earlier than the ‘preprint’ stage, which still has to be a completed paper;
  • Is open to input from / sharing to anywhere, and acknowledgement for anyone.

That makes it sound more planned than it was. This has evolved. Some background:

It’s been about 2.5 years since I started my secondment to the South Yorkshire Mayoral Authority (SYMCA) through Y-PERN. My elevator pitch has always been: we want to strengthen the glue between the region’s universities and other anchor institutions, as well as Yorkshire and the Humber’s communities. Some of this has been easy, gone with the grain. For other things - including research/report outputs - the cogs tend to grind more. Almost every single institutional structure and incentive differs between universities and regional government, which has meant progress is often made despite how we work, not because of it, built on relationships between people who care about the same things.

At the same time, change is happening stupidly fast. We’re firmly in a ‘no-one knows where this will land’ phase, I think. Universities are in crisis as old funding models fold, the government not showing much sign of comprehending the scale of that. Throw in this new LLM world where humans have just lost their monopoly on the written word. Just in the past couple of months, systems like Claude Code (more on that below) are realising the promise/threat of LLMs to upend how knowledge works (see Naomi Alderman’s Don’t Burn Anyone at the Stake Today for an amazing long view of this).

That’s all been sloshing around in the back of my skull as I’ve mulled how to digest my work with SYMCA and Y-PERN so far. (I’ve been keeping a list of open code and outputs here, and stuck much of the work including how-tos here.)

And personally, I’ve been trying to design a me/task/environment combo that actually goes with my own brain-grain. Traditional academic papers don’t do that so much. That’s true for many people, but we continue to bash ourselves against it, even while the whole system is making ominous creaking, groaning noises.

Oh no, LLMs

Yeah, afraid so. As I’ve mentioned before, I’m aware most of us are sick to death of People’s Opinions About LLMs, and the fact that 90% of social media posts now mention them.

The last two months have been a turning point for me. Claude Code has shifted this technology from ‘fascinating but deeply flawed’ to ‘my job and probably entire sectors just changed forever’. Pretending this isn’t happening probably won’t help, though I get the temptation. So we need some rules for ourselves as we try to navigate.

(I’ve hived off a whole ramble about LLM ethics, what it might to do to my sector and just how terrifyingly powerful Claude Code is, we’ll come back to that.)

So I’ve put ‘acknowlegement of LLM use’ fully into the project structure, and you can see LLM use acknowledgements listed in the first chunk here.

This piece has my first attempt at some principles I’m using (linking to some LLM-gathered sources on the subject) to make sure mine and anyone else’s work is clearly separable from robot output. As mentioned, the project folder also has a (Claude-Code-written) python tool that converts LLM back-and-forths taking place in the project folder locally into human-readable markdown automatically, so there’s a full trace of how work was produced.

The fundamental principle seems simple enough - it should be super-clear which words and code I did and didn’t make. This is just basic academic integrity, yes?

That piece on experimental principles acknowledges there are blurry and difficult cases, and that it pushes against the LLM marketing - which is “full of suggestions that co-pilot should quietly slip into your workflow and write all your words for you, whether email, teams comments, slide prep.”

There may be work situations where that’s OK. But for this kind of project, where it’s imperative everyone gets a correct nod for their input, it seemed like a good chance to test how we can try to (a) use these tools for our benefit while (b) making it super-visible who’s producing what.

This connects back to the ‘different incentives’ point above. The reason academic integrity rules exist, the reason plagiarism (including self-plagiarism) is a thing is precisely because credit is the lifeblood of academia. Credit should of course always go to the right people, university or not. But academia is particularly vulnerable, since credit equals career more explicitly than any other sector.

In that environment, the urge to use LLMs to appear more productive must be difficult to resist for many. But the fundamental point - you shouldn’t pass off anyone else’s work as your own - didn’t go away. LLMs can, in fact, make us more productive - that’s now very true for coding. But it needs to happen openly and ethically. Otherwise it’s no different to plagiarism.

I’m not saying I’ve got that right here, but I am trying to build it in. Feedback on this, and anything else, welcome.

…

OK, let’s see how this experiment goes. Now to harangue some people for views on the first chunk, and discover what dreadful mistakes I’ve made.

]]> regecon writingexperiment https://coveredinbees.org/posts/chasing_vampires_off_cliffs_plus_testing_modular_research/ Thu, 05 Mar 2026 00:00:00 GMT Why make things simple https://coveredinbees.org/posts/why-make-things-simple/ Douglas Adams said:

“I am rarely happier than when spending an entire day programming my computer to perform automatically a task that would otherwise take me a good ten seconds to do by hand.”

This is exactly the kind of productivity thinking the country needs, I think. I’ve been trying to follow his example recently. I am trying to disassemble how I do things / why I’m doing them / whether I should I just sit in my pants all day eating pringles / what sort of task-environment-brain re-jigs might help answer any of those questions. This is leading to many “all day programming for a ten second task” moments.

It’s all part of joining in the currently very fashionable and exciting pastime of “wondering what kind of brain I have and whether it really needs to have been so bloody difficult all this time.” (Analogy: as a left-hander, if I’d always been made to use my right hand, as was the case pre 1950ish.) Coming at it like a design problem, prototyping things that may end up on the scrapheap, is the only way.

So rather than get on with, say, actually working on the sci-fi book or writing about anything concrete, I’ve done the following:

  • Made this blog using Quarto in R and linked it to buttondown for subscribers. Autonomy! Open source! No readers!
  • Read a bunch about distributed knowledge systems and brain diversity, leading to –>
  • Moving to Obsidian for all note-taking / thinking stuff (it’s a delight to use, if you’re already a bit markdown-leaning, and again it lets you have transparent control over all your stuff; it can also do lovely clever things like collate tasks from all sources).
  • Using Obsidian’s Digital Garden plugin to host exobrain.coveredinbees.org via Vercel autopushing to Github, setting a YAML property to dg-publish for any md file I want live (what a super, pretentious two-decades-old-type word ‘exobrain’ is: see the ngrams, pic below also - it actually peaked in 2010, though its first ngrams appearance is in 1970, Arthur B. Bronwell’s ‘Science and Technology in the World of the Future’: “to carry out man’s informational processing demands in much the same way that an exoskeleton carries out his powerful tasks.”)
  • Then there’s a nice little obsidian-to-quarto plugin. Err why? Because just blogging in Quarto, I’d have to manually add exobrain URLs myself, which would take me a good ten seconds by hand, so instead I (and chatGPT) made some R functions that check for Obsidian exports and, if present, convert all Obsidian internal links to exobrain URLs, as well as reformatting the YAML so the tags work correctly. So for example I can use Obsidian’s own internal link system (which has very smooth autofind) to connect to exobrain bits (e.g. a list of starter questions connecting magic, the money system and ‘the game’).

So now the process of writing a blog post couldn’t be simpler. I just (1) write it in Obsidian as I’m doing now; (2) use the Obsidian-to-Quarto export function; (3) in RStudio, run the R function to convert exobrain links and YAML and move the files; (4) when in a state I’m happy to publish, move it manually to the posts folder (that could be streamlined); (5) run ‘render’ on that final post in RStudio so the HTML goes to the docs folder; (6) git stage/commit and push in Gitraken to get it live on coveredinbees. Done! Just like that. Substack schmubschmack.

Sigh. I am reminded a little of the Mars Curiosity Rover’s landing plan (in one of my all time favourite youtube videos, recorded before they knew whether it would succeed). Aim for a flat spot, use Mars’ atmosphere to slow down a little (it’s 1% the density of Earth’s), use a huge parachute, release a cap to view the ground, drop the rover carried by its own rocket stage because none of that can slow it down enough, but you can’t use rockets to land because the dust will bugger the rover’s systems - so hover above the ground and lower it on four wires, and then boost the rocket stage off to the side so it doesn’t crash on top of the now-on-Mars Curiosity. Oh and do all that entirely automatically because signals to Mars were a 14 minute round trip at the time. Done! Just like that. Elon Schmelon.

Here’s Jazz-student turned NASA entry/descent/landing lead Adam Steltzner:

“When people look at it, it looks crazy. That’s a very natural thing. Sometimes when we look at it, it looks crazy. It is the result of reasoned engineering thought. But it still looks crazy.”

Reasoned engineering thought, though, takes a lot of testing. The problem with testing human brains, or at least mine, is the tendency to default to “you’re just a useless idiot” and stop at that, rather than taking little steps, inching forward, playing, trying new things, seeing what goes with the grain, what rubs against it. It needs resetting daily as the self-blame module can be deeply wired in - previous perceived failures obviously a personal failure, says the module, rather than allowing consideration of the whole you/task/environment setup1.

So while I’m obviously getting those sweet sweet Douglas Adams Dopamine Hits from all this ridiculous tinkering, it’s… not entirely ridiculous. None of this is novel - here’s a nice writeup of the general digital garden idea, it’s in that ballpark. The world is littered with failed ‘Systems That Will Solve Everything’, but maybe there’s merit in this kind of slow, tentative gardening.

All the ‘digital garden’ tech is allowing is easy linkage - it’s been described as ‘your own personal wikipedia’. But replicating existing links would seem pointless, no? So the test here is: can it be used to add little modular chunks, questions, connections? In a way that isn’t just building an ideas oubliette? What, if anything, can it really add?

I’m finding that when you start playing with the task/environment/self parts, they all start affecting each other in unexpected ways. One of the most intriguing: what it’s doing to what I think words are actually for. That’s led to a List of writing questions (see, nice exobrain link; sorry, some LLM assistance in that list). There’s a long section on different functions of writing where “narrow professional words” are in the minority. There’s also writing for one’s own brain/ psychology/emotions; connection; experimentation; identity building…

That rather solipsistic view segues into bigger veins and arteries we all share. I tried “Writing” about that - it’s got some of the ideas there, but… too much. Let’s see if it’s possible to pick away at it more slowly. It feels like a very specific historical moment for language, when humans have lost their monopoly on the written word (people pointing that out here, here, here), when the collective lighthouses we used to steer away from the Deathly Rocks of Bullshit are under well-funded, sustained attack.

That’s enough to make it feel like finding your own words is pointless. Working theory: it isn’t, it’s more important than ever. So let’s add that one to the list for later:

Footnotes

  1. Hat tip to my partner Helen for bringing her OT background to bear.↩︎

]]> brainsystems https://coveredinbees.org/posts/why-make-things-simple/ Wed, 10 Dec 2025 00:00:00 GMT I wrote a sci-fi book about AI before chatGPT came out. Now what? https://coveredinbees.org/posts/i_wrote_a_scifi_book/

I wrote a sci-fi book about artificial intelligence. I started the research for it in 2018, from an idea I’d had in 2012. A first draft was done by mid 2021. After no luck pitching to agents, I lovingly disassembled that draft and put it back together by the following summer in 2022, then churned through another round of agent pitches (also nope).

A few months later, at the end of 2022, openAI released chatGPT 3.5 for public use. Three years have since passed. That Oatmeal panel above (from this piece) is how a lot of people feel now about AI - there’s a decent chance you’d be willing to pay to make everyone please shut up about it, please stop inserting it into our lives whether we ask for it or no.

Amid this ceaseless cacophony, I am planning to get the book out into the world. Great timing. I’ve set myself this aim:

Self-publish it, after one more churn cycle, and chart the process here on coveredinbees. I’m guessing it’ll take six to nine months. We’ll see how optimistic that is1. I may possibly attempt an Indiegogo launch campaign in the final weeks.

I’ll be re-assessing the book scene by scene in this final loop, writing about the ideas and the construction of it. It has to remain a product of its pre-chatGPT time. It’s a bad idea to let current events blow one’s ideas around like a weathervane. But our brave new LLM world (bubble pop or not) does offer a chance to reflect. How much has it (a) changed how I think about the book and (b) changed how I think about AI and its impact on us?

I’ll be journaling progress as I go in this planning/work doc (partly to be open, partly a post-LLM urge to demonstrate the flailing human frightfulness of the process).

Anyone who knows me knows I struggle to see the value of my own work. The book has plenty of flaws, but I’m super-proud of it. This never happens. I feel baffingly, DunningKrugerly confident in it. I’ve worked hard to mix ideas with pace, to write something fun. Your definition of fun may differ, but I think a certain kind of sci-fi fan (i.e. people like me) will dig it. I’m going to roll with that feeling before my brain notices.

So stick your email in the subscribe box if…

  • You’re interested in the nuts and bolts of fiction writing and self-publishing (a pretty convoluted process) and want to follow along as I get this done. Maybe you’re especially into sci-fi.
  • You’re into ideas about how technology and society thread through each other. You may be trying to get under the skin of what’s actually happening with AI. You might also find yourself drawn to the questions it raises about minds and intelligence, including the collective kind (despite exasperated protests from several / AI / researchers that things like AGI are a dumb distraction).

(I’ll mark sci-fi related posts, in case you want to avoid other stuff I write about.)

In my day-job, I’m now working on analysis of AI workplace impacts. Given that, writing about sci-fi feels a bit like running through town wearing nothing but a fez and a feather boa: equally mortifying and liberating, potentially illegal. But it’s how I was schooled - in the last year of my Sheffield politics degree, Mike Kenny paired social theory with sci-fi every week2. I was introduced there to what’s arguably still my favourite book, Le Guin’s Dispossessed. (We didn’t have to wear just a fez and feather boa in seminars though.) Causal arrows between our imaginations and reality go in all directions (though that has issues - see below).

What’s the book about?

People and technology co-evolve. The path they forge together is messy and unpredictable. Sci-fi slips to the edges of this a lot: AI heaven, liberating us from all Earthly drudgery; or a glowing-red-eyed hell where the best we can hope for is that AI wants us as pets. I wanted to dig into the mess more.

It’s a “two strands turn into one” book. Strand #1 explores just how intractably difficult it would be a re-create something as complex as a human mind. It’s the opposite of Matrix-style “stick a cable in this hole and you’re good to go” and other “brain upload = just pop this cap on / lie on this table” takes that make it look as clean as scanning an item at the checkout. (Recent example: Alien Earth.) Those do have an advantage - just do the upload and get on with the plot. I made a story out of the process itself.

Now, I’m very very not a neuroscientist. But this is fiction, so here’s what I did. If brain knowledge is a vast, carefully curated palace full of the most subtle, inscrutable paintings (see e.g. Damasio’s amazing Self Comes to Mind, a book I leaned on a lot), then I broke into that palace at 3am with a swagbag, cut as many pictures out of frames with a Stanley knife as I could and scarpered before the cops arrived.

With that contraband in hand, strand #1 is about what brain upload looks like if we take the complexity of the human meatsack remotely seriously. It naturally fits a wake up in a room3 setting - arguably a steep challenge for a first time fiction author to make that compelling, but I gave it a go. The dreaded C word4 does arise, but in a slightly novel way I hope.

Strand #2: a ripping yarn built on copyright and intellectual property law. Wild. No, stay with me. The idea is that AI will radically alter our relationship with language. We’re seeing an argument for that erupting into the world right now with LLMs, firmly in “whoa what an unpredictable mess” territory.

Again, sci-fi’s natural grain leans to the extremes, authoritarian in this case - e.g. Ma Boyong’s “City of Silence” short story in Invisible Planets or more recently Where the Axe is Buried; amazing takes on the “AI + 1984 style control and destruction of language = what?” question.

But if we start from where we are now, with the mire of politics, law and money we swim in, where might the evolving mess take us? Strand #2 is a thought experiment about one of those possible destinations. There’s plenty enough darkness in the result, but I’ve tried to get other shades in there.

There also ended up being a fair dose of Evil Robot. Turns out it’s really difficult to avoid.

Self-publishing?

Yep. And not just because all those failed agent pitches mean it’s the only option, no sirree. This is in no way like a jilted lover wailing, “I never wanted you anyway.” Here’s how I comfort myself: self-publishing does actually have some advantages. (“I don’t need you, my life’s better now!”) The book may have its niche - TBC - but it was probably never a very commercial proportion5. While I’ve worked very hard to make the plot bounce along, I have also given all the ideas-y stuff pride of place. That’s why I wrote it - I wanted that challenge of trying to make ideas and plot work together. Kill your darlings schmill your schmarlings. I love my darlings and want them to live. There’s also a level of control - including what I hope to do here on the blog in the coming months - that would be difficult otherwise. Control, yes. Fame, wealth, accolades, less of that. So it goes.

What next?

Stick around for sci-fi and other ramblings. As well as general progress reports, here’s some things I might try and write about.

  • Forecasting versus thought experiments in science fiction. Bouncing off Le Guin’s brilliant short intro to the Left Hand of Darkness. As I mentioned, part of my day job is attempting some predictive scenarios of AI’s impact on work (there’s a little over 30 other reports on this subject I’ve found, most out in the last two years - everyone’s at it). Le Guin makes a forceful case that sci-fi shouldn’t be splashing about in those waters: “Prediction is the business of prophets, clairvoyants, and futurologists. It is not the business of novelists. A novelist’s business is lying.” But the genre’s role in shaping how we conceive our present and future (for better or worse) is an interesting tangle.
  • Creativity. There’s a lot of insistence that AI can’t be creative like wot a human can. “Like a human” is probably true, but it’s harder to dismiss AI creativity entirely. There are overlaps between how AI and humans do it. A common line: “AI can’t go beyond its training like we can.” Really? I’ve written a similar-ballpark short story that explores this. The issue isn’t the creative divide between meatsack and machine, it’s how the structures we inhabit use and abuse creativity. I should read something by the late Prof. Margaret Boden before I write anything about this.
  • Intelligence. AI engenders some odd views about it. The singularity, for example - the notion that once AI surpasses humans, it’ll start an exponential feedback that’ll make us to AI as ants are to humans etc etc blah. Uh huh. But we already have systems more intelligent than a human - they’re called ‘collections of humans’ and their organisations. Who do you think built London or the 747? Why didn’t they start exponential feedbacks? Positive feedbacks, yes, but they seemed to stabilise - why? What, if anything, does that say about intelligence? That brings up some curious viewpoints - people who think, yes, we’ve made superintelligences like the market system, but only in the way that ants (them again) build complex nests, and we mustn’t presume to interfere etc etc blah. Uh huh. So yeah, fun to be had here.
  • Consciousness. Why there’s (almost) no point talking about it. We can mark out a space around it - what’s left after we’ve cut away what can rationally be discussed. But the thing itself? Consciousness is that kid in the Mickey Mouse hat in the Larson cartoon below. Don’t be one of the snakes. (Actually, that might be all there is to say on this subject. I will probably still go on about it.)

Footnotes

  1. I’ve also committed to not drinking alcohol until the book is out. I really want a beer, this is very motivating. I hope it’s not three or more years.↩︎

  2. Daily Mail headline: “Woke university teaching science fiction! This is why we lost the Empire!”↩︎

  3. Two TV Tropes links there, you may notice. Yep. I’ve read a lot of stuff on story structure - plenty of people want to sell you the one framework for how stories work - but out of everything I’ve looked at, TV Tropes is still top. It doesn’t mess around, and is way more up to date on how classic tropes have converted themselves into modern media. And the writing is comic genius.↩︎

  4. I met with a few friends in a little meaning-and-nature-of-consciousness discussion group for a while. You end up chasing yourself in circles very quickly indeed. I’ll come back to this, though. I have Things To Say. (See the list of possible topics too.)↩︎

  5. You learn how indispensable that is as you dig into traditional publishing - agents have to convince publishers with projected sales numbers alone, usually with “x is similar to y and z, and those two are selling well right now”.↩︎

]]> fiction scifi https://coveredinbees.org/posts/i_wrote_a_scifi_book/ Tue, 21 Oct 2025 23:00:00 GMT Prototyping open econ tools https://coveredinbees.org/posts/prototyping_open_econ_tools/ Over the past couple of years, I’ve been building up a loose ‘regional economics’ toolkit while digging into UK sectors, growth and other econ-adjacent data (working mainly with SYMCA wearing my Y-PERN hat)1.

I’m gathering data/code/explanations onto a regecontools website, though bits are spread all over. This live outputs page on the techie blog has gathered most of it in one place. (Fun2 example interactive: South Yorkshire’s job count from 2023 BRES data; click to drill down through local authority and sector granularity.)

Over the next six months or so, I want to make this a more focused project. The plan is roughly this:

  • Using open data/methods, test and improve these tools (and prototype new ones) so they’re as useful as they can be
  • Dig deeper into the ideas and questions we’re pursuing - iterate between these and the tools.
  • Build on existing relationships and search out new ones to refine the tools/ideas/questions cycle.
  • Help build capacity through things like this ONS Local session I ran introducing R for regional economic analysis (video here; the online slides here work as a how-to guide, including how to use R easily in a browser).

The least tested part of this for me, and maybe the most interesting, is the ‘tools/ideas/questions’ cycle. The tools/concepts we use to make sense of our regional economies have grown from a hodgepodge of ideas. These can be questioned at different levels - from narrowly methodological (“Do existing measures of public sector output capture their real value? Have we understood what impact boundary choice has?”) through to deeper, more political ideas about what economies are and what/who they’re for (see e.g. the latest doughnut economics Nature paper or this survey of over 200 eco-econs/wellbeing indicators).

Y-PERN’s whole purpose has been to experiment with breaking down barriers in this cycle - trying to push past deficit model thinking that just sees that cycle as a “transfer of information from experts to non-experts”. This is still a pretty entrenched view: academics toil in the dark for years, emerge pale and blinking, clutching new insights. A separate ‘knowledge exchange’ process then commences.

But that doesn’t align with the reality of data-driven policymaking: everyone involved are experts in what they do. It’s a deeply collaborative endeavour. We don’t all share the same knowledge, but we also won’t progress much without bringing everyone’s together.

The ‘open’ part of the toolkit is in symbiosis with the ‘ideas/questions’ part. Regions face many of the same economic questions, including identical asks from central government (e.g. to produce Local Growth Plans). The same data sources are re-used, the same assumptions. At the same time, we’re all working to develop our own identities as devolution deepens. Given that, here’s the philosophy I want to work with:

Do data work openly where we can. It will support collaboration and learning. It will help build a shared sense of ground truth. It will avoid wheel reinvention.

I don’t have a dogmatic fealty to open source — we have a data/analysis mixed economy where private providers can do things other can’t, and that’s a good thing. There are also essential datasets that can’t be open, for good reason. But where we can, working in the open just goes better with the grain of those three things: (1) collaborative learning/capacity building across silos; (2) developing our shared sense of ground truth; (3) avoiding wheel reinvention (a facet of reproducibility, which needs open methods; you can’t reproduce something if you can’t access it).

The only way this works is if we prototype. Or call that test and learn if you prefer. As that doc says:

At its simplest [it] involves (1) developing something; (2) making contact with reality; (3) learning from the results.

Sounds great in principle - but we have to be open to our preconceptions being challenged when we smack into reality. The Valve Games employee manual puts it like this3:

“—–ing up is a great way to find out that your assumptions were wrong or that your model of the world was a little bit off. As long as you update your model and move forward with a better picture, you’re doing it right. Look for ways to test your beliefs. Never be afraid to run an experiment or to collect more data.”

While not always comfortable, this is part of how to grow our shared sense of ground truth by working together. It’s perfect ‘blind people and elephant’ parable territory4. Brace for an extremely mixed elephant/economy metaphor.

  • We want to see as much of the elephant as we can. How is the economy structured, what’s growing, shrinking, changing? Why? (And a whole bag of less narrow questions, of course.)
  • Those lead naturally to “What data/methods do we have to answer these questions?” Any one data source helps us grab a single pachyderm part. Multiple sources give us a grip on different bits, but the whole beast still eludes us. We need enough knowledge to know our tools’ limitations - we will regularly hit the streetlight effect, mistaking the data for the whole animal5.

Combining openness and prototyping, though, we can exchange what we think we’ve seen with others, maybe learn from our errors, and move a step or two closer to a collective sense of shape.

Then there are awkward people saying things like, “Are we sure this is an elephant? I think maybe we have to go back to first principles here.” But again - an open, testing approach lets us argue our competing pachyderm theories out, and often discover diverse tools can be used in different places very effectively despite no total agreement.

That can hit up against very different incentives and cadences across academia and policymaking (as SYMCA’s Alice Rubbra lays out here really clearly) - but those are other elements to test as we go. What mix of relationships and structures can we grow to get closer to truly learning, adaptive regions?

So, what next? I’ll need to keep things fairly narrow to achieve anything at all, most likely by testing some ONS and Companies-House-extracted economy measures, see what can be improved, what blind spots there are. I’ll also try to find two or three focused problems to apply this to - including, I’m hoping, something on how we close the gap between quant views of sector mix and ground level knowledge of how firms interact, and what difference that makes to their success.

Underlying-idea-wise, I’d like to see how older theories of technical changes in production can help shed light on current AI fears (I’m also working on an AI impact project). That could help think through where we want economic progress to come from.

Let’s see how contact with reality changes things. Stick your email in the subscribe box to follow along…

Footnotes

  1. Github commit of this doc with cuttings at the bottom.↩︎

  2. If you find this kind of thing entertaining…↩︎

  3. Not an endorsement of Valve’s supposedly desk-on-wheels culture or anything similar. As this article explores, structures like this (including some co-ops without explicit rules) can often just obscure hidden hierarchies, making them harder to navigate or challenge.↩︎

  4. Elephant parable example applied to economics.↩︎

  5. Krugman: “We just don’t see what we can’t formalise”. [citation forgotten!]↩︎

]]> econ https://coveredinbees.org/posts/prototyping_open_econ_tools/ Mon, 06 Oct 2025 23:00:00 GMT