Category: Research

Friday, 7.03pm

Sheffield, U.K.

I personally think there’s going to be a greater demand in 10 years for liberal arts majors than there were for programming majors and maybe even engineering, because when the data is all being spit out for you, options are being spit out for you, you need a different perspective in order to have a different view of the data. – Mark Cuban

I was reading a paper by Chris Huxham and Siv Vangen titled “Researching organizational practice through action research: Case studies and design choices”, and came across the phrase “deliberate and systematic data collection is essential, whether it be data such as flip chart notes made at a meeting…”

This stuck with me because I hadn’t thought of notes on a flip chart or whiteboard as “data”, but I suppose it is, everything is – when it’s collected. We tend to assume that data is a particular thing, like numbers or statistics, but it is actually everything. What we find unnerving about Google and Facebook is that they collect it all, whatever we do is monitored and stored and then something happens with it later.

The thing about data, however, is that it is useful to have it, whatever you do. For example, I’ve realised that I have spent most of my life creating data – once I learned to write anyway. I wrote when I was younger because it helped me think and early in my career I learned that taking notes during every meeting was a good habit to get into. There’s no point asking whether something is worth recording or not. Record it first and ask questions later. This has become a twenty-year habit And I hope it continues.

The reason you write, however, is not just to record. It’s so that you have space in your brain to observe and see the total context – the widest possible picture of what is going on. Without notes you’ll remember only a few things. With notes, you can ask more and explore widely. And if you take good notes, visual ones, you can map out understanding very effectively indeed.

But once you have this data what do you do with it? In the collection of essays, “Analyzing qualitative data”, Christina Hughes talks about Schatzman and Strauss’ argument that notes can be Observational Notes (ON), Theoretical Notes (TN) or Methodological Notes (MN). I see this recommendation as one to take notes about what you’re seeing, notes about what you think about what you’re seeing, and notes on how you’re going about making notes and thinking about what you’re seeing.

The reason why you might want to get better at collecting data is that once you have it and develop the skills to understand it life becomes a whole lot easier. If you think of a sales meeting as an exercise in data collection you’ll approach it very differently to if you think of it as an exercise in psychological manipulation. Any meeting, in fact, from one with your child to one with your boss and even one with yourself, will go much more easily if you first seek to collect data and then look to understand and make decisions on the basis of what you’ve collected.

Cheers,

Karthik Suresh

Thursday, 8.28pm

Sheffield, U.K.

All our knowledge begins with the senses, proceeds then to the understanding, and ends with reason. There is nothing higher than reason. – Immanuel Kant

In Zen and the art of motorcycle maintenance the lead character, Phaedrus, recalls coming across the writing of Kant. He writes about studying it like a chess player, looking for openings. He writes about Kant’s “formidable logical fortification” of his position. But eventually he finds that Kant is lacking something – that the ideas about beauty seem ugly and somehow there’s something not quite right with reason itself.

We’re moving away from this idea that there we can reason our way to a better way to be, that logic is perhaps not all there is. The thing about reason is that it works within a frame. You define what your boundaries are, what your axioms and beliefs are, and then you can construct logical arguments. The thing is that those argument work only within the context of a frame. What if the frame is wrong – instead of being a rectangle, it’s actually a circle? That’s going to affect how you think and feel about what’s happening inside the frame.

We also have a habit of denying anything outside the frame. If it falls inside our belief system then it’s fine. If it’s outside, then it’s wrong.

I think we are moving to a system of understanding that is starting to look at the connections between frames rather than the frames themselves. It’s the interaction of belief systems that creates much of what we see in the world around us, whether it’s conflict based on age-old religious differences or modern differences of opinion on forms of government or the rights of people.

We spend too much of our time looking inside a frame – looking to do the best job we can do, or be a particular kind of person. If you want to make life easier and create more value – it’s worth understanding what’s going on more broadly. And the easiest way to do that is to start recognising the frames around you and seeing what kinds of connections link them.

Cheers,

Karthik Suresh

Wednesday, 8.39pm

Sheffield, U.K.

If you take a more Darwinian point of view the dynamics of the universe are such that as the universe evolved in time, complex systems arose out of the natural dynamics of the universe. – Seth Lloyd

MIT, The Massachusetts Institute of Technology, publishes research on its DSpace repository and I came across an interesting paper on systems dynamics by Linda Booth Sweeney and John D. Sterman.

Systems Dynamics is one flavour of systems thinking that tries to understand the behaviour of complex systems as they change over time. I don’t have much experience of this space but came across quite a nice application of it when I presented a paper at a conference in 2019. Dennis Sherwood from The Silver Bullet Machine Manufacturing Company Limited described climate change using a systems model and it made quite a lot of sense. The most interesting part was realising that the issue we have with carbon in the atmosphere is akin to having a leaking boat. If we do nothing, we sink. If we plug the leak, we float. But another option is to bale water out faster than it’s coming in – and Sherwood’s argument is that carbon capture can do that, if we remove more carbon from the atmosphere than we create, then we can limit rises in temperature.

Anyway, the thing about systems dynamics is that it’s complex. The image that start this post shows you the kinds of things you need to know. For example, you need to know what happens when things interact, what kind of complex behaviour can result. You need to understand how feedback works and how things move through a system in stocks and flows. For example, if you buy a toy a day then that looks like a constant line, one toy every day when you think about flow. When you think about stocks, however, you end up with a rising line, as the total number increases every day. Which is why when the kids got to 10 years old there are at least 3,650 pieces of unwanted plastic that I didn’t have when we started.

Another important element to understand is how time delays work – you make a change and it takes time to work through. Moving too fast is a problem. Then there are non-linearities, not everything is causal and it’s hard to model some kinds of changes. And then you have people and the models they carry around in their heads or on paper.

All this skills are necessary if you want to create good quality models and it turns out that they are not innate – we don’t have a natural ability to think in this way. Not even smart people, the paper suggests, find this obvious or easy. One suggestion is that it’s just too hard and we aren’t clever enough to deal with the challenges. Or maybe it’s because the people doing the tests weren’t being paid or didn’t have enough time. Maybe it’s the test itself, it was done in a lab while in the real world we seem ok at making these kinds of decisions. But the paper says that perhaps the problems are even more basic than that. We’re poor at doing this because we don’t know why we’re doing it.

I liked this sentence, “Nevertheless, there is only a weak relationship between education and performance. For a large fraction of the subjects, training and experience with calculus and mathematics did not translate into an intuitive appreciation of accumulations, of stocks and flows.” Modelling does not build intuition – working on real-world problems does. This may not be a situation that’s unique to this field – it feels like a problem that applies to much of education. We are left wondering what the point of it all is.

Cheers,

Karthik Suresh

Tuesday, 6.28pm

Sheffield, U.K.

The instant people specialize, it’s in their interest to dehumanize the people their specialized function operates upon. – William T. Vollmann

I’ve been reading a couple of papers by Colin Eden about action research and I’m glad I came across them a little later in my exploration of this research space. Eden writes about his experiences and what he’s learned along the way. Interestingly I think I discovered a lot of it myself, but he weaves it together and helps makes sense of it – and it starts to explain the journey I’ve been through.

There’s an important point that I haven’t really seen explained elsewhere and it comes down to the old joke that goes like this, “Why are academic arguments so vicious? Because the stakes are so small.” The nature of research has changed over time, a focus on measurable, positivist approaches that stress hypothesis and experimentation as a route to understanding have dominated thinking. That makes a lot of sense if you want to understand the material world – because it’s amenable to tests and measurement but it is much less useful when it comes to the inner world of the minds of humans and the forms of their societies. Quantitative research in such areas has been less successful, less valuable.

And I suppose the best example of that is mathematics – the purest kind of reasoning there is. Many mathematicians see their space as a pure art, untainted by any suggestion of real-world impact. I studied a lot of maths – from arithmetic to calculus and I can safely say that all the maths I need to know involves the basic operators we learn about in primary school. That was all that was needed to manage billions of pounds of trades.

The reason that any form of study that seeks to do something that is generally true – that works irrespective of people – is that in social systems the people are what matter. Eden points this point as one of the requirements for action research is that we research situations where people care about something enough to take action. If they don’t, then there is no value, no reason to invest and put money into the project. This is missed by lots of people – they study their areas and come up with a bright idea and are then surprised that no one cares. So they try and change policy, asking the government to force people to care. And that works – to the extent that people do the minimum needed to comply. What this means is you can spend a lot of time working on something that interests you but that people don’t care about enough to pay you.

The unsurprising takeaway is that if you’re smart then you’ll be interested in lots of things. But if you also want to be rich you have to work on things that matter enough to people so that they will take action.

Cheers,

Karthik Suresh

Sunday, 7.09pm

Sheffield, U.K.

The true art of memory is the art of attention. – Samuel Johnson

What is worth learning and what is not, and how do you tell the difference?

I have just cancelled our subscription to a well-known online library service because in its collection of over a million books there is very little of any quality. One of my children, after reading through a dozen or so titles, said that some were good but most were trash. I wonder if what happens to subscription services is the same thing that happens with subprime mortgage debt and junk bonds – where you package a ton of rubbish with a few nuggets of gold and sell the whole lot to a hopeful counterparty. Yes you can find a title that is very good in there – but if it’s very good the author probably wants to sell it as a standalone piece.

What this tells us about life is that hoping for a bulk order of good stuff is not a good idea. We have more content than ever before but the vast majority of it is of poor quality. But as a consumer you have to decide what’s good and what’s not because the heuristics that helped us decide have changed. For example, anyone can publish a book now, so the chances are that most self-published books are not going to be very good. Some will be excellent, of course, but how can you tell? Well, you look at peer reviews, of course – which can be hacked by some people but, on the whole, you hope that content with lots of reviews is better than stuff that hasn’t been noticed by anyone. But is that enough – what if you’ve missed something important in the overlooked material – what if there is a gem hiding there?

These sorts of concerns aren’t new ones. The German sociologist Niklas Luhmann wrote about this and came up with a system of note-taking that he called a zettelkasten, a slip box with notes that had a unique management system. In his essay on learning how to read he talks about the difference between memorising something and learning something – about how to see what is essential and new and sets out something of a process that aspiring learners can follow.

The first thing to do is read selectively. What does that mean? In my case it means peer-reviewed papers as being better than books, which are in turn better than a mate’s opinions or a WhatsApp forward. The Internet has made it possible for us to access information easily but this always-on aspect can lead us to collect more material than we can possibly process. I wonder if it will help to decide what’s worth reading by actually printing it out – if it’s a paper you think is worth spending time on then print it out. There’s a barrier – a small one – but it will filter out a whole lot of rubbish.

If you have a printout of something useful then you’re going to want to engage with it. That’s where annotation and highlighting come in – and where the margins play a role in learning. Writing in the margins and pointing to things you find interesting when you read the material is going to help fix it more firmly in your mind. Although it might actually be a good idea to slow yourself down a little bit more. For example John Locke’s advice on making commonplace books makes two suggestions: first, “extract only those things which are choice and excellent”; second, read the whole thing first then only on the second reading mark out what you want to make note of. This process of slowing yourself down, of insisting on taking your time and going through something again before you spend your time trying to understand and remember it has one huge benefit. You will not want to do it – so you will not do it for things that are not worth it. So, when you do force yourself to make a note it will be for something you want to remember.

This really comes back to how you want to use your time. The only thing that’s fixed is time – the one thing that you cannot change. So what you want to try and do is spend your time doing the highest quality work you can do and that means creating barriers, creating reasons why you should not do anything unless it’s absolutely worth doing.

Then, the last bit of the process of reading is to write what you’ve learned in your own words – not trying to remember what’s been said but to frame it in a way that makes sense to you. For Luhmann, this meant writing it down in his own words. For me, it often means drawing a model, like the one that starts this post. The important thing is that you use your own voice, you write something that is original. One obvious benefit of doing this is that you won’t breach copyright if you use those words later in a paper or publication. And, when you use your own words these days you might also want to consider whether you just use paper or consider other media. But that’s a different subject.

The takeaway is that in an always-on culture where you can have everything it’s up to you to create the habits and processes that make sure you focus on what is “choice and excellent.”

Cheers,

Karthik Suresh

Saturday, 6.17pm

Sheffield, U.K.

A decent man who doesn’t consider himself a bigot can indeed be trained to behave like a bigot if he welcomes feedback exclusively from those who consider bigotry no big deal or, indeed, an attribute to be admired. – Adam-Troy Castro

I’ve been thinking about research and learning for a few posts now and I’m starting to wonder if a few things I’ve been told are actually all they seem.

For example, take the idea of thinking in systems. The essential concept here is that we start thinking in terms of parts and wholes and the relationships between the parts, and the emergent nature of the whole that comes from the interactions of these parts. Russell Ackoff presented an approach to this, describing four systems that came from this model, as shown in the table below.

In this approach what’s important is whether the part or the whole has choice – whether it can decide to do one thing or the other. So a clock, for example, is a machine made up of parts. But the gears of the clock and the clock as a whole does not display any indication of wanting to do something else – there is no choice involved anywhere. An animal, on the other hand, from an amoeba to a human makes choices as a whole – where to go, what to do, even if that choice is simply to move towards a source of food. The part of that animal, like the heart and lungs of a human, don’t have any choice – they just do what they do. Then you have a social system where choices are made by individuals and choices are made by societies as a whole. And then finally you have systems where the parts have choice – like animals and humans – but the whole doesn’t – because it’s the ecology or environment.

We’ll come back to this in a second but the next model that I looked at was about Cybernetics, which is the study of control and communications. In essence, Cybernetics is about steering your way to where you want to go. In the image above you start with where you are and figure out where you want to get to. There is a direct route from start to finish – but in reality what happens is that you set off and then check if you’re on course and if you’re not you correct your course. Often this results in an over-correction and you go the other way and so you check again and in this way keep getting feedback and correcting until you get to the destination. Makes sense, right?

The third thing I wrote about was models of gods. The two approaches I described were god as the creator and god as everything – a Western and an Eastern approach in simple terms.

Now, I’m questioning the wisdom of accepting any of these theories. Take the first one, about the systems model. Although you can talk about systems that have no choice when it comes to the part or the whole, such systems do not self-assemble themselves in the absence of input from a creator. You don’t take a walk and find clocks embedded in the sides of hills. A mechanistic system cannot exist without a creator, whether that creator is a human building a clock or a beaver making a dam. And if it does assemble by accident then if there isn’t someone around to notice does it make any difference? In essence, the first element of the division, while logical, cannot exist without consciousness. In fact, all those four systems have something in common – human consciousness to notice their existence. This may be relevant in a bit.

Now, the Cybernetic idea that you can steer your way from a start to a finish assumes that you know where you are and you also know where you want to be. Neither one of these is assured, in my experience. Quick, take an inventory of your assets, experience and capabilities? Do you think you know exactly where you are or are you still figuring out your strengths and weaknesses? And what happens if you do everything to get to where you want to get to – will that make you happy? And why are so many successful people apparently not entirely content? What if you live your life trying to be what you think you should be – steering your life on the right path. But if you didn’t have that feedback, perhaps you could have gone somewhere else? What if you’d followed your interests rather than looking around and seeing what else was out there or what someone else thought would be a good thing to do? You’ll never know now, will you?

The Gods argument came from Ackoff as well, the idea that a deterministic universe must have a creator – if the world is a machine someone must have built it. But if it isn’t a machine then actually what that tells us is not that God is everywhere but that we don’t need a God to explain how things were created. Which brings us to Terry Pratchett and his third model of a God which I had forgotten. He argues that we create gods as we need them. If we’re tired or scared or lonely and we pile a few rocks up and light a little flame and it makes us feel better and we decide that a good spirit lives there and then others notice and add their offerings and little prayers and then a thousand years later that’s a sacred spot with a temple and monks and rituals – we’ve just ended up creating a another god. These gods are everywhere – all it takes to make one is a few people with a belief. And gods today encompass everything from two thousand year old avenging old men to the ones that live in the temples of agile programming. It all comes down to belief and gods don’t exist – they are created by humans. For a god to be, we have to want them to be.

What’s the takeaway here. Well, it’s something around the idea that what you believe in matters. And that means that every once in a while you should question everything you believe in.

Cheers,

Karthik Suresh

Friday, 8.46pm

Sheffield, U.K.

All coaching is, is taking a player where he can’t take himself. – Bill McCartney

What is a good way to look at developing your career or professional capability? Would it help you to have a coach? Jake Cornett and Jim Knight summarised the research on coaching and found that while we know less than we would like what’s there suggests that coaching can help you learn more effectively.

One of the approaches they review is called Peer Coaching, where colleagues work together to improve their capabilities. For example, a teacher can share what they’ve learned with other teachers. But information alone does not result in improvement – there are ways to get more effective at how you do professional development.

You start with the content – with the information. That’s typically the bulk of any session – the lecture content. In addition to the lecture, however, modeling what you’re talking about helps show how you do what you do. For example, if you talk about note-taking then showing your notes models your approach. Getting the audience to practice what you’re showing them helps them to take what they’re seeing and get used to doing it themselves. Once they’ve created something it helps to give them feedback, to show they where they’re on the right track and where there is room for improvement.

These four elements, providing information, modeling, practice and feedback can take the learning rate of the audience from under 10% to nearly 20%. But if you add the last element, coaching, then you can get a transfer rate of over 90%.

What this means is that if you’re designing a development programme for you or for others then in addition to the work you do on the day or in class you should also consider providing ongoing support and coaching if you want your students to get the best result.

Cheers,

Karthik Suresh

Thursday, 8.07pm

Sheffield, U.K.

Not only is women’s work never done, the definition keeps changing. – Bill Copeland

From dictionary.com

noun

1.the act of defining, or of making something definite, distinct, or clear: We need a better definition of her responsibilities.

1. the condition of being definite, distinct, or clearly outlined: His biceps have great muscle definition.
2. the formal statement of the meaning or significance of a word, phrase, idiom, etc., as found in dictionaries. An online dictionary resource, such as Dictionary.com, can give users direct, immediate access to the definitions of a term, allowing them to compare definitions from various dictionaries and stay up to date with an ever-expanding vocabulary.
3. Optics. sharpness of the image formed by an optical system. Radio and Television. the accuracy of sound or picture reproduction.

I find that lectures often get to a point, early in the session, where the speaker introduces definitions. I’ve also noticed that’s the exact same point where I start to switch off and drift away. There’s something unutterably boring about definitions, something that makes the idea in there wilt and lose its vitality.

Abbott reminds us that a definition is actually a way of looking at the world – in order to understand a definition you have to understand the context in which it’s being presented. Most of the rest of the paper would require closer reading to understand than I have time for right now.

Others, it seems, have also wondered about definitions and created slightly impenetrable papers about the concept. For example, Michael B. Abbott has a paper called “on definitions” (2002) where he talks about the history of definitions. It seems that when people use words to have conversations they can often end up having endless discussions that result in no outcome or an unintended outcome. This is because we don’t use words in normal conversation with precision – a word can mean different things to different people. I remember, for example, someone getting confused about the difference between having an argument and making an argument.

Another paper by Fodor, Garrett, Walker and Parkes called “Against definitions” (1980) talks about how definitions are “one of those ideas that hardly anybody ever considers giving up.” They talk about The Standard Picture (TSP) of the notion of a definition – which helps you do the following:

• what a word is being used to mean
• how a word can be used in a logical argument
• if you understand the word, you understand its definition
• a definition of a concept lets you understand the parts of the concept

The rest of the paper, which runs on for 105 pages, ends up saying, I think, that even when you use definitions you can have misunderstandings.

I wonder if definitions are most useful when we’re early in a learning journey or when we have to write answers for exams. For example, Jay Hall’s brilliant school videos start with definitions as do many other teaching videos. Or perhaps definitions are a legacy of Newtonian thinking – the idea that the world is mechanistic and therefore so is language and thought. When we think of a definition perhaps we think of it as a precise thing, like mass or acceleration or geometry – from which we can derive conclusions.

But maybe we should really be thinking of definitions in quantum terms, in the sense of a probability space or cloud. Instead of looking for a definition to give us a point we should look at it as a boundary inside which we search for shared meaning.

Cheers,

Karthik Suresh

Wednesday, 7.20pm

Sheffield, U.K.

It is not our differences that divide us. It is our inability to recognize, accept, and celebrate those differences. – Audre Lorde

It’s extremely hard to watch someone else do something differently from how you would do it and resist the urge to jump in and tell them how to do it properly – your way. If you’re competent at what you do it’s hard to watch someone else bumbling their way through a task, using the wrong approach, and the wrong method.

Of course, there isn’t really anything “wrong” about it most of the time. It’s a different problem, and one that rests with you and with me.

For example, I remember the first time another student and I worked on a maths problem and encountered the European way of using dots and commas in large numbers. For example, I would write 2 million and 2 as 2,000,000.02 while they would write it as 2.000.000,00. The “fault” lies with Leibniz but when you first encounter this it’s a shock – seeing something that runs entirely counter to your established patterns of thought – a realization that the whole world does not think the way you do.

I have learned that this happens quite a lot since then. People are different, but it’s difficult to keep that in mind. We understand some people more easily not because they make good points but because we are more familiar with their ideas. We tend to assume that people who speak with an accent also think with an accent. We believe that because something worked for us that means it will also work for someone else.

But the opposite doesn’t hold true either. Not every exotic idea or mystic utterance makes sense. Believing that past experience is no guide to the future is a good way to learn nothing. I was listening to someone talk about coaching and the idea that a coach should not contribute expertise – that’s for the client to bring to the party. And that sounded like an abdication of responsibility – surely a coach needs to know how to do something well, even if they can’t execute it themselves?

I started to critique their position and then reminded myself that there was no point – no benefit to be gained from getting riled and worked up over an idea. It’s dots and commas, in one sense.

But the other thing I remind myself is the proof that we cannot prove that one method works – that any particular way is the “right” way and that others are wrong. The proof goes something like this.

If I tell you to do something and it works for you – then how do we know that you wouldn’t have had better results doing it another way? And if you do what I say and it doesn’t work – I can very reasonably say that you must have made a mistake in the way you did it. For example, I’m very interested in visual thinking. One kind of visual thinker focuses on the “visual” bit – using their artistic skills to create stunning images of ideas and concepts. My own approach is to focus more on the “thinking” bit. For example, the sentence that starts this post is visual – you can see it. It’s legible but not beautiful.

Asking whether it’s better or worse doesn’t really help. There is no right or wrong here. What matters is whether the method is useful, whether it works for you in your situation. An approach that works for you may be one that does not work for me. But how will I know unless I am open to the idea of trying it out? If you really believe that your method is useful – that it’s better – then you should be willing to try out other methods if only to test if that is actually still the case for you once you’ve done some tests.

People get very attached to ideas and methods. I have. I like the way I do things. But we have to always remind ourselves that it’s not the method that matters, it’s whether you find it useful for the situation in which you find yourself.

Cheers,

Karthik Suresh

Tuesday, 8.47pm

Sheffield, U.K.

Anything not worth doing is worth not doing well. – Robert Fulghum

Have you ever had to do work that you once used to do but haven’t done for a while? The sort of thing that’s called the “coal face”, what you might have done when you started your career but hope other people do now well enough so that you don’t have to get involved.

I don’t know about you but I miss that kind of work sometimes – the real work of creation and problem solving. When you’re older you’re still creating and solving problems but they tend to be problems of what to do rather than how to do it and the how problems are often ones you can get on with on your own.

The problem, however, is that before you know how to solve something you need to know what you are solving and why it matters. And that’s something that comes with experience – by spending lots of time working on things that teach you skills but often make little or no difference at all.

A good example of this is certain kinds of analysis. Warren Buffett writes about the difference between being approximately right and precisely wrong. I took a long time to learn this difference, spending lots of time trying to be precise without asking whether it was right. That’s easy to do – you can build a bridge in the wrong place, write code that has no function and spend a life wasted in a job that gives you no happiness at all.

What you have to hope is that even though you’re doing something that isn’t really that important you’re building skills. After all, doctors practice on cadavers before they’re let loose on real patients. Artists use cheap printer paper before splurging on the expensive stuff. It’s okay to spend time working on these things as long as you’re building skills and learning how to do different things. That’s making up for inexperience.

But eventually you start to see the difference between what needs to be done and what doesn’t – why you need to do one thing and not another. Your skills can help you do the thing you decide to do but your growing experience helps you decide what’s important and what isn’t.

Ultimately what experience helps you do is decide what not to do. And that’s vital – because too many of the world’s problems are a result of people doing things they just shouldn’t do.

Cheers,

Karthik Suresh