What types of data analysis can we do?

types-of-data-analysis.png

We live in a world where we collect increasing amounts of data – but how many of us do anything with it?

At one extreme, we might do nothing at all, missing out on insights that could make a difference to the way in which we live and work.

At the other, we could create very detailed and sophisticated analyses that either no one understands or work under such specific conditions that they are not terribly useful.

The data out there includes information on customer behaviour, sales activity, operational production, energy use, waste generation – the list seems endless.

So, what can we do about improving the way we go about analysing the data?

An approach summarized by Dr. Jerry A. Smith and attributed to Jeffrey Leek suggests six types of analysis we can do.

We can start by describing the data – finding out more about its shape and characteristics.

How large is the data set? What is the average value? What does the distribution look like? Are there any outliers?

A large number of analyses stop here and go no further.

But what we should do next is explore the data. This means that we look for relationships between variables.

How does one variable correlate with another, or change over time?

For example, a classic use case is to look at energy consumption in relation to the outside air temperature.

Google correlate is an interesting tool that lets us see what kinds of search patterns match real world data.

The warning, as always, is correlation is not causation.

Our next, cautious, step is to see if we can infer something from our analysis to date.

Given what we have learned, can we say something about what might happen more widely?

So given the reactions of a sample of customers, can we be reasonably confident that the wider market will react in a certain way?

The level of certainty we have will make this method flow into the next as we predict what will happen.

At election time, this kind of data analysis always reaches fever pitch. All night analyses, updated with constant data feeds, update and predict the outcome.

Prediction is usually possible over relatively short time scales – we might be able to predict with accuracy the winner of a presidential contest in the next month, but not the winner from a pool of potential candidates five years from now.

Now we have to see if there is a causal relationship between two variables – a change in one will cause a specific change in the other.

This is the kind of analysis that happens in clinical trials. For example, a specific dose of a drug will result in a measurable improvement in a condition.

Finally we can look at a mechanistic analysis or an exact model, where we know what will happen as all the variables change.

This is usually the domain of engineering models – we know that a steam train will operate in a certain way once the water gets up to temperature and pressure and the various mechanical systems start to operate.

In a sense, the various methods of analysis progress from simple to complex.

A complex system – like human beings in a social environment – may only be understood with simpler analyses.

We can predict what someone will do, but we cannot say with certainty that a particular set of input stimuli will cause exactly certain neurons to fire and result in a defined activity.

An exact model may only be possible with engineering systems that operate within clear parameters and tolerances.

Analysing data isn’t something that comes naturally to most people.

We need to work on developing the skills, capabilities and toolkit needed to make sense of data.

And that probably starts with knowing what types of analyses we can do and understanding the situations where we can apply them.

Which business model will catch the next wave?

surfing-model.png

Charlie Munger talked about competitive destruction – the process by which new businesses come along and destroy older ones – often built using new and different technology.

Being one of the first to market can be a good thing in this situation.

Using a surfing model, if a business can get up and catch the wave, they could ride it for a long time, making profits on the way.

Intel did it with microprocessors, Microsoft with desktop operating systems, Apple with smartphones and Google with search.

Products based on artificial intelligence (AI) and machine learning might seem good candidates for the next wave.

Take the way in which we use mobile phones, for instance.

Tools like predictive text have been around for a while – but phones are used for much more than talking or texting.

Navigation systems on them have gone from route planning to real time route optimisation, with suggestions on how to change routes in the middle of a journey based on travel patterns in the area.

Translation is another area being transformed by technology.

For gist translation – where what we need is an understanding of what a document says in a different language – the systems built into browsers and search engines do a remarkable job.

Machine learning may provide the solution to spam emails.

Microsoft Outlook’s clutter service means that virtually all spam type emails are filtered out and never hit the inbox.

Generic newsletter, marketing and sales emails simply can’t interrupt us any more.

Some of us don’t worry about scheduling or planning things – the entries turn up in our diaries and we can rely on our phones to tell us where we are going and when to set off.

These tiny changes to the way in which machines help to organise and optimise our days are happening in a barely recognizable way.

But they are becoming also becoming an inextricable part of how we go about our daily business.

These changes signal a groundswell that is expected to turn into a tidal wave as AI affects everything from law and medicine to transportation and sustainability.

The question that individuals and organisations need to consider is how they will fit into a world where work requires hybrid human-machine skills.

Should we go for the easy option?

one-foot-hurdles.png

Warren Buffett wrote that after many years he and his partner, Charlie Munger, had not learned to solve business problems.

What they had learned to do was to avoid them, by looking for one-foot hurdles they could step over rather than seven-foot ones they needed to clear.

But how can this approach be applied day to day?

Take the emerging field of product management.

Is it better to create a new product and then try and sell it to potential users or to first try and understand the needs of potential users and then try and design an offering around those?

One school of thought argues that customers don’t know what they need before they see the product – if you had asked people what they wanted before the car was invented, they might have said a faster horse.

If the business we’re in is more humdrum – more exposed to competition – what approach can we take?

Let’s say we owned a food business – what advantages would help us beat the competition? Would it be better ingredients, better signage, widespread advertising or more delivery options?

The late Gary Halbert used this example and said people could choose any combination of advantages they wanted and he would still beat them if he had a single advantage – A starving crowd.

The test for any product is not how good it is or how glowing the reviews are – it’s how well it’s doing on gaining market share.

The energy efficiency business, for example, should really be an easy one to operate in.

Who wouldn’t want to cut their energy costs – after all savings go straight to the bottom line and how much product would a company need to sell to get the same result?

But many projects fail to go ahead because they don’t meet a 2-year payback?

But, if project developers thought like product managers, they might think about what the CEO and FD of the company really want to achieve.

If they are like most CEOs and FDs, their focus is on earnings growth and increasing shareholder value.

Payback to them is less important than what the project will contribute to EBITDA during its lifetime.

A McKinsey article shows how a modern approach to a portfolio of projects might evaulate them all, rank them on a risk/reward basis and select the ones close to the efficient frontier – essentially the best ones.

Cherry-picking makes it more likely that investments will return value in the long term.

We are often programmed to believe that anything worth doing must be hard – taking effort and sacrifice.

By going after the easy things, however, we may actually make a difference and create value.

How understanding fractals can help us decide

fractals.png

A fractal is a curious thing.

It is most commonly shown as a pattern, often mathematical, that is similar at different scales.

The Koch snowflake, for instance, is drawn by starting with a line and creating an equilateral triangle by splitting it into three parts.

Then, each segment of the triangle – a line – is turned into another triangle. Then that is repeated again.

What we end up with is similarity at different levels – as we zoom in we see the same pattern repeating itself.

Another fractal that is easier to visualize is a fern, or a lighting bolt.

A mountain is a fractal.

Seen from a distance it has a certain shape. Zoom in and the bumps and ridges are replicated on the surface all the way down to individual rocks and pebbles that show similar shapes under a magnifying glass.

So, what does a fractal have to do with decisions?

There are two ways we often approach decision problems.

One is through fundamental analysis – we look at the long term features of the problem or situation and come up with an approach to deal with it.

In investing, for example, this may involve looking at the stock price, earnings, assets, market sector, historical performance, management team and so on.

Or, in a business, it might involve looking at the accounts or the number of billable hours and making choices on where to invest or how to spend time.

A different approach might be to learn how to recognize patterns.

A price chart often shows similariy at different levels.

Performance during the year, during a month and during a day all show signs of the continuing battle between supply and demand.

Understanding these patterns and working out a logical approach to dealing with them can make the difference between good investment decisions and shooting in the dark.

It might also help us with managing people.

For example, an individuals career often follows a series of ups and downs – starting, learning, growing, plateauing and ending.

A company does the same thing – following a lifecycle.

Whether you look at it over the 40 year life of a company or the 40 year career of an individual, we’ll see similar patterns emerging.

And the way to deal with patterns is to recognize that things happen again and again.

From a decision making point of view, it means that there are very rarely crucial decision points.

We can take it easy – if we miss one opportunity, another will come along in a while.

It’s that whole thing about another door opening when one closes.

The best time to start a new project, for example, is 10 years ago.

The second best time is now.

When should we change lenses?

choice-of-lenses.png

There are at least two problems with how we go about solving problems.

The first is we approach them from our perspective – using the set of strategies, models, expectations, biases, experience and tactics that we have built up over time.

The second is our first attempt to come up with a a solution tends to narrow our thinking very quickly, as we look for patterns, evidence and reasoning to support that solution while forgetting about the rest of the options out there.

It is very hard to stop doing either of these things – it’s an approach we’re comfortable with and when we are faced with a problem – whether it’s doing some DIY or fixing a failing healthcare programme – we tend to fall back on our default programming.

So, can we change this or are we stuck with the way things are?

One technique that may help is problem restatement.

We restate the problem by taking the time to write out the problem again in as many ways as we can think of.

For example, perhaps we have a difference of opinion with someone at work on an issue.

It’s easy to make the fundamental attribution error – saying the problem is that person is rubbish because of who they are as a person rather than the situation they are in at the moment.

But, what if we tried to look differently and restated the problem.

We might be able to use a selection of lenses.

The reverse lens tries to look at the situation from the other person’s point of view.

What would they say about us and the situation and how would they justify their approach – and is there any merit in what they are saying?

The long lens helps us look at things with a longer term perspective.

Will this issue matter in a week, a month or a year?

The wide lens looks that the situation in context.

How does this issue affect everything else?

If it went the wrong way would the consequences be significant or not, and so how important is it in the wider scheme of things?

Restating the problem is one way to increase our chances of getting a good outcome.

As Charles Kettering said – a problem well stated is a problem half solved.

Why we should do more things that give us energy

energy-audit.png

Robert Kiyosaki in his book Rich Dad Poor Dad has an elegant way to define assets and liabilities.

Assets put money in your pocket. Liabilities take money out of your pocket.

This isn’t the way accountants look at things – but it’s a good way for the rest of us to visualize where we should put our money.

Investing in a rental property that gives us income every month is good.

Buying a flash car that costs us hundreds in payments every month is bad.

It’s a clear and simple model that should increase how much money we have if followed.

We can tell what kind of financial health our life is in by looking at the pattern money takes as it flows through our hands.

If, when money comes in, we invest in assets and the money flows back into our pockets, that leaves us with more at the end of the day.

On the other hand if, when money comes in, we have liabilities, then the money flows out to pay for them and we are left with less when done.

And the same model, it turns out, can be used to think about activities that we do every day.

It’s a bit of a cliche – but even the Harvard Business Review is happy to use an article title like Manage your energy, not your time.

Our days are made up of rituals – things that we do from the time we wake up to the time we go to bed.

Some of these things are going to leave us with more energy than we started with.

Others are going to drain us of energy when done.

And, while it’s easy to fall into a victim mentality and argue that what we have to do is driven by the demands of other people like colleagues and bosses, recognizing the patterns that aren’t good for us is still the right starting point.

Like doing an audit.

There are a couple of ways to do this.

One approach is to log what we do over the day and then look back to see how our energy levels have increased or decreased.

Another is to make a simple list of things and give them a score.

Then it’s time to start making changes.

Usually this starts with creating new rituals.

For example, some people find their energy levels go up if they check email an hour after getting into work rather than first thing or in the afternoons.

Or we could do work sprints – 90-120 minutes of focused activity with a 5-minute break every 25 minutes followed by a longer break at the end.

We need to increase the number of rituals we have that are assets, which leave us feeling more energized when we are done doing them.

We might work on liabilities with high energy, but when we are done we’re drained – so we need to keep those to a minimum.

The last word on this goes to Scott Adams, the creator of Dilbert, who says –

Manage your creativity, not your time. People who manage their creativity get happy and rich. People who manage their time get tired.

Which path would you take?

small-steps.png

Edward de Bono wrote about a small but significant difference between the way we think about the physical and mental world.

Let’s imagine we had to build a staircase.

Would it look like the one on the left in the picture – or the one on the right?

Most people, in the real world, understand that small steps are the way to go.

But, when it comes to mental work, we try and take shortcuts.

Take investing, for example.

Tom Dorsey wrote about a colleague of his who said – Tom, ain’t but one way to make money in the stock market. Slowly.

Most of the people and businesses that survive compounded their value over time – they didn’t simply skyrocket to fame and fortune.

The recent falls in the value of cryptocurrencies, although there has been a rise again, have left some people feeling elated, and some despondent.

If you bought at $16,000 hoping to hold until it went up, $12,000 is going to keep you nervous for a while.

But, the prices simply reflect supply and demand – and someone who takes a position needs to be aware that buying a currency is not a one-way ticket to riches.

Over the last year, a fairly sensible trading strategy could have resulted in a 6x return on Litecoin. Around 6-7 trades would have turned $20,000 into $122,000.

If you knew how to do it, had the money, were willing to take the risk and had the discipline to monitor markets at least daily that is…

On the other hand, a much easier approach would have been to buy a selection of low cost index funds five years ago and then get on with life.

That would have gained almost 30%. That’s not a bad return either given where interest rates are.

The same principles – going step by step – apply to most of the knowledge work we do now.

People who create anything of value do so piece by piece over time.

It’s tempting to take a shortcut.

But a shortcut may turn out to be the harder path.

The story of Mr. Market

mr-market.png

Benjamin Graham, the father of value investing and Warren Buffet’s friend and teacher wrote about the kind of mental attitude that would be most useful for investment success.

Imagine that we own part of a business – a privately held one – and have a partner called Mr. Market.

Each day, Mr. Market comes into our office and names a price where he will sell his share of the business to us, or buy our shares of the business.

Our business might be in something quite normal – a restaurant, retailer, a professional services firm.

Mr Market, on the other hand, is anything but normal.

Some days he is euphoric, wildly optimistic, and the price he gives us is high because by selling to you he may lose out on future gains.

At other times he is down and depressed – unable to see anything but problems ahead – and names a very low price because he doesn’t want us to dump our shares on him.

The other thing about Mr. Market is that he comes back day after day.

He doesn’t take offence if we ignore him for a while – he’s always willing to offer a price to buy and to sell.

So, the question is would we sell to Mr Market at his prices? Should we be up when he is up and down when he is down?

That would seem very odd in a business we owned.

We would know the value of the business – how much it makes and is likely to make. We understand the costs and profits.

So, if we were to go through all the hassle of selling, we’d only do it when we felt like it.

We’d let the results of the business tell us how well we were doing rather than rely on Mr. Market’s opinion.

Mr. Market is there to serve us – to allow us to buy and sell when we want – and not to guide us and tell us when to do something.

Benjamin Graham also said – In the short run, the market is a voting machine but in the long run it’s a weighing machine.

The story of Mr. Market is a useful one to keep in mind when trying to make our own investment decisions.

What is the flippening and would you take a position?

ico-fear-greed.png

It’s hard to get one’s head around how currencies like bitcoin are changing things.

Two things exploded in 2017 – cryptocurrencies and Initial Coin Offerings (ICOs) and will probably get more attention in the weeks and months ahead.

Take the choice of cryptocurrencies themselves. New ones are being added all the time – so what should we be buying or selling?

Warren Buffett says that he wouldn’t get involved.

But, what he would do is buy a five-year put on every one of the currencies out there.

What that means is that if he could buy an option to sell a currency at a fixed price within the next five years, then he would buy that for every currency.

That means that he believes that in five years the price of all currencies would be lower than they are today.

So – should we take the advice of the legendary investor?

Well – the counterpoint is Arthur C. Clarke’s comment that if an elderly but distinguished scientist says that something is possible, he is almost certainly right; but if he says that it is impossible, he is very probably wrong.

Is the world a different place now? More like it was in 1971?

That was when the gold standard ended, and dollars were no longer pegged to gold.

All of a sudden, currencies could be whatever value they wanted and the total amount wasn’t linked to the amount of gold a country held.

With a fixed supply of cryptocurrency – the idea of a standard comes back into play.

What do those that are bullish on the space say then?

The idea behind cryptocurrencies is that they are fair – they take away the control of the large financial institutions and redistribute power to the community and network in a democratic way.

In a sense, bitcoin’s founder had the idea of one-cpu one-vote.

Where are we now?

Some Initial Coin Offerings (ICO’s) have staggering valuations – they raise millions in minutes.

In 2017, Gnosis raised $12 million in 15 minutes, OMG raised $25 million, and Qtum raised $15.6 million.

The latter two sold tokens at at around $0.30 which have gone up by around 40 times in under a year.

That’s an interesting return.

So greed kicks in – a new technology that looks like it’s going to take over the world, fast returns and the ability to lock in incredibly high returns.

On the sidelines we have fear – this doesn’t make sense and the valuations are not tied to any real world value and revenue.

Are investors – and there are a few who seem to take up most of the offering value – focused on making money from volatility rather than from the underlying business?

And is the basic business case the greater fool theory – where we can find someone more dumb to buy the stuff we hold?

Just the idea of a company being involved in this space is like giving it steroids. Kodak – a 130 year old company – tripled its shares after announcing it was going to start a currency.

Back to the title – the flippening is what happens as sentiment flips from one asset class to another.

It’s used mostly to monitor the possible flip from bitcoin to ethereum as the latter becomes more valuable.

Will 2018 be the year when there is a flippening from the dollar and stock markets to cryptocurrencies and ICOs?

What happens to the time?

long-days-short-years.png

A single sentence, writes the author Gretchen Rubin, seems to resonate most deeply with people.

The days are long, but the years are short

People with children are acutely aware of this. The comedian John Bishop talks about walking along with his son, who reaches up to hold his hand.

One day – perhaps one that might not even be remembered – his son doesn’t reach up – he’s big enough to walk on his own.

That experience will never happen again.

Tim Urban, in his blog article, The Tail End, writes about how when we measure time in events rather than years or days, the impact is different.

Take holidays with children for example, if a child is 10, that might mean only 10 more family holidays before they want to do things with their own friends.

If someone is 30 and optimistically decides to leave to 90 – that’s 60 more winters they’ll experience.

If they build one snowman at least every winter – that’s 60 snowmen they will build.

Or – thinking of it another way, people on average might see their parents once grown up for around 10 days a year.

Let’s say they are lucky and live 20 more years – that’s 200 days we might spend with them.

Or around 7 months.

Or take health.

Edward Stanley said those who think they have not got time for bodily exercise will sooner or later have to make time for illness.

Then, of course there are the friendships and relationships and ways in which we pass time time.

After work.

Water runs downhill – we’ll do the things that are easiest – see more of the people who live closest to us.

After that is comes down what we do with time – what we choose to do, how we choose to do it.

What is time, then?

According to Ray Cummings, the author of The man who mastered time, time is what keeps everything from happening at once.

And a maxim worth keeping in mind comes from the thirteenth century Archbishop of Canterbury, Edmund of Abingdon.

Study as if you were to live forever, live as if you were to die tomorrow.