Category: Analytics

Hedge funds could look very different in a few years.

The Financial Times published an article about the rise of DIY algorithmic traders – people who develop automated investment strategies.

These people don’t work for hedge funds or banks on Wall Street.

Instead, they are mathematicians, progammers, physicists and data experts who are using their skills and cheap, powerful computers to tackel investing.

And this is happening everywhere we look.

Online sales are hyper-competitive, and the companies with an algorithmic edge can squeeze out more profits from their platforms.

Recommendation engines are key to keeping users interested, as algorithms work out personalised offers.

The energy business is fuelled by data – from meters recording generation output to those working out who has consumed it and what their bill should be.

In a world of abundant, cheap money, projects have to work on razor thin margins.

Getting the numbers wrong, over time, will mean that the project makes negative returns.

So, who is going to succeed in this new world?

Drew Conway came up with the Data Science Venn Diagram to explore the key skills needed in the world of Data Science – the field that will most likely underpin modern work.

In adapted form, the key is having three sets of skills.

Hacking skills are an entry requirement – being able to deal with and clean text and numeric data is part of every project.

Excel won’t hack it anymore – we’re going to have to use better tools to deal with more and messier data.

Then we need some maths and stats knowledge.

Knowing how to draw and interpret charts and understand the relationships between sets of numbers makes the difference between guessing and having a theory.

And a scientific approach is based on having hypotheses and running experiments.

Finally – many people think they can simply waltz into a new field and take it over.

Domain knowledge often makes the difference between success and failure in a field – it’s very hard for someone to build a tool to solve a problem that they have never experienced themselves.

That’s why we get lots of tools that look pretty, but end up doing little.

Curious people with good tools are what we need for modern work.

Science has been more successful at making life easier for us than any other system of thinking so far.

We have learned to control and adapt the material world to ourselves.

As George Bernard Shaw said, The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.

That’s worked for a few hundred years because of a particular way of thinking.

The positivist approach looks at reality and sees it as something that is independent of anything in it, including ourselves.

If we drop a stone, it falls the same way it will when dropped by anyone else.

That means we can look at objects, measure their properties and build concepts and ideas that exist independently of us.

Gravity would exist whether there was life or not. Once a building is constructed, the designer is no longer needed for people to live and use the building.

In the positivist’s world, there are things and other fuzzy things like people that don’t really compute.

We get into trouble when we try and apply positivist thinking to social structures like organisations and companies.

These structures exist because humans.

We can argue that if people didn’t exist, then there would still be moon rocks.

If people didn’t exist, there would be no companies to work for or carbon emissions to reduce.

Interpretivists see people as inseparable from reality. They are part of the world.

What we see around is constructed from what we see and the ideas we have – and how we interpret that.

This is why the assembly line organisation constructed by Ford and the lean manufacturing system constructed by Toyota both, on the surface, make cars – but have fundamentally different organisational philosophies.

Positivists run into trouble when they try and apply principles that work very well for things in the real world to organisations.

It’s easy to fix a problem in a machine – apply grease to a stuck part and it gets going.

An organisation’s equivalent of grease is harder to grasp – is it a meeting, a study, a team that works on a problem?

The extreme positivist approach says that everything can be fixed with a hammer and a spanner.

The extreme interpretive approach says everything is in our minds so nothing really matters.

A pragmatic view is somewhere in between.

There are technical solutions to some problems.

In many other situations, however, we need to have a model for how people fit in as well.

Without that, we can fool ourselves that we understand reality more than we actually do.

Thinking critically about something is not the same as criticising it – just like making an argument is very different from having an argument.

In day-to-day usage it appears negative or destructive – but from an academic point it’s simply a common sense approach.

How can we approach a new situation, idea or information with our eyes open and do something meaningful as a result?

This starts with being able to consider the situation critically – questioning rather than blindly accepting the things that are put in front of us.

This is especially important in a world where we face complex choices – from whether we should act quickly or slowly on our own contribution to climate change to enthusiastically adopt the latest technology fad.

Quite often we default to doing nothing – and that may be the worst of all outcomes. That leads to atrophy and failure.

John Mingers identified four aspects of critical thinking that act as a useful checklist for us.

The first is to be wary of rhetoric.

Is the argument fair, balanced and logical or is the speaker using language in a way that could appeal to emotions or mislead us?

Is it a sales pitch rather than an insight?

It’s not always easy to tell, because we can be swayed by passionate people who believe in what they are saying – but we need to try.

The second aspect is to question tradition.

Tradition can involve unquestioned assumptions that are made by people or the culture and practices that have sprung up around an idea.

This can be a difficult thing to approach as the existing position, or status quo, is something people will cling strongly to and resist changing.

It’s easier to go along with them – but that might not be the right thing to do.

The third aspect is not to accept authority unthinkingly.

Arthur C. Clarke wrote – If an elderly but distinguished scientist says that something is possible, he is almost certainly right; but if he says that it is impossible, he is very probably wrong.

Things change – and sometimes people that have built their reputation on a particular set of ideas find it difficult or impossible to accept that their contributions could now be overturned.

If they have power, they can direct resources and attention to other areas instead.

We see examples of this everywhere – most notably in politics across the world.

The final aspect is to question the objectivity of the people involved.

Robert Pirsig in Zen and the art of motorcycle maintenance writes about how scientific work can be like sorting grains of sand on the beach into piles.

The piles represent related ideas, concepts, theories. It’s the way that we approach and classify the world.

The thing we cannot forget is that the piles do not exist on their own.

There is a person kneeling there on the beach making them.

And we need to consider how value-free and objective that person is about the issue.

For example, we would not give a news report from a state that routinely censors information the same weight as a report from a respected investigative reporter.

So, in summary, critical thinking is not about criticising.

It’s about not blindly following persuasive, traditional, authoritarian or seemingly objective points of view.