Can a bot help us predict financial performance?


We are bombarded with information every day and don’t have enough capacity to process and analyse it all.

One way we try and simplify is to look at the numbers.

For example, we look at figures and statistics over time – the performance of markets, the change in interest rates, the purchasing managers index and year-on-year comparisons.

Numbers are easier to process, chart and analyse, so we focus on them – but are they telling us the full story?

Are we missing out on what the associated text is saying?

Numbers rarely exist in isolation. They are often accompanied by analysis and commentary in the form of text.

Take annual reports, for example.

New investors look at company annual reports as an accurate and faithful rendering of a company’s performance.

Seasoned investors know that an annual report is the starting point.

It says what the company officers want to say.

The real messages are buried in the text and the numbers have been “managed” to meet expectations.

Is it possible to automate text processing?

This paper by BangRae Lee, Jun-Hwan Park, Leenam Kwon, Young-Ho Moon, YoungHo Shin, GyuSeok Kim, and Han-joon Kim analyses the relationship between business text patterns and financial performance in corporate data.

Specifically, they use annual reports of US listed companies in 10-K format that report on financial performance, the state of the business, competitiveness and the risks the companies face in their industry.

These reports talk about the past. What can text analysis tell us about the future?

Text mining is a way to process and extract insights from text

Text mining techniques process text and analyse it using descriptive statistics, clustering and sentiment analysis.

For example, the length of text in company annual reports can be expressed in terms of the number of sentences, the number of words and the number of words per sentence.

Clustering involves grouping companies that have similar statistics and then comparing their performance.

For example, we could use their average compound annual growth rate (CAGR) and compare that with another set of companies.

Finally, sentiment analysis looks at how positive, negative or neutral the text is – a way of measuring the subjective content and tone of text.

Does it work?

It’s still early days for this kind of technology but some interesting things are pointed out in the paper.

Companies with good performance talk about products, services, users and business, while those with poor performance talk about the government, contracts, results and the future.

It’s possible that companies that do well write more – longer sentences and more words about how they are doing.

Finally – and an interesting result – the tone of the text has no relationship with sales performance.

The takeaway is – don’t get sucked in if the company officers predict good times ahead, or if they are pessimistic about things.

That says more about them than the company.

It’s possible that text mining techniques will help us make better forecasts as we continue to use and refine them.

Key principles for smart appliance standards


The appliances in our houses – washing machines, fridges and air conditioners – have a key role to play in the transition to a low-carbon economy.

Electric vehicles and electrification of heat could increase peak demand from 60-70 GW now by 18 GW by 2020.

That will need around 6 new nuclear reactors the size of Hinkley Point C and a huge investment in additional network reinforcement.

Smart appliances that can change when they use electricity could reduce that additional demand from 18 GW to only 6 GW, according to a government consultation on smart appliances that runs from March to June 2018.

The consultation is looking at defining standards – the principles and associated functionalities required for smart appliances – to give industry something to aim for.

A smart appliance is, according to the policy, a product which:

  1. Has communications: It can connect to a network and communicate
  2. Is flexible: It can change how it uses electricity based on a signal like price (automatic, responsive modulation)

The standards will not apply to all products – only the ones with the most potential for flexibility. In particular, lighting and cooking are not included.

The kinds of products that could be smart are:

  • Cold appliances: fridges, freezers
  • Wet appliances: Washing machines, dryers, dishwashers
  • Heating: Electric heaters, controls, heat pumps, air conditioners
  • Battery storage: Standalone or combined storage like PV-Solar systems

The standards for smart appliances look at five key areas.

It’s important that people can choose brands of product without having issues with how their machines talk to each other.

Open standards that promote interoperability help with this – and using a common data model that has a standard instruction set is a preferred approach in the policy for communications between machines and between machines and their controllers.

The point about smart appliances is that they help with grid stability, for example by shifting when they are on away from times when there is lots of stress on the grid.

Clearly, they can also do the opposite: add load to the grid if misused and put more stress on the grid – potentially leading to faults and blackouts.

So, cyber security is an essential part of all internet connected devices these days to make sure this doesn’t happen.

A secure by design approach is a recommended one.

The appliances will create and store data so data privacy requirements will be in the standards.

Finally, when it comes to consumer protection, the standards will cover product safety and end-of-life process – and these will cover how the physical product is recycled and how the data on the system is handled.

Products that comply with the standard will have a label, and the standards will also be aligned with what is happening internationally.

A smart energy system could save taxpayers £17-40 billion to 2050, so many people will be watching this space with interest over the coming years.

Why I sold my crypto holdings


Like many other people, crypto currencies weren’t really something I looked at seriously until the staggering rise in their valuations in 2017.

In December and January, the price of Ethereum went from under $500 to over $1,250, more than doubling in two months.

The entire world got very excited.

Everyone seemed to be looking at these currencies and talking about buying it.

Was crypto something I should get into as well?

Buying a crypto currency is not like buying a stock or an index fund.

With a stock, we are taking an ownership stake in a company, with underlying cash flows and the possibility of growth and more income over time.

As an owner, we share in the growth (or not) of the business – and it makes sense to buy good businesses and hold them over time.

An index fund that covers a market is taking a position that an entire economy will grow and we will share in that.

For example, a S&P tracker will simply follow the performance of the largest companies, and they will probably be worth more in 20 years than now.

Cryptos are a pure trading play

Currencies don’t work like that – they have no intrinsic value.

They are worth what someone else is willing to pay for them in another currency.

That means we need to understand how buyers and sellers in that market work.

I needed a trading method

In many situations, we only need to be good at the buy side.

We’re going to buy and hold for the long term – and as an investment method that works quite well for things like buying stocks or making decisions on commodities like electricity and gas that we actually need for our homes and businesses.

In trading, however, we need to be good at the sell side as well.

More than good actually.

Lets say that we are 70% right at picking when to buy. That seems good, right?

If we are also 70% right at picking when to sell, then for any trade that involves a buy and sell, our probability of success is 0.7 x 0.7 = 49%.

That means we have a less than 50% chance of being right over time on both elements.

That’s where a system comes in, so my first step was to code one up to use for crypto currencies and I decided to apply point and figure charting (P&F).

A P&F system is designed for long term traders that want to understand how buyers and sellers are operating and make decisions on taking positions in that market.

The picture above is a snapshot of my P&F chart for Ethereum.

A position with actual money makes it more real

It’s easy to talk about what we would do, but to really get a feel for how we will act in a trading situation, it’s necessary to put down real money.

After the fall in value in February 2018, I entered the market with a very small position on the first reversal – marked as BUY on the chart.

I bought at around $725, and set a stop-loss at $650 – the red line.

The price went up a bit, down a bit, up a bit and then started to come off consistently during late February and into March.

When it crashed through my stop – I sold.

There isn’t much point having a trading system if you don’t stick to it

The point is that my sell wasn’t due to market conditions or timing or emotional decision making.

At the time I bought, I had a sell in mind, both for the upside and the downside.

This was a trading play, not an investment one – so when it went against me, the only rational thing to do was cut my losses and stop playing.

The next thing to answer is – when should I enter the market again and have another go?

How to create an online marketplace


Markets connect buyers and sellers, bringing them together to carry out mutually beneficial transactions.

Village and town markets are still popular because they give people a chance to come and browse the goods on offer, compare between different sellers and choose something they like.

Usually, there is enough business for everyone – but the markets themselves are operated and regulated by something like the town council.

Online marketplaces take the same idea and create a platform to do this instead – and there is a rush to create new platforms at the moment.

We see marketplace systems to make transactions easier in everything from selling houses to paintings, from comparing energy suppliers to recruiting freelancers in a gig economy.

An article in the Harvard Business Review by Andrei Hagiu and Simon Rothman sets out some of the key things to look out for when creating an online marketplace.

The point out that the network effect – where reaching a critical mass of users that then results in exponential growth – is not enough.

What is more important in getting a market working is that it gets the buyer-seller fit right.

The platform will only work when both sides are happy.

This means that transactions need to be mutually beneficial. Both sides need to be happy with what they get.

But, the parties don’t know each other – it’s the marketplace that brings them together.

So, the market needs to create a safe, trusted environment – by providing contracts, guarantees and insurance.

The point of a market is to help people easily compare goods and services – reducing the search costs of finding information, reducing the friction involved in negotiating a transaction and lowering the total cost of doing a deal.

This is the equivalent of a well-lit showroom, where we can walk around and compare things and see the good and bad points.

Doing these basics correctly will mean that it is more likely that buyers and sellers will enter into deals and come back to do more of them.

Having lots of buyers and sellers then creates a liquid market, one that becomes self sustaining and is used repeatedly.

That’s the point where the marketplace has staying power and is likely to be around for a while, and the marketplace operator earns a good commission on providing it as a service.

The final point raised in the article is that marketplaces need to contribute to society.

They are centres of commerce online, and so the tax collectors and regulators are interested in them.

They want to get their share of the profits and make sure people’s rights are protected.

Marketplaces are the future of business – when we walk down high streets in towns now they are often a dismal collection of charity shops, pound stores and cash advance places.

Business is moving online to new marketplaces.

And, some of us will create some of them.

How could microgrid and peer-to-peer energy networks work?


Why is the energy business so heavily controlled and regulated?

Mostly, its history.

When you have a few large generators and millions of consumers, its big business – and that leads to operators trying to control markets which triggers political oversight, which inevitably leads to questions of control.

So what we have across the world is a system of generation, transmission and distribution over a grid system that connects where energy is made and where it is used, and a parallel system of metering and accounting to bill users.

Microgrids and peer-to-peer systems want to change that

Imagine a new housing development where the developers have decided to create a private network of electricity wires that connect the homes instead of using the cables and equipment provided by the grid.

There may be a few connections to the main grid, but the rest of the properties are effectively off-grid.

At the same time, each house has solar panels for electricity and hot water, excellent insulation, low running requirements and perhaps a micro-chp unit and battery storage.

The independent network forms a microgrid.

The existence of housing units with the ability to generate electricity and heat from a variety of sources and a population that uses energy creates a network of peers – equal participants.

The concept of peer is sometimes forgotten – the households of the future will be both producers and users of energy – so called prosumers.

What they need to work are markets

In a microgrid peer-to-peer system, there will need to be some way of keeping everybody happy – and that is done by a price system and a market.

If people are free to set prices (or the trading is automated and the machines trade among themselves) then the market will result in a price that matches supply and demand.

It avoids the cost of routing energy through the grid, so it should be cheaper.

Experiments like the Brooklyn microgrid set up by LO3 Energy are showing how this could be done.

A peer-to-peer network does not have to be part of a microgrid

We could have renewable generators, like a solar farm, connected to the grid that want to directly sell all their output to a user connected somewhere else on the grid.

They can currently enter into a bilateral contract that is settled and billed by a supplier.

A true peer-to-peer system could eliminate the need for a supplier, and simply have a separate contract – based for example on a contract for differences model – although these are still complex to create and agree on a one-to-one basis.

A start in this direction is Open Utility’s Piclo platform that matches users with local generators.

We are still in the early stages of a transition

We’re a long way away from having solar PV on every roof and local networks of users have yet to spring up.

Will there be a revolutionary peer-to-peer change, or is it likely that the majority of the system will still be controlled by a few producers.

If history is anything to go by – network effects and scale matter.

We may have lots of committed, small players, but Google style companies for energy will still probably emerge – a few highly connected hub players that aggregate and influence how everything else works.

We still operate in a winner-takes-all ecosystem, and peer-to-peer is a small part of it.

Will it be different this time?

What is emergence and how can we make it happen?


We’ve all seen a flock of birds wheeling and swooping as if it were a single, giant organism.

The same thing happens with shoals of fish, or even people trying to leave a train station at rush hour.

Why and how does this happen, and what does it mean for us?

The term emergence is used to describe complex phenomena or behaviour that emerges from the interaction of simpler elements – often in a way that can’t be predicted from the features of the simpler element.

We can simulate flocking behaviour by setting up a system that follows three rules:

  1. Don’t crowd neighbours
  2. Move in the average direction of where neighbours are moving
  3. Move in the average direction of where neighbours are

These three rules result in a swarm – see here for example.

In organisations, emergence can happen in two ways.

In a hierarchy, the rules are set by those in charge.

People are given jobs, roles and responsibilities. In most organisations now, they have latitude and discretion in how they do their roles but have rules to follow.

Take the flocking rules, for example, and recast them for a job role. This might say:

  1. Avoid doing the same work as someone else – create your own niche.
  2. Try and make sure what you do is aligned with the vision and mission of the organisation.
  3. Do work that feeds into and works with what others in the organisation are doing.

If company had a number of people who organised their work in line with these rules it’s very likely that they will do some very interesting things.

It’s that balance between individuals and the collective that creates the conditions for innovation and creativity to emerge.

It’s also why micromanagement doesn’t work.

We need freedom and control – too much of either results in very simple or chaotic behaviour, neither of which are useful.

The second way in which emergence happens is through markets

Take Ebay, for example.

By creating a platform where people can exchange things, they created a thriving ecosystem of buyers and sellers.

Products from bicycles to floor mats flow through the system, in bursts of transactions that spill out into the real world – triggering a flow of packages in white vans that then creates emergent behaviour in the flow of traffic.

On a macro-level, the most successful economies are those that let markets form – allowing people to freely exchange goods and services.

We are surrounded by emergence – and what it reminds us is that we cannot control everything.

The best stuff happens when find the space between simplicity and chaos.

Do you have the skills needed for modern work?


Hedge funds could look very different in a few years.

The Financial Times published an article about the rise of DIY algorithmic traders – people who develop automated investment strategies.

These people don’t work for hedge funds or banks on Wall Street.

Instead, they are mathematicians, progammers, physicists and data experts who are using their skills and cheap, powerful computers to tackel investing.

And this is happening everywhere we look.

Online sales are hyper-competitive, and the companies with an algorithmic edge can squeeze out more profits from their platforms.

Recommendation engines are key to keeping users interested, as algorithms work out personalised offers.

The energy business is fuelled by data – from meters recording generation output to those working out who has consumed it and what their bill should be.

In a world of abundant, cheap money, projects have to work on razor thin margins.

Getting the numbers wrong, over time, will mean that the project makes negative returns.

So, who is going to succeed in this new world?

Drew Conway came up with the Data Science Venn Diagram to explore the key skills needed in the world of Data Science – the field that will most likely underpin modern work.

In adapted form, the key is having three sets of skills.

Hacking skills are an entry requirement – being able to deal with and clean text and numeric data is part of every project.

Excel won’t hack it anymore – we’re going to have to use better tools to deal with more and messier data.

Then we need some maths and stats knowledge.

Knowing how to draw and interpret charts and understand the relationships between sets of numbers makes the difference between guessing and having a theory.

And a scientific approach is based on having hypotheses and running experiments.

Finally – many people think they can simply waltz into a new field and take it over.

Domain knowledge often makes the difference between success and failure in a field – it’s very hard for someone to build a tool to solve a problem that they have never experienced themselves.

That’s why we get lots of tools that look pretty, but end up doing little.

Curious people with good tools are what we need for modern work.

Why we find it hard to get creative ideas accepted


Most people feel that their organisation needs to nurture and develop creative ideas – because that’s where innovation and growth comes from.

So, why is it such a struggle to actually get new ideas and projects accepted and pushed through the organisational decision making process.

A paper by Jennifer S. Mueller, Shimul Melwani and Jack A. Goncalo gives us an insight into how creative ideas are seen by others.

Most people, if you ask them, will say they support creativity – it’s a good thing.

A creative idea is different from just doing a job well.

It is novel – there is something new or different about it and it is useful – it should help us in some way.

And, in general, people feel like they would support creativity – either because they feel it’s the thing that other people would support – a social norm, or because they think of themselves as creative.

The odd thing is that creative ideas also introduce uncertainty.

If we already do things in a certain way, and we’re used to a particular set of accepted ideas and beliefs, we may be biased against creativity without being aware of it.

We can see this in the NIH syndrome – the not invented here mindset that poo poohs anything that comes from a different team or company.

Even supposedly creative people find it hard to recognise other people’s creative ideas.

We may only accept that an idea is creative once it has been endorsed by someone we trust or when it has reached a critical mass of users and we, rather belatedly, decide to join the party.

In the paper, the authors cite the example of Robert Goddard who worked on rocket propulsion and was ridiculed by his peers.

In the early days of the internet, some people felt that it would simply stop working because of the number of connections that were being created and how communication would be impossible.

A few years ago, people felt that blockchain would never work – and now they are starting to become more aware of the potential but also the huge issues that still need to be solved.

Their interest has been sparked, however, not by the idea but by the enormous increase in the value of bitcoin and other crypto currencies.

The problem for organisations, the authors argue, may not be about coming up with creative ideas.

It’s that we automatically organise ourselves into a defensive wall anchored in familiar, traditional or approved ideas.

So, what we need to do is learn how to address the biases we have – and improve the way in which we think about and recognise creative ideas.

Why we don’t understand how we fit into reality


Science has been more successful at making life easier for us than any other system of thinking so far.

We have learned to control and adapt the material world to ourselves.

As George Bernard Shaw said, The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.

That’s worked for a few hundred years because of a particular way of thinking.

The positivist approach looks at reality and sees it as something that is independent of anything in it, including ourselves.

If we drop a stone, it falls the same way it will when dropped by anyone else.

That means we can look at objects, measure their properties and build concepts and ideas that exist independently of us.

Gravity would exist whether there was life or not. Once a building is constructed, the designer is no longer needed for people to live and use the building.

In the positivist’s world, there are things and other fuzzy things like people that don’t really compute.

We get into trouble when we try and apply positivist thinking to social structures like organisations and companies.

These structures exist because humans.

We can argue that if people didn’t exist, then there would still be moon rocks.

If people didn’t exist, there would be no companies to work for or carbon emissions to reduce.

Interpretivists see people as inseparable from reality. They are part of the world.

What we see around is constructed from what we see and the ideas we have – and how we interpret that.

This is why the assembly line organisation constructed by Ford and the lean manufacturing system constructed by Toyota both, on the surface, make cars – but have fundamentally different organisational philosophies.

Positivists run into trouble when they try and apply principles that work very well for things in the real world to organisations.

It’s easy to fix a problem in a machine – apply grease to a stuck part and it gets going.

An organisation’s equivalent of grease is harder to grasp – is it a meeting, a study, a team that works on a problem?

The extreme positivist approach says that everything can be fixed with a hammer and a spanner.

The extreme interpretive approach says everything is in our minds so nothing really matters.

A pragmatic view is somewhere in between.

There are technical solutions to some problems.

In many other situations, however, we need to have a model for how people fit in as well.

Without that, we can fool ourselves that we understand reality more than we actually do.

Do you know which strategic play is right for you?


It’s conference season in the energy sector – and it’s a good chance to look around and see what companies are doing to position themselves as the industry and markets change around them.

For over 20 years now, we have seen battles between incumbents and innovators.

Innovators come along and try to get market share

The innovators – let’s call them the Red Team – see a market and believe they can do better.

They come along with a cheaper substitute and capture new, low-end customers.

For example, domestic users now have a choice of switching portals that make it much easier to compare offerings from suppliers and switch.

The portals are moving upmarket, targeting increasingly larger users and higher end customers – and compete with a number of other software and portal based systems that address the same market.

They also compete with various framework type structures that try to make it easier for users to decide between options.

This happens at every level of the market – in energy it’s being seen from how we buy energy, to how we use it and check the bills are right, and how we use our assets to make money.

The incumbents are slow to recognise how things are changing

Incumbent companies – the Blue Team – with market share and good profits don’t really want things to change.

They can ignore the innovators, but the ones that do that tend to find that by the time they are awake to the danger it’s too late.

A Blue Team that is on the lookout for this kind of competition has to either acquire the innovator or invest time and resource into creating a competing business that tackles the innovator head on.

In today’s digitalised markets, however, this article from the Harvard Business Review by Larry Downes and Paul Nunes says that things are a little different.

The fight just turned unfair

Downes and Nunes point to the emergence companies that land with a Big Bang and take market share suddenly and completely, with no warning – let’s call them Green Teams.

The military like this approach – the old adage says if you find yourself in a fair fight, then you didn’t plan your mission properly.

The big example here is how smartphones with free maps have upset the market for navigation devices.

The Green Teams, however, don’t operate like a military unit.

Instead, they’re often a group of people working on cool stuff that unexpectedly takes over a completely different industry from the one they’re in.

Products that come out of hackathons and experimental product launches have an effect beyond expectations because they turn out to be cheaper, more inventive and better integrated than the stuff that is out there right now.

Products like Twitter, Whatsap and WordPress changed the way we communicate and build stuff on the internet all at once – people signed up in massive numbers very quickly, leaving no time for incumbents to react.

Which team are you on?

In the energy business, the Blue Teams include traditional suppliers who are dealing with a changing energy system that is decarbonising.

Some are getting rid of traditional generation and becoming completely green generators.

The Red Teams include a host of new suppliers and players in the supply chain – from developers of platforms to technology.

The energy industry is notoriously slow to change – and this time around no company jumps out as being a clear Green Team leader, although many are trying to position themselves in this space.

The game goes on – the essence of strategy is knowing which team and play we’re going with.