The problems implementing AI workflows in corporate data pipelines are becoming more apparent the more we try and use these tools in practice.
Their main advantage is speed – it can write a snippet of code, check and existing block, or rewrite very fast.
And it mostly works unless you’re working with a newish library that it doesn’t know about yet.
But when it comes to more tasks I’m struggling with reliability.
Anything you produce needs to be checked. And if it has to be checked, that needs someone who knows what they’re doing. So your more expensive resources get tied down.
And if you build anything significant there’s the issue of control – do you rely on one model or build for several?
Then there’s the last point, which it’s tricky to name but I’ll start with willfulness.
Many years ago, I learned electronics by taking things apart.
I learned that engineers started with expensive components for the first version.
Then they replaced things progressively, plastic wheels for ceramic in VCRs, for example.
The idea was to reduce cost while maintaining performance.
Software works differently – stuff that works well has to be limited in order to monetise it. Performance is tiered.
ChatGPT used to give you a list of 100 companies of a certain type without complaining.
Now it gives you five, refuses to repeat itself and points you to a reference.
It’s obstinate, willful, petulant, even.
I don’t know what the solution is becuause clearly companies have to experiment with pricing models to be sustainable, and limiting what you can do until you pay is one way to get the dollars rolling in.
It’s just tricky to stay a customer of something that seems to get worse over time rather than better.
