What is ailing the software job market is not AI taking away jobs (not yet anyway).
Here is my thesis, as a participant and observer of software for 30 years. Over those 30 years:
1. Massive over-capacity steadily developed in enterprise software due to a flood of VC, PE and IPO money.
2. Software vendors applied liberal doses of marketing spending to spread Fear (of missing out) and Uncertainty ("tech is changing, you need us") and Doubt ("are you confused? trust us") among corporate customers and the result was ever growing IT spending. Enterprise IT budgets kept rising because which CIO or Board would want to be seen as lagging? Major corporations in the West have layers and layers of duplicated IT systems, lots of money spent to acquire them and even more money spent to get the disparate systems to work together. The more inefficient IT systems ended up becoming permanent resource drains, requiring vast human resources to simply keep them running.
3. Large enterprises in the West found those needed human resources in India - transferring those inefficiencies to IT services firms in India, often multiplied by 3 or even 4! That "multiplied inefficiency" happened because IT budgets were kept fixed in dollar terms and more people got hired in India to "get more done".
Result? A large number of IT jobs in India came to depend on those original inefficiencies and the multiplied inefficiencies.
4. Note how Indian banks and financial institutions are more IT savvy, while spending far less, than established financial majors in the US - Indian firms did not have the budgets to splurge! Necessity made them highly efficient and today India's financial institutions are able to fend off foreign competition easily.
5. One of the deep facts about software is that often a 2 person team can solidly outperform a 20 person team and a 10 person team can do the work of a 200 person team. This is not just due to talent disparity - even when the large teams have equivalent talented people, they can easily end up being wasted on unproductive projects.
6. When people are billed by the hour or by the staff month (input metrics), as it happens often in IT services, there is zero incentive to remove those inefficiencies, to empower the 2 person team as opposed to hiring 20.
It is those multiplied inefficiencies in IT, built up over decades, that is facing a reckoning now.
Now enter AI. A large amount of code is boilerplate code in many projects and AI can eat such code for breakfast. Depending on the nature of a project AI can offer 10-20% productivity gains. Significant but not the 10x or 100x leap yet to destroy jobs on a vast scale.
But those AI gains of today pale in comparison to the "multiplied inefficiencies" built up over decades.
This reckoning nearly happened in the aftermath of the Global Financial Crisis (GFC 2008-9). It got postponed because the central banks flooded the world with money and a part of that money flooded into enterprise IT.
The pandemic unleashed another flood of money into enterprise IT.
Those floods are now history and we have a serious drought.
That is why I am pessimistic about the software job market, even before accounting for AI.
As I build AI-Basil, I’m realizing that food is deeply personal. It’s not something we can be forced to eat a certain way.
🍃✨Stay tuned for an exclusive 15-minute webinar where we’ll break it all down—simple, practical, and easy to apply.
that historically it were trusted family/community members who cooked what we ate and hence we have been wired to think, if it's food, it's good. This is not an AI generated post.
Sometimes, I can't stop to notice how easily we pick up food in store or restaurant to try just by its look or aroma. As the effect is not immediate, we really don't bother much thinking how it will impact our bodies. ...rest in comment
Then, I think some more that as a society that questions everything - from automobiles to AI and aspire to see transparency, explainability blah...blah..., how we become so indifferent when it comes to food. Where I have arrived time and again is the same realization...
I am right now working with extremely capable engineers on some advanced tech and we converse in Tamil because that is what they are most comfortable with. I am (very!) slowly learning Hindi so I can work with engineers in Hindi too. There is a lot of R&D talent in India if we get rid of the English barrier and the social stigma of not knowing English well.
As we achieve more, Indians some day will lose this idea that English is a mark of sophistication or achievement. It is a useful language to know but since 95% of Indians are not fluent in English, it cannot and should not be a barrier to advancing in India.
The Chinese are a major source of inspiration for me.
RIP Prof Donald Bitzer, inventor of the flat-panel plasma display among his many other accomplishments. Here is my personal story of Prof Bitzer.
I presented what would turn out to be my last academic paper at a conference in Trondheim, Norway in 1994, and that was a very low point in life for me. I had finished my PhD in Princeton and was feeling particularly useless about my academic work, feeling I had wasted my time. I was commiserating on this with an Indian student from NCSU I met at the conference and he told me about his professor who was also at the conference and that was Prof Bitzer. Shamefully I did not know about the highly accomplished Prof Bitzer but the student praised his professor so highly that I wanted to meet him. I am glad I did because that changed the direction of my life.
Prof Bitzer was extremely kind to my confused and depressed young self; he consoled me for how I felt and told me that the engineering PhD was becoming increasingly abstract, hiding an utter lack of relevance in mathematical sophistry; the focus had shifted to publish or perish.
He said we must get back to building things as engineers, rather than manipulating fancy mathematical expressions and obsess about publishing papers.
It truly felt liberating. I still remember that feeling of new energy after I talked to him. That chance meeting changed my life course and I gained profound respect for people who build things.
Thank you Prof Bitzer. RIP. Om Shanti 🙏