Back in @MeasureCampSyd I had the opportunity to reflect on lessons I picked up throughout my tenure as a data scientist at Atlassian. This led to a fun discussion at the APH on what it actually means to "get things right" in data analysis.
"Far better an approximate answer to the right question than an exact answer to the wrong question." So sayeth John Tukey! @arikf breaks down why analysts often need to value accuracy over precision, trust their intuition, and more! https://t.co/eeWGuW7Osc
Can confirm. Just claimed $50 USD Claude usage credits. I always remained within limits so I have no idea if that will last 5 minutes, 5 days or 5 weeks with my usage patterns.
Although AI allows disciplines to expand their reach into other domains and conduct their work more independently (shorter feedback cycles), I will be very surprised if it will wipe the value of each discipline's domain expertise, depth and focus.
AFAIK devs are expected to be On Call, refine code to handle edge cases properly, make technical tradeoffs (performance, efficiency, ...), grow other devs etc.
I assume most PMs would rather avoid those tasks, otherwise they would have opted to be devs in the first place.
On the other hand, PMs spend time talking to customers, studying competition, are involved in GTM and pricing decisions, make roadmap tradeoffs, coordinate across broader initiatives, and so on. Any time spent on technical endeavours comes at the expense of those activities.
Antigravity is great but "eager to get to work building" is an understatement, especially with Gemini.
Me: for <feature X> what are the pros and cons of implementing option A vs option B?
AG: I implemented option C.
5 files deleted. 7 files modified. git add . git commit bye
AI coding agents like @antigravity are eager to get to work building.
But if you want to slow it down, you can use spec-driven frameworks like SpecKit to labor more intensely on the product and engineering docs.
Good example here: https://t.co/9xU0qhkV32
Continuing to have AI build a weird game demo a day. Here is: "Make a game where you have to prevent the apocalypse, but the interface is just Jira tickets"
Pretty fun/funny branching storyline, all text is AI created with minor feedback from me. Play: https://t.co/Zr5OM7z3FN
Last quarter I rolled out Microsoft Copilot to 4,000 employees.
$30 per seat per month.
$1.4 million annually.
I called it "digital transformation."
The board loved that phrase.
They approved it in eleven minutes.
No one asked what it would actually do.
Including me.
I told everyone it would "10x productivity."
That's not a real number.
But it sounds like one.
HR asked how we'd measure the 10x.
I said we'd "leverage analytics dashboards."
They stopped asking.
Three months later I checked the usage reports.
47 people had opened it.
12 had used it more than once.
One of them was me.
I used it to summarize an email I could have read in 30 seconds.
It took 45 seconds.
Plus the time it took to fix the hallucinations.
But I called it a "pilot success."
Success means the pilot didn't visibly fail.
The CFO asked about ROI.
I showed him a graph.
The graph went up and to the right.
It measured "AI enablement."
I made that metric up.
He nodded approvingly.
We're "AI-enabled" now.
I don't know what that means.
But it's in our investor deck.
A senior developer asked why we didn't use Claude or ChatGPT.
I said we needed "enterprise-grade security."
He asked what that meant.
I said "compliance."
He asked which compliance.
I said "all of them."
He looked skeptical.
I scheduled him for a "career development conversation."
He stopped asking questions.
Microsoft sent a case study team.
They wanted to feature us as a success story.
I told them we "saved 40,000 hours."
I calculated that number by multiplying employees by a number I made up.
They didn't verify it.
They never do.
Now we're on Microsoft's website.
"Global enterprise achieves 40,000 hours of productivity gains with Copilot."
The CEO shared it on LinkedIn.
He got 3,000 likes.
He's never used Copilot.
None of the executives have.
We have an exemption.
"Strategic focus requires minimal digital distraction."
I wrote that policy.
The licenses renew next month.
I'm requesting an expansion.
5,000 more seats.
We haven't used the first 4,000.
But this time we'll "drive adoption."
Adoption means mandatory training.
Training means a 45-minute webinar no one watches.
But completion will be tracked.
Completion is a metric.
Metrics go in dashboards.
Dashboards go in board presentations.
Board presentations get me promoted.
I'll be SVP by Q3.
I still don't know what Copilot does.
But I know what it's for.
It's for showing we're "investing in AI."
Investment means spending.
Spending means commitment.
Commitment means we're serious about the future.
The future is whatever I say it is.
As long as the graph goes up and to the right.
Interesting thesis on what destroys software engineering productivity at large enterprises: it's that legibility is prioritized over everything else.
https://t.co/XvUZlJs4Oh
"Hey, can you write me a script that does X?"
Gemini: Sure, here are 50 LOC that do just that
ChatGPT: Sure, here are 400 LOC. I added some basic error handling in case a solar flare flips a bit during execution
I feel like vibe coding is the slot machine of the LLM era. Maybe the next prompt will make it work exactly like you wanted. In the meantime, please feed another coin into the machine.
@Yampeleg When they say the new model is "more conversational" does that mean "wastes more tokens"? It feels like LLMs are becoming more verbose than they used to, and quotas run out faster. It's not something I measured though, just an impression.
@vboykis I spent two hours this week trying to fix project configuration because the IDE didn't show unit tests (AI: oh, a common issue with IDE x and testing library Y, let's dig into this!). Turned out all I had to do was click the (accidentally disabled?) "show passed tests" button.
@vboykis@APodgayko I don't know about the world, but I see companies like OpenAI, Anthropic Google etc have a bunch of open software engineering roles, and annoyingly none of the job descriptions say "everybody welcome!" yet
This is an excellent read. It also aligns with my experience. I have my coding agents on a very short and tight leash, and no generated code gets pushed unless I have cleaned it up and understand it myself.
Everything else leads to pain and suffering.
https://t.co/Byuk20O7RN