AI
GitHub’s internal playbook for building an AI-powered workforce
This provides insight into how GitHub applies AI internally. What I found interesting was the section about metrics, most of which are activity or output metrics (read: not all that useful). “AI leverage” sounds interesting, but there’s no concrete means of calculating it.
Hidden Technical Debt in Agentic Systems
This article provides a good overview and checklist of sorts if you are building a product that uses AI. The author draws parallels to a 2015 paper about ML systems and the types of debt that can occur because of certain decisions. It’s also useful to see how the layers fit together. A telling example was API documentation that needs to be much more explicit. The description even looks like a skill description (This method does XYZ. Use when doing ABC.).
The ROI of AI-assisted Software Development
- This report was developed by the same Google group that reports on DORA.
- “Consequently, one of the most critical indicators of a software application’s health is how easily, safely, and quickly it can be modified.”
- It’s about reducing waste rather than reducing cost.
- “We don’t measure AI by the code it writes but by the bottlenecks it clears.”
- The most significant outcome of adopting AI was increased individual effectiveness. It’s worth noting that three negative outcomes increased: software delivery instability, burnout, friction.
- Value framework = cost efficiency, developer experience, business growth, user experience, productivity
- Measure the time saved and the revenue from extra feature deployments.
- The report strongly recommends not adopting a headcount-reduction strategy. The report gives simple equations for “headcount reinvestment capacity” and “revenue from accelerated feature delivery.”
- The last half provides instructions for calculating the ROI to get the J-curve effect.
Software engineering
Best Practices in .NET I Don’t Like Anymore
- Clean architecture. Now prefers vertical slices in feature folders that have all the layers within it. This arrangement also makes it easier for AI to find things (rather than hopping across horizontally sliced projects).
- Testing pyramid with tons of unit tests. Now prefers the testing diamond. A few E2E tests, more integration tests with test containers and web application factory to spin up a server, and some unit tests for hard-to-test edge cases.
- Mapping libraries (e.g., AutoMapper). Now prefers having an extension methods on the object.
- MediatR. Now prefers to just call the service from the controller and avoid multiple layers of abstraction. LLMs also work better with fewer layers of abstraction.
- Avoid Entity Framework Core. EF Core is performant and has many good features.
Why AI Won’t Replace Software Engineering
- It was never the code generation part that was expensive. The bottleneck was never the typing. It was understanding, designing, testing, integrating, and deploying solutions.
- AI accelerated the part of software development that wasn’t really the problem. The real problem is the enormous surface area of the problems we want to solve with computers.
- Jevon’s paradox — generating more code -> more complexity, more integration, more maintenance, more behaviors to evaluate, less time to understand the problem.
- It’s not a productivity gain; it’s a complexity bomb with a delayed fuse.
- Good engineering is about small, reversible steps with fast feedback.
- Software development is a process of discovery, not just a solution.
- Continuous delivery = system is always releasable
- Work in small steps. Evaluate releasability after every step (defined with tests). Organize the tests into a deployment pipeline.
- Automated testing forces us to be precise about what we want so we can verify we got what we wanted.
- Working small helps you make the best use of a limited context window.
- Being clear in criteria can help AI understand more quickly if it’s off track.
- Pipelines don’t care whether a human or AI wrote the code; either way it enforces the same standards.
- AI is good at taking general ideas and getting fairly close, but it’s usually a naive implementation.
- We need to teach AI tools TDD, and to not delete tests without asking a human first.
- The real skill is in engineering systems, not writing code.