AI
Agents Don’t Do Standups: Building the Post-Engineer Engineering Org
- The speaker’s company did a study about applying AI and was able to get a 25x increase in deployment frequency. Granted, this was from a small engineering team.
- They also measured output based on tickets and code complexity, giving a 10x increase.
- “Rituals designed for humans don’t work for agents.” Huddles were every other day with engineers, design, and product. They used specs, got AI to interview the team about it, created a lightweight design document, which turns into tickets and PRs.
- (Note: I think the speaker doesn’t understand Scrum or why certain ceremonies exist. The 2-person tiger team with high-power engineers that basically did Kanban, and could operate quickly with few dependencies.)
- Tasks need to be verifiable and deterministic (unit tests, E2E tests, linters, PR prerequisites, feature flags). Agents would review basic things (variable names, style) and not architectural things. (Note: I disagree with the previous idea.) Heavy human in the loop on design document review and product feel.
- Turn your engineering process into composable skills. Agents help with spec building, ticket creation, branching, coding, PR creation, and testing. Agent should be able to self-heal when mistakes happen. Let humans parallelize and be multipliers around security, product feel, and scaling complexity for the task (token spend, prevent over-engineering).
- The presenter recommends against: having everyone do this at first, creating a one-size-fits-all approach, doing too much at once.
Companies such as Microsoft and Uber that pushed employees to use AI are realizing the cost-per-token model is more expensive than they anticipated. Gartner estimates the cost of inference should reduce in the next 4-5 years, but the savings may be wiped out by the models requiring more tokens to do certain types of work.
Nobody Actually Wants AI Anymore
This video contains several clips of people trying to use or make sense of AI and getting frustrated. It mentions a pre-print paper about how AI reduces our persistence when learning, thus reducing our own skills. I’ve seen first hand people using AI for the most basic tasks, and have also heard about AI fatigue and brain fry from having to manage multiple agents. The ROI isn’t really there, but companies carry on regardless. There were also some clips of tech magnates being booed at commencement ceremonies, showing how out of touch they are with the generation entering the workforce.
Software Engineering
Can We Tackle Security & Compliance in Our Deployment Pipeline?
- Common topics companies think they can’t implement: trunk-based development, pair programming, compliance
- Many people grumble about compliance, but the spirit behind it comes with reasonable requests (e.g., code needs a reason to change, requirements should have an owner, you should know when the requirement is met).
- Compliance helps us create software that could do great damage (kill people, lose people’s money). First, it puts guardrails in place against doing dumb things. Second, how will we know something went wrong and how will we prevent that from recurring.
- Automation helps reduce paperwork. Continuous delivery helps you collect data and enforce the rules.
- Findings from penetration tests (done by a human) can often be turned into automated tests to ensure you don’t get hit with the same finding twice.
A Radical Approach to Long Term Software
- Maintenance mode – demand for new work declines to low or near zero; work is focused on defect fixes, vulnerability updates, and platform upgrades. Can be called business as usual (BAU), keeping the lights on (KTLO), or support.
- The “you build it you run it” model is still valid for team structure.
- Smaller multi-service teams are created when demand gets low for a particular area. Delivery teams can now work on new things or be resized. The service reliability stays high because the team maintaining it contains some of the people that built it.
- The catch is that senior engineers don’t like to do maintenance work. A solution is to bring juniors in to the multi-service teams to share knowledge and let the seniors mentor the juniors.
- Guardrails for keeping multi-service teams from becoming dumping grounds:
- Have a consistent definition of low demand in your org.
- Give the team a clear identity and purpose. Tip: Name them after the product or vertical, not “maintenance team.”
- Document service transfer criteria (deliver team to/from a multi-service team).
- “…think about outcomes first, solutions second…”