Tag: software engineering

The original launch of Copilot soured a lot of people on AI for coding

In 2023 I was one of the many software engineers whose management pushed them into trying Github Copilot. I was not impressed. At the time Copilot was basically pointless for Java developers. IntelliJ already had Intellisense which did everything Copilot did, but more deterministically. 

In retrospect autocomplete was not the use case for AI to assist in the programing process. Personally, I find the AI chatbots to be very useful coding assistants for writing scripts and functions. And this last year AI coding Agents have really come into their own. 

I’ve never been a python guy despite using it for light scripting tasks for a decade. But lately I’ve been using it a lot more because Claude can consistently create working 500~ line python scripts in a couple prompts.  The days when AI spit out code that didn’t even run are mostly in the past now. 

AI is just much better at coding now than it was in 2023. And I suspect the ‘autocomplete’ use case is just a bad one for AI. The more powerful models tend to produce output more slowly. You are looking at a 15+ second wait with edge models. In Claude Code and other agents, multiple 15+ second waits is pretty annoying. It’s like compiling huge java projects. For me I find the chatbot model works great. You write up a prompt, provide examples and context then Claude spits out a running program. You test it and iterate. Autocomplete has the problem that edge models are never going to have the latency that you want when you push ctrl-shift-enter. It just feels better to use an agent or chatbot. 

Code Without Learning to Code

https://codewithoutlearningtocode.com

I’m working on a new book on Vibe Coding for people who don’t know how to code. Using LLMs to build simple programs unlocks a lot of programming ability for people who either tried and failed to learn to program or never got started. You no longer need to learn the basics of programming logic or syntax to build useful programs to solve your problems. 

The current target of the book is people who don’t know how to code but are willing to learn to run and test computer programs they create through Vibe Coding. 

As part of this process I’m doing some research comparing free and paid LLM models for programming use. 

For each model I pasted the same prompt in and took the first result. I saved the code into a folder which already had pygame installed via pip and ran it directly.

“Please create a game for me using python and pygame. In the game the player should navigate a 2d space using the arrow keys. In this game there should be a maze like region with rocks and stalagmites. Inside the region should be chests which contain gold. The player should be able to navigate the maze and collect gold from the chests.”

Anthropic Claude Haiku 3.5 (free)

https://github.com/Sevii/vibecoding/blob/master/MakingGames/BlogPost_LLM_Comparison/haiku35_chest_game.py

Anthropic Claude Sonnet 3.7 (paid)

https://github.com/Sevii/vibecoding/blob/master/MakingGames/BlogPost_LLM_Comparison/claude37_sonnet_chest_game.py

Gemini 2.0 Flash (free)

https://github.com/Sevii/vibecoding/blob/master/MakingGames/BlogPost_LLM_Comparison/gemini_2_flash_chestgame.py

Gemini 2.5 Pro Experimental (paid)

https://github.com/Sevii/vibecoding/blob/master/MakingGames/BlogPost_LLM_Comparison/gemini_25_pro_experimental_chest_game.py

ChatGPT (free) 

https://github.com/Sevii/vibecoding/blob/master/MakingGames/BlogPost_LLM_Comparison/free_chatgpt_chest_game.py

It’s interesting to see how paid models differ from free models. But we are getting working code on the first pass from both free and paid models. 

Agile has gone from ‘people over process’ to ’no process’

In practice ‘agile’ means ‘we have no process in place and each team does whatever random thing the manager wants to try next’. Sometimes that is SAFE sometimes its SCRUM, usually it’s a combination of different things. This isn’t necessarily a bad thing, but there are trade offs. 

The first is standardization. If every team follows a different process it’s difficult to understand what is going on at a management level. Which teams are productive? Which teams are in downward spirals? If you don’t have a standard to judge against you can’t find out. 

Secondly, away team work is much harder. Working with a team that uses the same development process, pipeline setup, programming language and frameworks is easy. On the other hand working in the code base for a team which uses a different language, framework, architecture, etc is very difficult. Not supporting away team work severely limits your ability to integrate internal software components.

Thirdly, Estimates are not possible in this kind of environment. Since the process changes constantly historical data becomes useless. In response to this most companies don’t even keep historical data. The main use for estimates is ensuring that ‘burn down’ charts follow the 45 degree angle managers love. 

https://www.estimating.dev/there-is-no-team-calibrating-teams-vs-individual-estimators/

Subjective expert predictions are a valid form of estimating software tasks. But if you don’t have historical data to calibrate against estimates devolve into gaming the system. 

When you change how estimates are made, when tickets are considered done and the sprint cadence every 3-6 months there is no way you can have cross company data on productivity. The lack of process empowers management to obfuscate the productivity of their teams. The pursuit of the best process gives technical organizations a great excuse as to why they have no idea if their processes have improved over the last two years or not. 

In this type of environment all judgements have to be made based on subjective gut feelings and corporate politics. You don’t know which VP’s processes are better than the other because neither has any accountability. You don’t know which technical department is more efficient because neither the estimates or logged hours can be trusted. 

‘We’re being agile’ has become the excuse to follow whatever process you want. Instead of ‘people over process’ it has become  ‘no process’. 

What is your design budget

We all want to have good longterm software architecture. Build it right the first time. But some organizations fall into the trap of trying to get a perfect design before they start building.

They include more stakeholders, try to plan for contingencies, develop a high availability strategy. 

Those things are all good to have. But you don’t need them before you have users. If you have no users you shouldn’t be thinking about high availability. If your product doesn’t work yet you don’t need scalability.

Users > working product > scaling limitations > high availability

Another problem is having a committee to approve architecture proposals. One or two people is fine. But having three or more people who can block the start of the project is a recipe for pointless delays.

The reason we switched to agile methodologies is because it’s hard to know what the difficulties will be before you start building the software.

Be careful to structure your organization such that design doesn’t become more important than delivery.

A working product with customers is always more valuable than a product that doesn’t have customers but is highly available.

A design budget is a way to avoid falling into this trap. Simply budget a week or a month or whatever for designing up front. At the end of that time enforce a hard stop on doing more design. No more committee reviews or deliberation. If the architecture isn’t developed enough to start building, do a spike on a smaller scale. 

Away team work

Away team work is a critical component of high performance software organizations. It is a way for high priority teams to work around other teams’ manpower constraints to deliver software. Without a well established culture of away team project work your organization will default to a standard of ‘shut up and wait while your item is in the backlog’. 

What is away team work exactly? Away team work is when your team implements a feature or integration in another team’s codebase. Strictly speaking in away team work the host team does not review or approve pull requests. Your team’s senior engineers will get approval at an architectural level then handle pull request review themselves. 

The Away team process allows your team to unblock itself when dependencies don’t have manpower to spare. The key component is first that your team not the dependencies team does the work. Both the implementation, testing and code review. Some effort obviously is required of the host team, but it should be minimized. 

Away team work acts as a release valve on the conflict between the host team’s priorities and potential client’s goals of delivering features to customers. If the host doesn’t have manpower available your team can provide the engineers to make it happen. 

Without away team work your organization will have to make more tradeoffs on the scheduling side. More often than not you’ll have to pick a migration over the new feature. But Away team work would have enabled your organization as a whole to deliver both. 

What do you need to make away team work happen?

The key thing is that you have to make a formal process that specifies the maximum standards. You can’t allow teams to be squeamish about it. If the requirement is that two senior engineers approve that’s fine. Or maybe only senior engineers can do away team work in your company. Thats fine but you need a formal standard to avoid negotiations happening on a per project basis.