Category: Uncategorized

The original launch of Copilot soured a lot of people on AI for coding

In 2023 I was one of the many software engineers whose management pushed them into trying Github Copilot. I was not impressed. At the time Copilot was basically pointless for Java developers. IntelliJ already had Intellisense which did everything Copilot did, but more deterministically. 

In retrospect autocomplete was not the use case for AI to assist in the programing process. Personally, I find the AI chatbots to be very useful coding assistants for writing scripts and functions. And this last year AI coding Agents have really come into their own. 

I’ve never been a python guy despite using it for light scripting tasks for a decade. But lately I’ve been using it a lot more because Claude can consistently create working 500~ line python scripts in a couple prompts.  The days when AI spit out code that didn’t even run are mostly in the past now. 

AI is just much better at coding now than it was in 2023. And I suspect the ‘autocomplete’ use case is just a bad one for AI. The more powerful models tend to produce output more slowly. You are looking at a 15+ second wait with edge models. In Claude Code and other agents, multiple 15+ second waits is pretty annoying. It’s like compiling huge java projects. For me I find the chatbot model works great. You write up a prompt, provide examples and context then Claude spits out a running program. You test it and iterate. Autocomplete has the problem that edge models are never going to have the latency that you want when you push ctrl-shift-enter. It just feels better to use an agent or chatbot. 

Code Without Learning to Code

https://codewithoutlearningtocode.com

I’m working on a new book on Vibe Coding for people who don’t know how to code. Using LLMs to build simple programs unlocks a lot of programming ability for people who either tried and failed to learn to program or never got started. You no longer need to learn the basics of programming logic or syntax to build useful programs to solve your problems. 

The current target of the book is people who don’t know how to code but are willing to learn to run and test computer programs they create through Vibe Coding. 

As part of this process I’m doing some research comparing free and paid LLM models for programming use. 

For each model I pasted the same prompt in and took the first result. I saved the code into a folder which already had pygame installed via pip and ran it directly.

“Please create a game for me using python and pygame. In the game the player should navigate a 2d space using the arrow keys. In this game there should be a maze like region with rocks and stalagmites. Inside the region should be chests which contain gold. The player should be able to navigate the maze and collect gold from the chests.”

Anthropic Claude Haiku 3.5 (free)

https://github.com/Sevii/vibecoding/blob/master/MakingGames/BlogPost_LLM_Comparison/haiku35_chest_game.py

Anthropic Claude Sonnet 3.7 (paid)

https://github.com/Sevii/vibecoding/blob/master/MakingGames/BlogPost_LLM_Comparison/claude37_sonnet_chest_game.py

Gemini 2.0 Flash (free)

https://github.com/Sevii/vibecoding/blob/master/MakingGames/BlogPost_LLM_Comparison/gemini_2_flash_chestgame.py

Gemini 2.5 Pro Experimental (paid)

https://github.com/Sevii/vibecoding/blob/master/MakingGames/BlogPost_LLM_Comparison/gemini_25_pro_experimental_chest_game.py

ChatGPT (free) 

https://github.com/Sevii/vibecoding/blob/master/MakingGames/BlogPost_LLM_Comparison/free_chatgpt_chest_game.py

It’s interesting to see how paid models differ from free models. But we are getting working code on the first pass from both free and paid models. 

Reducing toil with AI

LLMs are at the point where they can reduce toil for software engineers across a host of use cases. Here we will explore a few I’ve thought of.

Reduces time spent on toil 

  • Resize this button -> AI
  • Refactor this function -> AI 
  • Connect these endpoints -> AI 
  • Write more tests -> AI 
  • Add more comments -> AI 
  • Create PlantUML diagrams from a sketch -> AI 
  • First pass code reviews -> AI

Reduces time spent researching small things

  • AI is a better stack overflow
  • Examine stack traces 
  • Easier to write architecture documents 
  • Faster development of small utilities

What AI still cannot do

  • Test if a library will work for your use case 
  • Respond to outages 
  • Decide the product direction
  • Argue with stakeholders 
  • Yell at people who want to do stupid things

A few opportunities to reduce operational toil

  • AI can review graphs and notice changes
  • AI can check if a website is down
  • File bug reports 

For me the most valuable use of Claude as a coding assistant has been how it makes getting started much easier. Usually, to program a swift game I’d have to spend a couple hours breaking into swift development and building up my program off of examples. Claude was able to create a basic version of the game I wanted off of a detailed prompt. It didn’t manage to create the game in one shot, I had to edit the code a bit to get it to run. But it turned a project that would have taken me a few days into one that took a few hours.

Links October 2024

Great article on mac setup

Lots of tools are outdated by default. Got to take advantage of this blog post since I got a new laptop this summer.

https://matt.sh/setup-2021-late

Operating Systems use out of date assumptions

Interesting talk on how modern systems on a chip differ from how we imagine computers to actually work. Over the last two decades hardware has gradually abstracted away many features from the actual operating system. The computer board is now a collection of sub-computers which cooperate to mimic the operation of an actual traditional computer. 

The system describe here is reminiscent of a micro service environment where you need to communicate with many different protocols to get the job done. I wonder if we can explore migrating ‘cloud’ microservice techniques down to the CPU level. 

Passive Radar for finding meteors

You can detect meteors using a set of clock synchronized radios. The way it works is you monitor a reference frequency at perhaps 180MHz and can detect changes in it as meteors burn up in the atmosphere. 

https://en.wikipedia.org/wiki/Passive_radar
https://britastro.org/wp-content/uploads/2019/11/Detection_of_meteors_by_RADAR.pdf

A clear sign you are overdoing microservices 

Fine grained services

Microservices have been the thing for over 15 years. They are great in large companies with CI/CD environments. But as your situation drifts farther away from the ideal microservice use case traps abound. 

Building a new service for one endpoint 

If you find yourself having a conversation where you need to create a new endpoint somewhere, but adding it to any of your existing services would break the concept of that microservice. Turning it instantly into a ball of mud with no clear purpose. You have fallen into this trap. microservice does not mean each service has only one HTTP endpoint. That use case is better served with Cloud functions like AWS lambda. 

The problem here is that we have gone too far in splitting up the monolith. Splitting a monolith with 100 HTTP endpoints into a dozen or so services with eight endpoints each is great. Splitting up a monolith with 100 endpoints into 100 services is counter productive. Instead of having an actual purpose the single endpoint microservice becomes the xyz endpoint microservice. 

Endpoints are things that microservices empower. An endpoint in of itself should never justify the creation of a microservice.