Why don’t people use functional languages more.

“There are only two kinds of languages: the ones people complain about and the ones nobody uses” — common quote

This is a fun statement if you think about it. Because the languages everyone complains about are obviously better than the ones nobody uses. It seems controversial but a simple thought example shows why it has to be this way.

Imagine a world where the mainstream languages were worse than the niche languages like Haskell or Crystal. In this world Haskell has a clear 2x productivity advantage over Java. Since Haskell is clearly better than Java most projects that use Haskell will execute roughly twice as fast. Double productivity is enough that companies using Haskell will be able to achieve the same results with less money spent on developers and thus achieve higher profits. We should see companies that don’t use Haskell going out of business. Eventually we would expect all greenfield development to happen in Haskell. 

You might say well, “Haskell isn’t really twice as good as Java, it is only 20% more productive overall”. Well, 20% averaged out over hundreds of thousands of software engineers is a huge amount of money. My employer would very happily take a 20% increase in productivity if it were on the table. 

However, in the real world we don’t see all greenfield development happening in Haskell. Instead lots of greenfield development happens in Java, Javascript or the .Net ecosystem. 

There has never been a consensus moment in the industry around niche functional languages. Instead people say they wish they could use them while making actual money writing Java code. 

Personally, I would really like to use a better language than Java. Unfortunately, we don’t seem to live in that world. 

Expecting people to become ‘Senior’ Engineers in 3 years is ridiculous.

Becoming a ‘Senior’ engineer in three years is a ridiculous concept. Programming is a complex enough endeavor that you can’t learn the basics of most areas in 3 years of work experience. By the 3 year mark great programmers will have achieved a high level of expertise in one programming domain. For example 3 years might be enough time to master web development with Django. But web development with Django is just one of many domains within computer programming. Mastering Django does not require a software engineer to learn anything about operating systems, low latency computing or graphics computing. In some environments web developers might not have any involvement with DNS or how their software is made available to users as deployment is handled by a infrastructure as code team. 

 In my case I was promoted to ‘Technical Architect’ and team lead in under three years at my first job. I was really good at my job and worked well in the consulting business. At that point in my career I knew a lot of stuff application development, frontend development, automated testing, devops, etc. But I felt under qualified and wanted more money so I ended up taking a demotion to move to FAANG. 

Why can people get promoted to ‘Senior’ Engineer so quickly in the software industry? The main reason is the expansion of the workforce. With double digit employment growth each year, there are a lot of inexperienced programmers in the workforce. Having three years of experience is a big deal in an environment where a large portion of working programmers have under three years experience. 

The issue is that three years is not really enough time to master software development. Honestly, in three years you are more of a ‘journeyman’ programmer than a senior one. You wouldn’t expect a three year senior to be able to bootstrap a new engineering team or to set the engineering culture at a new company or division. There is simply too much to learn about how software works. 

In my particular case, I felt that my ability to deliver projects was great. But when it came to designing software abstractions or building software architecture, I felt like I didn’t know enough to do a good job. I had read books like ‘Domain Driven Design’ on software design patterns, but knowing patterns and actually designing software are not the same thing. In my opinion software agency work is not the best place to learn software architecture design. You simply don’t stay embedded on projects long enough to see the longterm costs and evolution of your designs. 

Design Documents for Distributed Companies

How do you hash out architecture and design at a remote only company? Slack chats and phone calls are a slow way to communicate with an entire team. And miscommunication with people 1000s of miles away can cost you the time and money you were hoping to save by working remotely. 

The Design Document workflow

Some companies have a design document centric workflow. At the beginning of a project instead of writing Jira tickets or tasking out stories, an engineer will spend some time writing a 3-10 page document which describe the plan of attack for the project. This doc will include assumptions, design decisions and constraints, as well as a summary of the changes to be made. The document is not an exhaustive specification, but can include proposed interface or library changes. Then the team will have a review session to read and ask questions about the design. Next after some revisions, and possibly a follow up session, development will begin. 

Google Docs or any other shared document tool is sufficient to support the document review process. If you want to make the process truly asynchronous you could rely only on document comments and skip the meeting all together. 

Typically, a design document should be required both embarking on any significant project or feature addition. Adding a new button to a web form would not need a document, but building a new web service or adding a major feature would. 

When I worked in a distributed consulting company we used this workflow for proposals and Level of Effort estimates. But software design was usually handled by discussion and whiteboarding followed by implementation with half-hearted documentation occurring at the end of the project. A design document process would have helped share design between the east and west coasts keeping more of our projects on the same page.