Polyglot programming is a bad goal 

People have extended the goal of building micro service systems such that we aren’t locked into a language to a new goal of every repository being able to use any language.

It’s phrased as ‘shouldn’t we use the best language for the job.‘ But the implication is that we should be able to pick a language for this new project independent of the rest of our existing software stack.

The problem is that it’s not a free choice. Even if you’re in an ideal microservices environment where everything is API based, logging and metrics are handled by the platform and shared libraries are banned there are costs to having more languages in play.

The most basic cost is expertise splintering. If you only use one language and everyone knows it. Then everyone can read and extend all the code. But as you start throwing more languages and paradigms in that slowly becomes impossible.

You quickly reach a point where transferring people to a new team is expensive. Transferring from one Java project to another in the same stack is easy. Switching from Java to async JavaScript is trickier. Switching from Java to Haskell takes awhile.

Polyglot makes away team work more difficult. And it does it slowly so you won’t really notice until it’s too late. 

Shared libraries are another area polyglot sucks. You can’t always make a library an API. 

Whether it’s because you can’t externalize metrics and logging or you have high performance business logic you need to run locally. Often times you will need to duplicate shared libraries across languages.

Polyglot is nice in a lot of ways. I’d rather get to use Pandas for data science if I had the opportunity. We don’t really need to use only Java for everything. But when it comes to application code, I’d rather everyone used the same language, framework and RPC schemas. It doesn’t make any sense to write certain microservices in Java, others in Haskell, some in Python and a few in Typescript. If they are RPC or event based services you are just using five languages to do the same job instead of one language to do one job. 

Then go ahead and write sidecars in Go or Rust, frontends in Typescript and data science stuff in Python. Those are all different use cases with different trade offs. 

Design Documents for Distributed Companies

How do you hash out architecture and design at a remote only company? Slack chats and phone calls are a slow way to communicate with an entire team. And miscommunication with people 1000s of miles away can cost you the time and money you were hoping to save by working remotely. 

The Design Document workflow

Some companies have a design document centric workflow. At the beginning of a project instead of writing Jira tickets or tasking out stories, an engineer will spend some time writing a 3-10 page document which describe the plan of attack for the project. This doc will include assumptions, design decisions and constraints, as well as a summary of the changes to be made. The document is not an exhaustive specification, but can include proposed interface or library changes. Then the team will have a review session to read and ask questions about the design. Next after some revisions, and possibly a follow up session, development will begin. 

Google Docs or any other shared document tool is sufficient to support the document review process. If you want to make the process truly asynchronous you could rely only on document comments and skip the meeting all together. 

Typically, a design document should be required both embarking on any significant project or feature addition. Adding a new button to a web form would not need a document, but building a new web service or adding a major feature would. 

When I worked in a distributed consulting company we used this workflow for proposals and Level of Effort estimates. But software design was usually handled by discussion and whiteboarding followed by implementation with half-hearted documentation occurring at the end of the project. A design document process would have helped share design between the east and west coasts keeping more of our projects on the same page.  

Software Leviathans and the weird dominance of good enough.

One day in Spring 1989, I was sitting out on the Lucid porch with some of the hackers, and someone asked me why I thought people believed C and Unix were better than Lisp. I jokingly answered, “because, well, worse is better.” We laughed over it for a while as I tried to make up an argument for why something clearly lousy could be good.

https://www.dreamsongs.com/WorseIsBetter.html

It has long been wondered why Java took the crown for the ‘enterprise’ language. I can’t really argue on that topic since I came onto the scene long after Java was all there was. This article is about why software leviathans are written in Java more than anything else. 

You have a huge software project to build. What language do you build it in? The prototype was written in ruby on rails by one guy and an Adderal prescription. Now they want you to scale this thing to a 1000+ engineers over 5 years of development. You might think “aha this is my chance, lets save an order of magnitude lines of code and use lisp”, except this story happened in the past and they chose Java. 

Why is it always Java? Sure it’s reasonably fast, but Facebook made PHP work, can’t we at least use Haskell? Since we have the benefit of hindsight, we know that most of the biggest software systems are built in Java. Google built so many leviathans in Java that they bankrolled a new language like Java but with less features. Amazon is based on Java. Netflix is java again. Facebook made their own language and Microsoft is old enough to have existed before Java, but still made their own version of Java, C#. 

The real question should be, “What is Java’s secret?”. 

Java requires a lot of boiler plate

Java just plays well with the major constraints in a software leviathan and at Leviathan scale that is all that matters. 

This one is the corollary of “Java doesn’t support meta-programming”. Creating your own DSL is great, 1000 engineers creating their own DSL is 999 nightmares. Software Leviathans are too big for any one engineering team to understand. Any DSL you create makes your code unintelligible to the rest of the people working in hell with you. I can understand boilerplate written by a monkey, but a DSL written by another software engineer could take me days to understand. When your team gets poached to go work on a startup where the code base isn’t humongous, it’s a lot easier to bring in Java programmers to replace you lot than it would be to get Haskell engineers to figure out your undocumented dialect. 

Google got to the point where they figured Java had too much meta-programming ability so they created Go which is basically Java without inheritance. That is what happens when you work in a leviathan project. You begin to resent the ability of your peers to do anything unusual, because you know it’s just going to be more work for you. 

Adding more onboarding time to understand 1) the functional language and 2) the DSL your team created might push our already long 6 month on-boarding period closer to the 1 year mark. I wrote an article about onboarding time and functional languages aimed at startups, but honestly I don’t think the hiring market is the real reason Java dominates the top end. FAANG is already willing to train new grads to work on their giant software projects. 

It boils down to comprehension honestly. Humans can only comprehend so many things and at leviathan scale the max is a tiny fraction of the entire system. 

In a software leviathan your team constantly works with other teams’ systems. How does this API work? There isn’t any documentation and one 30 minute office hours isn’t going to explain that hair ball. If you all use the same language and that language is Java there is a chance you can open up their code base and figure out what is going on. They probably didn’t do anything you wouldn’t expect like pre-allocating all of their memory and storing all objects into a ring buffer. But if they did do something crazy you can probably figure it out. Besides Java doesn’t have anything like Scalaz so you won’t be surprised by a functor where you weren’t expecting it. 

Lets take the opposite side, away team work. You have been given the glorious task of implementing a new feature. But it’s impossible to do it cleanly without an API change in another team’s system. That team fully supports the change and has contributed 2 paragraphs to your architecture document describing the change to make in their system. But the change isn’t on their team’s roadmap so you are going to have to do it. 

Getting their service to run and pass integration tests in your virtual development machine takes a week. Now you need to navigate their system where they have conveniently used dependency injection to ensure that you can’t know which of the 5 implementations of this interface is in play. Do you still wish the other team could use Clojure? You might never figure out the DSL. 

Have you ever looked at somebody else’s Lisp code and wondered what was inside the variables? Now imagine this is your job and you will spend the next month making a 200 line change to a 100,000 line of code API service you didn’t know existed until this week. Except this will happen every quarter for the rest of your career. 

People complain about how Java forces you to write the type of things everywhere, but for software leviathans this is a benefit. I can see helpful type signatures everywhere, whether I’m reading your code in my IDE, an email, an excerpt in an arch doc, or in a Slack message you sent me at 3 am. 

Java and Go are great in Software Leviathans. You don’t have to worry about stumbling upon a programming mystery created 10 years ago by a disgruntled new grad. You can expect a consistent syntax and language whichever microservice you are working on. The code has self-documenting types that are ‘easy’ to understand. Honestly, they are a lot of benefits which make a tough coding environment a little more manageable.

Expecting end users to customize the experience is madness

 Don’t do it to yourselves

Don’t do it to customers 

Do the work to make a good product

Enterprise software sucks. Its not bought by the people using it, but by a guy wearing a suite on the 37th floor the day after eating a fabulous steak dinner paid for by Oracle sales guys.  By the time you start using it, it is bought and paid for. Suck it up and learn how this pile of code works. 

Internal enterprise software is another beast. Constantly underfunded, built by interns that just learned object oriented programming, and designed by the CEOs cousin, it is not the greatest. 

Know what will ensure that your internal software is never improved in a meaningful way? Make customizing it the default workflow. Just have every engineer at the company load up a GreaseMonkey script that adds in the features that PAAS should have by now. 

The problem is fixed for the graybeards. Sure, every new employee will spend six months realizing that all the people who are getting anything done have customized the UI so extensively its not recognizable as the same product. 

When they said go use ‘deployment ladder’, they meant use ‘deployment ladder’ with 12 GreaseMonkey scripts installed. Where are those scripts? You might ask, the answer is always ‘in the wiki’. Searching for the name of the thing in the wiki does not result in finding the thing, like it would in google for an open source project.

Having everyone customize the software does not result in a good product. It papers over a shitty UI by fragmenting it even more. After a while no one with any power in your organization realizes there is a problem because they have 50 GreaseMonkey scripts installed, and haven’t looked at the actual ‘base’ UI in 5 years. 

Save yourself millions in on-boarding. Invest in good tools. Put the work into offering a great default workflow. Don’t end up in a situation where the graybeards can’t even understand the workflows the new hires are dealing with.