We all know the feeling. You have just started a new contract and its time to find out the truth about their software stack. Do they still use Ant despite the fact that Maven and Gradle have been options for over 10 years? Are you about to find out that their website is ‘really’ built on a 20 year old Pearl web framework that hasn’t been used for anything anywhere else?
Once you get past the initial shock of finding out what they doubled down on after it was wildly known to be a horrible idea. You get to find out about the internal tooling. Maybe you hear someone say, “we use the tools that are right for the problem”. Or maybe someone says “We don’t want to force people to use tools that are not a fit for their use case.” That is a sign that there might be more than a few internal tools lurking in the company wiki.
In your first week you hear about how we have our own web framework and try to figure out how dependency management works here. You find it somewhat nifty that you can submit multi-repository Pull Requests to their custom version of bitbucket, except with a few less bells and whistles. Alright, we have our own RPC framework that uses XML kind of like how SOAP used to work.
Then over the next few months you realize all the tools are custom. The company hasn’t taken advantage of any of the advances the rest of the industry has over the last 20 years. They have continuous integration and continuous delivery but it is all custom. The end to end test framework your project relies on was built by another team that disbanded quietly last year and is now entirely unsupported. You have the options of taking over the project (fuck no) or migrating to another promising internal end to end test framework. That team promises they will fully support it once they hit 1.0.
Name a tool used in the industry. Go ahead, name one. “Alright, Terraform”, oh, Terraform is nice, but we have an internal tool Declarative Template Manager which allows you to write templates in Pearl. DTM templates are faster and supported as a first class citizen in our companies closed source deployment system.
-you- “Cool, I need to track down this bug, how can I search the logs?”
-old time dev- “We use grep.”
-you- “Alright, how do I get to the logs?”
-old time dev- “WoodShed, were you keep logs, here is the link”
Its a link to a wiki page with the standard internal documentation you know and love.
-you- “So how do I get the logs out of WoodShed?”
-old time dev- “Well, people who know what they are doing use Wood Chipper. WoodChipper is the best tool to use to grep logs from WoodShed. “
-you- “I just want to do a quick search, is there a web portal that can do that?”
-old time dev- “Well, we have SawMill, it can pool all the logs for your teams servers onto one host so you can ssh into one place and use grep!”
Now you might be thinking no serious software company would allow their engineers to ssh into production hosts just to grep logs, but you would be wrong. After all Devops means everyone has ssh access in production. You might think that the top software companies would have a better solution than STORING ALL THE LOGS IN S3 AND THEN GREP’ING THEM.
If you thought any of those things you were wrong. Every company has some private tooling, over time that ‘some’ goes from a couple tools to dozens and then hundreds. The reason software engineers love startups is because they get to dump all the legacy code (older than 2 years) and build something new. Thereby escaping the Chuthulu-esque horror building up in the company codebase.