Category: Product

RSS still works and is pretty awesome

Despite it being the age of social media, I get a lot of my content via RSS. I have the mac Reeder app which lets me subscribe to blogs. It is nice because without RSS I will see a blogger on Hacker News or reddit, enjoy their post and then their next post won’t be on the front page so I won’t see it. RSS fixes that because I can read their post and if I enjoy it, subscribe to their RSS feed. 

Most blogging software supports RSS natively. WordPress comes with RSS by default and so does Ghost. I find it pretty easy to use and really enjoy how it lets me control my ‘feed’ without influence from any algorithms or anything. 

Why not just subscribe to the email list? Well, I’ve found that I use my email inbox for different things than my RSS feed. When I’m going through emails, reading a long form email breaks my flow. Whereas when I’m going through my RSS feed I’m planning on reading longer form content. 

Spring into SledgeConf!

https://sledgeconf.dev

Our next SledgeConf is coming on Friday May 21st at 4:30pm PST!

Join us for discussion of the next 10 years of remote work. 

My presentation for this SledgeConf is titled. 

“The next decade in remote work.” 

I’m joined by Mason Traylor presenting!

“How to waste Everybody’s Time with Remote Meetings”

Agenda:

4:30 Introductions 

4:45 Presentation “How to waste Everybody’s Time with Remote Meetings”

5:00 Questions + Discussion

5:15 Presentation “The next decade in remote work.” 

5:45 Questions 

6:00 Promotional Round table 

6:15 Unstructured Discussion time

The zoom link will be sent to the mailing list the day of SledgeConf. Please go to SledgeConf.dev and sign up for the mailing list. If you are one of my connections on linkedin the zoom link will also be available there.

https://sledgeconf.dev

Software Leviathans

Dis-economies of scale, why FAANG pays high salaries, the dominance of Java

The top end of software engineering jobs are dominated by what I’ve started thinking of as ‘Software Leviathans’, large software systems that are staffed by thousands of engineers. A few that come to mind are Amazon Alexa, Amazon.com, Google Search, Salesforce, Facebook.com. These are not “monoliths’ or large services that do everything. Instead they are the result of combining 100s of smaller ‘micro-services’ into one massive software product. 

These leviathans do many many things, few people on the planet can claim to know all of the features of facebook.com. It is quite possible that there exists no single list that enumerates every feature in that product. 

Similarly, development on these systems happens in parallel across many teams. It it is essentially impossible for any one person to keep track of everything that is being added to the system. 

Leviathans are too big for anyone to understand. It doesn’t matter what architecture or runtime choices are made. It could be one massive JVM, a million lambda functions, a hundred thousand docker containers or thousands of micro-services. Even if you work on the leviathan, you won’t have any real understanding of the total state of the system. Each engineer will be aware of and communicate with a tiny fraction of the total number of people working inside the leviathan. 

Leviathans are heterogeneous systems. The do not do ‘one thing well’. Leviathans do everything you can think of. Google.com is a search engine, but it’s also a calculator, an advertising system, a web scraper, a hotel booking tool, a flight booking tool, and many more. Leviathans grow in parallel, across myriad tentacles of functionality. New features emerge all the time usually to the surprise of other engineers on the project. 

Leviathans are difficult to work in. Despite appearing to be a sea of constant change from the outside. Any change made inside the Leviathan is extremely expensive in engineering hours. There are thousands of potential interactions each engineering team has to consider when evaluating changes to their system. The architecture must be constrained heavily to support parallel development in environments where coordination between different teams is impossible due to scale. Engineers working on a software leviathan spend a relatively small fraction of their time actually writing code as compared to debugging issues, research, coordinating changes, and documenting. 

Leviathans are interesting because they are the ‘core’ services powering the digital world these days. Their scale is at top of the chart in the software engineering world and as a result they expose the limitations of software engineering. 

Software diseconomies of scale are at their most evident in these software leviathans. They are massive projects with huge numbers of the best engineers working on them. But development is slow per engineer and code quality is not clearly superior to industry best practices. 

Don’t use internal tooling, contribute to your tools.

There is a dichotomy in software engineering organizations, some only use public tooling, entirely avoiding building their owns tools. Other companies  follow the ‘Not invented here’ principle and try to only use software developed internally. 

There are two forces driving this split, first building software tooling is expensive, small companies often cannot afford to build and support internal tools. It is an expensive recurring cost that can easily get out of hand. Building 2 internal tools a year will set you on a path to supporting 20 tools 10 years from now eating huge chunks of your operations budget. 

The other force in play is that if you use public or off the shelf tooling you will encounter workflow discontinuities that are difficult to fix. Using off-the-shelf tool A with tool B might require an entire employee to bridge the gaps manually, while still be a pain for everyone involved. Decision makers at some companies think to themselves, “We can just build a tool B that works perfectly with our use case.” And that works great when you have one tool, but then when need C comes along your team  makes the same case again, “We already invested in custom tool B we can’t throw away that work, we need a new custom tool for need C”. And now your company is on the path towards building an alphabet’s worth of internal tools that aren’t useful outside of your business.

Luckily, there is a solution to this. Use Open Source tooling and when you run into a workflow problem, work with the maintainers to contribute a fix. Even in poorly managed projects extending the code to support your use case will be less costly than building an entirely new internal tool to solve the same problem. 

Software is a depreciating Good

“If customers are paying for the work, presumably by the hour, then you can’t expect them to pay for weeks (or more) of work every few years just to arrive back at the same product they started with.”
– From reddit user /u/PragmaticFinance

People have the idea that because the software has the same features as it did five years ago, it still is just as ‘good’ as it was five years ago. And if we can keep running it without modifications that is a reasonable practice. Unfortunately, while it feels intuitive that is not the reality with software. 

Sorry, a software product that is built on out of date tooling is strictly inferior to software built on up-to-date tooling. It is not ‘just as good’ it is significantly worse. The out of date software has increased vulnerability to security defects. Python 2.7 doesn’t get security patches anymore, if your software is built on it and a zero day is discovered. Your software could be unoperable for weeks while engineers upgrade it to function on Python 3. 

Software that can’t be patched to address security vulnerabilities in a reasonable time frame is strictly worse than up to date software. Anyone who was evaluating a purchase of these two programs would consider the expense of making changes. 

Aside from security issues, out of date software is harder to modify. What are the banks going to do when the last Cobol programmer dies? They will have to fund Cobol bootcamps which is not going to be cheap.

What are companies that had ‘feature complete’ software that didn’t need to change in years doing now that GDPR is a thing? That software is getting dusted off and updated or entirely replaced.

Roads and tractors require constant maintenance just to keep doing the same job. Software is the same. Maintenance costs should be estimated and included in the total cost of ownership for software. 

A lot of business software just encodes business processes. It can get out of date because the world shifted and the process changed, or the world shifted and the way the process is encoded is out of date.

Each time I write one of these technical debt posts it helps me understand why Software As A Service took over in such a big way. If you needed this year’s model of tractor every year you would lease it.