Category: Uncategorized

Few of the software leviathans are built in functional languages. Facebook uses PHP/Hack, Google Java, C++, Amazon Java, Netflix Java. The common consensus about functional languages is that they provide large benefits over object oriented and procedural languages like Java. One particular claim is that functional languages like Haskell can do the same work in 1/10th the lines of code. If functional languages really are better we would expect to see the big tech companies investing heavily in adopting functional languages. We might even expect them to create a functional programming language just for their use case, but instead Google created Go possibly the least functional programming language created in the 21st century. What is going on here? Why aren’t functional programming languages being adopted in the biggest software systems on the planet? 

People have argued that inertia is the explanation for the low adoption of functional programming languages in massive software projects, but I think the evidence is in the opposite direction. Google created an entire new language that was intentionally less functional than Java. Facebook started on PHP and then extended that language into Hack. They could have used that energy to completely adopt Haskell. 

My suspicion is that the real reason functional languages are not used in massive software leviathans is meta-programming. Meta-programming enables software developers to create custom domain specific languages, literally adding new programming syntax and expressions to the code base. This is an incredible power and can make a lot of problems much easier. But meta-programming does not scale.  

In a software project with 10,000 software engineers. At this scale the limiting factor is not our ability to write clean and concise code. The main issue is understanding the effects of changes to the code base. A change might take a month to research before changing 500 lines of code. Not doing your research upfront more likely then not will result in you starting the project than realizing 2 weeks in that your approach will never work. Then having to start over. 

Meta-programming falls under the set of programming constructs that are easier to write than they are to read. This is true for all code of course, but in large code bases reading Golang code is reliably easier than reading Lisp code. 

In a algorithmic metaphor, Golang code complexity scales at O(n^2) vs Lisp code scaling at O(n^3). 

I’ve been an avid twitter user for years, but had to stop this winter. I have been listening to ‘Deep Work’ while driving cross country and have done a lot of thinking about how to do better work. One of the things recommended in the book is to quit social media or at least exclude it from the part of your day when you work. I’ve typically just blocked twitter from my network during the workday then used it as much as I wanted afterwards. 

Well another thing I did in the pursuit of ‘deep work’ is to review my Qbserve stats for the last few months. My twitter numbers were way higher than I expected. I have been spending thousands of dollars worth of time using Twitter producing fun content that twitter then monetizes. I could have gotten a part time job or learned to paint. 

Track your time. There are a bunch of apps that can do it. I use Qbserve because it stores data locally and felt like a less heavy weight solution. I have also used RescueTime, but found logging in again when I need to restart tracking to be a pain. 

Once you have tracking going it gives you a lot of insight into what you are doing on your computer. Some people might think “ah, if I’m on the computer I’m working, what else would I use it for” but for millennials and digital natives who spend most of their lives on a computer it can really help. 

For example I know how much clock time I spent reading Xianxia, translated chinese pulp fiction, on wuxiaworld.co this year, four whole days. That is nearly double the amount of time I spent on news.ycombinator.com which came in at 1 day and 13 hours. I also know how much time I spent writing, note taking and journaling this year, around 30 hours so far. Admittedly, I haven’t run the app 24/7 and didn’t start until March so I only have around 8~ months worth of data.

I don’t think I would have made the realization of how much time I was spending on twitter, without a time tracking app. It is a lot like Television for normal people, it is just on all the time when you are home, you don’t really think about it’s effects on your life. Most people underestimate how much time they spend watching television, but you don’t have to underestimate how much time you spend on Youtube, just get Qbserve and review the data occasionally. 

In the week or so since I quit, I’ve already read a couple books and started writing on my blog again. 

I’ve dealt with burnout many times in my 6 years as a software engineer. Usually, it’s when I get bored of a project or there is a slow period where I don’t have a lot of work to do. Counter to what you would expect having less work makes it harder to get that work done than when there is more to do. I think it is because when there is less of it, the work feels less important and subconsciously it feels like I’m not really needed. Like when you are in a meeting trying to estimate the impact of an issue with the whole team, but only three out of ten people are actually able to do anything before people are duplicating work. Then you end up waiting for other people to do basic things like read logs and tell you what they say. You could just read them yourself, but do we need 3 people reviewing the same logs right now?

I intentionally stayed on my current team for a relatively long period of time just to see what it was like. Earlier in my career I worked as a consultant where my longest time on one project was 9 months. Here at a product company we have been working on essentially the same problems for years. This is great in a way because I have been able to develop deep expertise in my systems and tooling, but the cost is of course burnout. 

The pandemic has made this year significantly worse by forcing remote work. I’ve lived in studio apartments since college and rely on having an office to provide a distinction between working and other activities. Efficiencies, like eating at my desk, which make sense normally, serve to muddle work and play when everything happens in a 500sqft box. 

Having everything muddled together makes it much harder to maintain flow. The absence of which makes everything more difficult. Especially, when your general happiness is influenced by your self-perceived productivity and usefulness as mine is. A large reason for my career success so far is how I maintain focus in the office. I don’t let myself do certain activities in the office like use Facebook, Reddit, Twitter or almost anything non-work related. Figuring out how to extend those norms to a single room lifestyle has been very difficult. 

I haven’t been able to wait out burnout. In the past a team or job change alleviated the problem. This year it just got worse and worse by the end of my stay in Seattle I didn’t want to fix it.  

The good news is that I left Seattle, living there has never felt right to me, the winters are horrible. In the short term I will be itinerant, but eventually I will acquire a new permanent space which will be larger. I’m hoping to move into a house or condo, but might end up in a one or two bedroom apartment with a dedicated office.

Fixing one part of my life that I knew I didn’t like has helped. It hasn’t fixed everything, but I’ve had a lot of time to think of ways to improve my working situation which I think will pay off.  

Here are some links I have seen lately that were pretty interesting.

In window noise canceling speakers

They will integrate these speakers into windows/walls and make it smaller. Increasing the quality of life in dense cities. It is something thats needed since cities just keep getting larger.

Concept of ‘feature store’ for typed ML model inputs (tensors, vectors, etc)

https://www.logicalclocks.com/blog/feature-store-vs-data-warehouse

VM performance tests

Finally we have a pair of great posts from tratt.net about VM warmup with lots of data.

https://tratt.net/laurie/blog/entries/why_arent_more_users_more_happy_with_our_vms_part_1.html

https://tratt.net/laurie/blog/entries/why_arent_more_users_more_happy_with_our_vms_part_2.html