Vendoring and Dependency security 

In 2019 it has become standard practice to download dependencies straight off of the internet without verifying the code is secure and free of bugs. Software Engineers typically look up a dependency on github and follow the instructions to add it immediately to their project’s dependencies. More thorough engineers might take a few minutes to read through the code before adding it to their package.json file. 

This era of downloading dependencies off the internet without worrying about the security or integrity of the code has coincided with several notable hacks. The event-stream hack [1] which injected malicious bitcoin stealing code into a library used by thousands of NPM packages is only the latest in a series of hacks which take advantage of our loose standards for adding packages to our projects. 

Vendoring is the practice of embedding dependencies, binary or source code, into our codebases. The practice was popularized by the Go language, but had been used internally by Google and several other companies before that. Vendoring your dependencies adds an additional step, of committing new code to your repository, before new code can enter your software. And if you have all the source code for your dependencies vendor in your repository, you are at least able to verify all of the code in your system. 

Google stores all dependencies inside its custom version control system and has a process to vet new dependencies before they can be consumed by engineers. But most companies I have worked for had no restrictions on what dependencies could be added to a project whatsoever. 

The current status code of trusting without verifying code will continue to result in major hacks. The open-source ecosystem is too big to audit. And most companies do not even attempt to vet their dependencies. The many libraries created by independent developers which make open-source great are the least likely to be audited and most susceptible to custody hacks like what happened in the case of event-stream. 

 

[1] https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident

How long does interviewing take?

As a new Software Engineer it is hard to know how much time the job search should take. Most people  start by applying to 2-3 jobs and wait for recruiters to get back to them. Then 4 weeks later they assume that because no one got back to them they must not be very appealing on the job market. 

The reality is that if you are cold applying to job openings you will have a pretty low response rate. The most desirable engineers in the market get around a 50% response rate. Entry level engineers have to deal with a response rate closer to 10%. That means that for every ten jobs that you apply for, a recruiter will follow up with you once. 

With such a low success rate you have to adjust your approach to spend less time on each job application. I spend about 15 minutes customizing my cover letter for each job. Then upload the resume and cover letter. Once that is done move on to the next job application. Too much customization is counter productive because you could spend two hours customizing your resume and a recruiter might not even look at it. It is more effective to apply to large numbers of jobs with the same resume and cover letter than to apply only to a few. 

Once a recruiter gets back to you the recruiting process really starts. Typically a recruiter will email you and schedule a quick phone conversation where they make sure you know about the position and confirm a few things about your experience and goals. This conversation is usually low stress and unless the recruiter gets the impression you don’t really want the job you will move onto a technical phone screen. 

Some companies will ask you to solve a coding challenge which is essentially a timed leetcode style problem. The problems are similar to what you would be asked in a white boarding interview so your studying will carry over. Hackerrank and leetcode have an almost identical experience to most interview coding challenges to use them to prepare. 

Other companies will ask you to attempt a coding project. This is usually a relatively simple programming project that is intended to be unique enough that there are not any examples online. I have done projects ranging from creating a basic backend REST service to creating a full fledged cryptocurrency exchange. The key to these projects is to be ruthless about keeping the scope down, otherwise the projects will take up too much of your free time. 

The final obstacle before you go to the on-site interview is the Technical phone screen. This phone screen is typically done over video call with a shared notepad that you can write code on. There are some systems like coderpad.io that allow you to edit and compile code with your interviewers. You will be interviewed by 1-2 engineers at the company. Most of the questions will be about software engineering and this will include more leetcode style questions. 

If you made it past all those interviews you will finally end up with the on-site interview. The onsite interview will involve a mix of behavioral and white boarding questions. Every company does it a little bit differently, but it typically takes between 2-6 hours and is pretty exhausting. Make sure to sleep well and prepare thoroughly. 

How much time do all these interview take? I put together a list of companies that I had onsite interviews with and how long it took in total to decide whether I would get an offer. 

These 5 interviews resulted in 2 offers (22.5 hours per offer). Getting a new job takes a lot of work, that is why people stay in jobs that they do not like for so long.

Gusto

  • phone screen 2 hours
  • onsite 6 hours

Ibotta

  • coding project 6 hours
  • phone screen 2 hours
  • onsite 6 hours

GoSpotCheck

  • onsite 2 hours
  • phone screen 1 hour

Amazon

  • coding test 1 hour
  • onsite 5 hours

Salt Lending

  • coding project 12 hours
  • onsite interview 2 hours

Make your bosses look smart

What does it mean to make your boss look smart? At a basic level do a good job. Beyond that you want to make things on your boss’s wishlist happen. Fix a bug that is low priority but your boss wishes he could get it done now. Refactor a class that your boss or the lead engineer has mentioned being frustrated with. Don’t make the mistake of fixing something that you think is broken. Focus on things that you know your boss is frustrated with not things that you are frustrated with.  This is not to say that you shouldn’t fix the things that you are frustrated with, but do not expect to be rewarded for fixing your frustrations instead of other people’s frustrations. The reward for fixing your own frustrations is that you are not frustrated anymore. 

At the Vice President and director level bosses want to see proposals that align with their strategy. If machine learning is a market area of interest to the VP, make a proposal that uses machine learning to fulfill a business goal that either makes or saves money.

Learn when to shut up. If you disagree with the strategy find a new job. You are an employee you don’t get to have public opinions on strategy. If people want your opinions you will be promoted to VP or get a fat consulting contract. 

Make proposals that align with the strategy or wishlist. Don’t worry about stuff that doesn’t fall in those categories because it’s probably not important to the business. And if you see an iceberg you should be looking for a new job not wasting time sounding the alarm. Tell your direct boss about the problem and wipe your hands of it. Unless you have a substantial equity position in the company it’s not your problem.

If you don’t know what is on your boss’s wishlist, you don’t know your boss well enough. Start getting to know your boss better this week. 

Conventions and capabilities

Computer programmers have been arguing about programming languages for decades. Is functional programming better than object oriented programming. Should we all be using logical programming instead? In the midst of the language wars, mainstream object oriented languages have been quietly adopting functional language features. Java in particular has been adding functional paradigms like mapping and filtering over streams to the language in java 8. Now why would Java add streams now, when the concepts have been around for decades? Well part of the reason is that these language filters had become so mainstream it was hard to ignore the demand for them. 

Why not just use functional languages? They have numerous advantages over object oriented languages like Java, sum types, higher order polymorphism, etc. What is the hold up? Well, while functional languages have many language features that are better, their weaknesses are typically in the tooling and the programming conventions that have developed. 

When you create a new language you do not know what is the most effective way to build things in that language. In Java it has taken  10+ years to develop post Java EE conventions that are effective. And Java has been one of the most used languages in the world for that entire time. Many languages have features that are simply ignored in professional use. In Haskell the default String type is horrible and no one uses it for professional projects. Others like Scala require you to avoid higher order types if you want compile times to be reasonable. 

Tooling is the other side of the equation. Java’s tooling has evolved in support of the conventions. Java has tooling to block certain dependencies, to auto-format code and to determine the cyclomatic complexity of methods. Today most Java professionals use either Intellij or Eclipse as their editor. These editors have had millions of dollars invested in them to support professional software development in Java. All this tooling makes Java work well once you have it setup, but getting setup is difficult. 

The final stage is when you merge conventions back into the language. Golang is the best example I have worked with. Go has merged auto-formatting into the compiler. The ‘go’ tool supports formatting the same way it supports compiling and testing. There are no arguments on the subject. Your code is either formatted to spec with the tool or incorrect. The best part is that there is no setup. It can take days to get a new Java environment working even if installing Java takes minutes.  

Scaling in a Consulting firm 

The consulting firm or agency model in application development is based on programmer hours. We the agency have X developers we want to sell at Y margin. You the client either need development done urgently or are unable to competently run projects internally. The agency makes money by leasing development ability to the client on an hourly basis. Our developers work on the application, our project managers keep track of progress so that you the client have visibility and we bill you for each hour we spend working on the application. Our margin is a flat rate depending on seniority and skill set. 

Well, how do we scale this agency to maximize profits? We can either increase our margin or increase our revenue by selling more development hours. The rates agencies can charge are based on demand and how much value the client expects to get per engineer. In general application development can bill around $100-200 an hour, specialized talent might bill out up to $300 an hour. Billing beyond $300/hour requires very specialized skill sets that are hard to find people for. Most agencies will be billing in the $100-300 an hour range and have a mix of experienced and inexperienced engineers on staff. 

Once you have capped out your rates for the talent you can get, what does an agency do? Well, investors want their returns so you either need to expand or sell. If you are privately owned you might be able to grow to a sustainable level, but that is easier said then done. 

Instead of focusing on rates an agency can change its billing model to fixed price per project. Bidding a fixed price for a project lets you increase your margin by competing on efficiency with other agency bidders. Typically, fixed bid contracts are common in government and large corporations. The risk is that if the agencies estimates are bad you can lose substantial amounts of money on the contract. If your estimates are good you can potentially have bank busting margins. 

The final means of increasing profits in an agency setting is to increase billable hours. This typically means growing headcount and hiring a sales team to keep everyone busy. 

What is the best way to scale billable hours? You get a team of cheap junior developers or offshore developers and have them supervised by senior developers. Maybe you give the babysitters a fancy title like ‘Software Engineering Manager’.