An Essay For Aspiring Engineers
This article was originally posted in my Lithuanian (native tongue) blog and was received quite well among my colleague software engineers as well as non-technical friends. It was meant as an answer to a different blog post complaining about: aspiring programmers asking wrong questions about career bootstrapping, businesses and universities marketing constant lack of talent (encouraging too much people to study IT related fields) and all the worthless stuff the universities are teaching. Some of the bits apply only to situation in Lithuania, but I think the main point I tried to make is universal.
It all started with another blog post written by a person, who had studied computer science, does not work as a programmer, but has career in IT. I thought that I could walk through similar points she makes, providing my perspective, because I also work in IT and as they say - “I have seen some things.”
Questions
Seldom I receive questions on how to start a programming career. Where to start? What technologies to learn? Which ones generate the most revenue? It doesn’t happen very often, but when it happens I never give a straight answer, I give directions where to look for answers - books, blog posts, other people. Much like a solitary monk would say to his first apprentice: “You must find your path yourself, young one.”
Some could think that I am evasive, but I disagree. I do not know the correct answer to those and other questions: is my decision to be a developer - good? What are the qualities of a good programmer? How does a single day at work looks like? Honestly, I do not believe anyone could answer them - not even the last one. The difference between how my day looks like now and how it looked for someone, when I took my first steps learning a programming language, is like day and night.
However, I can give you one question with a constant right answer - who is a programmer? (software engineer & developer too - I’m using them interchangeably here).
A programmer is a person, efficiently solving someone else’s problem(s).*
* With each solved problem programmer is creating at least one for him/her-self.
When you look through this perspective - it is not important any more whether web or desktop programming jobs are better, but rather what problem a person wants and is able to solve efficiently. If you’re just starting out - internet is full of helpful resources, e.g. look for engineering blogs of tech companies[1]. Blogs full of articles about problems they were having and how they solved them. Read and learn! Get some perspective. If you are able to solve the same problem more effectively - you are guaranteed a job. If multiple companies have the same problem - maybe it is time to think about a startup.
Any technology you can think of is only a tool in developer hands, thus a question, like “which one to learn to maximize my income”, is obviously from absurd-land. The more tools you have - the bigger solution space you create for yourself to choose from. The bigger solution space you have - the more chances you will solve the problem effectively. Even this myth of “natural lifelong learning skill”, which engineers and developers are praised for, is nothing more than comprehending: “if I will always dig a well with a shovel, it is only a question of time until someone with an excavator will come to change me”. I know few people, who are developers but are far from enthusiastic about it, yet they have enough grit to follow this truth and keep their skillset up-to-date.
Sometimes, however, even importance of all available tools fade away in the light of someone’s ability to squeeze a maximum out of a single one. There are people[2], who are solving problems with Excel and/or Visual Basic (stereotypically an inferior language). I know at least one in person. And you know what… no one could beat them at what they do. Why? Switching would be too costly or breaking a human habit (users) - too hard[3].
Marketing
This was about various companies PR agenda too often being about lack of talent and boasting the size of income, which is usually couple of times than average monthly wage in Lithuania. Also, about universities agreeing on this agenda to boost number of student applications, because unis in Lithuania benefit from something called “student basket” - an amount of money they get for a single student. This, supposedly, had to increase the quality of studies. Don’t ask to explain…
Marketing is good and I will repeat the message - there is a lack of software engineers. Not any engineers, but those who are able to understand the scope and context of a problem, come up with a sensible solution and write source code to make computers (even thousands of them) solve that problem in an automated and efficient manner.
Companies, who are not picky enough about their engineers, often do not understand the cost[4].
On the contrary, companies that value their product quality are really harsh at picking talent. And I mean really harsh. Consider this: output of my university class was roughly 70-80%. Best case, a good company will hire only 1 out of 10. Best case.
In Silicon Milkroundabout[5] I had a chat with a colleague working at StackExchange (Disclaimer: I do not work for SE, but I call all engineers my colleagues). He said they hire less than ten out of hundred applicants. I couldn’t believe it at first, but the process is fine tuned to filter out only the best: a short call to acquaint with a candidate, a longer technical call, if there are doubts - another one, finally you’re invited on-site for even more technical interviews, sometimes lasting throughout a day.
On the other hand - universities are educating knowledge synthesizers and (ideally) - innovators. Before applying to university, a person must understand that there is not enough demand for that many people with higher education. (There is this absurd belief in my country, arguably a soviet relict embedded in my parents generation and they are pushing it further, that if you do not have diploma - you are worthless and won’t get a job.) For instance, some of my friends do not have a formal higher education in anything, but they are developing amazing things. I also had a chance to do an interview with a guy who, from the looks of it, dropped out of school to pursue a programming career. While I might not approve such approach, I was impressed with his ability to come up with optimal solutions to my questions and follow-ups, almost immediately. On the other end of the spectrum - some friends knew nothing about programming before uni and now are on a successful at software engineering career track.
Even better, when you are not a programmer by education, but know how to do it. I first heard this from a programming enthusiast in Copenhagen - he had quoted Zed Shaw[6]. Later, I had to hold my jaw closed when I found out who he was: general counsel at the worlds biggest shipping company - by day, Clojure hacker by night.
Studies
Disclaimer: this part is heavily based on quotes from that other article, but I’ll try my best to give context.
“Universities lag behind business requirements.” (This phrase is almost ubiquitous between students and businesses in Lithuania.) But following this logic it would mean that universities are training engineers almost for any business - be it template website “bakers” or banking systems providers. NO. Like, this NO!
Look:
Universities must prepare knowledge synthesizers who are, able to solve unique business problems.
Following this heuristic - it is the businesses that lag behind. It isn’t bad - some of them do not need brain-picking bespoke solutions. It is absolutely natural and normal. But imagine when after four years of ingesting all the important knowledge about software engineering and ready to work on the next big thing, one finally is tasked to mash together a website using a template and a CMS. That is why I felt cheated, didn’t write my thesis and dropped out after four(!) years. I completely lacked motivation and often sang the quote I started this section with. Took me couple of years, and career steps, until I understood what I missed, and I am fixing this mistake at my own expense now.
This next quote (about curriculum) also resonated with me:
However, you are forced to write a compiler, because “what if you’ll need to write one.” Maybe you could add aircraft flying to the curriculum? In case the whole crew passes out during a flight. You know, just in case.
It resonated with me, because once I was singing the same song - why the heck would I need to “write an operating system”, dig into “computer architecture” or how “computer networks” at a low level work? All operating systems are written too hard to comprehend for newbie, no new mainstream computers were invented recently (except[7]) and there are networking libraries ready to be used.
In the last 6 months I had to roll my sleeves and dive into Linux
Kernel source code to understand why packets sent over network to
rsyslog were being lost, when all the documentation I could find and
my understanding of the domain said they should go through. Do you
feel the irony? All three of them - low level code, operating system
and network. I could’ve brushed it away with an incompetent “this is
not my domain anymore, I don’t know why this is happening” and go
after another, more rewarding, task. However, my present self was
super happy that my past self wasn’t that dumb and didn’t skip all the
classes, and those skipped - were compensated by self-educating. And I
found the root cause and I fixed the problem: there is a limit to UDP
packet size, which you can trivially increase, but only to as much
memory as kmalloc can allocate, which, indeed, had a cap of 128k
bytes.
There is a saying - ignorance is a bliss, and it fits here perfectly: there’s a programming language and compiler or interpreter, nothing out of ordinary, but if something is not working - it becomes a language limitation, it is compiler fault or interpreter performance issue. This is where all the problems hide from ignorance and incompetence - this is the bliss.
Software engineer efficiency is limited to the depth of understanding his/her stack.
Sounds intuitive enough - if a developer only knows how to use a framework, his efficiency will be limited by that framework. If an engineer knows only his language, all optimizations will be limited by the language features. If, after everything else, they also know language internals - they are competent enough to consciously write the most efficient code.
If you ever dreamt of working at companies like Facebook (or build one), think of the problems they were having - Facebook website code was too slow. Rewriting 9.2mln[8] lines of code without stalling every day progress - is an abysmal step[9]. What else? Either buy a thousand more servers or rewrite the underlying language interpreter into a compiler. They successfully managed to do exactly that[10]. For the same purpose, they redefined how data centers are built - hardware and software[11]. I am doubtless - engineers who pulled all this off are heroes among their peers. And - they knew how to write a compiler.
That indistinguishable half a second saved, between you clicking “Like” and seeing a response, accounts for millions of dollars when we are talking about tens or hundreds of thousands of servers, generating Facebook wall for all of us.
Finally
I think I used different arguments to tell the same idea as in the other article. It doesn’t rain gold bricks for a software engineer. On the contrary - work is full of intensive brain picking and, often, very little reward, because with every problem solved you have a new one for yourself - maintain the code. It all depends on problem solving skills and understanding of the toolkit, whether you’re going to do boring tasks or push the boundaries, wherever they might be. So if you are an aspiring software engineer - start by finding a problem to solve. It doesn’t need to be grandiose - solve one for yourself, your family or friends. And if you got hooked - continue, you are on the right path.
Hope you liked this article, please do comment or upvote on Hacker News.
References
Some additional reading, explaining where my thoughts are coming from. If you trust me - no need to follow them, but it’s recommended if you want to check the facts.
The original article this was intended to reply (in Lithuanian) - here.
[1]: Facebook Engineering, Twitter Engineering, LinkedIn Engineering, Netflix Tech Blog, Code as Craft.
[2]: My friends call me a scumbag because I automate my work when I was hired to do it manually. Am I?
[7]: D-Wave Systems
[8]: How many lines of code is Facebook?
[9]: Things You Should Never Do, Part I
[10]: Facebook Speeds Development With “HipHop Virtual Machine”, A 60% Faster PHP Executor
[11]: Facebook Saved Over A Billion Dollars By Building Open Sourced Servers