Julius Seporaitis
on hobbies and work

Requiem for Phabricator

Last week Phacility, maintainers of Phabricator, announced that starting June 1st, Phabricator will not be maintained anymore.

Phabricator has been a tool I relied on on multiple occasions, made a couple of small contributions to and occasionally did small extensions in a private capacity when needed. It has genuinely been one of my favourite engineering tool suites.

Strange to say it, but Phabricator and Evan’s writing contributed to my growth as an engineer. I’ve never met Evan or the rest of the contributors, but I am immensely thankful to them for Phabricator.

This post is just a shortlist of things that I learned through using it, reading code, or blog posts.

All You Need Is Trunk

Back in 2012, a few months into the YPlan journey, I got my eyes on Chuck Rossi’s tech talk called “The Push.” One of the central ideas in the talk was this: to have a continuous flow of changes to production, engineers should not care about branches. They should be able to push to trunk/master/main branch. It is a less wild idea now, but back then, the mainstream was convoluted git-flow with feature, development and release branches. Chuck Rossi’s presentation turned the whole thing on the head. I was in awe, and we tried something like that at YPlan with GitHub. It was debilitating to do a sequence of small iterative changes using the pull request model. I almost thought there were no convenient tools to do that in the public domain or as a service. Except, there were a few moments in the talk where I thought I saw a familiar interface.

Cue Phabricator. Open-sourced a few years earlier. I talked about it with the team, and we decided to give it a go. It was effortless to set up an instance. We still used GitHub as the repository hosting but switched to Differential Revisions for code review. Worst comes to worst, we thought, we will go back to pull requests. We never looked back.

For a new engineer in the team, it took about a day to get used to the Arcanist tool. It took about a week to understand how powerful the setup was. Almost all engineers I worked with, and who worked with Phabricator, missed it after switching back to GitHub and the Pull Request based model. Those who never used it continuously - seldom get what’s so good about it.

After working in companies that used Phabricator and GitHub, my observation was that engineering teams using Phabricator were at least an order of magnitude more efficient at pushing material changes to production than with GitHub. Someone could effortlessly churn up to 10 changes a day to production.

Of course, the tool is just one aspect of the story, but there is something powerful in the ability to slice and dice and stack or drop your changes as needed that contributes to that story nonetheless.

Write, Review, Merge, Publish

Ever since I read “The Psychology of Computer Programming”, I was immensely fascinated by how seemingly inconsequential details can have a massive impact on engineers work. My favourite story is how a well-performing company’s HR department moved a watercooler away from a secretary’s desk. The secretary was annoyed by people chatting there all the time. In the aftermath, the engineering teams started missing deadlines and projects ground to a halt. Noone asked what those engineers were talking about and assumed they were wasting time drinking morning coffee. That watercooler was the informal standup for unblocking one another.

Similarly, I think there are assumptions and decisions encoded in the Pull Request based development model. What would happen if they got adjusted? These changes could be almost philosophical or cultural and yet profoundly affect work output. I think Evan Priestley’s article “Write, Review, Merge, Publish” is an article that had such an effect on me. It explains the differences between Phabricator and the Pull Request based - Write, Publish, Review, Merge model. I won’t even try to summarize it here. The difference is so subtle. Yet, it has a definitive psychological effect on how people perceive their changes, for example - “as finished.”

The Serendipity Of Relationships

One of the main things that I relied on Phabricator throughout the projects that used it - is the seamless linking of objects. If a task T123 is relevant to a revision D456 under code-review - you mention it in the code review description, and it is immediately a two-sided link. I can go to T123 and see D456, and all other revisions, blog posts, wiki pages, and so on that mentioned it. What is the point? It gives a fantastic opportunity to find low-effort and high-value improvements. Write down ideas into tasks, add some context in the description, mention the task id anywhere relevant. Later, come back, and you can see all the instances of relationships - tasks, code changes, blog posts, wiki pages - in one place.

It leads to a fantastic result that I called serendipity driven development. It works like this: if there is some bug or a problem that is not prioritized but annoying or impact is not clear, and fixing it would take effort. Leave it to brew, mention it where relevant, do work, read Hacker News. At some point, an idea will pop up, and a way to solve that problem in minimal time may come up. I’ve seen this in practice, and it feels fantastic doing it as well as watching others do it.

The Power Of Integrated Application Suite

Another thing that I loved about Phabricator is that it came as a suite of smaller applications. Task management, workboards, code review, code ownership, wiki, paste, internal blogging platform, if-this-then-that business rules, countdown timers, polling, build orchestrator, API.

All for free. All tightly integrated, allowing those relationships and more. Sure, there are plugins for GitHub, but many times if A and B work with GitHub, they don’t integrate well with one another or charge a fortune for that upper tier of service. If I ever get lucky to see a suite of apps that works as well with GitHub and among themselves, as Phabricator suite did, and do not cost a fortune - I might reconsider. But until then - I genuinely miss the zero effort things like that.

Multi-Repo What?

There was a time when multi-repo became this crazy hype and turned into a mono-vs-multi repository war. I read a lot on both, worked with both, and got to reflect on both, and I conjecture this:

The multi-repo appeal is in some cases driven by the popular continuous integration SaaS-es only allowing a single project configuration file per repository, or otherwise making the process of having multiple projects in one repo a significant nuisance to manage.

Some would say, “that’s the whole point.” Well, cue Phabricator and especially its code ownership tool with Harbormaster build orchestrator. It enabled the setup of multiple build triggers and build jobs, depending on which files were changed. Therefore, I had a choice. I could have had as many repos as I wanted, or one, with virtually no difference in the amount of effort in managing it. Build System at Genus AI was the prime example of that.

Scratching Below The Surface

When I started my professional career, I worked with PHP and 10-16 years ago, PHP was mocked for poor quality code of projects it was used in. It was never constructive criticism, but there was a point: PHP was easy to get started with but hard to master, and there was very little high-quality code to learn from.

Again, cue Phabricator. I remember spending afternoons and evenings just browsing and reading that code. There is a lot that can be learned from the way Phabricator code is structured. I wrote a few things at YPlan to link Jenkins with Phabricator, and it was a pleasure to work with. I can barely remember any other project that would be this amenable without presenting a massive amount of technical documentation first. Phabricator had little explanation in terms of its internal technical workings, but it was readable.

Database Design

I am not in the know, but I want to conjecture: the fact that it came out of Facebook, and the size of the Facebook organization and its projects, there must be many lessons embedded in Phabricator of how to scale databases.

Phabricator has quite an intriguing database(s) layout and relationship tracking model (edges). While at YPlan, I tried to replicate the edges model to drive the users’ timelines. I was the only enthusiast of that model, but it worked pretty well.

At YPlan, we also got rid of the database migration graph and relied on alphabetically sorting migrations. I got the idea after reading the Phabricator code, and it worked wonderfully. Up to this day, my conviction is that DAGs for database migrations in small and medium projects operated by small to medium teams causes more waste of time and annoyance than the value they provide.

A Sad Loss

I find it slightly ironic that there is a lot of talking about being cloud or platform-agnostic only to go back to their corporate GitHub accounts and sprint boards on Jira. The de facto monopoly of how software is made.

GitHub has done all the good things to make source control and open-source available to everyone, but it also made the open-source development model the new normal for private companies of 2, 10, 50, 100, & more engineers. There is essentially only one way to build software right now - the GitHub Pull Requests + Jira. Whether it’s GitLab Merge Requests or Monday/Asana is irrelevant - they are all essentially equivalent. I think this unseen monopoly is unfair.

Phabricator was the beacon of something radically different that feels uncomfortable at first, is rough around the edges, but might as well work better than what we have now. It feels sad and looks like an end of an era, but while the announcement said the official maintainers stepped down - I am keeping fingers crossed that there is hope in the community.

I think there always should be an alternative.