Follow @nemild
nemil
  
 
What Open Source Can Learn From Slack

As Slack has continued to grow, open source developers have had lengthy debates about using it rather than IRC. For some, the fact that Slack is closed source and a walled garden makes it unsuitable when building projects that are open.

I’ll take a different approach: in the age of software, why is open software1 not more competitive for many products used by non-engineers and what can be done?

When open projects are displaced or when tech companies overreach, there is universal anger amongst many open source developers. Non-tech2 users, on the other hand, heavily value features like the existing user base and usability.

For anyone interested in having OSS compete with proprietary alternatives, you have to think about what important user segments value, not simply the needs of contributors.

Rebellion

Many hardware/software engineers value open products/standards and care about privacy. As long as these values have existed, a number of actions have caused concern:

  • Apple reduces/removes the hackability of its earliest computers (1980s)
  • Facebook makes choices that worry those who care about privacy (late 2000s — present)
  • Slack displaces the IRC standard amongst open source projects (2010s)
  • MakerBot goes closed source, after beginning from the open source RepRap project (2010s)3

There’s a common thread: a company develops/exercises market power at a cost to their users. In the case of MakerBot, an open technology is copied and then co-opted from the open source community.4 In the case of social networks, a successful, proprietary product puts financial concerns over the privacy needs of its users.

At this point, anger builds leading to substantial developer interest in competing open source products. In some cases, the open tools win (e.g., Linux). But often, they lose (Diaspora, Thingiverse-competitors).

When that happens, it's usually because the incumbent is too entrenched — and non-tech users don’t value the features the open source competitors offer enough to let go of what they already have.

Open Source Developers and Our Users

It should come as no surprise that software engineers are exceptionally good at building tools for other software engineers.5 This becomes an issue when the primary users are not tech enthusiasts — or have needs/values different from ours.

First, for social networks and marketplaces especially, user acquisition strategy and community matters just as much, if not more, than technology or product features. An initial version of Twitter or Instagram or eBay or Airbnb can be created in a matter of days by capable engineers — but one of the hardest parts of these businesses is recruiting users and finding the right early adopter community.6

Each social network has its own unique growth story from Twitter’s hyper growth at the 2007 SXSW to Facebook’s use on college campuses to Airbnb’s early growth in NYC’s international traveler market. For each success, countless other companies have failed, not because of their technology or product, but because of user acquisition.

And unlike many developer products, some of these products would have failed had they started solely with engineers. Once these products are successful, their network effects are so powerful that they’re nearly impossible to displace for a period of time — which can make it hard for future open source efforts.7

Second, typical users value usability and simplicity more than developers. We often require a deep degree of power, customization, and speed — and will gladly trade off usability and increased learning time.8 Open source teams often aim to maximize a product's features, rather than focusing on the product experience.

The danger is that many of the users we’ll need to succeed care about things we don't value as developers. This may be one reason consumer products — where UI matters — are more challenging to attack with open source teams than enterprise products. Many great products (especially consumer products) are more about what you leave out than what you put in, which can be especially hard for consensus-based open source teams.

In the case of Slack, a key argument by IRC proponents is that Slack does what can be done by open source IRC clients. And yet, Slack's design and unique features make it a product with substantial uptake and stickiness, especially by non-engineers. And even engineers are realizing that usability matters in dev tools as the ranks of non-traditional developers expands (e.g., the success of of MongoDB is a good example).

This is a lesson that Mattermost, an open source Slack competitor, has to take to heart to succeed:

Open source software that is to succeed in this new world is going to have to be better than anything else. You can't sell just openness anymore; it is added value, not a unique selling point. Open source software now has to sell user experience.

Third, we generally overweight the importance of our personal values when developing a product — without considering how users will make their decision.

For example, users do value privacy, but many won’t switch to a product that protects their privacy if it means losing what they have — unlike so many of those commenting in Hacker News threads. A better approach for them would be to combine privacy with other valuable features that compensate for the switching costs. This allows our values to be in the products they use, while still ensuring the product is successful in the broader marketplace.

Unlike the earliest days of the tech industry, the majority of people using tech products are not engineers/scientists — meaning that their needs matter even more than our own, even when we're building and using the product.

In short, it isn’t enough to build tools/features targeted at technically savvy users if you want to compete in many markets. This doesn’t mean you can’t have impact: early interest in Diaspora arguably led to privacy changes at Facebook. But to truly provide an alternative, you need to think about what different sets of users want — especially in product categories that may support only one winner.

Making open source more competitive

I’m hopeful for a world where open software competes in many more product categories, but important factors prevent this from happening today.

All inventors solve for the problems they face. In open source, contributors are generally technically savvy, and often focused on the backend. What they build for themselves is often different from what key user segments need.

It is also easier to recruit for open source projects that work on technically compelling issues. An IPFS or a decentralized social network is an easy to recruit project, while a heavenly chat user interface with an uninteresting protocol or unexciting language choice may not be.

We also solve problems using what we know. As such, engineers like myself overweight the importance of technology (including speed, scalability, technical decisions that are immaterial to success), rather than other important considerations.9 We want to work on problems we enjoy and think of ourselves as the primary user. To succeed, it’s equally important to think about who your users actually are and what they want.

This is exacerbated by the fact that we’re surrounded by people like us, and so will naturally see just a subset of users. There is much anger and mistrust for voice assistants and Facebook on Hacker News. Now compare this to the actual usage of these products. By looking at just the former, you'll envision a very different product.

I’ll offer a few suggestions to broaden the impact of open software:

  • Start with objectives: Flesh out your objective with your open source project: is it pushing the technical needle? Offering an alternative for a group of tech enthusiasts? Beating an incumbent over time?
  • Make a plan: Learn about what it takes to win in your product category — and have a plan of attack for each of these areas, don't just focus primarily on technology or personal values that may not be important to the users you’ll need to win (this might be a perfect lesson for a tech-focused eBay competitor like OpenBazaar)
  • Recruit widely and actively: Recruit/collaborate with people who have skills critical to your project’s success — and realize that this may mean finding and inspiring designers, growth hackers, marketers, among others
  • Rethink decision making: You may have to change your governance model to make an open source project successful (e.g., consensus decisions can lead to compromises and larger feature sets, which is not always the best answer)
  • Learn from prospective users: If you’re building a product where tech users may not be the right early adopter group, spend time with potential user segments — and don’t overweight the feedback of the tech community
  • Focus on User Actions Not Just Words: As many good product designers know, focus on what your users want, not simply what they say they want (every developer would say they want a database that is reliable and stable, and yet usability was even more powerful for MongoDB in the early days)
  • Be paranoid: Even if you are content owning your niche as an open source project used by engineers, you risk being replaced in businesses where network effects, usability, and marketing matter (e.g., Slack vs. IRC)

To truly compete in a wider swathe of products, FOSS needs a big tent approach to recruit other perspectives to its ranks — namely, all the other skillsets it takes to make successful mass-market software in the 21st century, especially design, usability, user acquisition, and even great frontend engineering. Open source contributions are used in industry to identify expertise and find the best talent. The same benefits of open contributions could incentivize growth hackers, marketers, and designers.

Another strategy is to think about software development as just one leg in a longer race, where we need handoffs from engineers to teams with other skills. This means we don’t have to recruit different skillsets onto our team, but publicize our projects so others can take over the next leg (for example, a motivated party can fork your marketplace project or social network, and go after a different user persona, which may be the product that has a shot of truly winning).

I’m particularly excited by the growth of non-traditional engineers in software engineering (e.g., Hack Academy vs. college-educated computer scientists) because it augurs different thinking/skills. For example, usability may be much more valued by Hack Academy graduates than for me, including in dev tools.

In this age of Big Tech, the fight for the values of old is hardly lost — Linux and Firefox came after years of dominance by Microsoft and successfully competed against it. And I’m not content to cede open software to developer software alone or relegate it to deep within the world’s software stack. While developers expressing their anger at using Slack in OSS is natural, there is more to do.

Thanks to Siddhartha Dalal, Drew DeVault, Dave Fayram, Maxwell Salzberg (formerly Diaspora), and Ian Tien (Mattermost) for warm-spirited debates about this essay. All opinions are solely my own.

Footnotes

Note: I was in YCombinator’s Summer 2012 batch with both the Diaspora and Mattermost founders. This essay came together as I’ve been in the process of open sourcing a consumer product a few friends and I built.

  1. Even though there are big differences, I’ll use the term open software to encompass a number of different structures from free software to open source software to software with somewhat permissive licenses. The very real differences don’t impact my argument here. ↩︎
  2. “Non-tech” means people not involved in the technology industry, especially those without any technical training. ↩︎
  3. Much of my inquiries into open source began when I started a company in 3D printing and saw the debates about RepRap and MakerBot from afar ↩︎
  4. A separate debate is that even open systems – like email – are susceptible to being co-opted by a few large players, even if they are ostensibly considered open. ↩︎
  5. They are not exactly the same, as even amongst software engineers there is a wide range in values and skills. For example, a backend engineer and frontend engineer may have different needs. ↩︎
  6. For many an engineer who works on a marketplace side project, it’s often a surprise how small a piece of the puzzle technology is in marketplace business, relative to recruiting buyers and sellers. ↩︎
  7. Diaspora had to deal with the huge network effects that Facebook had. I once had a fun conversation with a chemical engineer turned software engineer, and he likened marketplace user behavior to a membrane, where low listing prices would over time make parties switch (in this case, he mentioned Open Bazaar vs. eBay). His model didn’t account for network effects. This makes me wonder if the models that many of my technical friends have are similarly flawed, leading to issues in the decisions the software engineering community can make. ↩︎
  8. It may also be one reason that encrypted email has still not taken off. ↩︎
  9. For example, designing a system for scale is a hard and fun problem for many great engineers. But often for early products, this is immaterial to success — and often counterproductive as it distracts us and takes up valuable resources. This means we’ll often prematurely scale a system, taking the entire engineering cost up front, without realizing tangible benefit. ↩︎