We're building a brand search engineGet early access
Psst we're building a brand search engine, get early access 👈

Automating browsers with browserless

Hey Joel. Can you walk us through your background, and what led you to start browserless?

Sure! Many, many years ago I was a young musician that wanted to get into programming. I had always been fascinated with computers, and had even “sold” programs on old TI-86 calculators in High School, so making a change into programming was something definitely in the cards. At the same time I always enjoyed the more creative-side of it, again being a musician, so that naturally led me to front-end development.

One of the things you encounter in front-end development is this notion of functional, or end-to-end tests, wherein you start a web-browser programmatically and “tell” it to do stuff to make sure the right workflow happens. The first time I encountered this I thought it was voodoo! I couldn’t believe this type of automation was possible! At the same time this very cool technology was extremely brittle, mostly due to it running on homemade browsers in order to execute headlessly.

Anyways, years down the road I was working on a side-project where folks could come and build out a wishlist from any website. This involved scraping information from webpages as users would simply enter their links into my site, which we’d then go off and get images and pricing data. One site in particular was architected as a single-page application, meaning you couldn’t just get that information with simple cURL or other libraries. You had to have a fully-running web browser to do so. Reflecting back upon the years of having done such a thing, I realized I didn’t want to do any of that and would readily pay someone to host the browser for me, if only such a thing had existed.

That was the spark that eventually led me to build browserless: it was something I wanted, would pay for, but didn’t yet exist.

How would you describe browserless to your grandma?

Such a great question, and every time I try and explain it to my family they seemingly get it wrong even though I try and simplify it down! I’ve had success with the “eBay” example now. Simply put: if you wanted to check the prices on eBay of a particular item each day you’d probably just login and do so manually. Pretty easy. However, let’s say you have a company that has to do this for thousands and thousands of products a day! That’s a really time consuming effort right? In comes browserless where you can automate this check, and avoid having to do all that work.

browserless website
The browserless site optimized for its audience of developers

Even though it’s a pretty common example, there’s actually a lot more you can do with the service. Let’s say your grandma is running a pretty neat recipe site with react as a front-end but doesn’t want to do server-side rendering (like a good grandma does). Now, grandma calls you and says that she wants to get some of that sweet sweet SEO, but can’t since there’s no good HTML for web-crawlers to parse and index! In that case, you can have browserless visit your site, snapshot the produced HTML after all the JavaScript has parsed and executed, and use that for your initial HTML payload. This way you can give web-crawlers the HTML they desperately crave, but still, give your users all the benefits that can come with a front-end application.

Now where’s my cookie, grandma?!

What part of the product are you the proudest of, and why?

There’s actually quite a bit here that I’m proud of, and some of it made it difficult to launch with due to my stubbornness on how it had to work. This probably sounds confusing, so let me give you a concrete example.

When I first got started I realized that most folks would probably want to just pay and use for the hosted service. But, I also knew that after working at a few enterprise companies, that having a product be on-premise is also incredibly valuable. I wanted to do both from the start and re-use all the core parts for both the hosted and DIY versions. I looked into many different ways of doing this: building a distributable and packaging it in a repo, offering up the source and running from that, and even building an ISO you could run the software with. These all had shortcomings, and many that were non-starters, namely the fact that it had to run everywhere to get as many users as possible, but I also needed tight control of the environment to make things like fonts and other issues work properly.

browserless debugger
The browserless live debugger debugging the debugger itself.

What I eventually landed on was docker. It just worked, was easy to get running everywhere, and gave me my own isolated world to install all the junk that makes stuff “just work.” Looking back on this decision I’m incredibly lucky and proud that I made the right choice.

The other part that I’m very content with is our load-balancing infrastructure. Because of the numerous libraries we support, which all have different ways of passing in authentication information, I pretty much had to write most of the load-balancing logic from scratch. This took quite some time to get right and test, but since it’s also in docker it’s really easy to start the whole software stack locally and test your changes! I guess what I’m really getting at is that containers are 100% the way to go, and if you’re not using them yet you’re really limiting yourself on what you can do.

How did you land your first paying customer? What are the main levers behind your growth?

The first customer was somewhat easy, however, the latter part of the question is deceptively nuanced and likely different for each and every product. However, where you get your first few customers is likely a good indicator of where your demand is, so I’ll start there!

Before actually creating browserless one of the things I was working on for this wishlist site was a headless driver of my own. Puppeteer and some of the more modern headless libraries didn’t exist at the time, and headless Chrome just got announced, so I was keen on taking advantage of all that. This little library was, of course, open-sourced with the hope that I’d be able to use it for my own application.

What actually ended up happening was quite the opposite: I had several people using it and giving feedback! This small network of consumers were doing something similar to what I was, and running it themselves since (again) nothing really existed in the market that met this demand. My first paying customer came out of this same group but in a more interesting way.

The headless driver I had written was doing pretty well, but Google came out with their own library (puppeteer), and as you can imagine everyone jumped on that. I don’t blame them: they have the resources and time to support such a huge undertaking. I was interested in helping them out, so I started looking at issue on their tracker. Being the good citizen that I am, I sorted by “Most commented” to see where the juiciest problems were. Lo and behold the #1 comment was running puppeteer on servers!

browserless github

Putting all this together, I decided it was time to pivot. It was abundantly clear that people wanted to run this work in the cloud, there was a huge amount of demand, and no solution existed. An obvious recipe for success, and where I landed my first few paying users.

Now, moving on to the second half of this question: growth levers.

If I was a more clever man I would have noticed that this product is really geared towards developers, especially considering that’s where I got my first customers from. However, it took me several months trying different landing-pages and copy changes before I figured out that this is really a developer-oriented product.

And it makes a lot of sense: I know a lot about my customers and potential customers by the sheer fact that I’m a developer just like they are. This definitely plays into our current growth strategy, which is really straightforward: blogs and no marketing. Blogs about fixing common problems, how to do XYZ with puppeteer, and even scaling stuff out on your own infrastructure are what people are searching for.

Advertisements don’t seem to work well with this audience, and I don’t blame them! Most are probably running ad-blockers of some kind, so it’s a fruitless effort in any case. Blogs, on the other hand, convey important information and raise awareness of the service. Even if they never touch our service or software we’re still giving back to the community in some way, which gives me immense satisfaction.

As you’re targeting a niche market, weren’t you afraid of not having enough customers when starting out?

This definitely was a concern early on, but as time goes on you begin to realize that even niche markets can be incredibly huge as they touch other tangential markets. For example, some of our biggest customers are from third-party logistics companies. Now, you and I might think that 3PL (as they’re called) wouldn’t have a lot of technical prowess, but the opposite is starting to come true. It’s an industry that is ripe for technical disruption as many of them still live in the dark ages with regards to technology.

With regards to browserless, this means that the solution follows them around. Engineers might have gained exposure to it via a past job, a blog post, or by a co-worker — but in the end, they need it for their current job.

It shares a lot of similarities to the game 6-degrees of separation. Even though a business might not be a direct user, a person in their organization might use it and know it’s a solution to a problem they now face. Since it has a lot in common with grass-roots type movements it’s important to build up a fanbase versus a user-base, especially when you’re getting started. This means a bunch of different things, but in essence, you have to go above and beyond in as much as you can to convert users to fans. It can be the way you help them troubleshoot issues, new integrations, or quick turnarounds on feature requests.

I can’t say this enough: if you’re starting with no known audience, and don’t have any traction anywhere, you need to build a fanbase! Click To Tweet

The cons to all of this is that your business isn’t going to skyrocket out of nowhere. Since it takes time for this process to occur, and it’s hard to trace it back to its origins, it might feel like you’re trying to move a mountain. It’s well worth the effort as it’ll establish you and your business as a domain-expert, and will help your brand recognition later on, which also pays out in dividends. Even if it all fails you’ll have likely helped someone somewhere and also increased your personal network, so it’s never a total loss. Sometimes throwing money at the problem nets you nothing back in return plus the loss of capital!

In any case, don’t let the size of addressable market scare you. Chances are it’s bigger than you think and one market will open the door to others, so it’ll grow with time.

What does a normal day at browserless look like? And what are your next big milestones?

Lots of writing!

Even though it’s not related: do more writing if you’re going to run a company! But, in general I do spend a lot of my time writing back to emails, helping folks on Slack, and writing up responses to issues on Github and Stackoverflow.

The highlights of my days are generally times where someone encounters a new problem. I’ve noticed that this can be a negative event for most folks, but I see it as both a learning opportunity and an area for expansion.

If they’re coming to us with an issue, then it’s likely that there are others out there with the same problem, or at least there will be. Because of this outlook, I look forward to working with folks through issues, and it’s something that I’m happy to say does happen.

In terms of milestones we’re working towards offering more capabilities to folks running on-premise plus the ability to record sessions as they’re running. Right now we have a pretty great set of tools to monitor, and even remotely view, your sessions as they’re going. However, if something bad happens it’s pretty hard to see what went down, so the ability to watch errors take place is of great importance to us.

The other big milestone is allowing folks to run different versions of Chrome in our product at the same time. This makes it easier to upgrade (or even downgrade) a version when you need to, plus it gives us better security as we can totally dispose of our containers when they’re done. Having the sessions be more ephemeral means there’s little left laying around, which means less reasons to be worried overall. I hop on any opportunity to increase the security of the product without question. There’s nothing worse, in my mind, than ruining the trust you have with your users given the frequent breaches we all seem to be accustomed to.

Looking back, what would you wish of doing sooner?

Two things come to mind: open-sourcing and blogging.

At first, the browserless container was closed-source since I was worried it would ruin us financially. It turns out that the whole “commoditize your competitors compliments” strategy really does work (more on that here, I highly recommend reading and understanding this!), but more specifically having software that people use tends to correlate to people paying for said software.

Once I open-sourced the image for browserless, more folks got exposure and (likely) ran it to see how it worked. This did two things for me: it gave me the ability to hear fast and frequently where things broke, plus it gave me permission to not have to offer free accounts.

I’m always interested in not having to have free accounts since I think they can be incredibly risky for the return they posses. Especially if you’re running a business by yourself! You can quickly end up in a place where free users dominate your time and energy, and potentially at a loss.

Having software open-source means that folks can take it, experiment with it, and decide what value they might get from it.

However, it does come at a cost to a certain degree, as you won’t likely be able to generate leads from this approach. Care needs to be taken here as this might have negative consequences depending on the character of your organization. Many places still live and die by leads and lead-generation, non surprisingly. I’m not saying it’s the right or wrong way to do it, just not the way that I myself am accustomed to.

Blogging, alongside open-source work, is another great investment that I should have been making much earlier. Even today I don’t think I write enough about what we’re doing and how we’re doing it. Doing it, however, results in a snowball of inbound traffic that it’s hard to ignore.

browserless blog
Joel shares his experience on the browserless blog

The one last thing I’ll mention about blogging is that it’ll force you to better understand your subject matter and make you a better writer. Both of which are so so so important for running a company that’s it’s hard to stress this enough. Write more and write often!

Kids today will never understand the struggle of what?

Setting IRQ channels to play Day of the Tentacle on a DOS system! (on a side-note: if you haven’t played these old LucasArts games you really should). More seriously, I’d say problem-solving without the internet. A lot of creative solutions to existing problems came about because of the lack of outside influence.

Sometimes not knowing why something is broken, and having to figure it out without outside help, can mean bigger opportunities and technologies later. Click To Tweet

Lots of things in life don’t have a blueprint so we shouldn’t get too accustomed to having them. Try fixing it yourself without looking for the answer! You’ll be surprised by what you find.

Want to stay in touch with Joel? Contact him, follow him on LinkedIn, and make sure to try browserless.

Get our latest posts straight to your inbox
Subscribe Now