From server crashes to unlimited traffic
In the course of just six months, an online community grew from 2 million to 20 million visitors per month. Front-end developer Simon Dragsbæk Petersen reveals how he and his team built a new site that could handle this massive traffic – without server crashes – on a very tight deadline.
“We’re too late.” This was Simon Dragsbæk Petersen’s first thought when he took over the reins as lead front-end developer for a global online community.
At that time, the number of visitors to the website had already exploded, from two million to 20 million unique pageviews per month in just six months. And on peak Fridays, when traffic from all over the world was at its height, the site had about 25 million visitors.
“That’s where we started lagging. The WordPress blog could just about handle 20 million, but we started getting server crashes when we hit 25 million. And once you’ve reached that point, you have to build something new in a hurry,” explains Petersen.
A strange stack
The task he faced was to lay a new foundation – a platform which could be scaled up to several hundred million active users per month. And for a number of reasons, that meant he had to construct a completely new system.
First of all, the original stack was “a strange hodgepodge,” as Petersen puts it. It consisted of a WordPress blog and an online store build on Magento, both written in PHP, the most widely used programming language for dynamic web solutions. The site also had a web application that was built in Yii, another PHP framework.
“Each framework comes with certain predefined functions which developers are constrained by. This means that these functions aren’t recognizable across the site. This happens as a result of bad management, ignorance or the start-up spirit, where you just charge ahead without thinking through what the purpose of the system is,” explains Petersen.
The original site was also supported by Varnish Cache software, which is designed for content-rich, dynamic websites, and is intended to relieve the back-end server to avoid overload. And when traffic was heavy, this is what kept the site’s wheels turning.
“But even though we cached most results, sometimes we just got so many hits that we couldn’t handle them. We simply needed an entirely new architecture.”
Users drawn in by Facebook
The online community had struck gold – the demand was out there. Another reason for the sudden, massive increase in interest in the community was that large numbers of visitors were being drawn into the site by the blog on Facebook – unlike many other businesses, which typically use SEO optimization to find their customers on Google.
“The business had found a way to use Facebook’s algorithms to boost content, which meant that the site was getting higher click rates than ever before,” remembers Petersen.
A solid approach
Together, Petersen and his development team concluded that the most important objective was to deliver fast, stable content on the site for users to read. So they decided to generate all blog posts as flat, static files – HTML, CSS, Javascript. The files were stored on a Simple Storage Device and distributed all over the world via a Content Delivery Network (CDN).
“The good thing about static files in our particular situation was that we had very large numbers of visitors who were reading the same articles. So we could avoid load on the server by serving content as HTML files. The files were created once and for all, and then we just paid for the data being transferred back and forth. That’s a pretty solid approach,” says Petersen with a smile:
“With the method it’s built on now, the site can potentially handle unlimited traffic.”
The only disadvantage to this approach is that all posts have to be generated again after major design changes to templates. It takes about 30 seconds to correct 10,000 posts. The writers have also noticed that there is a 10-15 second delay before their posts show up in the ‘recent fives’ column.
“This means that there’s a short interval with inconsistency on the site, but I think that’s an acceptable drawback,” says Petersen.
Structure and systematization
Petersen elected to completely discard the old site in parallel with the creation of the new site. First, the team built a new CMS in Angular, a module-based framework, which was integrated with the publication of posts in WordPress. Next, the developers built an entirely new front-end in React. The entire process took eight months.
“During the construction phase, we had to dial back the publication of the most popular posts to avoid getting too much traffic. And that’s just not cool when your company is peaking like never before.”
During the process, Petersen made it a high priority to make sure that everything they built was double-checked:
“Code reviews mean that you can catch and correct a lot of typos. It makes things a lot easier for the people who will be developing the site in future,” he explains.
Fundamentally, Petersen’s approach emphasizes structure and systematization from the very beginning. Because the hours spent before the building process starts are always a good investment:
“I get together with the team in front of a board, and we just draw – the architecture, the best-case scenario, and everything that could possibly go wrong. I’m a strong advocate of this approach.”
Who’s Who:
Name: Simon Dragsbæk Petersen
Age: 26
Career: Has just completed a project for Bolighed A/S through ProData Consult. Is also co-founder of introDus, where he serves as CTO.
Education: Media graphic designer from KTS
