Aha! is a leading roadmap software provider helping more than 400,000 users build products and counting north of 5,000 companies as customers. Founded in 2013 with an entirely distributed team, the company puts customer needs at the heart of its business model, responding to customer requests as quickly as possible is one of the company’s core values. This is part of an approach they call “The Responsive Method.”

“If someone writes in with a technical problem, we want to give them the solution in no more than four hours,” says Alex Bartlow, an Aha! Engineering Lead. “It doesn’t matter if it’s one of our largest enterprise customers or a smaller startup. We want everybody to have a good experience with us.”

Alex Bartlow, Engineering Lead at Aha!

Addressing customer problems often means drilling into the data logs to find out exactly what went wrong. But with the old system that was built using BigQuery, sifting through hundreds of terabytes of data could be a time-consuming process. “It wasn’t quite, ‘Go get a cup of coffee,’ but it was starting to get there,” Bartlow says of the time it would take engineers to run search queries. “Our log storage and search system was the only thing we were not running on AWS. Consolidating the infrastructure became a priority.”

By migrating onto Amazon Athena and Amazon S3, with a custom-built GUI, Aha!’s engineers are now able to diagnose problems much more efficiently. Searching through a week’s worth of data, which previously would have been cost-prohibitive, can now be done in a fraction of the time, and getting to the heart of a customer’s problem is a breeze.

“Our most common request is that a customer writes in and says, ‘This page crashed for me.’ We can find that in Datadog, and we have a link embedded in Datadog to our log search. So you click that link, we open up the page, and three seconds later I have the relevant logs for that request,” Bartlow explains.

Using Athena’s new partition projection feature allows data to be queried much faster inside of S3, and AWS also advised Aha! on how to fine-tune its file storage system in order to achieve optimal results. To ingest data, the company uses Fluentd, which is running on Amazon ECS. “We have this nice little daisy chain of our containers which sends their data to Fluentd, which stores it in S3, and then we query with Athena,” Bartlow says. “It’s a pretty elegant solution, just using the AWS tooling that already exists.”

Check out Aha!’s engineering blog for a deeper dive into the nuts and bolts of how they built this solution.

Building its data storage and search system on AWS has also proven to be an ideal solution for a company like Aha! that has long embraced remote working. Aha!’s engineers can quickly follow up on colleagues’ previous searches and requests—something they couldn’t do before. “Collaborating like that with our team really helps us provide that radically responsive support and support customers anywhere in the world,” Bartlow says.

Ultimately, being on AWS “just makes sense,” Bartlow says, and it’s a platform that Aha!’s engineers can continue to build on in all kinds of ways.

“This ability to take basically any kind of data stream that I want and dump it into S3 where encryption and retention policies are applied automatically, and then be able to query it and run analytics on it very inexpensively—that’s an incredibly powerful tool,” Bartlow says. “We’re excited to keep working on new, interesting projects like this, making the most of what AWS offers.”


(Excerpt) Read more Here | 2021-01-04 23:05:43
Image credit: source