At Acumen, our goal as an engineering team is simple - to make the best products on the market for our customers.
Acumen Invest™ is a trade promotion management and optimisation tool with an impressive and flexible set of features. One of its key capabilities is the ability to analyse P&L’s at any level of customer or product hierarchy at a daily granularity. For one of our customers, this means that for a typical 1 year, 1 customer report, the system had to deal with retrieving the data for, and calculating, nearly 1.5 million P&L’s.
As the scale of our clients was starting to grow, we identified the need for both highly flexible reporting with the speed of a modern web application. We then embarked on a programme of significant performance improvements across the platform.
With our goal in mind, we succeeded in hitting some impressive multi-user benchmarks. So in true retrospective fashion, here’s what I would tell myself if we were starting this initiative again:
Use tools and processes that suit your way of working, not vice versa
Our engineering team is proud of our dedication to being an agile team and have always found the SCRUM framework an extremely helpful way of working when developing new features. However, when we moved onto the application performance initiative we found ourselves struggling to articulate ourselves in user stories. The end-product of this was engineers expressing their ideas unclearly to try and fit a “Who, What, Why?” template, estimating things that we didn’t know a lot about and blindly committing these work items to 2 week sprints.
Identifying this and switching our way of working to a more KANBAN-esque style was a huge step forward in allowing the engineers to be more creative and clear with their priorities.
Create an environment where your people are not afraid to fail, and do so quickly
By removing the shackles of a process that wasn’t working for us, we enabled ourselves to really be creative. Frequent whiteboarding and discussions about the architecture of the system provided limitless ideas of ways to eke out more speed whilst leaving the flexibility of the system in place. All the developers, from the most experienced to the least, were unafraid to throw out even the most ridiculous of ideas and give them a go.
We take our flat team structure very seriously, finding that it removes all ego and provides the ideal environment for fostering creativity and awesome new ideas.
Be data driven in your approach
The above is all well and good, but at the end of the day computers don’t always work in the ways that you would expect of them to. One of my personal learnings was that most often, the areas that you suspect, aren’t necessarily the bottlenecks. Therefore, we insisted on having data before we entertained any discussion or ideas sessions, and utilised modern profiling and memory usage tools which make it so easy to achieve this picture.
This data driven approach was key to tracking our progression as a team and informing ways in which we could improve.
Take all variables into account
Sometimes getting data is not always enough, you need be sure you have the right data and all the data. This was typified in a very simple example. Our profiler was telling us that SQL was taking a long time to process a request, however, every time we tried to optimise this it was taking exactly the same amount of time. When we dug a bit a deeper into this we realised the bottleneck wasn’t actually the SQL itself, it was time taken to download the data. If we’d had that information upfront, we would have immediately looked at compression solutions, instead of spending time optimising the wrong thing.
Framework upgrades are free speed
This is rather self-explanatory, but over the course of this work we upgraded Invest™ to use the latest frameworks and libraries possible including .Net Core 2.1, ASP .Net Core and Entity Framework core. The speed gains from doing this all added up to a considerable chunk of our benchmarks and whilst it may seem like a “hit and hope” way of working, you’re essentially harnessing the work that others have already put in.
Keep the team fresh
Something that occurred almost accidentally was moving people in and out of the team. Getting new perspectives from new engineers and testers can cause complete paradigm shifts in how you work. Keeping the personnel fresh and engaged allowed us to constantly challenge our tech, our processes and ourselves to keep delivering high quality work.
Get in contact