Humio at M1 Finance: Real-time Observability for Intelligent Financial Solutions
This blog was originally published on humio.com. Humio is a CrowdStrike Company.
“Whether I wanted to move to Humio or not, I had pressure from my engineers that we had to move to Humio.” – Steven Gall, VP of Engineering at M1 Finance
The Company
M1 Finance is a rapidly growing fintech company on a mission to reinvent personal finance. They believe current tools used to invest, borrow, or spend money are woefully under-utilizing new technologies available. That’s why they’re building intelligent financial solutions so people can do exactly what they want with their money.
For M1 Finance, a real-time, scalable, yet still affordable log management solution was crucial for their development and operations teams, as well as for them to be able to meet the needs of their customers.
The Challenge
M1 was using their own ELK stack to manage their logs, but it was not meeting the company’s data aggregation needs. VP of Engineering Steven Gall explains their business model.
“We have a microservices architecture, meaning we have many services deployed in production. Each service is constructed of numerous pods – 20-30 services in 10-20 pods. That’s a lot of logs.”
Ingesting all these logs was taking from 5-15 minutes at peak times, so for engineers to observe what was going on in the system in real time, they would have to manually track data from 20 pods at once. This work-around was barely manageable, and took up a significant amount of their employees’ time.
The Search
M1 Finance went looking for a new solution. Their priorities were:
- Near real-time data aggregation that is easily searchable
- Cost
- Ingestion and retention rates
They first appraised Sumo Logic, but Steven found that the prices were too high for the services provided.
“There was more cost than initial value. Admittedly, after initial conversations, it wasn’t the right direction.”
Steven next considered getting managed services for their ELK stack, but that would only increase costs without taking care of the slow ingest problem. M1’s limited human-powered resources and money were best spent elsewhere.
“As you grow an ELK stack you need a whole team to make it work efficiently. It’s not a value proposition that we have a great ELK stack for our customers. They ask, “Give me my money.” or ‘Did my trades to go through?’”
The Solution
M1 ran a one-time trial of Humio in the periphery during production, after which the response from Steven’s team was overwhelmingly positive. Because of how much easier Humio made it to track problems in the logs in near real-time, the engineers wanted it over any other solution.
“After that, whether I wanted to move to Humio or not, I had pressure from my engineers that we had to move to Humio”.
The Results
Humio provided a solution for all three of M1 Finance’s needs.
Near-instant data aggregation gave them near-real-time observability.
“Having a solution that I can trace across different services and pods is very powerful for our use case. We see an error in production and we can see what’s happening in the system right now.”
Humio fixed rates for unlimited ingest met M1’s budgetary needs, allowing them to do more for the same cost.
“We were able to maintain the same cost structure as our ELK stack and improve retention from 7 days to 90 days. Ingestion went from 15 minutes to near real time.”
M1 benefited from Humio’s greater log retention times, enabling them to unlock unforeseen metric resources and better serve their customers.
“I wasn’t expecting the sheer amount of retention. When we were initially talking, Humio was going to retain 30 days. I was like ‘Wow! That’s a long time!’ We’re now up to 90 days retention. 3 months is a long time for a startup… It allows us to provide dashboards about our user interaction and logs — not just metrics shipping, but metrics about our static logs.”
Now, Steven no longer finds employees stuck trying to piece together what’s happening the hard way.
“I can’t remember the last time I saw anybody tail a log on a box in production. It’s a good thing for our efficiency schemes.”
“We were able to maintain the same cost structure as our ELK stack and improve retention from 7 days to 90 days. Ingestion went from 15 minutes to near real time.” – Steven Gall, VP of Engineering at M1 Finance
