Humio at Stibo Systems: Cloud Native log management for the world’s biggest vendors and retailers

A 225 year-old company improves its logging capabilities using Humio

Stibo Systems provides enterprise master data management solutions that enable the world’s largest retailers and vendors to better track their customer and product data. They have managed publishing information for other organizations since 1794, when they were founded as a printing company.

The thing that has kept Stibo Systems ahead of its competition for hundreds of years is how it adopts new leading technologies. The latest tool that they are focusing on is cloud hosting – they’re building a cloud native solution for product syndication, where they are syndicating production information to channels like Amazon, Walmart, The Home Depot, and others. Stibo Systems is also gradually migrating their customer’s master data management solutions from self-hosted storage to SaaS cloud models, which also is the preferred model for new customers.

Logging Stibo System’s serverless architecture

With the move to a dockerized microservice based cloud environment, Stibo Systems has to keep the needs of their enterprise customers in mind.

In order to control costs while supporting variable updates to their customers' product information systems, Stibo Systems is planning to use serverless channels. This enables Stibo Systems to shut down services they’re not actively using but also scale them up when their retail-based customers create a surge of usage. The services spin up as needed, running anywhere from one hour a day to ten hours a day. There can be hundreds of instances running during times of high usage such as the week before Black Friday.

The microservice design comes with a complication for monitoring. Docker containers go up and down, and you need the ability to investigate logs across many services and service instances.

“Aggregating logging naturally becomes important, because the instance comes up and goes down again. So, you have no way of looking at the log that’s in the instance, because the instance goes away — the request is gone.”

Challenges of cloud services

Dealing with variable availability demand, Stibo Systems requires a scalable log management solution. Onboarding large, international customers can involve a hundred thousand products, for one tenant alone. A monthly update can involve thousands of items, again for one of many tenants. Each individual product may have additional attributes which must be tracked, and also may have to be labeled in multiple languages. To address the need for scale for their log management, Stibo Systems initially attempted to implement a stack using Elasticsearch, Logstash, and Kibana – aka the ELK stack, but their engineers found it so unsustainable they reverted to looking for logs manually.

“Most developers actually turned directly to console logs instead of using ELK because it was too cumbersome and too tedious.”

In addition to meeting the variable availability needs of their customers, Stibo Systems needs to be able to provide immediate support at any time. To meet the demands of their enterprise clients, Stibo Systems maintains a live 24/7 support team who accesses Humio to help its customers.

Humio solves the observability problem of Cloud Native

Stibo Systems found that Humio’s search language and parameters were easier to use than Elasticsearch and Kibana to find logs associated with root causes, and it performed better at scale. This boost enabled their support team to respond faster.

It also helps with determining the causes behind issues after the instances in a microservice environment were no longer running. “With microservices, all customer issues become a murder mystery. To accurately determine what is happening, Humio is really essential.”

Humio also provides Stibo Systems with an overview of all their systems with live, real-time visualizations of data, allowing them to look across all customers at once from one place rather than one at a time. This enables them to identify systemic problems with their services that may otherwise go undetected.

“Having an ability to look across everything in the same tool set is absolutely vital.

So the ability to just create a field with a regex expression on the fly and to make a chart that looks at specific issues that way creates transparency. It gives a really good understanding of what is going on.”

Controlling Costs

The expenses of running cloud services drives Stibo Systems to constantly look for ways to boost efficiencies and cut costs. Not only does Humio help with this by monitoring ephemeral infrastructure that can be turned off to save costs, it also helps save costs by speeding up incident response.

Optimized for real-time search, alerting, and monitoring dashboards, Humio lets developers know the second an error occurs, and can direct them to the relevant logs and events.

In a production environment, Humio’s real-time streaming data enables Stibo Systems to find and proactively fix problems before customers experience them.

“The objective is to fix the customer issues preferably before they experience them themselves. That gives you more happy customers, and you save money.”

As Stibo Systems looks to the future, their developers are focusing on optimizing a system that is built around events rather than states. Humio supports this mission by providing an affordable way to log everything and have a complete record of all events that can be debugged in real-time or after the fact.

See Humio’s flexibility and performance

Gain a full understanding of how Humio’s index-free architecture suits enterprise needs by signing up for a free demo.