Side-Eye, a production debugger for Go systems
- Dynamic instrumentation
- Runtime tracing
- Production Debugging
- Distributed systems
Data Ex Machina is a US company focused on the monitoring and debugging of complex software systems, popularizing dynamic instrumentation tools.
With expertise in software architecture and product development, KOPERA assisted Data Ex Machina in bringing forward their newest Go instrumentation tool to a new level.
The vision of the project revolved around resolving both technical and product design questions, as well as bringing Side-Eye forward to the shape of a market testable offering.
The first phase validated the ability to model, ingest, analyze and serve collected runtime instrumentation data, including snapshots, function level tracing and scheduler tracing data in an efficient and versatile manner.
This proved complex given the abundance of data and its multidimensional aspect. Building the correct database schema for lossless storage and rapid querying to handle rich data was necessary, while introducing data querying client side through DuckDB-Wasm delayed data projection to where it mattered; at the fingertips of the end user.
We came to KOPERA looking to accelerate development and expand the scope of our production debugger. KOPERA helped us tremendously by building out the functionality around the visualization of Go scheduler execution traces. I was impressed by the ownership they took in the product and by the attention to the user experience and to details. We were very happy with the outcome — a challenging web app that’s pleasant to use. In the process, KOPERA also educated us on frontend development.
Andrei Matei, Co-founder
The second phase of the project was focused on providing intuitive and user friendly ways to traverse and serve the collected data in a meaningful manner.
This showcased two primary challenges: first in terms of creating a user experience that allowed insightful data observation and multidirectional data traversal; second was about safeguarding responsiveness in the face of considerable data size and introducing complex interactions that blend seamlessly together.
Challenges
- Data Multidimensionality proved challenging in terms of modeling, querying and projection while preserving data insight.
- Complex User Interactions needed implementation of custom UI components; This was necessary for the versatile data traversal required of the product.
- User Interface Performance proved a challenge when transforming tens of millions of data observations into snappy visual output client side.
Features
- Go Execution Traces Ingestor transforming system data into structured data entries for ease of manipulation.
- DuckDB architecture preserving data integrity and optimizing for data analysis and querying.
- Intuitive User Interface for snappy and rich traversal of complex data.