CarbonJ

CarbonJ is a drop-in replacement for carbon-cache and carbon-relay. It was designed with high performance read and write throughput in mind and supports writing millions of metric data points and serve millions of metrics datapoints per minute with low query latency. CarbonJ is designed to run on medium-size instances (AWS: c5.4xlarge or smaller, 500k-1M/m metrics storage capacity per core) and standard (no provisioned iops) GP2 EBS volumes.

This project seeks to sequentialize the random write I/O typically generated by carbon-cache. Carbon-cache writes ‘Whisper’-files for each metric. These files have to be updated in the frequency of arriving metrics data — for millions of metrics data points per minute carbon-cache needs to write millions of files in random order. On aggregation these files need to be rewritten as well when higher resolution gets rolled into lower resolution, causing even more churn.

CarbonJ instead uses JVM-embedded RocksDB to store all metrics datapoints once per interval and different buckets for different resolutions, avoiding churn on aggregation.

It is currently running the Salesforce CommerceCloud Grafana/Graphite metrics stack and…