Apache Cassandra and ScyllaDB in Production: Wide-Column Data Modeling, Hot Partitions, and Choosing Your NoSQL Store

Relational databases are where most engineering teams start, and for good reason: they handle most problems adequately. But there is a class of workloads where Postgres and MySQL simply stop working, and wide-column stores are often the right answer. In twenty years of designing distributed systems, I have reached for Cassandra and ScyllaDB in exactly those situations, and watched teams fight both databases when they applied relational thinking to a fundamentally different data model.

This article is the guide I wish I had on my first Cassandra project. I will cover Apache Cassandra 5.0 (as of October 2026, current patch is 5.0.8), ScyllaDB 2026.x with its new vector search and full-text search capabilities, wide-column data modeling that actually holds up under load, and the operational patterns that separate healthy clusters from ones that page engineers at 2 AM.

What Wide-Column Stores Actually Are

The name “wide-column database” is confusing because it sounds like columnar analytics databases. It is not that. Wide-column stores like Cassandra and ScyllaDB organize data as rows with a fixed partition key and a dynamic set of columns sorted by a clustering key. The partition key determines which node holds the data, and the clustering key determines the physical order of rows within that partition. Every read and write is designed to operate on a single partition, which means no cross-partition joins, no server-side aggregations across arbitrary dimensions, and no multi-row transactions in the traditional sense.

That sounds limiting. It is. But the constraint is also what makes these systems scale. When you design your data model around access patterns rather than normalization, a Cassandra cluster can absorb millions of writes per second with consistent single-digit millisecond latency while surviving node failures with no downtime. The tradeoff is that you often denormalize aggressively and maintain multiple tables for different query patterns, which feels uncomfortable if you have spent your career thinking in third normal form.

The workloads where wide-column stores shine: time-series data at high cardinality, user activity feeds and event logs, messaging backends where you write once and read by time range, recommendation signal stores, and IoT telemetry pipelines. If your workload is write-heavy, the data is naturally partitioned by some identifier (user ID, sensor ID, account ID), and you need predictable latency regardless of dataset size, you should evaluate Cassandra or ScyllaDB before defaulting to Postgres with clever sharding.

For workloads that require complex queries, rich secondary indexing across arbitrary fields, or multi-table transactions, you want a distributed SQL database like CockroachDB or YugabyteDB or a managed PostgreSQL service. Choosing the wrong store costs months of redesign, and I have watched teams try to retrofit Cassandra into a relational role with consistently bad results.

Apache Cassandra 5.0: What Actually Changed

Apache Cassandra 5.0 went GA in the fall of 2024 and introduced several changes that matter for production deployments. As of October 2026 the 5.0 patch series is in the 5.0.8 to 5.0.9 range; verify the current patch against the Apache Cassandra downloads page before deploying.

Storage-Attached Indexes (SAI) are the most impactful new feature for most teams. Before SAI, secondary indexing in Cassandra was a mess: the legacy 2i implementation was famously unreliable for high-cardinality data, and SASI (SSTable-Attached Secondary Index) was never promoted beyond experimental status. SAI is a different architecture that attaches the index at the SSTable level, similar in concept to how column stores handle projection indexes. In 5.0, SAI is production-ready and replaces 2i and SASI for new deployments. It handles equality, range, and now vector similarity queries, which means you can do approximate nearest-neighbor search inside Cassandra without a separate vector store. Whether that is useful depends on your scale requirements and tolerance for operational complexity, but the capability is real.

The Unified Compaction Strategy replaces the old confusing choice between Size-Tiered (STCS), Leveled (LCS), and Time-Window (TWCS) compaction. The new strategy is density-based and can be tuned to behave like STCS or LCS depending on your read/write ratio. Compaction is where many Cassandra clusters get into trouble: wrong compaction strategy for the workload means write amplification, read amplification, or both. The unified strategy simplifies the decision tree, though understanding the underlying mechanics still matters for tuning.

Trie-based memtables and the BTI SSTable format improve memory efficiency for write-heavy workloads. The old skiplist memtable implementation had high overhead per entry; tries compact much better for typical key distributions in time-series and IoT workloads.

One Cassandra 5.0 caveat worth noting: Debezium’s CDC connector compatibility with 5.0 lagged the GA release. If you depend on change data capture with Debezium for downstream data propagation, verify driver compatibility before upgrading. The Debezium team typically tracks new Cassandra releases within a few patch cycles, but this has historically been a source of surprises during upgrades.

The Hot Partition Problem: Cassandra’s Achilles Heel

Every team that runs Cassandra long enough encounters hot partitions. When multiple clients hammer the same partition key at the same time, one node in the ring becomes overwhelmed while the rest sit idle. The cluster’s aggregate throughput is healthy by the numbers, but one node is dropping requests and introducing latency spikes. This is not a Cassandra bug; it is a data model problem.

The canonical example is a message storage system partitioned by channel ID and a time bucket (say, by day). A high-traffic channel generates thousands of writes per second to one partition on one day’s bucket, funneling all that traffic to a single coordinator node. Discord ran into exactly this issue before their famous migration.

Discord’s public account (and InfoQ’s June 2023 coverage) describes how their largest Cassandra cluster reached a point where hot partitions tied to their message storage design were causing cluster-wide instability and limiting their ability to do maintenance. Their messages table used a partition scheme that created hotspots around popular channels. After migration, their largest cluster went from 177 Cassandra nodes to 72 ScyllaDB nodes, and historical message fetch latency at the 99th percentile dropped from 40-125ms to around 15ms. Those figures come from Discord’s own engineering account; I am attributing them there, not presenting them as universal benchmarks.

The solution to hot partitions is nearly always in the data model. Common patterns: add a random salt or bucket suffix to the partition key to spread writes across multiple partitions; use time-based bucketing with shorter windows so no single partition gets too large; or accept that your write pattern requires a different database entirely. For workloads with genuinely uniform write distribution, Cassandra works well. For workloads with power-law distributions (a small number of entities generating most of the traffic), you need to design defensively from the start.

This design discipline is similar to what you do in DynamoDB data modeling with single-table design, where access patterns drive the schema rather than normalization principles.

ScyllaDB: Why C++ and Shard-Per-Core Change the Math

ScyllaDB started as a Cassandra reimplementation in C++ using the Seastar async framework, and the architectural differences matter in production. The most important distinction is JVM elimination. Cassandra runs on the JVM, which means garbage collection pauses. At large heap sizes (Cassandra deployments commonly run with 16-32GB heaps), GC pause events can cause latency spikes of tens to hundreds of milliseconds, and tuning GC is a constant maintenance tax. ScyllaDB has no garbage collector; memory management is explicit and bounded.

The second distinction is shard-per-core scheduling. Cassandra uses a thread pool model where threads compete for shared resources. ScyllaDB assigns each CPU core a shard, and each shard owns a portion of the data and handles all I/O for that data without cross-core coordination. This design saturates modern multi-core hardware more efficiently and eliminates most of the lock contention that appears in high-throughput Cassandra deployments.

ScyllaDB maintains CQL compatibility with Apache Cassandra 3.11, which means most applications migrate without rewriting queries. The driver compatibility extends to the native protocol, so existing Cassandra drivers (Java, Python, Go, Rust) work against ScyllaDB with a connection string change. The major migration effort is operational: moving data, adjusting compaction and repair schedules, and validating that performance characteristics meet expectations.

ScyllaDB 2026.x: What Is New

ScyllaDB follows a calendar-based versioning scheme. The 2026.1 release is the current Long-Term Support version. The 2026.2 Short-Term Support release shipped in June 2026 with several notable changes.

Vector search on Cassandra SAI-compatible APIs is the headline feature in 2026.2. The vector search implementation now speaks the same API that Cassandra 5.0’s SAI vector extension uses, which means LangChain, LlamaIndex, and CassIO applications built against Cassandra SAI run against ScyllaDB without changes. Vector search was previously available only on ScyllaDB Cloud; as of 2026.2 it is available in self-hosted deployments as well. Whether a wide-column store is the right vector database for your use case is a separate question from whether the capability exists. For workloads where you already have user data in ScyllaDB and want to add lightweight semantic search without introducing a dedicated vector database like pgvector or Weaviate, this is a reasonable option. For pure vector-first use cases at large scale, dedicated vector stores still have a richer feature set.

Alternator Streams reaching GA in 2026.2 is significant for teams using ScyllaDB’s DynamoDB-compatible API layer. Alternator lets applications written against the DynamoDB API use ScyllaDB as the backend, and Alternator Streams enables change data capture from that path. If you are considering a migration away from DynamoDB to a self-hosted or cloud-agnostic alternative, ScyllaDB with Alternator is now a more complete option than it was a year ago.

ScyllaDB 2026.3, which shipped in September 2026, added full-text search via a CQL MATCH() function. I have not deployed this in production yet, so I will not characterize its maturity or performance. Evaluate against your own workload.

Strongly Consistent Tables are listed as experimental in 2026.2. ScyllaDB’s standard consistency model is the same eventual consistency framework as Cassandra, using tunable consistency levels (ONE, QUORUM, ALL). Strongly Consistent Tables aim to provide linearizable reads and writes for specific tables without running a separate Raft-based protocol, though the feature is experimental and not appropriate for production workloads yet. Watch this space; it closes one of the main architectural gaps between wide-column stores and distributed SQL systems.

Compaction Strategies: The Decision That Follows You

Cassandra and ScyllaDB both use LSM (Log-Structured Merge) tree storage, which means all writes are sequential appends, and reads merge data from multiple SSTables. Compaction is the background process that merges SSTables to reclaim space and improve read performance. The wrong compaction strategy for your workload causes either excessive write amplification, read amplification, or space amplification.

For write-heavy time-series workloads where data is accessed recently and then archived, Time-Window Compaction Strategy (TWCS) is the right choice. It groups SSTables by time window and compacts within windows, which means older data consolidates into fewer, larger SSTables and compaction activity stays proportional to recent write volume. The critical requirement for TWCS is that your data must be written in approximate time order. If you backfill historical data into the same table, TWCS can thrash badly.

For workloads with uniform read/write ratios and high read throughput requirements, Leveled Compaction Strategy (LCS) limits the number of SSTables a read must consult by maintaining levels of equal-sized files. The cost is higher write amplification; each write eventually cascades through multiple levels. LCS makes read latency more predictable at the expense of sustained write throughput.

For the common case of write-heavy workloads where reads are mostly recent, Size-Tiered Compaction Strategy (STCS), or its Cassandra 5.0 equivalent through the Unified Compaction Strategy, gives better write performance. Reads can be expensive if you have accumulated many SSTables, but bloom filters and key caching mitigate this substantially.

In practice: default to TWCS for any time-series workload where data arrives in time order. Default to LCS for read-heavy OLTP-like patterns with a stable working set. Use STCS or Unified for write-heavy workloads that have mixed access patterns.

Operational Realities: Repair, Multi-Datacenter, and Consistency

Cassandra and ScyllaDB use a gossip protocol to maintain cluster membership and a ring-based consistent hashing scheme to distribute data. Every piece of data is replicated to multiple nodes (typically three), and the consistency level on each operation determines how many replicas must agree. QUORUM is the most common production choice: a majority of replicas must acknowledge a write, and a majority must respond to a read, ensuring strong consistency without requiring all replicas to be available.

Repair is the maintenance operation that reconciles data across replicas when replicas drift due to node failures, network partitions, or compaction timing differences. In Apache Cassandra, repair is a manual or scheduled operation using nodetool repair. If repair falls behind, you accumulate hinted handoffs and potential data divergence. The recommended cadence is running incremental repair more frequently than the gc_grace_seconds setting (default ten days), which is the tombstone lifetime.

ScyllaDB introduced Automatic Repair in recent releases, which shifts repair into a continuous background process rather than a scheduled maintenance window. This is one of the operational advantages that makes ScyllaDB easier to run for smaller teams without dedicated Cassandra expertise.

Multi-datacenter deployments using NetworkTopologyStrategy are the standard for production systems that need geographic redundancy or separation of workloads (OLAP reads isolated from OLTP writes). The replication factor is set per datacenter, and LOCAL_QUORUM consistency level allows operations to satisfy quorum within a single datacenter for latency-sensitive paths. This is how you build multi-region active-active patterns for workloads where eventual consistency is acceptable between regions.

One operational hazard I have seen repeatedly: teams deploy Cassandra with a replication factor of one in development, then move to production with a replication factor of three without testing the consistency level implications. A write at consistency level QUORUM with replication factor three requires two acknowledgments. A read at QUORUM requires two nodes to respond. If one node is unavailable, the cluster is still operational but at reduced availability margins. Model this in load testing before production cutover.

Polyglot Persistence and Choosing Your Store

Cassandra and ScyllaDB are not general-purpose databases. They work best as part of a polyglot persistence strategy where each service uses the right database for its access pattern. The common pattern I see in large-scale deployments: PostgreSQL or Aurora for relational data and transactional workloads, Redis or Dragonfly for session caches and rate limiting state, Cassandra or ScyllaDB for high-throughput time-series and event data, and a separate search system for full-text queries.

When to choose Cassandra over ScyllaDB:

• You need Cassandra 5.0 features like Unified Compaction that are not yet in ScyllaDB’s stable release
• Your organization has existing Cassandra expertise and operational tooling (DataStax OpsCenter, Medusa for backups)
• You need JVM ecosystem integration (DSE, Spark Cassandra Connector at large scale)
• Vendor support contracts for Apache Cassandra through Instaclustr or DataStax are important to your procurement process

When to choose ScyllaDB over Cassandra:

• You are starting a new cluster and want lower operational overhead (automatic repair, better resource utilization per node)
• Your workload has high write throughput where JVM GC pauses are causing measurable p99 degradation
• You are migrating from Cassandra and need a drop-in path (note: ScyllaDB is compatible at the wire protocol level with Cassandra 3.11)
• ScyllaDB Cloud’s managed offering fits your operational model better than self-hosted Cassandra

The team’s operational expertise matters as much as the technical characteristics. A team that knows Cassandra well will be more productive on Cassandra, even if ScyllaDB offers better raw performance per node. Both databases are operationally complex relative to managed SQL databases, and that complexity cost is real.

Connection Pooling and Driver Configuration

Wide-column stores handle client connections differently from relational databases. The native protocol clients maintain persistent connections with multiplexed request streams, so the connection pooling patterns you would use with Postgres do not apply directly. Cassandra drivers maintain a connection per node in the data center (or across data centers), and requests are routed to the appropriate coordinator based on the partition key.

Token-aware load balancing in the driver is important: configure the driver to use TokenAwarePolicy with a local datacenter preference so requests route directly to the node that owns the partition. Without token-aware routing, every request incurs an additional network hop from the coordinator to the replica, which adds latency and load.

Set the speculative execution threshold in your driver for read-heavy workloads. Speculative execution fires a second request to a different replica after a configurable timeout if the first has not responded. This reduces tail latency at the cost of additional read load on the cluster, and the tradeoff is worth it for user-facing reads where p99 latency matters.

Monitoring: What to Watch

Both Cassandra and ScyllaDB expose JMX metrics (Cassandra) and Prometheus-compatible metrics (ScyllaDB). For Cassandra, the standard observability stack is Prometheus with the jmx_exporter agent, Grafana dashboards, and alerting on:

• Read and write latency at p99 and p999 per keyspace and table
• Compaction pending tasks (spikes indicate compaction is falling behind write rate)
• Dropped mutations (messages the coordinator could not deliver)
• Hints store size (growing hints indicate a down node is accumulating backlog)
• Heap occupancy and GC pause duration for Cassandra’s JVM (not applicable to ScyllaDB)

ScyllaDB’s /metrics endpoint uses the Prometheus exposition format natively, and the ScyllaDB Monitoring Stack (a published Grafana dashboard set) covers the key metrics. For ScyllaDB, pay particular attention to the shard-level latency histograms and reactor stall warnings, which indicate event loop blocking.

The ACID properties that relational databases guarantee do not apply uniformly to Cassandra and ScyllaDB. Lightweight Transactions (LWT) using Paxos can provide compare-and-set semantics for individual row operations, but they are substantially slower than normal writes (often by an order of magnitude) and should be used sparingly, only where the atomicity guarantee is necessary. Do not design a data model that depends heavily on LWT; if you find yourself doing that, evaluate whether a relational database is actually the right choice.

Migration Path: Getting Data Into and Out of Cassandra

Moving data into a fresh Cassandra or ScyllaDB cluster is straightforward with bulk loading tools (sstableloader, or ScyllaDB’s sstable-tools). The harder problem is online migration from another database to Cassandra.

The typical approach for migrating from a relational database is a dual-write period: application writes go to both the old and new database, and you use a backfill job to populate historical data in the new store. Once the backfill completes and you have validated consistency, you cut reads over to the new database and eventually remove the old write path. This is the same approach Discord used for their ScyllaDB migration (with a custom Rust migrator for the historical data), and it works for Cassandra migrations from relational stores as well.

If you are using Kafka and Kafka Connect in your architecture, there are Cassandra and ScyllaDB sink connectors that support streaming writes from Kafka topics, which simplifies the dual-write pattern for event-driven architectures.

For migrations between Cassandra versions or from Cassandra to ScyllaDB, use cassandra-stress or a representative load test against the target before cutover. Performance characteristics can vary by compaction strategy, hardware profile, and network topology in ways that are hard to predict without actual load.

What I Have Learned Running These in Production

The single biggest mistake I see with Cassandra and ScyllaDB is importing a relational data modeling mindset. Teams create normalized schemas because that is what they know, hit cross-partition query limits immediately, and conclude that the database does not work. The database works fine. The schema does not.

Start with your access patterns. Write down every query your application needs to execute, then design tables that answer those queries directly, one table per query pattern. You will end up with denormalized data, multiple copies of the same information in different tables, and no referential integrity constraints. That is correct for this database. Embrace it.

The second operational lesson: do not neglect repair. I have seen clusters develop subtle data inconsistencies because repair was treated as optional. For time-series data with mostly immutable records, the risk is low. For data with updates and deletes, un-repaired divergence shows up as stale reads or missing data in ways that are difficult to debug weeks later.

Cassandra and ScyllaDB are excellent tools for the workloads they are designed for. After twenty years of building distributed systems, I still reach for them when the write volume and access pattern fit, and I still see teams over-engineer their way into them for problems where a Postgres replica would have been adequate. Match the tool to the workload, model your data around your access patterns, and these databases will handle load that would make a relational cluster cry.

As of October 2026: Apache Cassandra’s 5.0 patch series is in the 5.0.8 to 5.0.9 range (verify against the Apache Cassandra downloads page). ScyllaDB 2026.1 is the current LTS release; 2026.2 and 2026.3 are STS releases. Verify current compatibility and feature status against official documentation for your deployment.