Cassandra “Operation timed out – received only 0 responses.” during write

cassandraPHP

I'm using Datastax PHP Cassandra Driver and Cassandra 2.2.5 on Windows 10. This is a single-node cluster, and the keyspace replication of

{
    'class': 'SimpleStrategy',
    'replication_factor': 1
}

This is the table I'm inserting to:

CREATE TABLE MyDocuments (
    "label" varchar,
    "fileName" varchar,
    "blob" blob,
    "createdTime" bigint,
    "lastUpdateTime" bigint,
    PRIMARY KEY("label", "fileName")
);

When I tried to insert a row with a blob that is 16419KB large, it gave me this error message Operation timed out - received only 0 responses.
The row is not inserted at all.

Inserting a row with a blob that is 16379KB worked fine.

The log files don't show anything. When the operation fails, nothing is appended to any log files inside Datastax / Cassandra folders.

After a few hours of Googling, I still have no clue as to why this is happening. I have tried increasing write_request_timeout_in_ms to 20000 in the settings (cassandra.yaml), but it does not help.

Does anyone know why a write seems to fail in this way when the size exceeds ~16MB?

Update:

I updated Cassandra to v3.7, same observation, but the error message now says Operation failed - received 0 responses and 1 failures.

Best Answer

You will need to increase commitlog_segment_size_in_mb to a higher value. Try 64MB instead.

It increases the size of the commit record segment. This property determines the maximum mutation size (max_mutation_size_in_kb), defined as half of the segment size. By default is 32MB, which made your record were overflowing the maximum mutation size.

From the docs:

(Default: 32MB) Sets the size of the individual commitlog file segments. A commitlog segment may be archived, deleted, or recycled after all its data has been flushed to SSTables. This amount of data can potentially include commitlog segments from every table in the system. The default size is usually suitable for most commitlog archiving, but if you want a finer granularity, 8 or 16 MB is reasonable.

This property determines the maximum mutation size, defined as half the segment size. If a mutation's size exceeds the maximum mutation size, the mutation is rejected. Before increasing the commitlog segment size of the commitlog segments, investigate why the mutations are larger than expected. Look for underlying issues with access patterns and data model, because increasing the commitlog segment size is a limited fix.

Related Question