Perhaps you should look into using tombstone tables.
Here is what I mean: Suppose you had this table
CREATE TABLE mytable
(
id INT NOT NULL AUTO_INCREMENT,
... other columns ....
PRIMARY KEY (id),
... other indexes ...
);
Now, create a table with the IDs that are marked for deletion
CREATE TABLE mytable_deleted
(
id INT NOT NULL,
PRIMARY KEY (id)
);
So instead of doing
DELETE FROM mytable WHERE MOD(id,100) = 0;
You would add the ids to mytable_deleted
INSERT IGNORE INTO mytable_deleted
SELECT id FROM mytable WHERE MOD(id,100) = 0;
From here, you simply have to add the mytable_delete to all queries
To see all rows
SELECT * FROM mytable;
To see non-deleted rows
SELECT * FROM mytable A LEFT JOIN mytable_deleted B USING (id) WHERE B.id IS NULL;
To see deleted rows
SELECT * FROM mytable A LEFT JOIN mytable_deleted B USING (id) WHERE B.id IS NOT NULL;
or
SELECT * FROM mytable A INNER JOIN mytable_deleted B USING (id);
To perform the actual deletion
DELETE A.* FROM mytable A INNER JOIN mytable_deleted B USING (id);
TRUNCATE TABLE mytable_deleted;
CAVEAT
This will means that lots of SELECT queries must incorporate the tombstone table.
I have discussed this before : Tombstone Table vs Deleted Flag in database syncronization & soft-delete scenarios. You should read the accepted answer from Leigh Riffel instead of mine for a more honest critique as to which method is better in your case.
- Use the
maxmemory
to set a limit to how much your Redis database can grow too. Failing to do so, Redis will grow until the OS will kill it once memory is exhausted (per your current experience).
- The usage of
maxmemory
should be coupled with maxmemory-policy
- you can choose from different eviction policies depending on your use case's requirements. For example, if you use the allkeys-lru
eviction policy, Redis will indeed start evicting (the least recently used) data once maxmemory
has been reached. Alternatively, you can instruct Redis to evict only expire-able data with the volatile-lru
or volatile-random
policies. Lastly, you can set the policy to noeviction
but that would mean that once memory has been exhausted, Redis will deny further writes with an OOM message.
Edit:
First disable swap - Redis and swap don't mix easily and this can certainly cause slowness.
Also do free -m
instead of top for the full picture of your RAM's state (http://www.linuxatemyram.com/).
Best Answer
Yes. Every modification (also delete) of data gets written to AOF. See Redis persistence demystified blog posting.