redis - mass inserts and counters

Question

This is my first question here on stackoverflow. Dear community, thanks a lot for your combined knowledge and expertise !

I'm new to Redis so please bear with me, as I'm sure there is a simple solution.

redis-server --version

=> Redis server v=2.6.14 sha=00000000:0 malloc=libc bits=64

redis-cli --version

=> redis-cli 2.6.14

I have read "How to use Redis mass insertion?" How to use Redis mass insertion?
I have googled and read a lot of praise for the redis INCR feature. But I don't really understand everything and it does not help me to do this redis-internally only.

My goal: I want to import 'n' lines of text into redis and retrive them later in exactly this order.

For each line I set a unique key, like key:1, key:2, key:3 etc. By using an increasing counter as part of the key I am able to later retrieve the lines in the same order in which they were stored in redis.

---

Now (without redis mass insert) I solve this easily by using an awk script to generate redis-cli calls, like:

cat data.txt | awk -f myscript.awk | bash

---

"data.txt" looks like this:
This is the first line.
This here is the much longer second line.

---

"myscript.awk" looks like this:

#!/usr/bin/awk -f

### This section is being applied before any line is read:
BEGIN {
# Set counter initially to Zero
print "redis-cli SET counter 0"
}

### This section is being applied per line read into awk:
{
# Increase counter by One
print "redis-cli INCR counter"
# Read current counter from redis into an variable
print "MYCOUNTER=`redis-cli GET counter`"
# Use this variable as counter for the key
print "redis-cli SET key:$MYCOUNTER \"" $0 "\""
# Retrive stored value from redis for illustration
print "redis-cli GET key:$MYCOUNTER"
}

The output of "cat data.txt | awk -f myscript.awk | bash" is:

OK
(integer) 1
OK
"This is the first line."
(integer) 2
OK
"This here is the much longer second line."

So everything is fine.

---

But instead of calling "redis-cli" twice per imported line I want to use the redis "mass insert" feature. Here I need your help:

How would I do something like this in redis only ?

SET counter 0
=> OK
INCR counter
=> (integer) 1
GET counter
=> "1"
SET KEY:{counter} "Content of line 1"
=> OK
INCR counter
=> (integer) 2
GET counter
=> "2"
SET KEY:{counter} "Different content of line 2"
=> OK

etc., etc.

The "GET counter" lines are just for illustration.

Any help is appreciated. Thanks again !

bernie

Answer 1

Use a list for this. There's no reason to use a new key for each line. All the list commands are here, but the one you want is RPUSH. You can rpush many values at once in the same line, so you'll just do that:

RPUSH some_key line1 line2 ... lineN

Then to retrieve:

LRANGE some_key 0 -1

Quick and easy!