Pagination search in Erlang Mnesia

Question

For example, given record

-record(item, {
  id,
  time,
  status}).

I want to search the 1000 to 1100 items, ordered by time and status =:= <<"finished">>

Any suggestions?

Answer 1

It depends on what your queries look like. If you need to order by lots of different columns, then I'd consider using SQL instead of Mnesia.

But if you only need the kind of query you described, you should be able to use the ordered_set type of table to handle the ordering and mnesia:select/4 to handle pagination and the constraint.

Here's some untested code to give you the gist:

% time goes first because it's our primary sort key
-record(statuses, {time, id, status}).
...
create_table() ->
  mnesia:create_table(statuses, [
                        {attributes, record_info(fields, statuses)}
                       ,{type, ordered_set}
                       ]).

-spec fetch_paged(integer()) -> {[tuple()], continuation()}|'$end_of_table'.
fetch_paged(PageSize) ->
  MatchSpec = {#statuses{id = '$1', status = <<"finished">>, _ = '_'}, [], ['$1']},
  mnesia:select(statuses, [MatchSpec], PageSize, read).

-spec next_page(continuation()) -> {[tuple()], continuation()}|'$end_of_table'.
next_page(Cont) ->
  mnesia:select(Cont).

Basically, mnesia:select/4 gives you a page of results and a continuation for the next page of results. There's not a built-in way to jump to the 1000th result like in a SQL DB, so if you needed that ability you'd build it yourself (keep an index of times so you can quickly look up that the 1000th time in your table is {{2015,4,12},{23,53,8}} and then use that as a guard when running select.

Answer 2

You might to use QLC Cursor, something like this:

order(table_name,[id,Direction]) ->
case Direction of
    descent -> fun(A,B) -> A#table_name.id > B#table_name.id end;
    ascent -> fun(A,B) -> A#table_name.id < B#table_name.id end
end;

select(universal,[Data_module,Table,Minor,Major,Order_by,Direction,Query]) ->
try
    if
        Minor == Major -> {error};
        Minor == 0; Major == 0 -> {error};
        true ->
            case apply(Data_module,order,[Table,[Order_by,Direction]]) of
                error -> {error};
                Order ->
                    Transaction = fun() ->
                        Query = qlc:q([X||X <- mnesia:table(Table)]),
                        Handler = qlc:sort(Query,[{order,Order}]),
                        Cursor = qlc:cursor(Handler),
                        if
                            Minor == 1 ->
                                Result = qlc:next_answers(Cursor,Major),
                                qlc:delete_cursor(Cursor),
                                Result;
                            true ->
                                qlc:next_answers(Cursor,Minor-1),
                                Result = qlc:next_answers(Cursor,Major-Minor+1),
                                qlc:delete_cursor(Cursor),
                                Result
                        end
                    end,
                    mnesia:transaction(Transaction)
            end
    end
catch
    Exception:Reason -> {error}
end.

The manual of QLC is here Qlc:reference but this way isn't fastest way in production, for middle heavy load is enought (10000 connection per second it tested), but more you will got a problems within memory free space, you will need to rise it up

Answer 3

I think, you should use mnesia:select/2 to select the objects which status =:= <<"finished">>, then you can use lists:sort to sort the results by time, at last you can use lists:sublist/3 to subtract the element from 1000 to 1100.

Here is the link: http://www.erlang.org/doc/man/mnesia.html#select-2 http://www.erlang.org/doc/man/lists.html#sort-2 http://www.erlang.org/doc/man/lists.html#sublist-3