Erlang lists:keyfind or proplists:get_value?
May 2010

Some days ago I quickly went through a post by Sergio Veiga, stating an interesting difference in speed between two Erlang functions which basically have the same functionality of retrieving a value from a list: lists:keyfind/3 and proplists:get_value/2.

Intrigued, I decided to perform additional testing, by performing a small benchmark of a random key retrieval on lists of different sizes, using those two different functions.

Here’s the code I used:

-module(pvsl).
-define(LIST_SIZES, [10000, 100000, 1000000]).
-define(RETRIES, 1000).
-compile(export_all).

start() ->
	% test for different list sizes
	lists:foreach(fun(N) -> test_list(N) end, ?LIST_SIZES).

test_list(ListSize) ->
	% generate a list of size ListSize of {Key, Val} entries
	KeyList = [{K, K} || K <- lists:seq(1, ListSize)],
	% test this list against both functions
	lists:foreach(fun(Type) -> get_val(Type, now(), KeyList, ListSize, ?RETRIES) end,
		[proplists, lists]).

% test getting values, compute necessary time and output print results
get_val(Type, Start, _KeyList, ListSize, 0) ->
	T = timer:now_diff(now(), Start),
	io:format("computed ~p random key searches on a ~p-sized list in ~p ms using ~p~n",
		[?RETRIES, ListSize, T/1000, Type]);
get_val(proplists, Start, KeyList, ListSize, Tries) ->
	proplists:get_value(random:uniform(ListSize), KeyList),
	get_val(proplists, Start, KeyList, ListSize, Tries - 1);
get_val(lists, Start, KeyList, ListSize, Tries) ->
	lists:keyfind(random:uniform(ListSize), 1, KeyList),
	get_val(lists, Start, KeyList, ListSize, Tries - 1).

I ran this test on my MacBook Pro, Intel Core i5 2.4GHz with 4GB Memory, and Erlang R13B04, with Kernel Polling enabled. These are the results.

roberto$ erl +K true +P 1000000
Erlang R13B04 (erts-5.7.5) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:true]

Eshell V5.7.5  (abort with ^G)
1> c(pvsl).
{ok,pvsl}
2> pvsl:start().
computed 1000 random key searches on a 10000-sized list in 323.373 ms using proplists
computed 1000 random key searches on a 10000-sized list in 12.897 ms using lists
computed 1000 random key searches on a 100000-sized list in 3273.973 ms using proplists
computed 1000 random key searches on a 100000-sized list in 130.592 ms using lists
computed 1000 random key searches on a 1000000-sized list in 34131.905 ms using proplists
computed 1000 random key searches on a 1000000-sized list in 2050.627 ms using lists
ok
3>

Of course, this is far from being a complete benchmark, however it does show that proplists:get_value/2 is considerably slower [up to 25 times on smaller lists on this benchmark run] than lists:keyfind.

Not sure why this is happening, though I would recommend using lists:keyfind/3 on all speed-critical parts of your code.

Comments (5)

Hi Roberto, keyfind and keysearch are BIFs, while proplists:get_value is not. We recently switched all internal KV lookups in CouchDB to use use keysearch (for R12 compatibility).

1. Posted by Adam Kocoloski — May 20, 2010 @ 3:46 pm

thank you adam for this feedback.

yes, that is indeed true. what may be an additional reason, is that proplists:get_value performs additional tests on list size and key being an atom, all things which do slow down code.

2. Posted by Roberto Ostinelli — May 20, 2010 @ 3:57 pm

The proplists module is written in Erlang, while lists:keyfind/3 is implemented in C. The proplists module is mainly meant for situations like lists of user options, and lets you do things such as writing ‘foo’ to represent a tuple {foo,true}. Hence, it is a bit more complicated than keyfind/3. Typically, you’d use the proplists module to extract specific parameters from a list of options and then you’d store those parameters in a record for quick access within loops. It’s reasonably efficient, considering the extra things that it does, but if you know that your lists contain only tagged tuples, it can be better to use lists:keyfind/3 and friends.

See also: http://erlang.org/pipermail/erlang-patches/2011-May/002137.html
and
http://erlang.org/pipermail/erlang-patches/2011-May/002140.html.

3. Posted by Richard Carlsson — June 5, 2011 @ 11:21 am

hi richard, indeed.

btw, you are referring to erlang discussions which are one year younger than this very thread ;)

r.

4. Posted by Roberto Ostinelli — June 5, 2011 @ 11:27 am

Hi Roberto. I know, but I recently started seeing tweets referring to this old blog article, and wanted to give newcomers some pointers.

5. Posted by Richard Carlsson — June 5, 2011 @ 1:05 pm
Leave a comment