Originally, someone complained about protobuf_c_message_unpack() using
alloca() for the allocation of the temporary bitmap used to detect that
all required fields were present in the unpacked message (Issue #60).
Commit 248eae1d eliminated the use of alloca(), replacing the
variable-length alloca()'d bitmap with a 16 byte stack-allocated bitmap,
treating field numbers mod 128.
Andrei Nigmatulin noted in PR #137 problems with this approach:
Apparently 248eae1d has introduced a serious problem to protobuf-c
decoder.
Originally the function of required_fields_bitmap was to prevent
decoder from returning incomplete messages. That means, each
required field in the message either must have a default_value or be
present in the protobuf stream. The purpose of this behaviour was to
provide user with 100% complete ProtobufCMessage struct on return
from protobuf_c_message_unpack(), which does not need to be checked
for completeness just after. This is exactly how original protobuf
C++ decoder behaves. The patch 248eae1d broke this functionality by
hashing bits of required fields instead of storing them separately.
Consider a protobuf message with 129 fields where the first and the
last fields set as 'required'. In this case it is possible to trick
decoder to return incomplete ProtobufCMessage struct with missing
required fields by providing only one of the two fields in the
source byte stream. This can be considered as a security issue as
well because user's code do not expect incomplete messages with
missing required fields out from protobuf_c_message_unpack(). Such a
change could introduce undefined behaviour to user programs.
This patch is based on Andrei's fix and restores the exact detection of
missing required fields, but avoids doing a separate allocation for the
required fields bitmap except for messages whose descriptors define a
large number of fields. In the "typical" case where the message
descriptor has <= 128 fields we can just use a 16 byte array allocated
on the stack. (Note that the hash-based approach also used a 16 byte
stack allocated array.)
