The overall goal is to keep the code-generator as simple as possible. Hopefully performance isn't sacrificed to that end!

Anyways, we generate very little code: we mostly generate structure definitions (for example enums and structures for messages) and some metadata which is basically reflection-type data.

The serializing and deserializing is implemented in a library, called libprotobuf-c rather than generated code.

For each enum, we generate a C enum. For each message, we generate a C structure which can be cast to a ProtobufCMessage.

For each enum and message, we generate a descriptor object that allows us to implement a kind of reflection on the structures.

First, some naming conventions:

We also generate descriptor objects for messages and enums. These are declared in the .h files:

   extern const ProtobufCMessageDescriptor
                     foo__bar__baz_bah__descriptor;
  

The message structures all begin with ProtobufCMessageDescriptor* which is sufficient to allow them to be cast to ProtobufCMessage.

We generate some functions for each message:

This library is used by the generated code; it includes common structures and enums, as well as functions that most users of the generated code will want.

There are three main components:

For example, enums are described in terms of structures:

    struct _ProtobufCEnumValue
    {
      const char *name;
      const char *c_name;
      int value;
    };

    struct _ProtobufCEnumDescriptor
    {
      const char *name;
      const char *short_name;
      const char *package_name;

      /* sorted by value */
      unsigned n_values;
      const ProtobufCEnumValue *values;

      /* sorted by name */
      unsigned n_value_names;
      const ProtobufCEnumValue *values_by_name;
    };

Likewise, messages are described by:

      struct _ProtobufCFieldDescriptor
      {
        const char *name;
        int id;
        ProtobufCFieldLabel label;
        ProtobufCFieldType type;
        unsigned quantifier_offset;
        unsigned offset;
        void *descriptor;       /* for MESSAGE and ENUM types */
      };
      struct _ProtobufCMessageDescriptor
      {
        const char *name;
        const char *short_name;
        const char *package_name;

        /* sorted by field-id */
        unsigned n_fields;
        const ProtobufCFieldDescriptor *fields;
      };

And finally services are described by:

      struct _ProtobufCMethodDescriptor
      {
        const char *name;
        const ProtobufCMessageDescriptor *input;
        const ProtobufCMessageDescriptor *output;
      };
      struct _ProtobufCServiceDescriptor
      {
        const char *name;
        unsigned n_methods;
        ProtobufCMethodDescriptor *methods;             // sorted by name
      };

We defined typedefs for a few types which are used in .proto files but do not have obvious standard C equivalents:

We also define a simple allocator object, ProtobufCAllocator that let's you control how allocations are done. This is predominately used for parsing.

There is a virtual buffer facility that only has to implement a method to append binary-data to the buffer. This can be used to serialize messages to different targets (instead of a flat slab of data).

We define a base-type for all messages, for code that handles messages generically. All it has is the descriptor object.

   struct _ProtobufCBuffer
   {
     void (*append)(ProtobufCBuffer     *buffer,
                    size_t               len,
                    const unsigned char *data);
   }

   typedef struct
   {
     ProtobufCBuffer base;
     FILE *fp;
   } BufferAppendToFile
   static void my_buffer_file_append (ProtobufCBuffer   *buffer,
                               unsigned         len,
                               const unsigned char *data)
   {
     BufferAppendToFile *file_buf = (BufferAppendToFile *) buffer;
     fwrite (data, len, 1, file_buf->fp);  // XXX: no error handling!
   }

     ...
     BufferAppendToFile tmp;
     tmp.base.append = my_buffer_file_append;
     tmp.fp = fp;
     protobuf_c_message_pack_to_buffer (&message, &tmp);
     ...

    unsigned char pad[128];
    ProtobufCBufferSimple buf = PROTOBUF_C_BUFFER_SIMPLE_INIT (pad);
    ProtobufCBuffer *buffer = (ProtobufCBuffer *) &simple;
    protobuf_c_buffer_append (buffer, 6, (unsigned char *) "hi mom");
  

   assert (buf.len == 6);
   assert (memcmp (buf.data, "hi mom", 6) == 0);
 

    PROTOBUF_C_BUFFER_SIMPLE_CLEAR (&buf);
  

To pack messages one first computes their packed size, then provide a buffer to pack into.

    size_t protobuf_c_message_get_packed_size
                                     (ProtobufCMessage *message);
    void   protobuf_c_message_pack   (ProtobufCMessage *message,
                                      unsigned char    *out);

Or you can use the "streaming" approach:

    void   protobuf_c_message_pack_to_buffer
                                     (ProtobufCMessage *message,
                                      ProtobufCBuffer  *buffer);

where ProtobufCBuffer is a base object with an append metod. See the section called “Buffers”.

To unpack messages, you should simple call

      ProtobufCMessage *
         protobuf_c_message_unpack (const ProtobufCMessageDescriptor *,
                                    ProtobufCAllocator  *allocator,
				    size_t               len,
				    const unsigned char *data);

If you pass NULL for allocator, then the default allocator will be used.

You can cast the result to the type that matches the descriptor.

The result of unpacking should be freed with protobuf_c_message_free().

Dave Benson.