How Do You Encode and Decode Protocol Buffers Messages in C Using Delimited I/O?

C Equivalents for Java's Protocol Buffers Delimited I/O Functions

In both C and Java, the need arises to read and write multiple Protocol Buffers messages from files. Java version 2.1.0 offers a set of "Delimited" I/O functions for this purpose:

• parseDelimitedFrom
• mergeDelimitedFrom
• writeDelimitedTo

These functions facilitate the attachment of length prefixes before each message. However, it remains unclear whether such capabilities exist in C .

Existence of C Equivalents

Initially, there were no direct C equivalents to these Java functions. However, as of version 3.3.0, C now features delimited message utility functions in google/protobuf/util/delimited_message_util.h.

Format of Size Prefixes

For users seeking to implement their own parsers in C before the release of these official utilities, it is important to understand the wire format for the size prefixes attached by the Java API. The format adheres to the following guidelines:

• The delimiters must be present even before the first message.
• The size of the message is encoded as a 32-bit varint.
• A 1-byte delimiter byte (0x0A) terminates each message, and the next length-prefixed message begins immediately afterward.

Optimized C Implementations

Following the release of the official C utility functions, several optimizations were discovered that are missing from the originally proposed implementations. These optimized functions, which are provided below, offer improved performance and avoid potential errors:

<code class="cpp">bool writeDelimitedTo(
    const google::protobuf::MessageLite& message,
    google::protobuf::io::ZeroCopyOutputStream* rawOutput) {
  // Create a new coded stream for each message.
  google::protobuf::io::CodedOutputStream output(rawOutput);

  // Write the message size.
  const int size = message.ByteSize();
  output.WriteVarint32(size);

  // Serialize the message directly to the output buffer if possible.
  uint8_t* buffer = output.GetDirectBufferForNBytesAndAdvance(size);
  if (buffer != NULL) {
    message.SerializeWithCachedSizesToArray(buffer);
  } else {
    // Use a slower path if the message spans multiple buffers.
    message.SerializeWithCachedSizes(&output);
    if (output.HadError()) return false;
  }

  return true;
}

bool readDelimitedFrom(
    google::protobuf::io::ZeroCopyInputStream* rawInput,
    google::protobuf::MessageLite* message) {
  // Create a new coded stream for each message.
  google::protobuf::io::CodedInputStream input(rawInput);

  // Read the message size.
  uint32_t size;
  if (!input.ReadVarint32(&size)) return false;

  // Set a read limit to enforce the 64 MB per-message size constraint.
  google::protobuf::io::CodedInputStream::Limit limit =
      input.PushLimit(size);

  // Parse the message.
  if (!message->MergeFromCodedStream(&input)) return false;
  if (!input.ConsumedEntireMessage()) return false;

  // Remove the read limit.
  input.PopLimit(limit);

  return true;
}</code>

The above is the detailed content of How Do You Encode and Decode Protocol Buffers Messages in C Using Delimited I/O?. For more information, please follow other related articles on the PHP Chinese website!