The Google protobuf project is currently experimenting with a new syntax
for .proto files called "editions". Since protobuf-c is a proto2/proto3
compiler, after the previous commit reimplementing `FieldSyntax()`, the
protobuf compiler will abort like this if presented with an "editions"
syntax .proto file due to the safety check in `FieldSyntax()`:
$ protoc --experimental_editions --c_out=. test.proto
protoc-gen-c: ./protoc-c/c_helpers.h:178: int google::protobuf::compiler::c::FieldSyntax(const google::protobuf::FieldDescriptor*): Assertion `syntax == "proto2" || syntax == "proto3"' failed.
--c_out: protoc-gen-c: Plugin killed by signal 6.
On protobuf 26, our `CodeGenerator` can implement certain methods to
declare that we "support" editions, and then reject any other edition
except proto2 and proto3, which have apparently been retroactively
declared to be "editions". Of course this needs to be wrapped in a
version guard.
With this protection in place, the protobuf compiler cleanly exits with
a nice error message like this:
$ protoc --experimental_editions --c_out=. test.proto
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1710988958.296200 20022 descriptor.cc:4620] Invalid proto descriptor for file "test.proto":
E0000 00:00:1710988958.296239 20022 descriptor.cc:4623] test.proto: Edition 2023 is later than the maximum supported edition PROTO3
--c_out: protoc-gen-c: Plugin failed with status code 1.
Recent versions of Google protobuf have broken the interfaces for
determining the syntax version of a .proto file. The current protobuf-c
1.5.0 release does not compile with Google protobuf 26.0 due to the most
recentage breakage. There is a possible workaround involving the Google
protobuf `FileDescriptorLegacy` class, which is documented as:
// TODO Remove this deprecated API entirely.
So we probably shouldn't rely on it.
Instead, this commit obtains the `FileDescriptorProto` corresponding
to the passed in `FieldDescriptor` and interrogates the `syntax` field
directly. This is a single implementation with no version-specific
workarounds. Hopefully this won't break in the next Google protobuf
release.
I tested the `FieldSyntax()` implementation in this commit across a
number of different Google protobuf releases and found that it worked
(`make && make check`) on all of them:
- Google protobuf 3.6.1.3 (Ubuntu 20.04)
- Google protobuf 3.12.4 (Ubuntu 22.04)
- Google protobuf 3.21.12 (Debian 12 + Debian unstable)
- Google protobuf 3.25.2 (Debian experimental)
- Google protobuf 26.1-dev
Previously, we were conditionally trying to set `min_header_version` to
the lowest possible value, and relying on a "legacy" Google interface to
determine the file descriptor's syntax version as part of that
determination.
Instead, simply bump the minimum version to 1003000 (1.3.0). This
release was almost 7 years ago. In practice protobuf-c users should not
be shipping pre-compiled .pb-c.c/.pb-c.h files, anyway.
Signed-off-by: Daniel Egger <daniel@eggers-club.de>
[edmonds: From https://github.com/protobuf-c/protobuf-c/pull/523.]
[edmonds: Adjust find string from "0 |" to "0 | ", test against `== 0`
rather than `!= std::string::npos` so that the find/erase calls are
anchored to the same portion of the string.]
This commit fixes the following compiler diagnostics:
../protoc-c/c_helpers.cc: In function ‘void google::protobuf::compiler::c::PrintComment(google::protobuf::io::Printer*, std::__cxx11::string)’:
../protoc-c/c_helpers.cc:221:25: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<std::__cxx11::basic_string<char> >::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
for (int i = 0; i < comment_lines.size(); i++)
~~^~~~~~~~~~~~~~~~~~~~~~
../protoc-c/c_helpers.cc: In function ‘std::set<std::__cxx11::basic_string<char> > google::protobuf::compiler::c::MakeKeywordsMap()’:
../protoc-c/c_helpers.cc:273:21: warning: comparison of integer expressions of different signedness: ‘int’ and ‘long unsigned int’ [-Wsign-compare]
for (int i = 0; i < GOOGLE_ARRAYSIZE(kKeywordList); i++) {
^
Use the newer "legacy" way of determining whether a file descriptor is
using proto2 or proto3 syntax on protobuf >= 23.0.
Based on
66574f3fd8
but continues to support older versions of protobuf.
Unfortunately, since this is a "deprecated", "legacy" API it'll probably
disappear in about five seconds.
According to the protobuf migration guide as of June 2023 [0], protobuf
22.0 (aka 4.22.0) took a dependency on something called "abseil" and as
a result the "stubs" have been removed. This apparently caused all the
uses of GOOGLE_* identifiers in protoc-c to fail when building against
newer versions of protobuf.
This commit introduces compatibility definitions when building against
protobuf >= 4.22.0 so that protobuf-c can build against older and newer
versions of protobuf.
[0] https://web.archive.org/web/20230611151200/https://protobuf.dev/support/migration/#abseil
Looking at where these identifiers are ultimately defined [0], it looks
like "DFATAL" means either "ERROR" or "FATAL" depending on whether
NDEBUG is defined. However, looking at the actual code sites in protoc-c
where DFATAL is used, it's not clear why we couldn't just use FATAL
unconditionally.
This is aimed at supporting newer versions of protobuf where the DFATAL
identifier apparently no longer exists.
[0] https://github.com/protocolbuffers/protobuf/blob/v21.12/src/google/protobuf/stubs/logging.h#L61-L65
protobuf has removed the definition of this macro as of commit
1595417dd3859bbff7d3d61ad0b6e39044d47489, so the invocation of this
macro in protobuf-c breaks the build when building agaist the protobuf
22.x or 23.x series.
Simply removing the macro invocations seems to be safe and doesn't break
the build on Debian's protobuf 3.21.12 nor Debian's protobuf 3.6.1.3.
FullName* methods are designed to split a full name based on '.' in the
name. name() does not have '.' separators, so call CamelToLower and
CamelToUpper in place of FullNameToLower and FullNameToUpper.
Allows for generic oneof offset lookups based on the name of the oneof,
instead of having to know the field name. All oneof fields should have
the same offset, since they are members of a union.
Fixes#204
Allows treating proto strings as ProtobufCBinaryData to work around
limitations such as NULL characters in strings, which are allowed in
protobuf, but not allowed in 'char *' types.
Fixes#203
Add options controller helper function generation. Preserve existing
behavior of not generating pack/unpack functions for sub-messages if
option is not explicitly set.
Fixes#240Fixes#442
Allows suppression of .pb-c.{c,h} file generation on a .proto file
basis.
protobuf-c.proto: Use the new option in itself, because the options are
never used by protobuf-c runtime, only by the protoc-gen-c compiler.
t/test-full.proto: import protobuf-c.proto to test that
protobuf-c.pb-c.h dependency does not get included in test-full.pb-c.h.
The protobuf common.h header currently has a "using std::string;"
statement that pulls std::string into the google::protobuf namespace:
ce66f6047d/src/google/protobuf/stubs/common.h (L195)
I plan to delete that line soon. To keep protobuf-c working, this commit
updates the generator to fully qualify std::string.
Fixes#188.
Implements suggested fix from the issue:
The struct tag namespace is different from the type namespace. Use the
same symbol names (i.e. without leading underscores) in every namespace.
Upstream protobuf removed scoped_ptr.h, which provided the scoped_ptr
that we depended on, in commit 67952fab2c766ac5eacc15bb78e5af4039a3d398
(67952fab2c).
This commit converts uses of scoped_ptr type to idiomatic C++11
std::unique_ptr, similar to the changes in the protobuf commit.
This has been tested on protobuf 3.0.0 (the version currently in Debian)
as well as the latest protobuf 3.6.1 release.
The change in commit 712154b912de824741381c0bb26c2fbed54515a3 ("Bump
minimum required header version for proto3 syntax") uses functionality
only exposed by protobuf-3.x, breaking the build when compiling against
protobuf-2.x.
Since we still want to support building against protobuf-2.x, this
commit makes the proto3 syntax check in the file generator dependent on
building against protobuf-3.x.
The changes in #274 (proto3: make strings default to "" instead of NULL)
add a new symbol to the library and add an 'extern' declaration to the
protobuf-c header file.
Since the compiler may generate files that depend on that new
'protobuf_c_empty_string' declaration in protobuf-c.h, we need to bump
the min_header_version value that is written into generated header
files. But since the 'protobuf_c_empty_string' symbol can only be
referenced when generating proto3 syntax files, we only need to use the
stricter min_header_version value when processing a proto3 file.
The protobuf project removed "using namespace std" namespace pollution
from their stubs/common.h header file. This caused build failures for us
since we relied on their namespace pollution.
This commit updates protobuf-c to convert:
'map' → 'std::map'
'set' → 'std::set'
'back_insert_iterator' → 'std::back_insert_iterator'
The spec talks about "empty string" and other languages like C#
return a zero length string and not null.
This is useful because when moving from proto2's "required string"
to a proto3's plain string we will be guaranteed that we
never get a null pointer.
The tradeoff is adding a special case to the library but avoiding
a lot of null checks in the calling code.
The current code is already special casing "" in pack_string.
This commit fixes#251, which causes incorrect code to be generated when
there are multiple oneofs in the same message.
In #221, we added code to force int-sizing for oneof enums, but we only
set vars["opt_comma"] initially, before entering the loop over the
message's oneofs. This caused commas to be omitted when generating the
enums for subsequent oneofs after the first oneof.
This commit resets vars["opt_comma"] every time through the loop that
generates the enum declarations for the message's oneofs.
This commit fixes a problem identified in #251. When generating the
PROTOBUF_C__FORCE_ENUM_TO_BE_INT_SIZE directive at the end of the enum
values generated for a oneof, we inadvertently only included the
protobuf class name in the call to the directive. This breaks if there
is more than one oneof per protobuf class, which causes duplicate enum
names to be generated.
This issue is fixed by also appending the oneof's name to the protobuf
class name when generating the PROTOBUF_C__FORCE_ENUM_TO_BE_INT_SIZE
directive, which should cause unique enum names to be generated.
This changes the enum name that is generated, but these names are
implementation details since they are sentinel values that only exist to
force the compiler to size the enum type correctly.
(The test case for #220 is updated since that test relies on the
oneof implementation details in the code generator.)
This makes __free_unpacked() consistent with free(), and simplifies
callers by allowing them to indiscriminately free message objects
without regard to whether they have been allocated or not.
This is a first cut at adding proto3 support.
As far as I understand protobuf-c already has pretty much everything
needed once it is built using a new version of protobuf itself.
The only missing thing is that in proto3 all fields are optional and
having to manually set has_foo is inconvenient.
This patch special cases the proto3 syntax files so that structs for the
bytes, enum and primitive fields do not emit the has_ field.
It also adds PROTOBUF_C_LABEL_NONE to the label enum that is used for
proto3 fields. When a fields has this label, the quantifier is not
consulted and instead the field is packed/unpacked depending on
whether it has a value different from NULL/0.
From https://developers.google.com/protocol-buffers/docs/proto#optional:
If the default value is not specified for an optional element, a
type-specific default value is used instead: for strings, the default
value is the empty string. For bools, the default value is false. For
numeric types, the default value is zero. For enums, the default value
is the first value listed in the enum's type definition. This means care
must be taken when adding a value to the beginning of an enum value
list.
Prior to this change, protoc-c set the default enum value to 0, whether
or not 0 was the first value listed in the enum's type definition (or if
it even was listed at all).
