This job builds abseil-cpp and protobuf from source using Cmake and
tries to build protobuf-c against those dependencies using Cmake.
The "latest" build dependencies are being targeted, currently Ubuntu
22.04, abseil-cpp from the lts_2024_01_16 branch, protobuf from the 27.x
branch. Update as newer versions become available.
This uses Cmake's Ninja generator which should be equivalent to the
legacy makefiles that it generates. The Cmake Ninja generator is useful
primarily because it automatically compiles using the maximum amount
of concurrency and it actually shows you the command that failed by
default.
PROTOBUF_PROTOC_EXECUTABLE is not set in the Protobuf CMake config file.
Set it properly in case we use CMake protobuf config file.
Signed-off-by: Clément Péron <peron.clem@gmail.com>
APT cache could be out of date leading to a:
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Fix this by always calling apt-get update before an apt-get install.
Signed-off-by: Clément Péron <peron.clem@gmail.com>
The CMake module is used by default but isn't compatible with
recent protobuf version.
Try to first look for a protobuf config then fallback to legacy
cmake module.
Signed-off-by: Clément Péron <peron.clem@gmail.com>
The Google protobuf project is currently experimenting with a new syntax
for .proto files called "editions". Since protobuf-c is a proto2/proto3
compiler, after the previous commit reimplementing `FieldSyntax()`, the
protobuf compiler will abort like this if presented with an "editions"
syntax .proto file due to the safety check in `FieldSyntax()`:
$ protoc --experimental_editions --c_out=. test.proto
protoc-gen-c: ./protoc-c/c_helpers.h:178: int google::protobuf::compiler::c::FieldSyntax(const google::protobuf::FieldDescriptor*): Assertion `syntax == "proto2" || syntax == "proto3"' failed.
--c_out: protoc-gen-c: Plugin killed by signal 6.
On protobuf 26, our `CodeGenerator` can implement certain methods to
declare that we "support" editions, and then reject any other edition
except proto2 and proto3, which have apparently been retroactively
declared to be "editions". Of course this needs to be wrapped in a
version guard.
With this protection in place, the protobuf compiler cleanly exits with
a nice error message like this:
$ protoc --experimental_editions --c_out=. test.proto
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1710988958.296200 20022 descriptor.cc:4620] Invalid proto descriptor for file "test.proto":
E0000 00:00:1710988958.296239 20022 descriptor.cc:4623] test.proto: Edition 2023 is later than the maximum supported edition PROTO3
--c_out: protoc-gen-c: Plugin failed with status code 1.
Recent versions of Google protobuf have broken the interfaces for
determining the syntax version of a .proto file. The current protobuf-c
1.5.0 release does not compile with Google protobuf 26.0 due to the most
recentage breakage. There is a possible workaround involving the Google
protobuf `FileDescriptorLegacy` class, which is documented as:
// TODO Remove this deprecated API entirely.
So we probably shouldn't rely on it.
Instead, this commit obtains the `FileDescriptorProto` corresponding
to the passed in `FieldDescriptor` and interrogates the `syntax` field
directly. This is a single implementation with no version-specific
workarounds. Hopefully this won't break in the next Google protobuf
release.
I tested the `FieldSyntax()` implementation in this commit across a
number of different Google protobuf releases and found that it worked
(`make && make check`) on all of them:
- Google protobuf 3.6.1.3 (Ubuntu 20.04)
- Google protobuf 3.12.4 (Ubuntu 22.04)
- Google protobuf 3.21.12 (Debian 12 + Debian unstable)
- Google protobuf 3.25.2 (Debian experimental)
- Google protobuf 26.1-dev
Previously, we were conditionally trying to set `min_header_version` to
the lowest possible value, and relying on a "legacy" Google interface to
determine the file descriptor's syntax version as part of that
determination.
Instead, simply bump the minimum version to 1003000 (1.3.0). This
release was almost 7 years ago. In practice protobuf-c users should not
be shipping pre-compiled .pb-c.c/.pb-c.h files, anyway.
A memory corruption in protobuf_c_message_free_unpacked happens at the
following line:
if (message->unknown_fields != NULL)
do_free(allocator, message->unknown_fields);
The do_free will free ->unknown_fields. This is may be wrong, because
protobuf_c_message_unpack uses malloc as the default allocator, allocates
rv with malloc. At the end, however, ->unknown_fields is only initialized
if there are some. That means if there are no such fields ->unknown_fields
is an uninitialized pointer.
The patch initializes the pointer to NULL to ensure the check before free
is performed on initialized memory in case there is no unknown_field.
This fixes https://github.com/protobuf-c/protobuf-c/issues/690
Signed-off-by: Stephan Mueller <smueller@chronox.de>
This file uses declarations after statements and fixing it would make
the code less readable, so silence this particular diagnostic in this
particular file.
Since we now require protobuf >= 3.0.0, we don't have to support older
protobuf versions that don't recognize the `syntax` syntax, so we can
put `syntax = "proto2";` on these proto files to silence the protoc
compiler's diagnostic "No syntax specified for the proto file".
Now that we require protobuf >= 3.0.0, we are always building with
proto3 syntax support.
Note that the test-generated-code3 test keeps `-DPROTO3` which is
separate from the HAVE_PROTO3 identifier. This test is building built
from the same source file as test-generated-code but the source file
supports proto2 and proto3 syntax depending on how it's compiled.
Signed-off-by: Daniel Egger <daniel@eggers-club.de>
[edmonds: From https://github.com/protobuf-c/protobuf-c/pull/523.]
[edmonds: Adjust find string from "0 |" to "0 | ", test against `== 0`
rather than `!= std::string::npos` so that the find/erase calls are
anchored to the same portion of the string.]
This commit updates a few tests where we aren't using the correct C
signature for main() or are not using any of its parameters (in the
case of t/version/version.c.)
This commit fixes the following compiler diagnostics:
../protoc-c/c_helpers.cc: In function ‘void google::protobuf::compiler::c::PrintComment(google::protobuf::io::Printer*, std::__cxx11::string)’:
../protoc-c/c_helpers.cc:221:25: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<std::__cxx11::basic_string<char> >::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
for (int i = 0; i < comment_lines.size(); i++)
~~^~~~~~~~~~~~~~~~~~~~~~
../protoc-c/c_helpers.cc: In function ‘std::set<std::__cxx11::basic_string<char> > google::protobuf::compiler::c::MakeKeywordsMap()’:
../protoc-c/c_helpers.cc:273:21: warning: comparison of integer expressions of different signedness: ‘int’ and ‘long unsigned int’ [-Wsign-compare]
for (int i = 0; i < GOOGLE_ARRAYSIZE(kKeywordList); i++) {
^
There are some recent reports of strange build failures that might have
been fixed by compiling in C++17 mode, so it might be a good idea to use
C++17 in general, and not just when building against very recent
protobuf versions.
Since it looks like we've lost protobuf < 3.0.0 support, configure.ac
can be simplified a bit if we just use C++17 by default.
Use the newer "legacy" way of determining whether a file descriptor is
using proto2 or proto3 syntax on protobuf >= 23.0.
Based on
66574f3fd8
but continues to support older versions of protobuf.
Unfortunately, since this is a "deprecated", "legacy" API it'll probably
disappear in about five seconds.
It's unclear from looking at the online protobuf documentation whether
C++14 or C++17 is now required when building against newer versions of
protobuf (or perhaps I'm looking in the wrong place), but experimentally
it did not work for me with C++14.
This commit rewrites the versioned PKG_CHECK_MODULES checks in
configure.ac to enforce the following policies:
* Require protobuf >= 2.6.0.
* On protobuf >= 3.0.0, require C++11.
* On protobuf >= 4.22.0, require C++17.
According to the protobuf migration guide as of June 2023 [0], protobuf
22.0 (aka 4.22.0) took a dependency on something called "abseil" and as
a result the "stubs" have been removed. This apparently caused all the
uses of GOOGLE_* identifiers in protoc-c to fail when building against
newer versions of protobuf.
This commit introduces compatibility definitions when building against
protobuf >= 4.22.0 so that protobuf-c can build against older and newer
versions of protobuf.
[0] https://web.archive.org/web/20230611151200/https://protobuf.dev/support/migration/#abseil
Looking at where these identifiers are ultimately defined [0], it looks
like "DFATAL" means either "ERROR" or "FATAL" depending on whether
NDEBUG is defined. However, looking at the actual code sites in protoc-c
where DFATAL is used, it's not clear why we couldn't just use FATAL
unconditionally.
This is aimed at supporting newer versions of protobuf where the DFATAL
identifier apparently no longer exists.
[0] https://github.com/protocolbuffers/protobuf/blob/v21.12/src/google/protobuf/stubs/logging.h#L61-L65
protobuf has removed the definition of this macro as of commit
1595417dd3859bbff7d3d61ad0b6e39044d47489, so the invocation of this
macro in protobuf-c breaks the build when building agaist the protobuf
22.x or 23.x series.
Simply removing the macro invocations seems to be safe and doesn't break
the build on Debian's protobuf 3.21.12 nor Debian's protobuf 3.6.1.3.
protoc depends on protobuf libraries for execution. Order libraries
accordingly. Observing link errors with yocto's meta-mingw project
during static linkage.
Signed-off-by: Sinan Kaya <sinan.kaya@microsoft.com>
The steps to add this functionality are from the cmake tutorial:
https://cmake.org/cmake/help/latest/guide/tutorial/index.html
The include directories being set on the targets is necessary
for dependent targets. There is some redundancy with the
INCLUDE_DIRECTORIES calls alreday present, but I left it alone
to make the PR smaller and more approachable.
The project currently uses mixed cases in functions. This uses
lowercase on calls I touched, as this is what CMake documentation
uses and seems to be the preferred way.
I also started wrapping long lines, as some of the lines I touched
were over 170 characters.
I tested this on both Mac OS 10.15 Catalina and CentOS 7 (with
devtoolset-7) with CMake 3.10.3.
Will work on tests in CI next.
From https://developers.google.com/protocol-buffers/docs/proto3#updating:
int32, uint32, int64, uint64, and bool are all compatible – this means
you can change a field from one of these types to another without
breaking forwards- or backwards-compatibility. If a number is parsed
from the wire which doesn't fit in the corresponding type, you will
get the same effect as if you had cast the number to that type in C++
(e.g. if a 64-bit number is read as an int32, it will be truncated to
32 bits).
Until this fix, protobuf-c did not conform to the rule above when it
came to deserializing non-boolean packed repeated data into a
protobuf_c_boolean array. Fully scan the varint and use parse_boolean to
truncate the resulting value.
Fixes#440
FullName* methods are designed to split a full name based on '.' in the
name. name() does not have '.' separators, so call CamelToLower and
CamelToUpper in place of FullNameToLower and FullNameToUpper.
Allows for generic oneof offset lookups based on the name of the oneof,
instead of having to know the field name. All oneof fields should have
the same offset, since they are members of a union.
Fixes#204
Allows treating proto strings as ProtobufCBinaryData to work around
limitations such as NULL characters in strings, which are allowed in
protobuf, but not allowed in 'char *' types.
Fixes#203
