67 Commits

Author SHA1 Message Date
Scott Graham
60ae9eeadb Implementation of HTTPTransport via raw socket
Partial implementation: Currently only handles http (i.e. no TLS), only
POST, and only certain response types (only when Content-Length is
specified, and not chunked). Used for Linux and Fuchsia lacking anything
better (that's shippable). Removes libcurl HTTPTransport, since it isn't
available in the Chromium sysroot anyway.

This is an intermediate step until BoringSSL is available in the Fuchsia
SDK. Once that's available, it should be "relatively straightfoward" to
make http_transport_socket.cc secure its socket using BoringSSL or
OpenSSL depending on the platform.

Bug: crashpad:196, crashpad:227, crashpad:30
Change-Id: If33a0d3f11b9000cbc3f52f96cd024ef274a922f
Reviewed-on: https://chromium-review.googlesource.com/1022717
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Joshua Peraza <jperaza@chromium.org>
2018-04-27 04:04:30 +00:00
Scott Graham
78689cc810 fuchsia: Depend on fdio_config when in-tree
Fixes .so link errors from
https://logs.chromium.org/v/?s=fuchsia%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8948324805139372352%2F%2B%2Fsteps%2Fbuild%2F0%2Fsteps%2Fbuild_fuchsia%2F0%2Fsteps%2Fninja%2F0%2Fstdout

[7302/11459] SOLINK x64-shared/crashpad_snapshot_test_module_small.so

FAILED: x64-shared/crashpad_snapshot_test_module_small.so x64-shared/lib.unstripped/crashpad_snapshot_test_module_small.so
/b/s/w/ir/kitchen-workdir/cipd/goma/gomacc ../../buildtools/linux-x64/clang/bin/clang++ -shared -Wl,--threads --sysroot=/b/s/w/ir/kitchen-workdir/out/debug-x64/zircon_toolchain/obj/zircon/public/sysroot/sysroot --target=x86_64-fuchsia -march=x86-64 -mcx16 -g -Og -Wl,--icf=all -Wl,--no-undefined -o "x64-shared/lib.unstripped/crashpad_snapshot_test_module_small.so" -Wl,-soname="crashpad_snapshot_test_module_small.so" @"x64-shared/crashpad_snapshot_test_module_small.so.rsp" && ../../buildtools/linux-x64/clang/bin/llvm-objcopy --strip-all "x64-shared/lib.unstripped/crashpad_snapshot_test_module_small.so" "x64-shared/crashpad_snapshot_test_module_small.so"
/b/s/w/ir/kitchen-workdir/out/debug-x64/../../buildtools/linux-x64/clang/bin/ld.lld: error: undefined symbol: dirfd
>>> referenced by directory_reader_posix.cc:72 (../../third_party/crashpad/util/file/directory_reader_posix.cc:72)
>>>               libutil.directory_reader_posix.o:(crashpad::DirectoryReader::DirectoryFD()) in archive x64-shared/obj/third_party/crashpad/util/libutil.a
/b/s/w/ir/kitchen-workdir/out/debug-x64/../../buildtools/linux-x64/clang/bin/ld.lld: error: undefined symbol: fdio_ioctl
>>> referenced by sysinfo.h:32 (../../out/debug-x64/zircon_toolchain/obj/zircon/public/sysroot/sysroot/include/zircon/device/sysinfo.h:32)
>>>               libutil.koid_utilities.o:(ioctl_sysinfo_get_root_job(int, unsigned int*)) in archive x64-shared/obj/third_party/crashpad/util/libutil.a
clang-7.0: error: ld.lld command failed with exit code 1 (use -v to see invocation)

Bug: crashpad:196
Change-Id: I50b50a8ba668d75d40e86d863976c386bb382f65
Reviewed-on: https://chromium-review.googlesource.com/1028840
Reviewed-by: Joshua Peraza <jperaza@chromium.org>
Commit-Queue: Scott Graham <scottmg@chromium.org>
2018-04-25 21:28:29 +00:00
Joshua Peraza
9c89cd99f3 gn: add templates for executables and loadable_modules
When building in chromium, executables and loadable_modules should
depend on:
//build/config:exe_and_shlib_deps
which, among other things, may be needed to introduce a
dependency on a custom libc++.

Bug: crashpad:30
Change-Id: Ic46a3cf5b46bdac09cca22950f9236e0776ba44a
Reviewed-on: https://chromium-review.googlesource.com/974713
Reviewed-by: Mark Mentovai <mark@chromium.org>
2018-03-23 16:50:58 +00:00
Scott Graham
8e80a575d1 Linux: Pull a sysroot if pulling a local clang
Didn't actually end up being too bad. Also requires setting GN arg of

  target_sysroot = "//third_party/linux/sysroot"

when building.

Bug: crashpad:220
Change-Id: I4d4b282f165d454b5d32fc8cc11287ff665b943d
Reviewed-on: https://chromium-review.googlesource.com/935981
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Joshua Peraza <jperaza@chromium.org>
2018-02-24 04:31:12 +00:00
Scott Graham
3030ae5417 fuchsia: Fix ninja auto-regen after run
After the avoidance of abspath(), automatic regeneration of ninja files
was broken following a test run. The problem is that the
--runtime-deps-list-file argument gets saved into the regeneration rule,
but it's relative to the cwd. The cwd is CRASHPAD_DIR on the first run,
but the binary_dir on regenerations, so either way it doesn't work (this
should probably fixed in either GN or ninja).

We could abspath the path the runtime deps targets file to avoid this.
However, it's a bit cluttery to have that --runtime-deps-list-file in
the regeneration rule anyway, when really it's only required to extract
runtime deps at test-running time. (Also, if you happened to delete only
targets.txt from the out dir, the regeneration would mysteriously fail.)
So since generation only takes tens of milliseconds, the best thing to
do is just remove it from the regeneration rule by re-running gn gen
without the flag after we've extracted the .runtime_deps to prepare for
the run.

Bug: crashpad:196, chromium:814816
Change-Id: I009851d8b821fef5c953d463ba9c4880e5cc082a
Reviewed-on: https://chromium-review.googlesource.com/929887
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2018-02-22 18:05:55 +00:00
Scott Graham
8ee14eef08 fuchsia: Fix some packaging when run isn't from Crashpad source root
- Use gn --root rather than setting the cwd when running GN so it can
find //.gn.

- Use source-relative path for assets when building the target path, see
for example the failure in:
https://build.chromium.org/p/client.crashpad/builders/crashpad_fuchsia_x64_rel/builds/77/steps/run%20tests/logs/stdio

Bug: crashpad:196
Change-Id: If95636fcb826c22d9d9543cad02f815780621414
Reviewed-on: https://chromium-review.googlesource.com/923436
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2018-02-16 21:58:18 +00:00
Scott Graham
798f654043 gn win: Use new toolchain names to support both x86 and x64
Rolls mini_chromium to include:
75b5697 gn win: Template toolchain to support x86 and x64
428a555 gn win: Automatically find location of Visual Studio

Bug: crashpad:79
Change-Id: Ie7ff10f8b4853071028e3a9580e5be8e5ca5c0ec
Reviewed-on: https://chromium-review.googlesource.com/905625
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2018-02-06 23:35:49 +00:00
Scott Graham
14dbd3531d gn win: Get main test binaries building
- default to subsystem:console
- don't build posix/timezone.*
- add some missing libs

This gets all the main binaries building and running. Most configs pass,
but there's some offsets that seem different in some builds; need to
investigate more. Additionally, the binaries used by end_to_end_test.py
aren't yet built, so that script fails.

Includes mini_chromium roll to 46eeaf9:
46eea49 gn win: Add debug info and pdb to cc/cxx
902a29f gn win: Various fixes towards making GN build work

Bug: crashpad:79
Change-Id: Ie56a469b84bed7b0330172cec9f1a8aeb95f702e
Reviewed-on: https://chromium-review.googlesource.com/902403
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2018-02-06 19:12:18 +00:00
Scott Graham
1f1657d573 Read either DT_HASH or DT_GNU_HASH to determine the size of DT_SYMTAB
Without the section headers for the symbol table, there's no direct way
to calculate the number of entries in the table.

DT_HASH and DT_GNU_HASH are auxiliary tables that are designed to make
symbol lookup faster. DT_HASH is the original and is theoretically
mandatory. DT_GNU_HASH is the new-and-improved, but is more complex.

In practice, however, an Android build (at least vs. API 16) has only
DT_HASH, and not DT_GNU_HASH, and a Fuchsia build has only DT_GNU_HASH
but not DT_HASH. So, both are tried.

This change does not actually use the data in these tables to improve
the speed of symbol lookup, but instead only uses them to correctly
terminate the linear search.

DT_HASH contains the total number of symbols in the symbol table fairly
directly because there is an entry for each symbol table entry in the
hash table, so the number is the same.

DT_GNU_HASH regrettably does not. Instead, it's necessary to walk the
buckets and chain structure to find the largest entry.

DT_GNU_HASH doesn't appear in any "real" documentation that I'm aware
of, other than the binutils code (at least as far as I know). Some
more-and-less-useful references:
- https://flapenguin.me/2017/04/24/elf-lookup-dt-hash/
- https://flapenguin.me/2017/05/10/elf-lookup-dt-gnu-hash/
- http://deroko.phearless.org/dt_gnu_hash.txt
- https://sourceware.org/ml/binutils/2006-10/msg00377.html

Change-Id: I7cfc4372f29efc37446f0931d22a1f790e44076f
Bug: crashpad:213, crashpad:196
Reviewed-on: https://chromium-review.googlesource.com/876879
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
Reviewed-by: Joshua Peraza <jperaza@chromium.org>
2018-01-30 22:40:17 +00:00
Scott Graham
48abd4a60f Add CRASHPAD_CHILD_TEST_MAIN() helper for multiprocess tests
Extends MultiprocessExec to support running functions registered via
CRASHPAD_CHILD_TEST_MAIN() as the main of a new child process.

Additionally, implements Fuchsia exit code checking, and adds a
CRASHPAD_CHILD_TEST_MAIN()-based test for that.

Bug: crashpad:196, crashpad:215
Change-Id: I49ce3f4d95a3b9823813e6df5a602cee2583bcf8
Reviewed-on: https://chromium-review.googlesource.com/879563
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2018-01-25 22:59:20 +00:00
Scott Graham
7e7193d5f1 fuchsia: Update expected mini_chromium location for in-Fuchsia build
The initial plan was to have crashpad and mini_chromium be siblings in
the Fuchsia tree, but the Fuchsia team sensibly preferred to nest
mini_chromium inside Crashpad
https://fuchsia-review.googlesource.com/c/garnet/+/115120, so update the
expected location to match. Conveniently this is where mini_chromium
lives in a normal standalone build too.

Bug: crashpad:196
Change-Id: Iedab0dd557fa248c6419380b676b71427279bba1
Reviewed-on: https://chromium-review.googlesource.com/883569
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Joshua Peraza <jperaza@chromium.org>
2018-01-24 18:59:05 +00:00
Scott Graham
c9b41a4d15 fuchsia: Fix removal of /tmp dir on test run, and location of .so files
1. .so files should (I believe) be in /pkg/lib according to
https://fuchsia.googlesource.com/docs/+/master/namespaces.md#typical-directory-structure
(None of the tests that load .so actually work currently, so it doesn't
make anything better or worse to fix this.)

2. The test directory structure looks like:
- /tmp/<guid>/pkg/...
- /tmp/<guid>/tmp/...

Previously /tmp/<guid>/pkg/ and /tmp/<guid>/tmp/ were being removed, but
/tmp/<guid>/ wasn't so there was a bunch of empty directory garbage
being left in /tmp. Clean up from the root instead.

Bug: crashpad:196
Change-Id: I4e82198721f329d597e14a89da8bc77fcc8647c6
Reviewed-on: https://chromium-review.googlesource.com/880884
Reviewed-by: Mark Mentovai <mark@chromium.org>
Commit-Queue: Scott Graham <scottmg@chromium.org>
2018-01-23 19:49:10 +00:00
Scott Graham
3a7b80995d fuchsia: Default crashpad_dependencies based on is_fuchsia_tree
Bug: crashpad:196
Change-Id: I0ff0157287e8b84323068741e060ff3c10f9719a
Reviewed-on: https://chromium-review.googlesource.com/871044
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2018-01-18 23:48:58 +00:00
Scott Graham
4431f76d86 Switch run_tests.py to argparse, and support --gtest_filter
Starting to work on individual tests on Fuchsia, and it's nice to be
to run only one.

Bug: crashpad:196
Change-Id: I1c5d924d2c93ee943673883de0a6022a3666f98c
Reviewed-on: https://chromium-review.googlesource.com/865103
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2018-01-12 22:47:33 +00:00
Scott Graham
54dc88724e fuchsia: Fix run when gn binary path non-absolute
Oops. I thought GN abspath()d this in the regen rule, but it does not.

Bug: crashpad:196
Change-Id: I33dadc5502a5e56f20ba7e4c0403fbc138f052c9
Reviewed-on: https://chromium-review.googlesource.com/864629
Reviewed-by: Mark Mentovai <mark@chromium.org>
Commit-Queue: Scott Graham <scottmg@chromium.org>
2018-01-12 18:59:13 +00:00
Scott Graham
3bc4156c1e fuchsia: Fix qemu script return code on failure
https://build.chromium.org/p/client.crashpad/builders/crashpad_fuchsia_x64_dbg/builds/6
failed in "start qemu", but it wasn't clear that it did because the
script returned 0 when it did so.

Bug: crashpad:196
Change-Id: I7b8f42bc58c273b5b8e9a64e52288b32e9f9addf
Reviewed-on: https://chromium-review.googlesource.com/862990
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2018-01-12 01:45:21 +00:00
Scott Graham
f35799cb41 fuchsia: Ensure qemu instance is ready before continuing after 'start'
Ensures that enough networking is set up before returning so that the
VM will be ready to accept commands to run tests. Otherwise requests to
do so immediately after the VM is started can flakily fail.

Bug: crashpad:196
Change-Id: Idc231e7fc418ad054bb9ec115c1598c804055a96
Reviewed-on: https://chromium-review.googlesource.com/862709
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2018-01-11 23:53:31 +00:00
Scott Graham
2a51cb5fd5 fuchsia: Fixes for finding GN when in bot configuration
On buildbot, GN isn't in the path, and must be run with a full explicit
path. GN saves this to the build.gn for regeneration purposes, so
extract it out of there.

Additionally, set cwd when calling GN, otherwise, it will be unable to
find the .gn in the root of the source tree.

Bug: crashpad:196
Change-Id: Ia14db175ba0af6dc61b215da6ba5c624bca56886
Reviewed-on: https://chromium-review.googlesource.com/862547
Reviewed-by: Mark Mentovai <mark@chromium.org>
Commit-Queue: Scott Graham <scottmg@chromium.org>
2018-01-11 22:47:41 +00:00
Scott Graham
ab153f7e1b gn: Avoid depending on BUILDCONFIG.gn globals
Goes with https://chromium-review.googlesource.com/c/chromium/mini_chromium/+/834648.

Includes mini_chromium DEPS roll to pull in edfe51ce81

Bug: crashpad:79, crashpad:196
Change-Id: Ib45cc738aecf9ae727f8faeff81f3b71e2dc9de8
Reviewed-on: https://chromium-review.googlesource.com/834543
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2017-12-19 23:38:36 +00:00
Mark Mentovai
99b0030616 gn, linux: Build for Linux with GN
This is sufficient for a native Linux build using GN. Android is not yet
supported.

mini_chromium side: https://crrev.com/c/833407

This also updates mini_chromium to 404f6dbf9928.

c913ef97a236 gn, linux: Build for Linux with GN
404f6dbf9928 gn: Don’t use .rsp files; rationalize descriptions and
             output dirs

Bug: crashpad:79
Change-Id: I4f3b72fd02884d77812e520fb95231b35815677d
Reviewed-on: https://chromium-review.googlesource.com/833408
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
2017-12-19 19:31:20 +00:00
Scott Graham
1bc07b76ed gn: Use mini_chromium_is_posix in preference to global is_posix
Goes with
https://chromium-review.googlesource.com/c/chromium/mini_chromium/+/833328.

Also roll mini_chromium:

scottmg@around:/work/crashpad/crashpad/third_party/mini_chromium/mini_chromium$ git log 20182dd263312db9fad52042fc92c33331ec6904..e182031 --oneline
e182031 gn: Add is_posix.gni to define local is_posix variable
4cb1344 gn: Enable proper release-mode optimizations for POSIX-non-Mac
9c0eb0c Remove reference to ptr_util.h
c5ae5aa gn: Configure the sysroot in target_sysroot, not sysroot
f7e5654 gn, mac: Honor mac_sdk_min, sysroot, and mac_deployment_target
7701901 Remove the deprecated sparse_histogram.h header.
e2f0160 Use Chromium copyright notice and BSD license in mini_chromium


Bug: crashpad:79, crashpad:196
Change-Id: Ie41d971e0e769db2ed18861da07021c071f6c650
Reviewed-on: https://chromium-review.googlesource.com/833329
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2017-12-19 00:07:06 +00:00
Scott Graham
457cc6a34f gn: Refactor build files to avoid build/secondary
In doing standalone bringup of Crashpad targeting Fuchsia, it seemed
tidy to keep the same literal paths to the dependencies that Chromium
needed and add stubs/forwarding to build/secondary in the Crashpad tree
as required to make those work.

However, when trying to build Crashpad in the Fuchsia tree itself, that
would require adding forwarding files to the Fuchsia tree to match the
Chromium directory structure, which would be awkward. Instead, have
explicit dependencies in the Crashpad tree that select the locations
for various dependencies.

Bug: crashpad:79, crashpad:196
Change-Id: Ib506839f9c97d8ef823663cdc733cbdcfa126139
Reviewed-on: https://chromium-review.googlesource.com/826025
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2017-12-18 22:53:58 +00:00
Mark Mentovai
3650d56570 gn: Build gtest and gmock tests
Bug: crashpad:79
Change-Id: Iea78fcb6a758f57d2b550b214b947ca5aabad036
Reviewed-on: https://chromium-review.googlesource.com/827732
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
2017-12-15 23:10:52 +00:00
Scott Graham
05dee13e84 fuchsia: Add QEMU and helper script to start/stop instance
This is mostly intended to be used for waterfall/trybots.
Fuchsia-on-metal isn't available as a Swarming dimension, so in order to
run tests, use QEMU-with-KVM on the host. It might also be useful for
local development for those without a Fuchsia hardware device.

Bug: crashpad:196, crashpad:212
Change-Id: I88170bc95bd532676b787b50a94f7fa3c69b1ac7
Reviewed-on: https://chromium-review.googlesource.com/822523
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2017-12-13 23:18:05 +00:00
Mark Mentovai
13d0defbfb android, win: Enable colored test output when available
Following the discussion at https://crrev.com/c/818125/3//COMMIT_MSG#17,
this sets GTEST_COLOR=yes when running tests on an Android device via
“adb” being driven from a Windows host. This is only done when standard
output is attached to a console and when
ENABLE_VIRTUAL_TERMINAL_PROCESSING is supported (it is on Windows 10).
As usual, colored output can be suppressed by setting GTEST_COLOR=no.

This is only partially tested. Instead of running on-device tests via
adb, I substituted:
  print('\x1b[0;31mred\x1b[32mgreen\x1b[34mblue\x1b[0m')

Change-Id: I3ef67f3890f18f7012111171a5e0eab4addca7b8
Reviewed-on: https://chromium-review.googlesource.com/819597
Reviewed-by: Scott Graham <scottmg@chromium.org>
Commit-Queue: Mark Mentovai <mark@chromium.org>
2017-12-11 17:21:46 +00:00
Mark Mentovai
0cc63d4a45 android: Make run_tests.py work on Android versions before 7.0 (N)
I took yesterday’s work and tried using it to run tests on a Nexus 4
running 5.1.1 (L), and absolutely nothing worked. The highlights:

 - There’s no /system/bin/mktemp.
 - There’s no /system/bin/env.
 - “adb shell” doesn’t know what the command’s exit status was.

While I’m in here, I’ll also make colored gtest output work, although it
won’t work on the normal Windows console which doesn’t understand ANSI
color codes. (It might work in Cygwin?)

Plus some bonus bloopers:
 - I was trying to catch an exception that isn’t even defined in Python
   2!
 - The part of the script that tells you what test it’s about to run had
   fallen into a conditional block, preventing it from being shown
   except when running end_to_end_test.py.

Bug: crashpad:30
Change-Id: I98fc410f90a2b4e91cb3cacb6a8decf2a8c2252b
Reviewed-on: https://chromium-review.googlesource.com/818125
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
2017-12-11 16:15:04 +00:00
Mark Mentovai
659420bc7d android: Run tests by running run_tests.py on the build host
Bug: crashpad:30
Change-Id: Ie432c58c4a2505b6434861276512a5011fd285d4
Reviewed-on: https://chromium-review.googlesource.com/811891
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
2017-12-08 16:24:37 +00:00
Scott Graham
a4fc880278 gn, mac: Get basic generation working
A couple targets compile, but not linking yet.

Requires https://chromium-review.googlesource.com/c/chromium/mini_chromium/+/812410

Bug: crashpad:79
Change-Id: Ifc472628ec5233fc88c746e74a1f7b3ce0637f91
Reviewed-on: https://chromium-review.googlesource.com/811884
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2017-12-07 00:19:36 +00:00
Scott Graham
15c4fff902 Get crashpad_client_test and crashpad_handler_test building
Stubs a variety of classes (CrashReportExceptionHandler,
ExceptionHandlerServer, HTTPTransport, CrashReportDatabase).

Bug: crashpad:196
Change-Id: I4772f90d0d2ad07cc2f3c2ef119e92fde5c7acef
Reviewed-on: https://chromium-review.googlesource.com/809940
Reviewed-by: Mark Mentovai <mark@chromium.org>
Commit-Queue: Scott Graham <scottmg@chromium.org>
2017-12-06 18:39:24 +00:00
Scott Graham
6719610b8c fuchsia: Get crashpad_snapshot_test building
ProcessSnapshotFuchsia is just a stub, so running fails immediately.

Bug: crashpad:196
Change-Id: Ie281cc13c4ff4a6e9699e882dbd6207daaab346d
Reviewed-on: https://chromium-review.googlesource.com/809234
Reviewed-by: Mark Mentovai <mark@chromium.org>
Commit-Queue: Scott Graham <scottmg@chromium.org>
2017-12-06 17:31:57 +00:00
Scott Graham
bfeb194b01 fuchsia: Get crashpad_util_test building
Links, but various tests fail.

Also adds support to run_tests.py to run a single binary, likely only
useful on Fuchsia.

Bug: crashpad:196
Change-Id: Ie82ef26ec214ff4262194e877469953aa8fb367e
Reviewed-on: https://chromium-review.googlesource.com/809467
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2017-12-05 23:23:30 +00:00
Scott Graham
2403d066c4 fuchsia: Support runtime_deps of directories
This is necessary for crashpad_util_test, which has a GN data
specification that includes "net/util/testdata/".

Bug: crashpad:196
Change-Id: I7e03c8cbe448fd90c2481ad6a7e541827efebb0d
Reviewed-on: https://chromium-review.googlesource.com/809328
Reviewed-by: Mark Mentovai <mark@chromium.org>
Commit-Queue: Scott Graham <scottmg@chromium.org>
2017-12-05 20:14:57 +00:00
Scott Graham
741a84a298 fuchsia: Add runner, get crashpad_test_test building and running
- Implement build/run_tests.py to run on Fuchsia device
- Implement paths_fuchsia.cc using standard Fuchsia namespace layout
- Exclude multiprocess tests, currently unimplemented
- Don't use unnecessary O_ flags on Fuchsia in open() call.

Bug: crashpad:196, chromium:726124, ZX-797
Change-Id: Ie59dce685b4c3fe54f3e36f357c1101d402ee8b7
Reviewed-on: https://chromium-review.googlesource.com/802180
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2017-12-05 18:59:26 +00:00
Mark Mentovai
5969d6b1eb android: NDK r16 compatibility
This updates build/gyp_crashpad_android.py to define the
android_api_level GYP variable whenver unified headers are in use.
Previously, it was only set when compiling with GCC and using unified
headers. This pairs with https://crrev.com/c/804574 to allow proper
detection of when _FILE_OFFSET_BITS=64 would be inappropriate.

Since there’s no longer any possibility of using a 64-bit off_t with API
< 21, this also drops the compatibility wrapper for mmap() that allowed
this configuration to work. Too bad, I liked this, but it’s pointless to
carry now.

The development documentation is also updated to refer to NDK r16.

mini_chromium is updated to 88e056258a01450b07414642fa5fb98493c1f6ce.

f609089390cd fuchsia: Add ZX_LOG, et al. to mini_chromium
0a8c5de30c67 fuchsia: Fix RandBytes() ZX_CHECK message string
88e056258a01 android: Don’t use _FILE_OFFSET_BITS=64 until API 21

Change-Id: I932116e0c01bcddd5719f9091a070d504eae600f
Reviewed-on: https://chromium-review.googlesource.com/804555
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Joshua Peraza <jperaza@chromium.org>
2017-12-04 16:26:00 +00:00
Scott Graham
eeb31321f7 gn, fuchsia: Add //testing forwarding and GN build file
third_party/gtest/BUILD.gn mostly written by rsesek.

Also includes DEPS roll of mini_chromium for Wexit_time_destructors config.

$ git log --oneline dd0c3e96..fa146c12
fa146c1 (HEAD, origin/master, origin/HEAD) gn: Extract Wexit-time-destructors into separate config
95bfddb fuchsia: Fix base::RandBytes() after 5a1c5f82ce75
b79608a fuchsia: Use llvm-ar from the Fuchsia clang package
c34725b fuchsia: Look for the toolchain and SDK in per-build-host directories
e8e1ee4 fuchsia: Don't assume that kernel will provide all requested rand bytes
5a1c5f8 fuchsia: Implement RandBytes()
df359ca fuchsia: Enable -fPIC
bd50c95 Restore accidentally changed license
a70db15 Improvements to GN build config
7de4d23 fuchsia: Fix compile of base/logging.cc
25a8b57 Add link GN rules to non-win build, set c++14 in CC flags.
7d15806 fuchsia: The very basics of compiling mini_chromium/base with GN

Bug: crashpad:79, crashpad:196
Change-Id: I3e741f185b028a96705eefc1f993037830d97448
Reviewed-on: https://chromium-review.googlesource.com/797414
Commit-Queue: Scott Graham <scottmg@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
Reviewed-by: Robert Sesek <rsesek@chromium.org>
2017-11-30 20:58:46 +00:00
Mark Mentovai
593ede52e0 Fixes for drive-by review comments after ef3ce94fbfbb
Change-Id: I28edc00549d51576ab553f401235aa1d9f669232
Reviewed-on: https://chromium-review.googlesource.com/797335
Reviewed-by: Scott Graham <scottmg@chromium.org>
Commit-Queue: Mark Mentovai <mark@chromium.org>
2017-11-29 19:52:08 +00:00
Scott Graham
3f5c939c84 gn: Next step in getting compilation working
- Correctly sets target_cpu and current_cpu so correct toolchain
  can be used on Fuchsia.
- Introduces GN argument "crashpad_in_chromium" which defaults to
  false. Used to set CRASHPAD_IN_CHROMIUM define, determine which
  zlib path to use, and how to package the test targets into
  binaries (one big one in Chromium, separate in Crashpad).

Bug: crashpad:79, crashpad:196
Change-Id: If6560dc064308ed6f8bf7c75cf74f684a3522e8b
Reviewed-on: https://chromium-review.googlesource.com/797354
Reviewed-by: Mark Mentovai <mark@chromium.org>
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Commit-Queue: Scott Graham <scottmg@chromium.org>
2017-11-29 19:46:08 +00:00
Mark Mentovai
ef3ce94fbf Python 3 support for Python scripts (without compromising Python 2)
It’s better to be prepared for the future than…to not be.

This is mostly the result of running 2to3 on all .py files, with some
small shims to maintain compatibility with Python 2.
http_transport_test_server.py was slightly more involved, requiring many
objects to change from “str” to “bytes”.

The #! lines and invokers still haven’t changed, so these scripts will
still normally be interpreted by Python 2.

Change-Id: Idda3c5650f967401a5942c4d8abee86151642a2e
Reviewed-on: https://chromium-review.googlesource.com/797434
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Commit-Queue: Mark Mentovai <mark@chromium.org>
2017-11-29 18:47:38 +00:00
Scott Graham
2bb56fafe3 Rework GN files to start to support building standalone, and also in Chromium
- Adds a .gn and a build/BUILDCONFIG.gn that uses mini_chromium's
  build/BUILD.gn.
- Adds some stub BUILD.gn files in locations where Chromium expects them
  (in //build, //testing, //third_party) containing empty targets/configs.
  These are no-ops in standalone builds, but add functionality when
  building in Chromium.  This is in preference to having a global bool
  that conditionally does Chromium-y things in the Crashpad build files.
  These stub files are all contained in a secondary source root in
  build/chromium_compatibility, referred to by //.gn.
- Adds //base/BUILD.gn which forwards to mini_chromium/base. This is
  only used when building standalone so that both Chromium and Crashpad
  can refer to it as "//base".
- Changes references to other Crashpad targets to be relatively
  specified so that they work when the root of the project is //, and also
  when it's //third_party/crashpad/crashpad as it is in Chromium.
- Moves any error-causing Mac/Win-specific files into explicit if (is_mac)
  or if (is_win) blocks as part of removing the dependency on
  set_sources_assignment_filter().

As yet unresolved:
- CRASHPAD_IN_CHROMIUM needs to be removed when standalone; to be tackled
  in a follow up.
- Not sure what to do with zlib yet, the build file currently assumes
  "in Chromium" too, and similarly having Crashpad //third_party/zlib:zlib
  pointing at itself doesn't work.

Bug: crashpad:79
Change-Id: I6a7dda214e4b3b14a60c1ed285267ab97432a1a8
Reviewed-on: https://chromium-review.googlesource.com/777410
Reviewed-by: Mark Mentovai <mark@chromium.org>
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Commit-Queue: Scott Graham <scottmg@chromium.org>
2017-11-28 20:29:35 +00:00
Robert Sesek
79e2dd843e Include string annotation objects when uploading crash reports.
This extracts string annotation objects from the minidumps and includes
them as form POST key-value pairs.

This change also starts building a crashpad_handler_test binary on Mac.

Bug: crashpad:192
Change-Id: I68cbf6fda6f1e57c1e621d5e3de8717cfaea65bf
Reviewed-on: https://chromium-review.googlesource.com/749793
Commit-Queue: Robert Sesek <rsesek@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2017-11-02 16:39:06 +00:00
Mark Mentovai
5e9ed4cb9f win: Dynamically disable WoW64 tests absent explicit 32-bit build output
Rather than having the 64-bit build assume that it lives in
out\{Debug,Release}_x64 and that it can find 32-bit build output in
out\{Debug,Release}, require the location of 32-bit build output to be
provided explicitly via the CRASHPAD_TEST_32_BIT_OUTPUT environment
variable. If this variable is not set, 64-bit tests that require 32-bit
test build output will dynamically disable themselves at runtime.

In order for this to work, a new DISABLED_TEST() macro is added to
support dynamically disabled tests. gtest does not have its own
first-class support for this
(https://groups.google.com/d/topic/googletestframework/Nwh3u7YFuN4,
https://github.com/google/googletest/issues/490) so this local solution
is used instead.

For tests via Crashpad’s own build\run_tests.py, which is how Crashpad’s
own buildbots and trybots invoke tests, CRASHPAD_TEST_32_BIT_OUTPUT is
set to a locaton compatible with the paths expected for the GYP-based
build. No test coverage is lost on Crashpad’s own buildbots and trybots.

For Crashpad tests in Chromium’s buildbots and trybots, this environment
variable will not be set, causing these tests to be dynamically
disabled.

Bug: crashpad:203, chromium:743139, chromium:777924
Change-Id: I3c0de2bf4f835e13ed5a4adda5760d6fed508126
Reviewed-on: https://chromium-review.googlesource.com/739795
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
2017-10-26 18:31:57 +00:00
Mark Mentovai
7a849482ea Switch the language standard to C++14 and use std::make_unique
Update mini_chromium to 7d6697ceb5cb5ca02fde3813496f48b9b1d76d0c

47ff9691450e Switch the language standard to C++14
7d6697ceb5cb Remove base/memory/ptr_util.h and base::WrapUnique

base::WrapUnique and std::make_unique are similar, but the latter is
standardized and preferred.

Most of the mechanical changes were made with this sed:

for f in $(git grep -l base::WrapUnique | uniq); do
  sed -E \
      -e 's%base::WrapUnique\(new ([^(]+)\((.*)\)\);%std::make_unique<\1>(\2);%g' \
      -e 's%base::WrapUnique\(new ([^(]+)\);%std::make_unique<\1>();%g' \
      -e 's%^#include "base/memory/ptr_util.h"$%#include <memory>%' \
      -i '' "${f}"
done

Several uses of base::WrapUnique that did not fit on a single line and
were not matched by this sed were adjusted manually. All #include
changes were audited manually, to at least move <memory> into the
correct section. Where <memory> was already #included by a file (or its
corresponding header), the extra #include was removed. Where <memory>
should have been #included by a header, it was added. Other similar
adjustments to other #includes were also made.

Change-Id: Id4e0baad8b3652646bede4c3f30f41fcabfdbd4f
Reviewed-on: https://chromium-review.googlesource.com/714658
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Leonard Mosescu <mosescu@chromium.org>
2017-10-12 19:07:13 +00:00
Joshua Peraza
b953388b95 Add SystemSnapshotLinux
Bug: crashpad:30
Change-Id: Ic1162c6637708492a5a9903a221cdd9266d3fd97
Reviewed-on: https://chromium-review.googlesource.com/601028
Commit-Queue: Joshua Peraza <jperaza@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2017-08-08 22:31:46 +00:00
Sigurdur Asgeirsson
30385d4e47 handler: Add user extensibility stream call-out.
Bug: crashpad:167
Test: Add crashpad_handler_test.
Change-Id: I79b0b71dc4f61e6dce6bc10083e2f924dc83c940
Reviewed-on: https://chromium-review.googlesource.com/463746
Commit-Queue: Sigurður Ásgeirsson <siggi@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
2017-04-11 19:06:00 +00:00
Mark Mentovai
f8ef834ef5 android: Support “unified headers” with GCC build
Previously, the __ANDROID_API__ macro was provided by
<android/api-level.h>. With unified headers (expected to become the
default in the next NDK release and the sole option in the subsequent
release), this macro is not set properly by headers. When building with
Clang, the standalone toolchain’s clang and clang++ wrappers set the
macro properly. GCC isn’t accounted for in this way, so the build system
must assume the responsibility of setting it.

This change fishes the value of __ANDROID_API__ out of the standalone
toolchain’s clang wrapper and sets the android_api_level GYP variable
appropriately. From there, it will be picked up by common.gypi in
mini_chromium and used to define __ANDROID_API__.

This updates mini_chromium to 62e6015f633dd4acb1610db15a064889315cadaa
which understands this new GYP variable.

62e6015f633d android: Support “unified headers” with GCC build

https://android.googlesource.com/platform/ndk/+/master/docs/UnifiedHeaders.md
https://android.googlesource.com/platform/ndk/+/ndk-r14/CHANGELOG.md

Bug: crashpad:30
Change-Id: I33e66eba8394e32ced8dca80c8226b85e0e786f3
Reviewed-on: https://chromium-review.googlesource.com/458021
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
2017-03-23 19:20:35 +00:00
Mark Mentovai
db8c54e142 android: Add gyp_crashpad_android.py for easier Android development
Bug: crashpad:30
Change-Id: Idf7e6db944bf946e6571064306897848222cd36f
Reviewed-on: https://chromium-review.googlesource.com/458078
Reviewed-by: Joshua Peraza <jperaza@chromium.org>
2017-03-23 00:15:26 +00:00
Mark Mentovai
39f13a77a4 Make run_tests.py work with paths instead of configuration names
This is co-dependent with
https://chromium-review.googlesource.com/457736.

I’ve been trying to share a source tree between different platforms,
using gyp_crashpad.py --generator-output to put the build output in the
right place. On Windows, it winds up in
out\win\out\{Debug,Release}{,_x64}. This makes it tricky to use
run_tests.py, which expects to find build output right in the “out”
directory. It’s not impossible to use, because you can tell it
“win\out\Debug_x64”, but it’s really awkward to use that path when we
all know that it’s not relative to anything that makes sense, like the
current directory.

This simplifies run_tests.py to work directly with the
configuration-specific output directory. For most users, this means
including “out/” or “out\” when running the script.

Bug: chromium:703890
Change-Id: Ic7de82fabd2adda7ae00558844cb3ce91aa4a5ed
Reviewed-on: https://chromium-review.googlesource.com/457716
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
2017-03-22 01:45:16 +00:00
Mark Mentovai
4c6f6e52e2 Remove vestigial support for in-Chromium GYP build
Chromium’s GYP build has been removed. The support added to Crashpad’s
GYP build to integrate with Chromium’s is now unused and unnecessary.
Chromium builds Crashpad as part of its own GN build.

https://groups.google.com/a/chromium.org/d/topic/chromium-dev/NZkPr-CXvQ0

Change-Id: I30f2d3453f4476037c9afe0714a780456f0bbcd6
Reviewed-on: https://chromium-review.googlesource.com/444044
Reviewed-by: Robert Sesek <rsesek@chromium.org>
2017-02-16 18:33:54 +00:00
Mark Mentovai
dd85381a32 GCC fix: Disable -Wmultichar warning throughout Crashpad
-Wmultichar is enabled by default with GCC (but not clang). It is
impossible to disable this warning with #pragma GCC diagnostic ignored.
See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53431

While compiling, for example, minidump_file_writer.cc:

In file included from ../../minidump/minidump_extensions.h:25:0,
                 from ../../minidump/minidump_file_writer.h:27,
                 from ../../minidump/minidump_file_writer.cc:15:
../../util/misc/pdb_structures.h:45:38: error: multi-character character constan
t [-Werror=multichar]
   static const uint32_t kSignature = '01BN';
                                      ^~~~~~
../../util/misc/pdb_structures.h:106:38: error: multi-character character consta
nt [-Werror=multichar]
   static const uint32_t kSignature = 'SDSR';
                                      ^~~~~~
../../minidump/minidump_file_writer.cc:190:23: error: multi-character character
constant [-Werror=multichar]
   header_.Signature = MINIDUMP_SIGNATURE;
                       ^~~~~~~~~~~~~~~~~~

doc/developing.md is also updated to provide GCC build instructions for
Android.

Tested with:
 - GCC 4.9 from NDK r13 targeting arm with SDK 16
 - GCC 4.9 from NDK r13 targeting arm64 with SDK 21
 - GCC 6.2 targeting x86_64

BUG=crashpad:30

Change-Id: I9e7993761f5461281c9f4d8b4c56e8407e2c5b47
Reviewed-on: https://chromium-review.googlesource.com/409776
Reviewed-by: Robert Sesek <rsesek@chromium.org>
2016-11-11 17:35:25 +00:00
Mark Mentovai
e18f6a6e66 Nest crashpad and mini_chromum deeper in the external-dependencies build
crashpad and mini_chromium both have top-level “build” directories.
These would conflict with top-level “BUILD” files in google3 when
checked out on a case-sensitive filesystem. Although Crashpad’s “build”
directory can be moved easily, mini_chromium’s matches Chromium’s, which
is much more difficult to move. “build” is also the best and most
obvious name for these directories.

To avoid this problem, in the external-dependencies build, crashpad and
mini_chromium are placed one level deeper, just as crashpad is in
Chromium, and mini_chromium is in the standalone Crashpad build. This
allows true pristine unmodified copies to be checked in to google3,
without comingling locally-added files such as BUILD with external
source.

The directory structure adopted for the external-dependencies build is
now

      root/crashpad/crashpad[/README]
      root/gmock[/include/gmock/gmock.h]
      root/gtest[/include/gtest/gtest.h]
      root/gyp[/pylib/gyp]
      root/mini_chromium/mini_chromium[/build/common.gypi]

Change-Id: Idbc8f1b0d87da0cbceab3c15e059e839c1fb6a3f
Reviewed-on: https://chromium-review.googlesource.com/323991
Reviewed-by: Robert Sesek <rsesek@chromium.org>
2016-01-27 21:59:50 +00:00