% yapf --in-place $(git ls-files **/*.py)
% yapf --version
yapf 0.30.0
Note that this is not using the “chromium” yapf style because Chromium
is moving to PEP-8.
https://groups.google.com/a/chromium.org/d/topic/chromium-dev/RcJgJdkNIdg
yapf 0.30.0 no longer recognizes “chromium” as a style option.
22ef70f3c4
Since this is a mass reformatting, it might as well move things all the
way into the future all at once.
This uses the “google” style, which is a superset of “pep8”.
Change-Id: Ifa37371079ea1859e4afe8e31d2eef2cfd7af384
Reviewed-on: https://chromium-review.googlesource.com/c/crashpad/crashpad/+/2165637
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
The “and not memory after range” follow-up to the “extra memory range
captured” test is very flaky in debug builds
(https://ci.chromium.org/p/crashpad/builders/ci/crashpad_win_dbg/388),
and occasionally flaky in release builds too
(https://ci.chromium.org/p/crashpad/builders/ci/crashpad_win_rel/363).
As with the already-disabled “extra memory removal” test, it’s likely
that Crashpad is discovering a pointer that references the memory
immediately beyond the intended range, and capturing that too. Unless
the environment is very tightly controlled, it’s difficult to prevent
valid memory from being pointed to, and thus to guarantee that Crashpad
won’t capture a particular block of memory.
These are probably never going to be fixed. The flaky tests are removed.
Bug: crashpad:101
Change-Id: I629ed8ce3c901507689218baaa102d8737db3f5b
Reviewed-on: https://chromium-review.googlesource.com/c/crashpad/crashpad/+/2150055
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
This allows clients to use the database to handle uploads themselves,
e.g. on Android, where Crashpad does not yet provide an uploader.
The handler does not launch an upload thread when no url is supplied.
Previously, the handler would move these reports to
completed and record the upload as skipped with kUploadsDisabled.
With this change, these reports would remain pending until pruned,
with no metrics recorded for them in regard to their upload.
Bug: crashpad:30
Change-Id: I4167ab1531634b10e91d03229018ae6aab4103aa
Reviewed-on: https://chromium-review.googlesource.com/1010970
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Commit-Queue: Joshua Peraza <jperaza@chromium.org>
It’s better to be prepared for the future than…to not be.
This is mostly the result of running 2to3 on all .py files, with some
small shims to maintain compatibility with Python 2.
http_transport_test_server.py was slightly more involved, requiring many
objects to change from “str” to “bytes”.
The #! lines and invokers still haven’t changed, so these scripts will
still normally be interpreted by Python 2.
Change-Id: Idda3c5650f967401a5942c4d8abee86151642a2e
Reviewed-on: https://chromium-review.googlesource.com/797434
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Commit-Queue: Mark Mentovai <mark@chromium.org>
Rather than having the 64-bit build assume that it lives in
out\{Debug,Release}_x64 and that it can find 32-bit build output in
out\{Debug,Release}, require the location of 32-bit build output to be
provided explicitly via the CRASHPAD_TEST_32_BIT_OUTPUT environment
variable. If this variable is not set, 64-bit tests that require 32-bit
test build output will dynamically disable themselves at runtime.
In order for this to work, a new DISABLED_TEST() macro is added to
support dynamically disabled tests. gtest does not have its own
first-class support for this
(https://groups.google.com/d/topic/googletestframework/Nwh3u7YFuN4,
https://github.com/google/googletest/issues/490) so this local solution
is used instead.
For tests via Crashpad’s own build\run_tests.py, which is how Crashpad’s
own buildbots and trybots invoke tests, CRASHPAD_TEST_32_BIT_OUTPUT is
set to a locaton compatible with the paths expected for the GYP-based
build. No test coverage is lost on Crashpad’s own buildbots and trybots.
For Crashpad tests in Chromium’s buildbots and trybots, this environment
variable will not be set, causing these tests to be dynamically
disabled.
Bug: crashpad:203, chromium:743139, chromium:777924
Change-Id: I3c0de2bf4f835e13ed5a4adda5760d6fed508126
Reviewed-on: https://chromium-review.googlesource.com/739795
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
end_to_end_test.py was producing these error messages 6 times (32-bit
x86) and 7 times (x86_64) per run:
[pid:tid:yyyymmdd,hhmmss.mmm:ERROR file_io.cc:89] ReadExactly: expected
36, observed 0
These messages were being produced by crashpad_handler, in the
LoggingReadFileExactly() call in
ExceptionHandlerServer::ServiceClientConnection().
sizeof(ClientToServerMessage) is 36. crashpad_handler believed that a
client was connecting, but the client sent no data.
This was tracked down to the use of os.path.exists() in
end_to_end_test.py to wait for crashpad_handler’s named pipe to be
created. Checking named pipe existence in this way appeared to be a
client connecting to the the pipe server in crashpad_handler, although
of course no real client was connecting and no message was forthcoming.
I found that running “dir” on the named pipe’s path produced the same
result.
Using WaitNamedPipe() is an alternative that can be used to signal when
the named pipe’s path exists. Furthermore, it tests more than mere
creation, it indicates that the pipe server has become ready to service
clients. That’s not necessary in this case as proper clients already
need to deal with this on their own, but checking it in
end_to_end_test.py should be harmless.
Test: end_to_end_test.py
Change-Id: Ida29a3d2325368f58930cdf8fb053449f621ea52
Reviewed-on: https://chromium-review.googlesource.com/703276
Reviewed-by: Leonard Mosescu <mosescu@chromium.org>
In the 64-bit version of the structure, padding is needed between
ShowWindowFlags and WindowTitle.
The CurrentDirectores (yes, that’s how it’s spelled) members would have
been interpreted incorrectly because STRING was defined incorrectly. The
length fields are USHORT, not DWORD. In the 64-bit version of the
structure, a padding member ensured that the structure was at least the
correct size. In the 32-bit version of the structure, this caused the
structure size to be inflated, so all but the first CurrentDirectores
element and any struct member that followed would appear at incorrect
offsets, and the overall struct size being read was larger than
appropriate.
This resolves crashpad_handler logging (usually) three errors while
handling a 64-bit process crash, such as:
[pid:tid:yyyymmdd,hhmmss.mmm:ERROR process_info.cc:632] range at
0x780f24de00000000, size 0x275 fully unreadable
[pid:tid:yyyymmdd,hhmmss.mmm:ERROR process_info.cc:632] range at
0x780f24fe00000000, size 0x275 fully unreadable
[pid:tid:yyyymmdd,hhmmss.mmm:ERROR process_info.cc:632] range at 0x0,
size 0x275 fully unreadable
Bug: crashpad:198
Test: end_to_end_test.py
Change-Id: I1655101de01cf46b4b50eda45a11f8d0f3bca8b3
Reviewed-on: https://chromium-review.googlesource.com/701736
Reviewed-by: Leonard Mosescu <mosescu@chromium.org>
hanging_program.exe is used by crash_other_program.exe, which is in turn
used by end_to_end_test.py. It hangs by loading loader_lock_dll.dll,
which squats in its entry point function while the loader lock is held.
hanging_program.exe needs to do some work in its Thread1() before the
loader lock is taken (a SetThreadPriority() call), and needs to do some
work in its main thread once the loader lock is held (it needs to signal
crash_other_program.exe that it’s successfully wedged itself).
Previously, proper synchronization was not provided. A 1-second Sleep()
was used to wait for the loader lock to be taken. Thread1() pre-work was
only achieved before the loader lock was taken by sheer luck. Things
didn’t always work out so nicely.
This uses an event handle to provide synchronization. An environment
variable is used to pass the handle to loader_lock_dll.dll, because
there aren’t many better options available. This eliminates both flake
and the unnecessary 1-second delay in hanging_program.exe, and since
this program runs twice during end_to_end_test.py, it improves that
test’s runtime by 2 seconds.
Bug: crashpad:197
Test: end_to_end_test.py
Change-Id: Ib9883215ef96bed7571464cc68e09b6ab6310ae6
Reviewed-on: https://chromium-review.googlesource.com/700076
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Leonard Mosescu <mosescu@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
This is like 270490ff79df, but for things run by end_to_end_test.py, and
things run for it by crash_other_program.exe.
Change-Id: Iabf3c762c50f41eb61ab31f714c646364196e745
Reviewed-on: https://chromium-review.googlesource.com/458822
Commit-Queue: Mark Mentovai <mark@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
The Windows 10 loader starts a few extra threads before main(). In a few
of the test cases, the tests were relying on thread ordering (generally,
the test thread being at index #1). Instead, use other signals to find
the correct thread to verify.
R=mark@chromium.org
Change-Id: Icb1f5a8fdf3a0ea6d82ab65960dbcb650965f269
Reviewed-on: https://chromium-review.googlesource.com/407073
Reviewed-by: Mark Mentovai <mark@chromium.org>
Previously, StartHandler() launched the handler process, then connected
over a pipe to register for crash handling. Instead, the initial client
can create and inherit handles to the handler and pass those handle
values and other data (addresses, etc.) on the command line.
This should improve startup time as there's no need to synchronize with
the process at startup, and allows avoiding a call to CreateProcess()
directly in StartHandler(), which is important for registration for
crash reporting from DllMain().
Incidentally adds new utility functions for string/number conversion and
string splitting.
Note: API change; UseHandler() is removed for all platforms.
BUG=chromium:567850,chromium:656800
Change-Id: I1602724183cb107f805f109674c53e95841b24fd
Reviewed-on: https://chromium-review.googlesource.com/400015
Reviewed-by: Mark Mentovai <mark@chromium.org>
This switches the default behaviour of crashpad_handler.exe to be a
/subsystem:windows app, so that normal usage won't cause a console to be
popped up. At the same time, creates a copy of crashpad_handler.exe in
the output dir named crashpad_handler.com. The .com doesn't affect
normal operation, as the way StartHandler() uses CreateProcess()
requires a real path to a file. However, when run from a command prompt,
.com are found before .exe, so editbin the .com to be to a console app,
which will be run in preference to the exe when run as just
"crashpad_handler", as one tends to do from a command prompt when
debugging. That is:
d:\src\crashpad\crashpad\out\Debug>where crashpad_handler
d:\src\crashpad\crashpad\out\Debug\crashpad_handler.com
d:\src\crashpad\crashpad\out\Debug\crashpad_handler.exe
d:\src\crashpad\crashpad\out\Debug>crashpad_handler --help
Usage: crashpad_handler [OPTION]...
...
d:\src\crashpad\crashpad\out\Debug>crashpad_handler.exe --help
<no output>
d:\src\crashpad\crashpad\out\Debug>crashpad_handler.com --help
Usage: crashpad_handler.com [OPTION]...
...
We also use the .com file in test invocations so that output streams
will be visible.
R=mark@chromium.org
Change-Id: I1a27f88472d491b2a1d76e63c45e6415d9f679c0
Reviewed-on: https://chromium-review.googlesource.com/371578
Reviewed-by: Mark Mentovai <mark@chromium.org>
When crashy_test_program's SomeCrashyFunction is inlined into
CrashyMain, cdb doesn't demangle the decorated form of an anonymous
namespace (?A0x12345678) into the expected `anonymous namespace' string.
I experienced this in Release_x64 and Release modes using MSVS 2015
update 3 (14.0.25420.1, cl 19.00.24213.1) and cdb versions 10.0.10240.9
and 10.0.14321.1024.
BUG=crashpad:129
Change-Id: I0a665b88891c271253adccd9b2b414fcaac26c8f
Reviewed-on: https://chromium-review.googlesource.com/368730
Reviewed-by: Scott Graham <scottmg@chromium.org>
Because DumpAndCrashTargetProcess() suspends the process, the thread
suspend count is one too high for all threads other than the injection
one in the thread snapshots. Compensate for this when we detect this
type of exception.
BUG=crashpad:103
Change-Id: Ib77112fddf5324fc0e43f598604e56f77d67ff54
Reviewed-on: https://chromium-review.googlesource.com/340372
Reviewed-by: Mark Mentovai <mark@chromium.org>
Adds a new client API which allows causing an exception in another
process. This is accomplished by injecting a thread that calls
RaiseException(). A special exception code is used that indicates to the
handler that the exception arguments contain a thread id and exception
code, which are in turn used to fabricate an exception record. This is
so that the API can allow the client to "blame" a particular thread in
the target process.
The target process must also be a registered Crashpad client, as the
normal exception mechanism is used to handle the exception.
The injection of a thread is used instead of DebugBreakProcess() which
does not cause the UnhandledExceptionFilter() to be executed.
NtCreateThreadEx() is used in lieu of CreateRemoteThread() as it allows
passing of a flag which avoids calling DllMain()s. This is necessary to
allow thread creation to succeed even when the target process is
deadlocked on the loader lock.
BUG=crashpad:103
Change-Id: I797007bd2b1e3416afe3f37a6566c0cdb259b106
Reviewed-on: https://chromium-review.googlesource.com/339263
Reviewed-by: Mark Mentovai <mark@chromium.org>
In debug builds, the extra memory is sometimes getting captured
(probably by a stale stack pointer), so disable this test for now to
un-red the bots. We can probably fix it by moving this one test to a
separate binary (or perhaps just removing it, I'm not sure it's that
useful anyway above and beyond the unit test.)
R=mark@chromium.org
BUG=crashpad:101
Change-Id: I98a58a467fb4a4d9f84d2e0d020a031a0ea9743c
Reviewed-on: https://chromium-review.googlesource.com/334821
Reviewed-by: Scott Graham <scottmg@chromium.org>
The previous approach was nice for its simplicity, but unfortunately
didn't work when the compiler decided to do some of its confounded
"optimization".
R=mark@chromium.org
BUG=crashpad:86, chromium:571144
Review URL: https://codereview.chromium.org/1563273004 .
On Win 7 in a debug configuration, walking all locks was gathering
hundreds of thousands of locks, causing test timeouts to be exceeded in
debug. On user machines, UnhandledExceptionHandler() could have timed
out too. For now, only grab the loader lock as it's the most interesting
one. Unfortunately, this means that !locks won't work for now.
In the future, we may want to figure out a signalling mechanism so that
the client can note other interesting locks to be grabbed, and just
avoid walking the list entirely.
R=mark@chromium.org
BUG=chromium:546288
Review URL: https://codereview.chromium.org/1475033005 .
This is better because now end_to_end_test.py fails immediately with
[1180:9020:20151106,145204.830:ERROR registration_protocol_win.cc:39] CreateFile: The system cannot find the file specified. (0x2)
R=mark@chromium.org
BUG=crashpad:75
Review URL: https://codereview.chromium.org/1409693011 .
This is necessary for 64 bit tools installed on a 64 bit OS, but with
the tests run from a 32 bit Python. (sigh)
Doesn't happen on bots, but comes up occasionally testing on VMs.
R=mark@chromium.org
Review URL: https://codereview.chromium.org/1425153003 .
I considered writing the CodeView records to the minidump, but I didn't
find a ton of docs and debugging is only lightly supported (e.g.
http://www.debuginfo.com/articles/gendebuginfo.html#debuggersandformats
and it doesn't attempt to load at all on more recent Visual Studios).
As we won't be generating symbols in this format, and we don't expect to
have symbols for any weird modules that get injected into us in the
wild, it seems like we don't lose anything by just ignoring them.
R=mark@chromium.org
BUG=crashpad:47
Review URL: https://codereview.chromium.org/1430773003 .
I thought I had confirmed that this still allocated and ignored the flag
on older OSs, but I must have not had the PLOG active yet? I'm not sure
what I did. (I might try to blame VMware as it has an annoying habit of
caching old binaries when you use it's "Shared Folders" feature to point
at the dev machine's build dir.)
I confirmed that it does work on Win8 and Win10 but doesn't on Win XP
and Win 7.
R=mark@chromium.org
BUG=crashpad:52
Review URL: https://codereview.chromium.org/1405243002 .
Capture the memory for the loader lock (can be inspected by !cs), as
well as all locks that were created with .DebugInfo which can be viewed
with !locks.
e.g.
0:000> !cs ntdll!LdrpLoaderLock
-----------------------------------------
Critical section = 0x778d6410 (ntdll!LdrpLoaderLock+0x0)
DebugInfo = 0x778d6b6c
NOT LOCKED
LockSemaphore = 0x0
SpinCount = 0x04000000
0:000> !locks -v
CritSec ntdll!RtlpProcessHeapsListLock+0 at 778d7620
LockCount NOT LOCKED
RecursionCount 0
OwningThread 0
EntryCount 0
ContentionCount 0
CritSec +7a0248 at 007a0248
LockCount NOT LOCKED
RecursionCount 0
OwningThread 0
EntryCount 0
ContentionCount 0
CritSec crashy_program!g_critical_section_with_debug_info+0 at 01342c48
LockCount NOT LOCKED
RecursionCount 0
OwningThread 0
EntryCount 0
ContentionCount 0
CritSec crashy_program!crashpad::`anonymous namespace'::g_test_critical_section+0 at 01342be0
WaiterWoken No
LockCount 0
RecursionCount 1
OwningThread 34b8
EntryCount 0
ContentionCount 0
*** Locked
Scanned 4 critical sections
R=mark@chromium.org
BUG=crashpad:52
Review URL: https://codereview.chromium.org/1392093003 .
Getting closer... Some tests passed on the last run, but the ones that
rely on having ntdll symbols fail on the bot. With `_NT_SYMBOL_PATH`
set, cdb will be able to download the PDBs so will be able to dump
data for `ntdll!_PEB`, etc.
R=mark@chromium.org
BUG=crashpad:46
Review URL: https://codereview.chromium.org/1402643002 .
Oops, was passing the out dir (...\crashpad\out), not the binary dir
(...\crashpad\out\Debug). Didn't notice because I was running the
script directly, rather than via run_tests.py. :/
R=mark@chromium.org
BUG=crashpad:46
Review URL: https://codereview.chromium.org/1394343005 .
I'd like to write some `expect(1)`-style tests (possibly using
http://pexpect.readthedocs.org/en/stable/) to verify that various windbg
commands that I'm adding support for do actually work when consuming
minidumps in real life.
For the moment, this is just the beginnings of a stub as I don't know if
bots even have windbg/cdb installed.
R=mark@chromium.org
BUG=crashpad:20, crashpad:46, crashpad:52
Review URL: https://codereview.chromium.org/1396943002 .
