0
0
mirror of https://github.com/zeux/pugixml.git synced 2024-12-26 21:04:25 +08:00

1353 Commits

Author SHA1 Message Date
Arseny Kapoulkine
ea544eb48b tests: Add fuzzing dictionaries
Hopefully this will allow for better fuzzing coverage
2017-02-11 13:17:02 -08:00
Arseny Kapoulkine
8c62fa9121 tests: Add XPath fuzzing
Only fuzz the parser for now.
2017-02-09 07:37:38 -08:00
Arseny Kapoulkine
8b15ae8015 tests: Add a script to set up fuzzing tools
This downloads a clang build that has support for instrumentation, and also
downloads and compiles libFuzzer.a.
2017-02-09 07:37:04 -08:00
Arseny Kapoulkine
00ef791078 fuzz: Use libFuzzer instead of afl-fuzz
This allows us to have faster fuzz cycles since the fuzzer is in-process.
2017-02-09 07:36:32 -08:00
Arseny Kapoulkine
e748f435e5 tests: Increase the number of translate calls
This should make the test fail on a 32-bit target.
2017-02-09 07:36:32 -08:00
Arseny Kapoulkine
4bab082a27 tests: Fix clang build 2017-02-09 07:36:32 -08:00
Arseny Kapoulkine
ba39838ab5 tests: Add more XPath out of memory tests 2017-02-09 07:36:31 -08:00
Arseny Kapoulkine
d4c456bdef Add invalid type assertion for offset_debug
This will make sure we don't forget to implement offset_debug for new
node types if they ever happen (really it's mostly for consistency).
2017-02-09 07:36:31 -08:00
Arseny Kapoulkine
2162a0d80c XPath: Simplify sorting implementation
Instead of a complicated partitioning scheme that tries to maintain the
equal area in the middle, use a scheme where we keep the equal area in
the left part of the array and then move it to the middle.

Since generally sorted arrays don't contain many duplicates this extra
copy is not too expensive, and it significantly simplifies the logic and
maintains good complexity for sorting arrays with many equal elements
nonetheless (unlike Hoare partitioning).

Instead of a median of 9 just use a median of 3 - it performs pretty
much identically on some internal performance tests, despite having a
bit more comparisons in some cases.

Finally, change the insertion sort threshold to 16 elements since that
appears to have slightly better performance.
2017-02-07 00:05:50 -08:00
Arseny Kapoulkine
774d5fe9df XPath: Optimize insertion_sort
The previous implementation opted for doing two comparisons per element
in the sorted case in order to remove one iterator bounds check per
moved element when we actually need to copy. In our case however the
comparator is pretty expensive (except for remove_duplicates which is
fast as it is) so an extra object comparison hurts much more than an
iterator comparison saves.

This makes sorting by document order up to 3% faster for random
sequences.
2017-02-06 19:28:33 -08:00
Arseny Kapoulkine
8cc3144e7b XPath: Remove redundant calls from xml_node::select_nodes et al
Instead of delegating to a method that just forwards the call to
xpath_query call the relevant method directly.
2017-02-05 21:52:30 -08:00
Arseny Kapoulkine
00e39c581a XPath: Remove evaluate_string_impl
It adds one stack frame to string query evaluation and does not really
simplify the code.
2017-02-05 21:50:13 -08:00
Arseny Kapoulkine
a9fe2bb62e Merge pull request #131 from zeux/xpath-noeh
XPath: Remove exceptional control flow
2017-02-05 21:34:54 -08:00
Arseny Kapoulkine
10676b6b85 tests: Add more XPath sorting tests
Cover empty node case - no XPath query can result in that but it's
possible to create a node set with empty nodes manually.
2017-02-05 21:12:55 -08:00
Arseny Kapoulkine
bcc7ed57a2 XPath: Simplify evaluation error flow
Instead of having two checks for out-of-memory when exceptions are
enabled, do just one and decide what to do based on whether we can
throw.
2017-02-03 20:33:40 -08:00
Arseny Kapoulkine
33159924b1 XPath: Clean up out-of-memory parse error handling
Instead of relying on a specific string in the parse result, use
allocator error state to report the error and then convert it to a
string if necessary.

We currently have to manually trigger the OOM error in two places
because we use global allocator in rare cases; we don't really need to
do this so this will be cleaned up later.
2017-02-02 18:40:20 -08:00
Arseny Kapoulkine
faadd460c4 tests: Add more out of memory tests for XPath evaluation 2017-02-02 08:57:02 -08:00
Arseny Kapoulkine
c28ff128d8 tests: Add more embed_pcdata tests 2017-02-02 08:40:34 -08:00
Arseny Kapoulkine
f9f1c86716 tests: Improve parsing coverage
Add tests for PI erroring exactly at the buffer boundary with
non-zero-terminated buffers (so we have to clear the last character
which changes the parsing flow slightly) and a test that makes sure
parse_embed_pcdata works properly with XML fragments where PCDATA can be
at the root level but can't be embedded into the document node.
2017-02-01 21:07:46 -08:00
Arseny Kapoulkine
0e3ccc7396 Remove redundant branch from xml_node::path()
The code works fine regardless of the *j->name check, and omitting this
makes the code more symmetric between the "count" and "write" stage;
additionally this improves coverage - due to how strcpy_insitu works
it's not really possible to get an empty non-NULL name in the node.
2017-02-01 21:05:37 -08:00
Arseny Kapoulkine
e56686f1e5 tests: Remove redundant coverage test
The only point was to try to test all paths where we can run out of
memory while decoding something. It seems like it may be impossible to
actually do this given that we can't run all paths as wchar_t size
detection is done at runtime...
2017-02-01 20:21:14 -08:00
Arseny Kapoulkine
1a3e92a7cc tests: Add more tests to increase coverage
This change adds more thorough tests for attribute conversion as well as
some assorted tests that fix gaps in coverage.
2017-01-31 20:40:50 -08:00
Arseny Kapoulkine
094a0c8ebe tests: Add compact hash table reserve test
This makes sure all .reserve calls failure paths are covered. These
tests don't explicitly test if reserve is present on all paths - this is
much harder to test since not all modifications require reserve to be
called, so we'll have to rely on a combination of automated testing and
sanity checking for this.

Also add more parsing out of memory coverage tests.
2017-01-31 19:19:04 -08:00
Arseny Kapoulkine
41fb880bf0 tests: Add coverage tests for encoding detection
Enumerate successfull cases and also cases where the detection stops
half-way and results in a different detected encoding.
2017-01-31 07:50:39 -08:00
Arseny Kapoulkine
ef64bef5c3 tests: More XPath coverage tests 2017-01-31 00:35:15 -08:00
Arseny Kapoulkine
6ffd2ffc41 tests: Add more DOM coverage tests
Add tests for various corner cases of DOM inspection and modification
routines.
2017-01-31 00:10:20 -08:00
Arseny Kapoulkine
9c7897b8d2 Remove null pointer test from first_element_by_path
All other functions treat null pointer inputs as invalid; now this
function does as well.
2017-01-30 23:55:31 -08:00
Arseny Kapoulkine
a1bc15c8d5 tests: Add more coverage tests
Expand out of memory coverage during XPath parsing and evaluation and
add some other small tests.
2017-01-30 23:52:32 -08:00
Arseny Kapoulkine
f500435cb4 XPath: Remove (re)allocate_throw and setjmp
Now error handling in XPath implementation relies on explicit error
propagation and is converted to an appropriate result at the end.
2017-01-30 22:31:57 -08:00
Arseny Kapoulkine
9e40c58532 XPath: Replace all (re)allocate_throw with (re)allocate_nothrow
This generates some out-of-memory code paths that are not covered by
existing tests, which will need to be resolved later.
2017-01-30 22:28:57 -08:00
Arseny Kapoulkine
60d5688a87 tests: Make predicate out-of-memory test less aggressive
Currently this test has very large runtime and relies on the fact that
the first memory allocation error causes the test to terminate. This
does not work with new behavior of running the query through and
reporting the error at the end, so make the runtime reasonable but still
generate enough memory to blow past the budget.
2017-01-30 22:27:48 -08:00
Arseny Kapoulkine
c370d1190d XPath: Fix reallocate_nothrow to preserve existing state
Instead of rolling back the allocation and trying to allocate again,
explicitly handle inplace reallocate if possible, and allocate a new
block otherwise.

This is going to be important once we use reallocate_nothrow from a
non-throwing context.
2017-01-30 22:10:13 -08:00
Arseny Kapoulkine
1a2e4b88ee XPath: Use nonthrowing allocations in duplicate_string
This requires explicit error handling for xpath_string::data calls.
2017-01-30 21:58:53 -08:00
Arseny Kapoulkine
ac150d504e XPath: Throw std::bad_alloc if we got an out-of-memory error
This allows us to gradually convert exception handling of out-of-memory
during evaluation to a non-throwing approach without changing the
observable behavior.
2017-01-30 21:58:53 -08:00
Arseny Kapoulkine
02cee98492 tests: Add more tests for branch coverage
gcov -b surfaced many lines with partial coverage, where branch is only
ever taken or not taken, or one of the expressions in a complex
conditional is always either true or false. This change adds a series of
tests (mostly focusing on XPath) to reduce the number of partially
covered lines.
2017-01-30 21:58:48 -08:00
Arseny Kapoulkine
cac1d8ad9f tests: Add an error propagation test for XPath
This test is supposed to test error coverage in different expressions
that are nested in other expressions to reduce the number of never-taken
branches in tests (and make sure we aren't missing any).
2017-01-30 11:51:07 -08:00
Arseny Kapoulkine
1b3e8614e7 XPath: Reword brace mismatch errors for clarity 2017-01-30 11:51:07 -08:00
Arseny Kapoulkine
1ed6d2102b XPath: Improve error message for expressions like .[1]
W3C specification does not allow predicates after abbreviated steps.
Currently this results in parsing terminating at the step, which leads
to confusing error messages like "Invalid query" or "Unmatched braces".
2017-01-30 11:51:07 -08:00
Arseny Kapoulkine
bc1e444694 XPath: Track allocation errors more explicitly
Any time an allocation fails xpath_allocator can set an externally
provided bool. The plan is to keep this bool up until evaluation ends,
so that we can use it to discard the potentially malformed result.
2017-01-30 11:51:07 -08:00
Arseny Kapoulkine
298996df11 Enable branch probabilities for gcov 2017-01-30 11:51:07 -08:00
Arseny Kapoulkine
635fe02801 XPath: Provide non-throwing and throwing allocations in xpath_allocator
For both allocate and reallocate, provide both _nothrow and _throw
functions; this change renames allocate() to allocate_throw() (same for
reallocate) to make it easier to change the code to remove throwing
variants.
2017-01-29 22:02:58 -08:00
Arseny Kapoulkine
8aa8e11ba6 tests: Add query out of memory test 2017-01-29 21:53:40 -08:00
Arseny Kapoulkine
6abf1d7c1a XPath: Minor error handling refactoring
Handle node type error before creating expression node
2017-01-29 21:53:23 -08:00
Arseny Kapoulkine
34b8c5908b Add NO_EXCEPTIONS build to Travis 2017-01-29 21:52:42 -08:00
Arseny Kapoulkine
4fa2241d7b XPath: Route out-of-memory errors through the exceptionless path
We currently need to convert error based on the text to a different type
of C++ exceptions when C++ exceptions are enabled.
2017-01-29 21:09:12 -08:00
Arseny Kapoulkine
bd8e2d782e XPath: Forward all node constructors through alloc_node
This allows us to handle OOM during node allocation without triggering
undefined behavior that occurs when placement new gets a NULL pointer.
2017-01-29 21:09:12 -08:00
Arseny Kapoulkine
293fccf3b0 XPath: Do not use exceptions to propagate parsing errors
Instead, return 0 and rely on parsing logic to propagate that all the
way down, and convert result to exception to maintain existing
interface.
2017-01-29 21:09:12 -08:00
Arseny Kapoulkine
7bb433b141 XPath: Assume that every function can fail and return 0
Propagate the failure to the caller manually. This is a first step to
parser structure that does not depend on exceptions or longjmp for error
handling (and thus matches the XML parser). To preserve semantics we'll
have to convert error code to exception later.
2017-01-29 21:09:12 -08:00
Arseny Kapoulkine
d72c0763f9 XPath: Minor parsing refactoring
Simplify function argument parsing by folding arg 0 parsing into the
main loop, reuse expression parsing logic for unary expression
2017-01-29 20:15:14 -08:00
Arseny Kapoulkine
60e580c2a8 XPath: Remove parse_function_helper
It was only used in three places and didn't really make the code more
readable.
2017-01-29 20:04:34 -08:00