Solution: do not rely __cplusplus >= 201103L to detect whether the
compiler supports thread safe static initialisation, but check only
the proper feature preprocessor macro.
GCC introduced it in version 8, and Clang in version 6.
Solution: initialise variable in options.cpp to dummy value to fix
compiler complaint:
src/options.cpp: In function
'int zmq::do_setsockopt_int_as_bool_strict(const void*, size_t, bool*)':
src/options.cpp:121:5: error: 'value' may be used uninitialized in
this function [-Werror=maybe-uninitialized]
src/options.cpp: In function
'int zmq::do_setsockopt_int_as_bool_relaxed(const void*, size_t, bool*)':
src/options.cpp:135:31: error: 'value' may be used uninitialized in
this function [-Werror=maybe-uninitialized]
The zero copy decoding strategy implemented for 4.2.0 can lead to a large
increase of main memory usage in some cases (I have seen one program go up to
40G from 10G after upgrading from 4.1.4). This commit adds a new option to
contexts, called ZMQ_ZERO_COPY_RECV, which allows one to switch to the old
decoding strategy.
ZeroMQ currently supports location independent addressing using TIPC
Port Names with tipc://{type,instance}. This commits adds support for
connecting and binding using TIPC Port Identity addresses. To connect
using Port Identities the expected format is tipc://<Z.C.N:Ref>, e.g.
"tipc://<1.2.3:123123>". To bind using TIPC Port Identities the expected
format is "tipc://<*>".
reuse
Solution: extract into functions defined in ip.hpp
Problem: signaler_t::make_fdpair not reusable
Solution: move make_fdpair to ip.hpp
Problem: epoll worker with no fds cannot be stopped
Solution: use interruptible epoll_pwait call
Problem: insufficient unit tests for poller
Solution: add test cases
* Problem: build failure with select as polling mechanism
Solution: cast mailbox_handle argument to (poller_t::handle_t) like in
the reaper thread class.
* Problem: build failure due to INT_MAX use without include
Solution: include limits and climits in src/select.cpp where INT_MAX is
used
* Problem: build failure due to unused variable in select.cpp
Solution: move the declaration of int rc inside the ifdef block where
it is actually used
* Problem: reference to wrong variable in select.cpp breaks build
Solution: fix it
* Problem: family_entry_t constructor has no body, build fails on *nix
Solution: add empty inline function in the struct
* Problem: no test coverage for poll and select
Solution: add Travis jobs for them on Linux
* Problem: Travis jobs cannot run in container infra
Solution: set sudo: false as it is not required anymore
* Background thread scheduling
- add ZMQ_THREAD_AFFINITY ctx option; set all thread scheduling options
from the context of the secondary thread instead of using the main
process thread context!
- change ZMQ_THREAD_PRIORITY to support setting NICE of the background
thread when using SCHED_OTHER
Solution: add ZMQ_ZAP_ENFORCE_DOMAIN to hide backward incompatible
change and make it disabled by default.
In a future release that breaks API compatibility we can then switch
the default to enabled in order to achieve full RFC compatibility.
Fixes#2762
On FreeBSD the sysmbol __FreeBSD_kernel__ is only defines if a
specific param.h file is included, unlike Debian/kFreeBSD where this
symbol is always defined. So also compile the FreeBSD specific code
if __FreeBSD__ is defined for FreeBSD 11 & 12 compatibility.
Signed-off-by: Christopher Hall <hsw@ms2.hinet.net>
Solution: when a connection breaks and ZMQ_RECONNECT_IVL is set to -1,
which means a reconnection will not be attempted, send a message from
the I/O thread to the application thread to make the socket call
term_endpoint, which is the equivalent of manually calling
zmq_disconnect.
This way subsequent zmq_connect call to the same endpoint will attempt
again to do a connection.
Otherwise, for some socket types like SUBs, those new connects will
fail as the endpoint is recorded, despite the connection having been
permanently closed.
Add test cases to exercise this corner case with TCP and IPC.
zmq::socket_poller_t::wait() had an important set of common lines
between POLL and SELECT variant.
Solution: refactor zmq::socket_poller_t::wait() and add the
following methods:
zmq::socket_poller_t::zero_trail_events()
zmq::socket_poller_t::check_events()
zmq::socket_poller_t::adjust_timeout()
Signed-off-by: Marc Sune <mardevel@gmail.com>
Solution: do not assert() the values returned; if we failed to set the thread name - we just have harder debugging, not flawed production conditions.
Closes github issue #2679
Signed-off-by: Jim Klimov <EvgenyKlimov@eaton.com>
Solution: added tests for CURVE, add emitting of client-side event in curve_client_t; add ZAP code 300/500 tests for all mechanisms; suppress sending an error message for ZAP code 300
Solution: use check_basic_command_structure in curve_client_t::decode, also prepare other client mechanisms to use that method by rearranging inheritance hierarchy
* Problem: no test case with CURVE encryption and large identity
Solution: added test case (currently crashing)
* Problem: possible buffer overflow in mechanism_t::add_property
Solution: add target buffer length parameter and check the buffer is sufficiently large
* Problem: test cases accidentally excluded from build
Solution: remove #if/#endif
* Problem: possible buffer overruns related to metadata at various locations
Solution: allocate buffer large enough for actual metadata, reduce code duplication
* Problem: syntax error related to pointer type conversion
Solution: change argument type of make_command_with_basic_properties to const char *
* Problem: large metadata may cause an assertion in produce_initiate
Solution: Allow metadata of arbitrary size in produce_initiate
* Extracted connect_vanilla_socket function
* Problem: no tests for ZMTP-CURVE protocol errors
Solution: added two test cases with erroneous HELLO commands
* Problem: insufficient tests for ZMTP-CURVE protocol errors
Solution: added two test cases with erroneous HELLO command version
* Problem: test HELLO message is invalid apart from deliberate errors
Solution: create cryptographically correct HELLO message
add tweetnacl.c to test_security_curve
* Problem: nonce is incorrect, build fails with GCC
Solution: use correct non prefix
* Problem: make builds are failing
Solution: transfer CMake changes to (auto)make files
* Problem: nonce is incorrect, build fails with GCC
Solution: use correct non prefix
* Problem: make builds are failing
Solution: transfer CMake changes to (auto)make files
* Problem: no test with INITIATE command with invalid length
Solution: added test case
* Problem: code duplication between test_security_curve.cpp and curve_client.cpp
Solution: extracted parts of zmq::curve_client_t::produce_hello into reusable function
* Problem: code duplication between test_security_curve.cpp and curve_client.cpp
Solution: extracted further parts of zmq::curve_client_t into reusable functions
added missing file
* Problem: mechanism_t::add_property can be declared static
Solution: declare mechanism_t::add_property static
* Problem: intermediate crypto data needs to be passed between static function calls to curve_client_tools_t
Solution: add non-static member functions
* Problem: msg_t instance may be closed twice
Solution: remove offending close
* Problem: prepare_hello uses static curve_client_tools_t::produce_hello
Solution: Use non-static curve_client_tools_t::produce_hello
* Problem: no test with invalid command name where INITIATE command is expected
Solution: added test case
* Problem: make builds are failing due to curve_client_tools.hpp not being found
Solution: add curve_client_tools.hpp to list of source files
* Problem: wrong initializer order in zmq::curve_client_t
Solution: reorder
* Problem: under non-Windows systems, test fails because random_open was not called
Solution: call random_open/random_close within test
* Problem: conflict between custom function htonll and macro definition on Darwin
Solution: define htonll function only if not defined as a macro
* Problem: nullptr not defined on all platforms
Solution: replace nullptr by NULL
* Problem: libsodium builds not working
Solution: adapt compile and link file sets for libsodium builds
* Problem: Makefile.am broken
Solution: Fix syntax
* Problem: no tests for garbage encrypted cookie or content in INITIATE
Solution: added test cases
* Problem: test cases accidentally excluded from build
Solution: remove #if/#endif
* Solution: some error cases are unreachable
Problem: for the time being, added some comments without changing the code
* Added comments on hard-to-test cases
Solution: force the compiler to make the atomic_counter_t alignment
friendly.
This will ensure that the pointers inside the buffers allocated by
shared_message_memory are aligned, at the cost of growing the memory
size of atomic_counter_t from 4 to 8 bytes on 64 bit (when not using
mutexes).
Note that although content_t contains an atomic_counter_t, the
compiler already padded the struct so there is no change in the
buffer sizes used by the engines, save for the extra 4 bytes for the
buffer's own single atomic counter.
Fixes#2588
Solution: remove requirement to manually define macro and just check
for the C++ supported version.
Note that compiler intrinsics still have priority if available, to
avoid changes unless necessary.
* Problem: missing test for status code 300, inadequate assertion for status code 500
Solution: add test, change assertion (currently test fails)
* Problem: gcc compiler error deprecated conversion from string constant
Solution: declare variable as const
* Problem: in case of ZAP handler returning a status code other than 200, no appropriate event is emitted
Solution: immediately emit event after receiving reply from ZAP handler
* Problem: endpoint address is not included in zap-reply monitor event
Solution: added functions to retrieve endpoint address in zmq::i_engine and zmq::session_base_t
removed unused code block in zmq::stream_engine_t::next_handshake_command
* Problem: wrong formatting
Solution: fix formatting
* Problem: test fails because of EPIPE
Solution: add EPIPE/ECONNRESET/ECONNAGAIN handling for more test cases
* Fixing #2002 one way of doing it
* Mechanisms can implement a new method `error_detail()`
* This error detail have three values for the moment: no_detail
(default), protocol, encryption.
+ generic enough to make sense for all mechanisms.
- low granularity level on information.
* Fixing #2002: implementation of the error details
The ZMQ_EVENT_HANDSHAKE_FAILED event carries the error details
as value.
* Removed Microsoft extenstion for enum member access
This was leading to compilation error under linux.
* Adaptation of CURVE test cases
* Monitoring event: changed API for detailed events
Removed ZMQ_EVENT_HANDSHAKE_FAILED and replaced it by:
- ZMQ_EVENT_HANDSHAKE_FAILED_NO_DETAIL,
- ZMQ_EVENT_HANDSHAKE_FAILED_PROTOCOL,
- ZMQ_EVENT_HANDSHAKE_FAILED_ENCRYPTION
Adaptation of text case `security_curve`
* Removed event value comparison
This was introduced for the previous API model adaptation
* Removed the prints in std output and added missing details
`current_error_detail` was not set in every protocol error cases
* Fixed initialization of current_error_detail
* Fixed error in greeting test case
The handshake failure due to mechanism mismatch in greeting is actually
a protocol error. The error handling method consider it like so and
send a protocol handshake failure monitoring event instead of no_detail.
Fixed the test_security_curve expectation as well.
* Upgraded tests of monitoring events
The tests check the number of monitoring events received
* Problem: does not build under Linux or without ZMQ_DRAFT_API
Solution:
- properly use ZMQ_DRAFT_API conditional compilation
- use receive timeouts instead of Sleep
* Problem: duplicate definition of variable 'timeout'
Solution: merged definitions
* Problem: inconsistent timing dependencies
Solution: reduce timing dependency by using timeouts at more places
* Problem: assertion failure under Linux due to unexpected monitor event
Solution: output event type to aid debugging
* Problem: erroneous assertion code
* Problem: assertion failure with a garbage server key due to an extra third event
Solution: changed assertion to expect three events (needs to be checked)
* Problem: extra include directive to non-existent file
Solution: removed include directive
* Problem: assertion failure on appveyor for unknown reason
Solution: improve debug output
* Problem: no build with libsodium and draft api
Solution: add build configurations with libsodium and draft api
* Problem: assertion failure on CI
Solution: change assertion to reflect actual behaviour on CI (at least temporarily)
* Problem: error in condition in assertion code
* Problem: assertion failure on CI
Solution: generalize assertion to match behavior on CI
* Problem: assertion failures on CI
Solution: removed inconsistent assertion on no monitor events before flushing
improved debuggability by converting function into macro
* Problem: diverging test code for three analogous test cases with garbage key
Solution: extract common code into function
* Problem: does not build without ZMQ_BUILD_DRAFT_API
Solution: introduce dummy variable
* Attempt to remove workaround regarding ZMQ_EVENT_HANDSHAKE_FAILED_NO_DETAIL again
* Problem: EAGAIN error after handshake complete if there is no more data in inbuffer
Solution: Skip tcp_read attempt in that case
* Problem: handshaking event emitted after handshaking failed
Solution: use stream_engine_t::handshaking instead of mechanism_t::status() to determine whether still handshaking
* Include error code in debug output
* Improve debugging output: output flushed events
* Split up ZMQ_EVENT_HANDSHAKE_FAILED_PROTOCOL into ZMQ_EVENT_HANDSHAKE_FAILED_ZMTP and ZMQ_EVENT_HANDSHAKE_FAILED_ZAP
* Fixed compilation without ZMQ_BUILD_DRAFT_API
* Renamed ZMQ_EVENT_HANDSHAKE_SUCCEED to ZMQ_EVENT_HANDSHAKE_SUCCEEDED for language consistency
* Renamed ZMQ_EVENT_HANDSHAKE_SUCCEED to ZMQ_EVENT_HANDSHAKE_SUCCEEDED for language consistency
* Renamed ZMQ_EVENT_HANDSHAKE_SUCCEED to ZMQ_EVENT_HANDSHAKE_SUCCEEDED for language consistency
* Fixed assert_monitor_event (require event instead of allowing no event)
Reverted erroneous change to handshaking condition
Renamed test_wrong_key to test_garbage_key
Generalized assumption in test_garbage_key to allow for ZMQ_EVENT_HANDSHAKE_FAILED_NO_DETAIL with error == EPIPE
* Better isolate test cases from each other by providing a fresh context & server for each
* Added diagnostic output
* Changed assertion to reflect actual behavior on CI
* Fixed formatting, observe maximum line length
* Fixed formatting, observe maximum line length
* Increase timeout to check if this fixes valgrind run
* Close server with close_zero_linger
* Increase timeout to check if this fixes valgrind run
* Increase timeout to check if this fixes valgrind run
* Generalize assertion to also work with valgrind
* Fixed formatting
* Add more diagnostic output
* Generalize assertion to also work with valgrind
Linux now supports Virtual Routing and Forwarding (VRF) as per:
https://www.kernel.org/doc/Documentation/networking/vrf.txt
In order for an application to bind or connect to a socket with an
address in a VRF, they need to first bind the socket to the VRF device:
setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
Note "dev" is the VRF device, eg. VRF "blue", rather than an interface
enslaved to the VRF.
Add a new socket option, ZMQ_BINDTODEVICE, to bind a socket to a device.
In general, if a socket is bound to a device, eg. an interface, only
packets received from that particular device are processed by the socket.
If device is a VRF device, then subsequent binds/connects to that socket
use addresses in the VRF routing table.
Solution: add a crypto [de-]initialiser, refcounted and serialised
through critical sections.
This is necessary as utility APIs such as zmq_curve_keypair also
call into the sodium/tweetnacl libraries and need the initialisation
outside of the zmq context.
Also the libsodium documentation explicitly says that sodium_init
must not be called concurrently from multiple threads, which could
have happened until now. Also the randombytes_close function does
not appear to be thread safe either.
This change guarantees that the library is initialised only once at
any given time across the whole program.
Fixes#2632
Before this commit, xhas_out() was returning true regardless. This
was correct before the ZMQ_ROUTER_MANDATORY flag as introduced.
However, ZMQ_POLLOUT.
With this commit, _if_ ZMQ_ROUTER_MANDATORY is set, xhas_out() will
return false if ALL peer's outgoing pipes are full.
There is an outstanding high-level design question:
If ZMQ_ROUTER_MANDATORY is set, and zmq_poll() waits for ZMQ_POLLOUT
events, zmq_poll() will immediately wake up if only 1 pipe has
room to send, regardless of the peer, creating a busy loop of
zmq_poll() wake-up, zmq_send() (EAGAIN). There is no way for
the application to selectively wait for ZMQ_POLLOUT for specific
peer(s), which seems somehow necessary in ZMQ_ROUTER_MANDATORY.
This discussion will be addressed in a separate issue.
Signed-off-by: Marc Sune <marc@voltanet.io>
Signed-off-by: Fredi Raspall <fredi@voltanet.io>
Solution: if a CURVE server is using zmq_connect, the same session
will be used for any client "reconnect" (actual binds). This is
acceptable, so do not assert if zap_pipe already exists during the
handshake, but simply reuse it.
Fixes#2608
Solution: remove the pipe from the real trie when a peer disconnects.
Also add a unit test that exercises the behaviour by reconnecting
a different socket and sending a message that matches.
Fixes#2601 and introduced by #2042
Solution: revert change that made ZAP mandatory.
The "Stonehouse" pattern, where CURVE is used only for encryption and
without authentication, is a valid use case so we should still
support it.
Also restore CURVE testing in the test_heartbeat.
Fixes#2594
Solution: if inproc://zeromq.zap.01 exists, which means ZAP is
enabled, abort immediately if it cannot be used (eg: out of memory)
or it is configured incorrectly (eg: wrong socket type).
Otherwise authentication failures will simply be ignored and
unauthorised peers will be allowed to slip in.
problem: for zmq radio/dish pattern, if the radio process restarts, the dish will not resend subscriptions to radio. And the result is that the dish will never receive any more messages.
solution: in session_base_t::reconnect (), take ZMQ_DISH into consideration when invoking hiccup method.
In function session_base_t::reconnect, if we set immediate to 1 and set linger, we will get into first block of reconnect function, and set pipe to NULL, but we forget to cancel timer of linger. Once timer tiggered, we will get coredump. Solution: cancel timer in the end of set pipe to NULL
Solution: don't set thread name on Android
Setting a thread name on Android may fail with "permission
denied" error and abort the process due to failed assertion.
Tested on Android 5 and 6 (two phones).
Strangely enough, it only happens on signed APKs and is fine
in debug. Using JeroMQ is not an option as we need TCP keepalive
settings and authentication which JeroMQ doesn't support.
Solution: roll back the pipe if writing messages other than the
first fails in router::xsend. Roll it back also when the pipe is
terminating.
Also add test case that reproduces the memory leak when ran with
valgrind.
Fixes#2567
Solution: roll back the pipe if writing messages other than the
first fails in router::xsend.
Also add test case that reproduces the memory leak when ran with
valgrind.
Fixes#2567
Solution: do not define ZMQ_HAVE_POLLER in src/zmq_drafts.h otherwise
src/zmq.cpp will implement zmq_poll using the new poller classes.
Same for ZMQ_HAVE_TIMERS, even though it has no internal effect, but
to be safe against future development.
Problem: GSSAPI DRAFT code was made conditional on
ZMQ_BUILD_DRAFT_API, but zmq_draft.h duplicates the DRAFT
symbols definitions from zmq.h so this is unnecessary.
Solution: drop the extra ifdefs
Problem: GSSAPI NAMETYPE socket option numbers were modified
+1000 when moved to DRAFT section, but should use the definitive
values while in DRAFT to minimize disruption later.
Solution: renumber the socket options
Problem: The new GSSAPI NAMESPACE options should have been
added to the DRAFT section of the API so they can be changed
until stabilized.
Solution:
- Move defines to the DRAFT section of zmq.h
- Duplicate them in zmq_draft.h, as is the local custom
- Compile only if defined (ZMQ_BUILD_DRAFT_API)
- Refactor internals slightly to avoid #ifdef hell
Problem: principals are looked up unconditionally
with the GSS_C_NT_HOSTBASED_SERVICE name type.
Solution: Add two new socket options to set the name type
for ZMQ_GSSAPI_PRINCIPAL and ZMQ_GSSAPI_SERVICE_PRINCIPAL:
ZMQ_GSSAPI_PRINCIPAL_NAMETYPE
ZMQ_GSSAPI_SERVICE_PRINCIPAL_NAMETYPE
They take an integer argument which must be one of
ZMQ_GSSAPI_NT_HOSTBASED (0) - default
ZMQ_GSSAPI_NT_USER_NAME (1)
ZMQ_GSSAPI_NT_KRB5_PRINCIPAL (2)
These correspond to GSSAPI name types of:
GSS_C_NT_HOSTBASED_SERVICE
GSS_C_NT_USER_NAME
GSS_KRB5_NT_PRINCIPAL_NAME
Fixes#2542
Problem: one call to gss_import_name() includes the terminating
NULL in a gss_buffer_desc.length, and one doesn't.
According to the examples at:
http://docs.oracle.com/cd/E19253-01/816-4863/overview-22/index.html
the NULL should be included in the length.
Solution: Fix one case to include the terminating NULL in the length.
Problem: if client sets ZMQ_GSSAPI_PRINCIPAL to a name
for which credentials cannot be obtained, authentication
proceeds with default credentials.
Solution: Before initializing the security context, check
whether there was a failed attempt to acquire credentials
for a specific principal and bail out if so.
Fixes#2531
Problem: if client sets the ZMQ_GSSAPI_PRINCIPAL to a valid
principal, authentication fails.
When an application sets ZMQ_GSSAPI_PRINCIPAL, whether as a
client or a server, libzmq internally calls gss_acquire_cred()
with cred_usage=GSS_C_ACCEPT. This cred_usage setting is for
acceptors (servers) only, thus it doesn't work for initiators
(clients).
Solution: Change the cred_usage parameter to GSS_C_BOTH to allow
initiators to set ZMQ_GSSAPI_PRINCIPAL.
* change macOS < 10.12 clock to SYSTEM_CLOCK, fixes#2537
* remove clock_id option from alt_clock_gettime since we always want a monotonic clock.
* update header definition for alt_clock_gettime
* pass clock definition down to host_get_clock_service for macOS < 10.12
* change to monotonic clocks
Treat ECONNRESET, ENETRESET and EINTR as non-fatal
failures in TCP Tuning on Windows as they are in the
Unixes. Also adds ENETRESET to the Unix list that was missing.
Detect AI_V4MAPPED is not supported in getaddrinfo() on the fly
and retry query. This issue was seen on ancient FreeBSD releases
and current implementation does support this flag correctly.
* Prevent DOS by asserts in TCP tuning
-Propagates socket option errors from the
tuning functions to the callers.
-Asserts a subset of error conditions during tuning,
excluding external network causes.
-Checks tuning results in 3 call sites and treats
them like failures to connect, accept, etc.
* Fix variable name
* Remove lambda requiring C++11
* add define for windows/UWP
* prevent issue with COM references
* gettickcount not available on uwp
* add compiler definitions
* add convenitnece cmake file
* brute force uwp compilation
* fix compiler version
* cosmetics
