tqcq/sled
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

feat add profiling
All checks were successful
linux-x64-gcc / linux-gcc (Debug) (push) Successful in 48s
Details
linux-x64-gcc / linux-gcc (Release) (push) Successful in 48s
Details

Browse Source
This commit is contained in:
tqcq 2024-03-16 22:56:10 +08:00
parent e05c1f4894
commit 719fecd4bc
30 changed files with 4673 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
3party/cppuprofile/.gitignore vendored Normal file
View File

@ -0,0 +1,5 @@
build
CMakeFiles
CMakeCache
.vscode
.idea

14
3party/cppuprofile/.pre-commit-config.yaml Normal file
View File

@ -0,0 +1,14 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.2.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: v16.0.6
hooks:
- id: clang-format

13
3party/cppuprofile/CMakeLists.txt Normal file
View File

@ -0,0 +1,13 @@
PROJECT(cppuprofile)
CMAKE_MINIMUM_REQUIRED(VERSION 2.8.12)
OPTION(PROFILE_ENABLED "Whether library performs profiling operations or does nothing" ON)
OPTION(SAMPLE_ENABLED "Whether sample binary is built or not" OFF)
OPTION(BUILD_SHARED_LIBS "Build shared libraries" OFF)
OPTION(GPU_MONITOR_NVIDIA "Whether NVidiaMonitor class for monitoring NVidia GPUs is compiled and embedded to the library" OFF)
ADD_SUBDIRECTORY(lib)
IF(SAMPLE_ENABLED)
ADD_SUBDIRECTORY(sample)
ENDIF()

26
3party/cppuprofile/LICENSE Normal file
View File

@ -0,0 +1,26 @@
Copyright (c) 2022, Orange
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS AS IS AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

153
3party/cppuprofile/README.md Normal file
View File

@ -0,0 +1,153 @@
# cppuprofile
![GitHub tag](https://img.shields.io/github/v/tag/orange-opensource/cppuprofile)
![Licensing](https://img.shields.io/github/license/Orange-OpenSource/cppuprofile)
[![Documentation](https://img.shields.io/badge/documentation-ok-green)](https://orange-opensource.github.io/cppuprofile/)
![Linux](https://img.shields.io/badge/Linux-full_support-green?logo=linux&logoColor=white)
![Windows](https://img.shields.io/badge/Windows-partial_support-orange?&logo=windows&logoColor=white)
![MacOS](https://img.shields.io/badge/MacOS-not_tested-orange?&logo=apple&logoColor=white)
This project provides a tiny C++ profiling library for monitoring:
* execution time
* CPU usage
* memory usage
* GPU usage and memory
This library aims at collecting metrics on embedded devices to monitor device
performance while operating heavy tasks or booting for example. Those metrics can
be useful to check that load is properly spread onto all CPU cores or
that memory is not starved.
This library can also run on non-embedded devices like servers or desktop PCs. It is
compatible with Linux and Windows.
Metrics are stored in a CSV file (the path is configurable).
## Usage
The library is lightweight, simple and easy to use. It can be easily used from an existing application or integrated in a dedicated monitoring application.
Full API documentation is available [here](https://orange-opensource.github.io/cppuprofile/).
### System memory and CPU usage monitoring
```cpp
#include <uprofile/uprofile.h>
...
uprofile::start("uprofile.log");
uprofile::startSystemMemoryMonitoring(200);
uprofile::startCPUUsageMonitoring(200);
...
uprofile::stop();
```
### Record time execution
```cpp
uprofile::timeBegin("my_custom_function");
...
uprofile::timeEnd("my_custom_function");
```
### GPU monitoring
The library also supports GPU metrics monitoring like usage and memory. Since GPU monitoring is specific to each vendor, an interface `IGPUMonitor` is available to abstract each vendor monitor system.
To monitor a specific GPU, you must subclass `IGPUMonitor`:
```cpp
#include <uprofile/igpumonitor.h>
class MyGPUMonitor: public uprofile::IGPUMonitor {
public:
float getUsage() override;
void getMemory(int& usedMem, int& totalMem) override;
}
```
And then inject it at runtime to the `uprofile` monitoring system:
```cpp
uprofile::addGPUMonitor(new MyGPUMonitor);
uprofile::start("uprofile.log");
uprofile::startGPUMemoryMonitoring(200);
uprofile::startGPUUsageMonitoring(200);
```
#### Supported GPU monitoring
Here is the list of GPUs supported by `cppuprofile`
* NVidia Graphics Cards (through `nvidia-smi`). Pass `-DGPU_MONITOR_NVIDIA=ON` as compile option and inject `uprofile::NvidiaMonitor` from `monitors/nvidiamonitor.h` as `GPUMonitor`. The `nvidia-smi` tool should be installed into `/usr/bin` directory.
## Build
The build process is based on CMake. Minimum version is 2.8.
```commandline
$ cmake --configure . -B ../build-cppuprofile
$ cmake --build ../build-cppuprofile
```
### Shared/dynamic library
By default, it generates a shared library on Linux and a dynamic library (DLL) on Windows. To link with this library on Windows, you must
pass `-DUPROFILE_DLL` definition to CMake.
### Static library
If you want to generate a static library, you must use `-DBUILD_SHARED_LIBS=OFF` CMake option.
### Disable profiling in Release mode
If you want to disable profiling in Release mode or if you want to only enable profiling in particular cases, you can use the `PROFILE_ENABLED` option (set to `ON` by default).
To disable the profiling:
```commandline
$ cmake --configure . -B ../build-cppuprofile -DPROFILE_ENABLED=OFF
```
## Tools
The project also brings a tool for displaying the different metrics in
a single view:
![ScreenshotShowGraph](doc/show-graph-screenshot.png)
This tool is written in Python3. It requires a set of dependency packages. To install them:
```commandline
$ pip3 install -r requirements.txt
```
Then
```commandline
$ ./tools/show-graph uprofile.log
```
Note that you can filter the metrics to display with `--metric` argument.
## Sample
The project provides a C++ sample application called `uprof-sample`
that shows how to use the `cppuprofile` library. You can build it with `SAMPLE_ENABLED` option:
```commandline
$ cmake --configure . -B ../build-cppuprofile -DSAMPLE_ENABLED=ON
$ cmake --build ../build-cppuprofile
$ ../build-cppuprofile/sample/uprof-sample
```
## Windows support limitations
The library compiles on Windows but only time execution is supported so far. Monitoring metrics like CPU Usage and system, process and nvidia GPU memory are not supported.
Contributions are welcomed.
## License
This project is licensed under BSD-3-Clause license. See LICENSE file for any further information.

2579
3party/cppuprofile/doc/doxygen.cfg Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
3party/cppuprofile/doc/show-graph-screenshot.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 54 KiB

114
3party/cppuprofile/lib/CMakeLists.txt Normal file
View File

@ -0,0 +1,114 @@
PROJECT(cppuprofile DESCRIPTION "CppProfiling library")
CMAKE_MINIMUM_REQUIRED(VERSION 2.8.12)
SET( CMAKE_USE_RELATIVE_PATHS ON)
IF(CMAKE_COMPILER_IS_GNUCXX)
ADD_DEFINITIONS( -std=c++0x )
ADD_DEFINITIONS( -D_GLIBCXX_USE_NANOSLEEP )
ENDIF()
IF(WIN32)
add_compile_options(/W4)
ELSE()
add_compile_options(-Wall -Werror)
ENDIF()
IF(BUILD_SHARED_LIBS)
IF(CMAKE_VERSION VERSION_GREATER_EQUAL 3.12.0)
ADD_COMPILE_DEFINITIONS(_UPROFILE_BUILD_SHARED)
ELSE()
ADD_DEFINITIONS(-D_UPROFILE_BUILD_SHARED)
ENDIF()
ENDIF()
IF(PROFILE_ENABLED)
IF(CMAKE_VERSION VERSION_GREATER_EQUAL 3.12.0)
ADD_COMPILE_DEFINITIONS(PROFILE_ON)
ADD_DEFINITIONS( -DPROFILE_ON )
ELSE()
ADD_DEFINITIONS(-DPROFILE_ON)
ENDIF()
ENDIF()
SET(UProfile_PUBLIC_HEADERS
api.h
uprofile.h
timestampunit.h
igpumonitor.h
)
SET(UProfile_IMPL
uprofile.cpp
uprofileimpl.cpp
util/timer.cpp
util/cpumonitor.cpp
)
IF(GPU_MONITOR_NVIDIA)
LIST(APPEND UProfile_PUBLIC_HEADERS monitors/nvidiamonitor.h)
LIST(APPEND UProfile_IMPL monitors/nvidiamonitor.cpp)
ENDIF()
SET(UProfile_HEADERS
${UProfile_PUBLIC_HEADERS}
uprofileimpl.h
util/timer.h
util/cpumonitor.h
)
SET(UProfile_SRCS
${UProfile_HEADERS}
${UProfile_IMPL}
)
SET(LIBRARY_NAME ${PROJECT_NAME})
ADD_LIBRARY(${LIBRARY_NAME} ${UProfile_SRCS}) # STATIC or SHARED are determined through BUILD_SHARED_LIBS flag
IF(BUILD_SHARED_LIBS AND UNIX)
# Hide symbols not explicitly tagged for export from the shared library
TARGET_COMPILE_OPTIONS(${LIBRARY_NAME} PRIVATE "-fvisibility=hidden")
ENDIF()
# Specify the include directories exported by this library
TARGET_INCLUDE_DIRECTORIES(${LIBRARY_NAME} PUBLIC
${CMAKE_CURRENT_SOURCE_DIR}
)
IF(UNIX)
TARGET_LINK_LIBRARIES(${LIBRARY_NAME} pthread)
ENDIF()
# Set specific pkg-config variables
SET(PKG_CONFIG_LIBDIR
"\${prefix}/lib"
)
SET(PKG_CONFIG_INCLUDEDIR
"\${prefix}/include/${LIBRARY_NAME}"
)
SET(PKG_CONFIG_LIBS
"-L\${libdir} -l${LIBRARY_NAME}"
)
SET(PKG_CONFIG_CFLAGS
"-I\${includedir}"
)
# Generate the pkg-config file
CONFIGURE_FILE(
"${CMAKE_CURRENT_SOURCE_DIR}/pkg-config.pc.cmake"
"${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}.pc"
)
# Install library project files
INSTALL(FILES ${UProfile_PUBLIC_HEADERS}
DESTINATION include/${LIBRARY_NAME}
)
INSTALL(TARGETS ${LIBRARY_NAME}
LIBRARY DESTINATION lib
ARCHIVE DESTINATION lib
)
INSTALL(FILES "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}.pc"
DESTINATION lib/pkgconfig
)

31
3party/cppuprofile/lib/api.h Normal file
View File

@ -0,0 +1,31 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2023 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#pragma once
// UPROFAPI is used to export public API functions from the DLL / shared library.
#if defined(_UPROFILE_BUILD_SHARED)
#if defined(_WIN32)
/* Build as a Win32 DLL */
#define UPROFAPI __declspec(dllexport)
#elif defined(__linux__)
/* Build as a shared library */
#define UPROFAPI __attribute__((visibility("default")))
#endif // if defined(_UPROFILE_BUILD_SHARED)
#elif defined(UPROFILE_DLL)
#if defined(_WIN32)
/* Call uprofile as a Win32 DLL */
#define UPROFAPI __declspec(dllimport)
#endif // if defined(_WIN32)
#endif // if defined(UPROFILE_DLL)
#if !defined(UPROFAPI)
#define UPROFAPI
#endif

44
3party/cppuprofile/lib/igpumonitor.h Normal file
View File

@ -0,0 +1,44 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2023 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#ifndef IGPUMONITOR_H_
#define IGPUMONITOR_H_
namespace uprofile
{
/**
* Interface to implement for monitoring GPU usage and memory
*
* No generic abstraction of GPU metrics exists
* on Linux nor Windows. So specific IGPUMonitor class should
* be defined to retrieve metrics from GPU vendor (Nvidia, AMD, Broadcom
* for RPI...)
*/
class IGPUMonitor
{
public:
virtual ~IGPUMonitor() {}
// Start monitoring
virtual void start(int period) = 0;
// Stop monitoring
virtual void stop() = 0;
// Return if monitor is currently watching data
virtual bool watching() const = 0;
// Usage should be in percentage
virtual float getUsage() const = 0;
// usedMem and totalMem should be returned as KiB
virtual void getMemory(int& usedMem, int& totalMem) const = 0;
};
}
#endif /* IGPUMONITOR_H_ */

171
3party/cppuprofile/lib/monitors/nvidiamonitor.cpp Normal file
View File

@ -0,0 +1,171 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2023 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#include "nvidiamonitor.h"
#include <algorithm>
#include <errno.h>
#include <fcntl.h>
#include <iostream>
#include <sstream>
#include <string.h>
#if defined(__linux__)
#include <sys/wait.h>
#include <unistd.h>
#endif
const string errorMsg = "Failed to monitor nvidia-smi process";
#if defined(__linux__)
int read_nvidia_smi_stdout(int fd, string& gpuUsage, string& usedMem, string& totalMem)
{
string line;
while (line.find('\n') == string::npos) { // full line read
char buffer[4096];
ssize_t count = read(fd, buffer, sizeof(buffer)); // if child process crashes, we gonna be blocked here forever
if (count == -1) {
return errno;
} else if (count > 0) { // there is something to read
line += string(buffer, count);
}
}
// Remove colon to have only spaces and use istringstream
auto noSpaceEnd = remove(line.begin(), line.end(), ',');
if (noSpaceEnd == line.end()) { // output trace does not have comma so something went wrong with the command
return ENODATA;
}
line.erase(noSpaceEnd, line.end());
std::istringstream ss(line);
ss >> gpuUsage >> usedMem >> totalMem;
return 0;
}
#endif
uprofile::NvidiaMonitor::NvidiaMonitor()
{
}
uprofile::NvidiaMonitor::~NvidiaMonitor()
{
stop();
}
void uprofile::NvidiaMonitor::start(int period)
{
watchGPU(period);
}
void uprofile::NvidiaMonitor::stop()
{
abortWatchGPU();
}
float uprofile::NvidiaMonitor::getUsage() const
{
std::lock_guard<std::mutex> lock(m_mutex);
return m_gpuUsage;
}
void uprofile::NvidiaMonitor::getMemory(int& usedMem, int& totalMem) const
{
std::lock_guard<std::mutex> lock(m_mutex);
usedMem = m_usedMem;
totalMem = m_totalMem;
}
void uprofile::NvidiaMonitor::watchGPU(int period)
{
if (m_watching) {
return;
}
#if defined(__linux__)
char* args[5];
args[0] = (char*)"/usr/bin/nvidia-smi";
string period_arg = "-lms=" + to_string(period); // lms stands for continuous watching
args[1] = (char*)period_arg.c_str();
args[2] = (char*)"--query-gpu=utilization.gpu,memory.used,memory.total";
args[3] = (char*)"--format=csv,noheader,nounits";
args[4] = NULL;
string output;
int pipes[2];
// Create the pipe
if (pipe(pipes) == -1) {
cerr << errorMsg << ": pipe creation failed" << endl;
return;
}
// Create a child process for calling nvidia-smi
pid_t pid = fork();
switch (pid) {
case -1: /* Error */
cerr << errorMsg << ": process fork failed" << endl;
return;
case 0: /* We are in the child process */
while ((dup2(pipes[1], STDOUT_FILENO) == -1) && (errno == EINTR)) {
}
close(pipes[1]);
close(pipes[0]);
execv(args[0], args);
cerr << "Failed to execute '" << args[0] << "': " << strerror(errno) << endl; /* execl doesn't return unless there's an error */
exit(1);
default: /* We are in the parent process */
int stdout_fd = pipes[0];
// Start a thread to retrieve the child process stdout
m_watching = true;
m_watcherThread = unique_ptr<std::thread>(new thread([stdout_fd, pid, this]() {
while (watching()) {
string gpuUsage, usedMem, totalMem;
// if the child process crashes, an error is raised here and threads ends up
int err = read_nvidia_smi_stdout(stdout_fd, gpuUsage, usedMem, totalMem);
if (err != 0) {
cerr << errorMsg << ": read_error = " << strerror(err) << endl;
m_mutex.lock();
m_watching = false;
m_mutex.unlock();
break;
}
m_mutex.lock();
m_gpuUsage = !gpuUsage.empty() ? stof(gpuUsage) : 0.f;
m_usedMem = !usedMem.empty() ? stoi(usedMem) * 1024 : 0; // MiB to KiB
m_totalMem = !totalMem.empty() ? stoi(totalMem) * 1024 : 0; // MiB to KiB
m_mutex.unlock();
}
}));
}
#else
cerr << errorMsg << endl;
#endif
}
void uprofile::NvidiaMonitor::abortWatchGPU()
{
#if defined(__linux__)
if (m_watcherThread) {
m_mutex.lock();
m_watching = false;
m_mutex.unlock();
m_watcherThread->join();
m_watcherThread.reset();
}
#endif
}
bool uprofile::NvidiaMonitor::watching() const
{
std::lock_guard<std::mutex> lock(m_mutex);
return m_watching;
}

49
3party/cppuprofile/lib/monitors/nvidiamonitor.h Normal file
View File

@ -0,0 +1,49 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2023 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#ifndef NVIDIAMONITOR_H_
#define NVIDIAMONITOR_H_
#include "api.h"
#include "igpumonitor.h"
#include <mutex>
#include <string>
#include <thread>
#include <vector>
using namespace std;
namespace uprofile
{
class NvidiaMonitor : public IGPUMonitor
{
public:
UPROFAPI explicit NvidiaMonitor();
UPROFAPI virtual ~NvidiaMonitor();
UPROFAPI void start(int period) override;
UPROFAPI void stop() override;
UPROFAPI bool watching() const override;
UPROFAPI float getUsage() const override;
UPROFAPI void getMemory(int& usedMem, int& totalMem) const override;
private:
void watchGPU(int period);
void abortWatchGPU();
mutable std::mutex m_mutex;
std::unique_ptr<std::thread> m_watcherThread;
bool m_watching = false;
int m_totalMem = 0;
int m_usedMem = 0;
float m_gpuUsage = 0.f;
};
}
#endif /* NVIDIAMONITOR_H_ */

8
3party/cppuprofile/lib/pkg-config.pc.cmake Normal file
View File

@ -0,0 +1,8 @@
Name: ${PROJECT_NAME}
Description: ${PROJECT_DESCRIPTION}
Version: ${PROJECT_VERSION}
prefix=${CMAKE_INSTALL_PREFIX}
includedir=${PKG_CONFIG_INCLUDEDIR}
libdir=${PKG_CONFIG_LIBDIR}
Libs: ${PKG_CONFIG_LIBS}
Cflags: ${PKG_CONFIG_CFLAGS}

23
3party/cppuprofile/lib/timestampunit.h Normal file
View File

@ -0,0 +1,23 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2022 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#ifndef TIMESTAMP_UNIT_H_
#define TIMESTAMP_UNIT_H_
namespace uprofile
{
enum class TimestampUnit {
EPOCH_TIME, // Time since epoch
UPTIME // Time since boot
};
}
#endif /* TIMESTAMP_UNIT_H_ */

113
3party/cppuprofile/lib/uprofile.cpp Normal file
View File

@ -0,0 +1,113 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2022 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#include <chrono>
#include <fstream>
#include <iostream>
#include <sstream>
#include "uprofile.h"
#include "uprofileimpl.h"
using namespace std::chrono;
#ifdef PROFILE_ON
#define UPROFILE_INSTANCE_CALL(func, ...) \
UProfileImpl::getInstance()->func(__VA_ARGS__);
#define UPROFILE_INSTANCE_CALL_RETURN(func, ...) \
return UProfileImpl::getInstance()->func(__VA_ARGS__);
#define UPROFILE_DESTROY_INSTANCE() \
UProfileImpl::destroyInstance();
#else
#define UPROFILE_INSTANCE_CALL(func, ...) (void)0;
#define UPROFILE_INSTANCE_CALL_RETURN(func, ...) \
return {}
#define UPROFILE_DESTROY_INSTANCE() (void)0;
#endif
namespace uprofile
{
void start(const char* file)
{
UPROFILE_INSTANCE_CALL(start, file);
}
void stop()
{
UPROFILE_INSTANCE_CALL(stop);
UPROFILE_DESTROY_INSTANCE();
}
void addGPUMonitor(IGPUMonitor* monitor)
{
UPROFILE_INSTANCE_CALL(addGPUMonitor, monitor);
}
void removeGPUMonitor()
{
UPROFILE_INSTANCE_CALL(removeGPUMonitor);
}
void setTimestampUnit(TimestampUnit tsUnit)
{
UPROFILE_INSTANCE_CALL(setTimestampUnit, tsUnit);
}
void timeBegin(const std::string& step)
{
UPROFILE_INSTANCE_CALL(timeBegin, step);
}
void timeEnd(const std::string& step)
{
UPROFILE_INSTANCE_CALL(timeEnd, step);
}
void startProcessMemoryMonitoring(int period)
{
UPROFILE_INSTANCE_CALL(startProcessMemoryMonitoring, period);
}
void startSystemMemoryMonitoring(int period)
{
UPROFILE_INSTANCE_CALL(startSystemMemoryMonitoring, period);
}
void startCPUUsageMonitoring(int period)
{
UPROFILE_INSTANCE_CALL(startCPUUsageMonitoring, period);
}
void startGPUUsageMonitoring(int period)
{
UPROFILE_INSTANCE_CALL(startGPUUsageMonitoring, period);
}
void startGPUMemoryMonitoring(int period)
{
UPROFILE_INSTANCE_CALL(startGPUMemoryMonitoring, period);
}
void getProcessMemory(int& rss, int& shared)
{
UPROFILE_INSTANCE_CALL(getProcessMemory, rss, shared);
}
void getSystemMemory(int& totalMem, int& availableMem, int& freeMem)
{
UPROFILE_INSTANCE_CALL(getSystemMemory, totalMem, availableMem, freeMem);
}
std::vector<float> getInstantCpuUsage()
{
UPROFILE_INSTANCE_CALL_RETURN(getInstantCpuUsage);
}
}

142
3party/cppuprofile/lib/uprofile.h Normal file
View File

@ -0,0 +1,142 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2022 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#ifndef UPROFILE_H_
#define UPROFILE_H_
#include <fstream>
#include <list>
#include <map>
#include <string>
#include <vector>
#include "api.h"
#include "igpumonitor.h"
#include "timestampunit.h"
/**
* @defgroup uprofile Functions for monitoring system metrics
* @{
*/
namespace uprofile
{
/**
* @ingroup uprofile
* @brief Start profiling to periodically record monitored events to a file
* @param file: file path where events will be saved
*/
UPROFAPI void start(const char* file);
/**
* @ingroup uprofile
* @brief Stop all monitorings
*
* Note: stopping profiler without prior call to start() has no effect
*/
UPROFAPI void stop();
/**
* @ingroup uprofile
* @brief Inject a GPUMonitor object that will be responsible for monitoring GPU metrics (usage and memory)
* @param monitor: custom GPUMonitor object
*
* Note: uprofile takes ownership of the passed object.
*/
UPROFAPI void addGPUMonitor(IGPUMonitor* monitor);
/**
* @ingroup uprofile
* @brief Destroy injected GPUMonitor object
*/
UPROFAPI void removeGPUMonitor();
/**
* @ingroup uprofile
* @brief Change the timestamp unit to record profiling metrics
*
* It should be called before calling start() method.
*
* Note: default value is EPOCH_TIME
*/
UPROFAPI void setTimestampUnit(TimestampUnit tsUnit);
/**
* @ingroup uprofile
* @brief Start monitoring the execution time of the given event
* @param title: event key
*/
UPROFAPI void timeBegin(const std::string& title);
/**
* @ingroup uprofile
* @brief Stop monitoring the execution time of the given event
*
* The library computes the duration for the given event and saves it into the report file.
*
* If no timeBegin() has been called with the given title, the call is ignored.
*/
UPROFAPI void timeEnd(const std::string& title);
/**
* @ingroup uprofile
* @brief Start monitoring of the memory used by the process
* @param period: period between two memory dump (in ms)
*/
UPROFAPI void startProcessMemoryMonitoring(int period);
/**
* @ingroup uprofile
* @brief Start monitoring of the global memory used on the system
* @param period: period between two memory dump (in ms)
*/
UPROFAPI void startSystemMemoryMonitoring(int period);
/**
* @ingroup uprofile
* @brief Start monitoring of the usage percentage of each CPU
* @param period: period between two cpu usage dump (in ms)
*/
UPROFAPI void startCPUUsageMonitoring(int period);
/**
* @ingroup uprofile
* @brief Start monitoring of the usage of the GPU
* @param period: period between two gpu usage dump (in ms)
*/
UPROFAPI void startGPUUsageMonitoring(int period);
/**
* @ingroup uprofile
* @brief Start monitoring of the usage of the GPU memory
* @param period: period between two gpu usage dump (in ms)
*/
UPROFAPI void startGPUMemoryMonitoring(int period);
/**
* @ingroup uprofile
* @brief memory used by the current process
*/
UPROFAPI void getProcessMemory(int& rss, int& shared);
/**
* @ingroup uprofile
* @brief dump global system memory
*/
UPROFAPI void getSystemMemory(int& totalMem, int& availableMem, int& freeMem);
/**
* @ingroup uprofile
* @brief get usage of all cpu cores
* @return vector holding the usage percentage of each CPU core
*/
UPROFAPI std::vector<float> getInstantCpuUsage();
}
/** @} */ // end of uprofile group
#endif /* UPROFILE_H_ */

353
3party/cppuprofile/lib/uprofileimpl.cpp Normal file
View File

@ -0,0 +1,353 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2022 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#include <chrono>
#include <fstream>
#include <iostream>
#include <sstream>
#if defined(__linux__)
#include <sys/sysinfo.h>
#include <unistd.h>
#elif defined(_WIN32)
#include <windows.h>
#elif defined(__APPLE__)
#include <sys/resource.h>
#include <sys/time.h>
#endif
#include "uprofileimpl.h"
using namespace std::chrono;
namespace uprofile {
UProfileImpl *UProfileImpl::m_uprofiler = NULL;
UProfileImpl::UProfileImpl() : m_tsUnit(TimestampUnit::EPOCH_TIME), m_gpuMonitor(NULL) {}
UProfileImpl *
UProfileImpl::getInstance()
{
if (!m_uprofiler) { m_uprofiler = new UProfileImpl; }
return m_uprofiler;
}
void
UProfileImpl::destroyInstance()
{
delete m_uprofiler;
m_uprofiler = NULL;
}
UProfileImpl::~UProfileImpl() { removeGPUMonitor(); }
void
UProfileImpl::start(const char *file)
{
m_file.open(file, std::ios::out);
if (!m_file.is_open()) { std::cerr << "Failed to open file: " << file << std::endl; }
}
void
UProfileImpl::addGPUMonitor(IGPUMonitor *monitor)
{
if (!monitor) {
std::cerr << "Invalid GPU Monitor" << std::endl;
return;
}
if (!m_gpuMonitor) { removeGPUMonitor(); }
m_gpuMonitor = monitor;
}
void
UProfileImpl::removeGPUMonitor()
{
delete m_gpuMonitor;
m_gpuMonitor = NULL;
}
void
UProfileImpl::timeBegin(const std::string &title)
{
std::lock_guard<std::mutex> guard(m_stepsMutex);
m_steps.insert(make_pair(title, getTimestamp()));
}
void
UProfileImpl::timeEnd(const std::string &title)
{
unsigned long long beginTimestamp = 0;
// Find step in the map
m_stepsMutex.lock();
auto it = m_steps.find(title);
bool found = it != m_steps.end();
if (found) {
beginTimestamp = (*it).second;
m_steps.erase(it);
}
m_stepsMutex.unlock();
if (found) {
write(ProfilingType::TIME_EXEC, {std::to_string(beginTimestamp), title});
} else {
write(ProfilingType::TIME_EVENT, {title});
}
}
void
UProfileImpl::startProcessMemoryMonitoring(int period)
{
m_processMemoryMonitorTimer.setInterval(period);
m_processMemoryMonitorTimer.setTimeout([=]() { dumpProcessMemory(); });
m_processMemoryMonitorTimer.start();
}
void
UProfileImpl::startSystemMemoryMonitoring(int period)
{
m_systemMemoryMonitorTimer.setInterval(period);
m_systemMemoryMonitorTimer.setTimeout([=]() { dumpSystemMemory(); });
m_systemMemoryMonitorTimer.start();
}
void
UProfileImpl::startCPUUsageMonitoring(int period)
{
m_cpuMonitorTimer.setInterval(period);
m_cpuMonitorTimer.setTimeout([=]() { dumpCpuUsage(); });
m_cpuMonitorTimer.start();
}
void
UProfileImpl::startGPUUsageMonitoring(int period)
{
if (!m_gpuMonitor) {
std::cerr << "Cannot monitor GPU usage: no GPUMonitor set!" << std::endl;
return;
}
m_gpuMonitor->start(period);
m_gpuUsageMonitorTimer.setInterval(period);
m_gpuUsageMonitorTimer.setTimeout([=]() { dumpGpuUsage(); });
m_gpuUsageMonitorTimer.start();
}
void
UProfileImpl::startGPUMemoryMonitoring(int period)
{
if (!m_gpuMonitor) {
std::cerr << "Cannot monitor GPU memory: no GPUMonitor set!" << std::endl;
return;
}
m_gpuMonitor->start(period);
m_gpuMemoryMonitorTimer.setInterval(period);
m_gpuMemoryMonitorTimer.setTimeout([=]() { dumpGpuMemory(); });
m_gpuMemoryMonitorTimer.start();
}
void
UProfileImpl::dumpProcessMemory()
{
int rss = 0, shared = 0;
getProcessMemory(rss, shared);
write(ProfilingType::PROCESS_MEMORY, {std::to_string(rss), std::to_string(shared)});
}
void
UProfileImpl::dumpSystemMemory()
{
int total = 0, available = 0, free = 0;
getSystemMemory(total, available, free);
write(ProfilingType::SYSTEM_MEMORY, {std::to_string(total), std::to_string(available), std::to_string(free)});
}
void
UProfileImpl::dumpCpuUsage()
{
vector<float> cpuLoads = m_cpuMonitor.getUsage();
for (size_t index = 0; index < cpuLoads.size(); ++index) {
write(ProfilingType::CPU, {std::to_string(index), std::to_string(cpuLoads.at(index))});
}
}
void
UProfileImpl::dumpGpuUsage()
{
if (!m_gpuMonitor || !m_gpuMonitor->watching()) { return; }
float usage = m_gpuMonitor->getUsage();
write(ProfilingType::GPU_USAGE, {std::to_string(usage)});
}
void
UProfileImpl::dumpGpuMemory()
{
if (!m_gpuMonitor || !m_gpuMonitor->watching()) { return; }
int usedMem, totalMem;
m_gpuMonitor->getMemory(usedMem, totalMem);
write(ProfilingType::GPU_MEMORY, {std::to_string(usedMem), std::to_string(totalMem)});
}
vector<float>
UProfileImpl::getInstantCpuUsage()
{
// To get instaneous CPU usage, we should wait at least one unit between two polling (aka: 100 ms)
m_cpuMonitor.getUsage();
this_thread::sleep_for(std::chrono::milliseconds(100));
return m_cpuMonitor.getUsage();
}
void
UProfileImpl::stop()
{
m_processMemoryMonitorTimer.stop();
m_systemMemoryMonitorTimer.stop();
m_cpuMonitorTimer.stop();
m_gpuUsageMonitorTimer.stop();
m_gpuMemoryMonitorTimer.stop();
if (m_gpuMonitor) { m_gpuMonitor->stop(); }
m_file.close();
}
void
UProfileImpl::setTimestampUnit(TimestampUnit tsUnit)
{
m_tsUnit = tsUnit;
}
void
UProfileImpl::write(ProfilingType type, const std::list<std::string> &data)
{
std::lock_guard<std::mutex> guard(m_fileMutex);
if (m_file.is_open()) {
const char csvSeparator = ';';
std::string strType;
switch (type) {
case ProfilingType::TIME_EXEC:
strType = "time_exec";
break;
case ProfilingType::TIME_EVENT:
strType = "time_event";
break;
case ProfilingType::PROCESS_MEMORY:
strType = "proc_mem";
break;
case ProfilingType::SYSTEM_MEMORY:
strType = "sys_mem";
break;
case ProfilingType::CPU:
strType = "cpu";
break;
case ProfilingType::GPU_USAGE:
strType = "gpu";
break;
case ProfilingType::GPU_MEMORY:
strType = "gpu_mem";
break;
default:
strType = "undefined";
break;
}
m_file << strType.c_str() << csvSeparator << getTimestamp();
for (auto it = data.cbegin(); it != data.cend(); ++it) { m_file << csvSeparator << *it; }
m_file << "\n";
m_file.flush();
}
}
unsigned long long
UProfileImpl::getTimestamp() const
{
return (m_tsUnit == TimestampUnit::EPOCH_TIME ? getEpochTime() : getTimeSinceBoot());
}
unsigned long long
UProfileImpl::getEpochTime()
{
return duration_cast<milliseconds>(system_clock::now().time_since_epoch()).count();
}
unsigned long long
UProfileImpl::getTimeSinceBoot()
{
#if defined(__linux__)
double uptime_seconds;
if (std::ifstream("/proc/uptime", std::ios::in) >> uptime_seconds) {
return static_cast<unsigned long long>(uptime_seconds * 1000.0);
}
return 0;
#elif defined(_WIN32)
return GetTickCount64();
#else
return 0;
#endif
}
void
UProfileImpl::getSystemMemory(int &totalMem, int &availableMem, int &freeMem)
{
#if defined(__linux__)
// /proc/meminfo returns the dump here:
// MemTotal: 515164 kB
// MemFree: 7348 kB
// MemAvailable: 7348 kB
ifstream meminfo("/proc/meminfo");
string line;
while (std::getline(meminfo, line)) {
if (line.find("MemTotal") != std::string::npos) {
stringstream ls(line);
ls.ignore(256, ' ');
ls >> totalMem;
} else if (line.find("MemFree") != std::string::npos) {
stringstream ls(line);
ls.ignore(256, ' ');
ls >> freeMem;
} else if (line.find("MemAvailable") != std::string::npos) {
{
stringstream ls(line);
ls.ignore(256, ' ');
ls >> availableMem;
break;
}
}
}
#endif
}
void
UProfileImpl::getProcessMemory(int &rss, int &shared)
{
#if defined(__linux__)
int tSize = 0, resident = 0, share = 0;
ifstream buffer("/proc/self/statm");
buffer >> tSize >> resident >> share;
buffer.close();
long page_size_kb = getpagesize() / 1024;
rss = resident * page_size_kb;
shared = share * page_size_kb;
#elif defined(__APPLE__)
struct rusage usage;
if (0 == getrusage(RUSAGE_SELF, &usage)) {
rss = usage.ru_maxrss;
shared = usage.ru_ixrss;
}
#endif
}
}// namespace uprofile

91
3party/cppuprofile/lib/uprofileimpl.h Normal file
View File

@ -0,0 +1,91 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2022 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#ifndef UPROFILEIMPL_H_
#define UPROFILEIMPL_H_
#include "igpumonitor.h"
#include "timestampunit.h"
#include "util/cpumonitor.h"
#include "util/timer.h"
#include <fstream>
#include <list>
#include <map>
#include <mutex>
#include <string>
namespace uprofile
{
class UProfileImpl
{
public:
enum class ProfilingType {
TIME_EXEC,
TIME_EVENT,
PROCESS_MEMORY,
SYSTEM_MEMORY,
CPU,
GPU_USAGE,
GPU_MEMORY
};
static UProfileImpl* getInstance();
static void destroyInstance();
virtual ~UProfileImpl();
// Implementation
void start(const char* file);
void stop();
void addGPUMonitor(IGPUMonitor* monitor);
void removeGPUMonitor();
void setTimestampUnit(TimestampUnit tsUnit);
void timeBegin(const std::string& title);
void timeEnd(const std::string& title);
void startProcessMemoryMonitoring(int period);
void startSystemMemoryMonitoring(int period);
void startCPUUsageMonitoring(int period);
void startGPUUsageMonitoring(int period);
void startGPUMemoryMonitoring(int period);
void getProcessMemory(int& rss, int& shared);
void getSystemMemory(int& totalMem, int& availableMem, int& freeMem);
vector<float> getInstantCpuUsage();
private:
static UProfileImpl* m_uprofiler;
UProfileImpl();
void write(ProfilingType type, const std::list<std::string>& data);
unsigned long long getTimestamp() const;
static unsigned long long getEpochTime();
static unsigned long long getTimeSinceBoot();
void dumpCpuUsage();
void dumpProcessMemory();
void dumpSystemMemory();
void dumpGpuUsage();
void dumpGpuMemory();
TimestampUnit m_tsUnit;
std::map<std::string, unsigned long long> m_steps; // Store steps (title, start time)
std::ofstream m_file;
Timer m_processMemoryMonitorTimer;
Timer m_systemMemoryMonitorTimer;
Timer m_cpuMonitorTimer;
Timer m_gpuUsageMonitorTimer;
Timer m_gpuMemoryMonitorTimer;
CpuMonitor m_cpuMonitor;
IGPUMonitor* m_gpuMonitor;
std::mutex m_fileMutex;
std::mutex m_stepsMutex;
};
}
#endif /* UPROFILEIMPL_H_ */

100
3party/cppuprofile/lib/util/cpumonitor.cpp Normal file
View File

@ -0,0 +1,100 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2022 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#include "cpumonitor.h"
#include <fstream>
#include <sstream>
using namespace std;
uprofile::CpuMonitor::CpuMonitor() :
m_nbCpus(getNumberOfCPUCores())
{
m_lastIdleTimes.resize(m_nbCpus, 0);
m_lastTotalTimes.resize(m_nbCpus, 0);
}
uprofile::CpuMonitor::~CpuMonitor()
{
}
size_t uprofile::CpuMonitor::getNumberOfCPUCores()
{
size_t nbCores = 0;
#if defined(__linux__)
ifstream meminfo("/proc/cpuinfo");
string str;
while (getline(meminfo, str)) {
if (str.rfind("processor", 0) == 0) {
nbCores++;
}
}
#endif
return nbCores;
}
void uprofile::CpuMonitor::extractCpuTimes(const string& cpuInfo, size_t& idleTime, size_t& totalTime)
{
// Remove 'cpu<index' word and
// extract the idle time and sum all other times
size_t spacePos = cpuInfo.find(' ');
if (spacePos == string::npos) {
return;
}
stringstream times(cpuInfo.substr(spacePos + 1));
int index = 0;
for (size_t time; times >> time; ++index) {
if (index == 3) { // idle time i s the 4th param
idleTime = time;
}
totalTime += time;
}
}
vector<float> uprofile::CpuMonitor::getUsage()
{
vector<float> usages;
#if defined(__linux__)
ifstream procStat("/proc/stat");
// /proc/stat dumps the following info:
// user nice system idle iowait irq softirq
// cpu 2255 34 2290 22625563 6290 127 456
// cpu0 1132 34 1441 11311718 3675 127 438
// cpu1 1123 0 849 11313845 2614 0 18
// ...
// Each numbers represents the amount of time the CPU has spent performing
// different kind of work
for (size_t cpuIndex = 0; cpuIndex < m_nbCpus; ++cpuIndex) {
string cpuName("cpu");
cpuName += to_string(cpuIndex);
// Look in /proc/stack the CPU info
string line;
while (getline(procStat, line)) {
if (line.find(cpuName) != std::string::npos) {
size_t idleTime = 0, totalTime = 0;
extractCpuTimes(line, idleTime, totalTime);
// To compute CPU load, we compute the time the CPU has been idle since the last read.
float cpuLoad = 100.0 * (1.0 - (float)(idleTime - m_lastIdleTimes[cpuIndex]) / (totalTime - m_lastTotalTimes[cpuIndex]));
usages.push_back(cpuLoad);
// Save the times value for the next read
m_lastIdleTimes[cpuIndex] = idleTime;
m_lastTotalTimes[cpuIndex] = totalTime;
break;
}
}
}
#endif
return usages;
}

41
3party/cppuprofile/lib/util/cpumonitor.h Normal file
View File

@ -0,0 +1,41 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2022 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#ifndef CPUMONITOR_H_
#define CPUMONITOR_H_
#include <string>
#include <vector>
using namespace std;
namespace uprofile
{
class CpuMonitor
{
public:
explicit CpuMonitor();
virtual ~CpuMonitor();
vector<float> getUsage();
private:
static size_t getNumberOfCPUCores();
static void extractCpuTimes(const string& cpuInfo, size_t& idleTime, size_t& totalTime);
// Store the last idle and total time for each CPU
vector<size_t> m_lastIdleTimes;
vector<size_t> m_lastTotalTimes;
size_t m_nbCpus;
};
}
#endif /* CPUMONITOR_H_ */

64
3party/cppuprofile/lib/util/timer.cpp Normal file
View File

@ -0,0 +1,64 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2022 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#include "timer.h"
Timer::Timer(int interval) :
m_th(NULL),
m_interval(interval)
{
}
Timer::~Timer()
{
stop();
}
void Timer::setInterval(int interval)
{
m_interval = interval;
}
void Timer::setTimeout(const std::function<void(void)>& timeout)
{
m_timeout = timeout;
}
bool Timer::isRunning()
{
std::lock_guard<std::mutex> lock(m_mutex);
return m_running;
}
void Timer::start()
{
if (m_interval > 0 && m_th == NULL) {
m_running = true;
m_th = new thread([=]() {
while (isRunning()) {
this_thread::sleep_for(chrono::milliseconds(m_interval));
if (m_timeout) {
m_timeout();
}
}
});
}
}
void Timer::stop()
{
m_mutex.lock();
m_running = false;
m_mutex.unlock();
if (m_th) {
m_th->join();
delete m_th;
m_th = NULL;
}
}

41
3party/cppuprofile/lib/util/timer.h Normal file
View File

@ -0,0 +1,41 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2022 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
#ifndef TIMER_H_
#define TIMER_H_
#include <chrono>
#include <functional>
#include <iostream>
#include <mutex>
#include <thread>
using namespace std;
class Timer
{
public:
explicit Timer(int interval /* ms */ = 0);
virtual ~Timer();
void setTimeout(const function<void(void)>& timeout);
void setInterval(int interval);
void start();
void stop();
bool isRunning();
private:
thread* m_th;
bool m_running;
int m_interval;
std::mutex m_mutex;
std::function<void(void)> m_timeout;
};
#endif // TIMER_H_

3
3party/cppuprofile/requirements.txt Normal file
View File

@ -0,0 +1,3 @@
numpy
plotly
pandas

37
3party/cppuprofile/sample/CMakeLists.txt Normal file
View File

@ -0,0 +1,37 @@
PROJECT(uprof-sample DESCRIPTION "Sample using uprofile library")
CMAKE_MINIMUM_REQUIRED(VERSION 2.8.12)
SET( CMAKE_USE_RELATIVE_PATHS ON)
IF(CMAKE_COMPILER_IS_GNUCXX)
ADD_DEFINITIONS( -std=c++0x )
ENDIF()
IF(BUILD_SHARED_LIBS)
IF(CMAKE_VERSION VERSION_GREATER_EQUAL 3.12.0)
ADD_COMPILE_DEFINITIONS(UPROFILE_DLL)
ELSE()
ADD_DEFINITIONS(-DUPROFILE_DLL)
ENDIF()
ENDIF()
IF(WIN32)
add_compile_options(/W4)
ELSE()
add_compile_options(-Wall -Werror)
ENDIF()
SET(Sample_SRCS
main.cpp
)
ADD_EXECUTABLE(${PROJECT_NAME}
${Sample_SRCS}
)
# Specify here the libraries this program depends on
TARGET_LINK_LIBRARIES(${PROJECT_NAME}
cppuprofile
)
INSTALL(TARGETS ${PROJECT_NAME} DESTINATION bin)

74
3party/cppuprofile/sample/main.cpp Normal file
View File

@ -0,0 +1,74 @@
// Software Name : cppuprofile
// SPDX-FileCopyrightText: Copyright (c) 2022 Orange
// SPDX-License-Identifier: BSD-3-Clause
//
// This software is distributed under the BSD License;
// see the LICENSE file for more details.
//
// Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al
#include <chrono>
#include <stdlib.h>
#include <thread>
#include <uprofile.h>
void printSystemMemory()
{
int total = 0, free = 0, available = 0;
uprofile::getSystemMemory(total, free, available);
printf("Memory: total = %i MB, free = %i MB, available = %i MB\n", total / 1000, free / 1000, available / 1000);
}
int main(int argc, char* argv[])
{
uprofile::start("./test.log");
// --- DUMP CPU USAGE ---
printf("CPU usage = (");
std::vector<float> loads = uprofile::getInstantCpuUsage();
for (auto it = loads.cbegin(); it != loads.cend(); ++it) {
printf("%0.2f%% ", *it);
}
printf(")\n");
// --- START MONITORING ---
uprofile::startCPUUsageMonitoring(200);
uprofile::startSystemMemoryMonitoring(200);
uprofile::startProcessMemoryMonitoring(200);
// --- USE MEMORY ---
printSystemMemory();
uprofile::timeBegin("UseMemory");
int lengthBuff = 100000000;
char* buf;
printf("Allocating %f MB\n", lengthBuff / sizeof(char) / 1000000.0f);
buf = (char*)malloc(lengthBuff + 1);
for (int i = 0; i < lengthBuff; i++) {
buf[i] = rand() % 26 + 'a';
}
buf[lengthBuff] = '\0';
uprofile::timeEnd("UseMemory");
printSystemMemory();
// --- WAIT 5 SECONDS ---
uprofile::timeBegin("Sleep1");
std::this_thread::sleep_for(std::chrono::seconds(5));
uprofile::timeEnd("Sleep1");
// --- RELEASE MEMORY ---
uprofile::timeBegin("FreeMemory");
free(buf);
uprofile::timeEnd("FreeMemory");
printf("Releasing memory\n");
printSystemMemory();
// --- WAIT 5 SECONDS ---
uprofile::timeBegin("Sleep2");
std::this_thread::sleep_for(std::chrono::seconds(5));
uprofile::timeEnd("Sleep2");
uprofile::stop();
return 0;
}

239
3party/cppuprofile/tools/show-graph Executable file
View File

@ -0,0 +1,239 @@
#!/usr/bin/env python3
# Software Name : uprofile
# SPDX-FileCopyrightText: Copyright (c) 2022 Orange
# SPDX-License-Identifier: BSD-3-Clause
#
# This software is distributed under the BSD License;
# see the LICENSE file for more details.
#
# Author: Cédric CHEDALEUX <cedric.chedaleux@orange.com> et al.
import argparse
import pandas as pd
import plotly.express as px
import plotly.figure_factory as ff
import plotly.graph_objects as go
from plotly.subplots import make_subplots
METRICS = {
'time_exec': 'Execution task',
'cpu': 'CPU load',
'sys_mem': 'System memory (in MB)',
'proc_mem': 'Process Memory (in MB)',
'gpu': 'GPU load',
'gpu_mem': 'GPU memory (in MB)'
}
def read(csv_file):
"""
Generate a DataFrame from the CSV file
Metrics event can have up to 3 extra parameters in addition to its type and its timestamp
:param csv_file:
:return: Dataframe
"""
return pd.read_csv(csv_file, sep=';', names=['metric', 'timestamp', 'extra_1', 'extra_2', 'extra_3'])
def filter_dataframe(df, metric):
"""
Filter the datafram by metric type
:param df:
:param metric:
:return: Dataframe
"""
return df[df['metric'] == metric]
def gen_time_exec_df(df):
"""
Format the dataframe to represent time exec data as gant tasks
:param df:
:return: dataframe with gant tasks
"""
if df.empty:
return None
# 'time_exec' format is 'time_exec:<end_timestamp>:<start_timestamp>:<task_name>')
time_exec_df = df[['extra_2', 'extra_1', 'timestamp']].copy()
time_exec_df.rename(columns={"extra_2": "Task", "extra_1": "Start", "timestamp": "Finish"}, inplace=True)
time_exec_df['Description'] = time_exec_df.apply(lambda row: "Task: {} (duration = {} ms)"
.format(row['Task'], int(row['Finish']) - int(row['Start'])),
axis=1)
time_exec_df['Start'] = pd.to_datetime(time_exec_df['Start'], unit='ms')
time_exec_df['Finish'] = pd.to_datetime(time_exec_df['Finish'], unit='ms')
return time_exec_df
def create_gantt_graph(df):
"""
Create a gant graph to represent the tasks
(see https://chart-studio.plotly.com/~empet/15242/gantt-chart-in-a-subplot-httpscommun/ and https://plotly.com/python/gantt/)
:param df:
:return: graph
"""
size = len(df['Start'])
colors = px.colors.sample_colorscale("turbo", [n / (size - 1) for n in range(size)])
return ff.create_gantt(df, colors=colors, show_colorbar=True, showgrid_x=True, showgrid_y=True,
show_hover_fill=True)
def create_cpu_graphs(df):
if df.empty:
return None
# 'cpu' metrics (format is 'cpu:<timestamp>:<cpu_number>:<percentage_usage>')
cpus = pd.unique(df['extra_1'])
for cpu in cpus:
cpu_df = df[df['extra_1'] == cpu]
yield go.Scatter(x=pd.to_datetime(cpu_df['timestamp'], unit='ms'),
y=pd.to_numeric(cpu_df['extra_2']),
name="CPU {}".format(cpu),
showlegend=True)
def create_sys_mem_graphs(df):
# 'sys_mem' metrics (format is 'mem:<timestamp>:<total>:<available>:<free>')
if df.empty:
return None
names = ["Total", "Available", "Free"]
for index in range(3):
yield go.Scatter(x=pd.to_datetime(df['timestamp'], unit='ms'),
y=(pd.to_numeric(df["extra_{}".format(index + 1)], downcast="integer") / 1024),
name=names[index],
showlegend=True)
def create_proc_mem_graphs(df):
# 'proc_mem' metrics (format is 'mem:<timestamp>:<rss>:<shared>')
if df.empty:
return None
names = ["RSS", "Shared"]
for index in range(2):
yield go.Scatter(x=pd.to_datetime(df['timestamp'], unit='ms'),
y=(pd.to_numeric(df["extra_{}".format(index + 1)], downcast="integer") / 1024),
name=names[index],
showlegend=True)
def create_gpu_mem_graphs(df):
# 'gpu_mem' metrics (format is 'gpu_mem:<timestamp>:<total>:<used>')
if df.empty:
return None
names = ["Used", "Total"]
for index in range(2):
yield go.Scatter(x=pd.to_datetime(df['timestamp'], unit='ms'),
y=(pd.to_numeric(df["extra_{}".format(index + 1)], downcast="integer") / 1024),
name=names[index],
showlegend=True)
def create_gpu_usage_graphs(df):
# 'gpu' metrics (format is 'gpu:<timestamp>:<percentage_usage>')
if df.empty:
return None
yield go.Scatter(x=pd.to_datetime(df['timestamp'], unit='ms'),
y=pd.to_numeric(df['extra_1']),
name="GPU usage",
showlegend=True)
def build_graphs(input_file, metrics):
# Use a multiple subplots (https://plotly.com/python/subplots/) to display
# - the execution task graph
# - the CPU usage
# - the memory usage (system and process)
# - the GPU usage and memory
graph_titles = []
for metric in metrics:
graph_titles.append(METRICS[metric])
figs = make_subplots(rows=len(metrics),
cols=1,
shared_xaxes=True,
vertical_spacing=0.025,
subplot_titles=graph_titles
)
with open(input_file) as file:
global_df = read(file)
row_index = 1
for metric in metrics:
if metric == 'time_exec':
# Display a grant graph for representing task execution durations
time_exec_df = gen_time_exec_df(filter_dataframe(global_df, metric))
if time_exec_df is not None:
for trace in create_gantt_graph(time_exec_df).data:
figs.add_trace(trace, row=row_index, col=1)
elif metric == 'cpu':
# Display all CPU usages in the same graph
for trace in create_cpu_graphs(filter_dataframe(global_df, metric)):
figs.add_trace(trace, row=row_index, col=1)
elif metric == 'sys_mem':
# Display memory usage
for trace in create_sys_mem_graphs(filter_dataframe(global_df, metric)):
figs.add_trace(trace, row=row_index, col=1)
figs.update_yaxes(row=row_index, col=1, rangemode="tozero")
elif metric == 'proc_mem':
# Display process memory usage
for trace in create_proc_mem_graphs(filter_dataframe(global_df, metric)):
figs.add_trace(trace, row=row_index, col=1)
figs.update_yaxes(row=row_index, col=1, rangemode="tozero")
elif metric == 'gpu':
# Display gpu usage
for trace in create_gpu_usage_graphs(filter_dataframe(global_df, metric)):
figs.add_trace(trace, row=row_index, col=1)
figs.update_yaxes(row=row_index, col=1, rangemode="tozero")
elif metric == 'gpu_mem':
# Display gpu memory
for trace in create_gpu_mem_graphs(filter_dataframe(global_df, 'gpu_mem')):
figs.add_trace(trace, row=row_index, col=1)
figs.update_yaxes(row=row_index, col=1, rangemode="tozero")
row_index += 1
figs.update_layout(
height=1200,
xaxis_tickformat='%M:%S',
showlegend=False
)
return figs
def main():
"""
Show the duration task graph and the cpu usage graph on a single view
The tools reads the metrics file generated by the uprofile library
"""
parser = argparse.ArgumentParser()
parser.add_argument('INPUT_FILE', type=str, help='Input file that contains profiling data')
parser.add_argument('--output', '-o', type=str,
help='Save the graph to the given HTML file')
parser.add_argument('--metric', type=str, dest='metrics', choices=METRICS.keys(), action='append', default=[],
help='Select the metric to display')
args = parser.parse_args()
if args.INPUT_FILE is None:
parser.error('no INPUT_FILE given')
if not args.metrics:
args.metrics = METRICS.keys()
graphs = build_graphs(args.INPUT_FILE, args.metrics)
graphs.show()
if args.output is not None:
print("Saving graph to '{}'".format(args.output))
graphs.write_html(args.output)
if __name__ == '__main__':
main()

5
CMakeLists.txt
View File

@ -15,6 +15,7 @@ set(BUILD_UNIT_TESTS OFF)
set(BUILD_EXAMPLES OFF) set(BUILD_EXAMPLES OFF)
add_library(sled STATIC "") add_library(sled STATIC "")
add_subdirectory(3party/cppuprofile EXCLUDE_FROM_ALL)
add_subdirectory(3party/protobuf-3.21.12 EXCLUDE_FROM_ALL) add_subdirectory(3party/protobuf-3.21.12 EXCLUDE_FROM_ALL)
if (NOT TARGET marl) if (NOT TARGET marl)
add_subdirectory(3party/marl EXCLUDE_FROM_ALL) add_subdirectory(3party/marl EXCLUDE_FROM_ALL)
@ -44,6 +45,7 @@ target_sources(
src/network/physical_socket_server.cc src/network/physical_socket_server.cc
src/network/socket_address.cc src/network/socket_address.cc
src/network/socket_server.cc src/network/socket_server.cc
src/profiling/profiling.cc
src/strings/base64.cc src/strings/base64.cc
src/strings/utils.cc src/strings/utils.cc
src/synchronization/event.cc src/synchronization/event.cc
@ -70,7 +72,7 @@ target_sources(
# set(BUILD_WITH_STATIC_RUNTIME_LIBS ON) set(BUILD_WITH_DOCUMENTATION OFF) # set(BUILD_WITH_STATIC_RUNTIME_LIBS ON) set(BUILD_WITH_DOCUMENTATION OFF)
# add_subdirectory(3party/rttr EXCLUDE_FROM_ALL) # add_subdirectory(3party/rttr EXCLUDE_FROM_ALL)
target_link_libraries(sled PUBLIC rpc_core fmt marl protobuf::libprotobuf) target_link_libraries(sled PUBLIC rpc_core fmt marl protobuf::libprotobuf cppuprofile)
if(SLED_BUILD_BENCHMARK) if(SLED_BUILD_BENCHMARK)
if (NOT TARGET benchmark) if (NOT TARGET benchmark)
@ -97,6 +99,7 @@ if(SLED_BUILD_TESTS)
add_executable(sled_tests add_executable(sled_tests
src/any_test.cc src/any_test.cc
src/filesystem/path_test.cc src/filesystem/path_test.cc
src/profiling/profiling_test.cc
src/strings/base64_test.cc src/strings/base64_test.cc
src/cleanup_test.cc src/cleanup_test.cc
src/status_or_test.cc src/status_or_test.cc

49
include/sled/profiling/profiling.h Normal file
View File

@ -0,0 +1,49 @@
#pragma once
#ifndef SLED_PROFILING_PROFILING_H
#define SLED_PROFILING_PROFILING_H
#include "sled/system/location.h"
#include <fmt/format.h>
#include <string>
namespace sled {
class Profiling {
public:
static Profiling *Instance();
inline bool Started() const { return started_; }
bool Start(const std::string &file);
void Stop();
void TimeBegin(const std::string &name);
void TimeEnd(const std::string &name);
void StartProcessMemoryMonitoring(int period_ms);
void StartSystemMemoryMonitoring(int period_ms);
void StartCPUUsageMonitoring(int period_ms);
static void GetSystemMemory(int &total_mem, int &available_mem, int &free_mem);
static void GetProcessMemory(int &rss, int &shared);
private:
bool started_ = false;
};
class FunctionSampler final {
public:
inline FunctionSampler(const Location &location = Location::Current())
{
std::string filename = location.file();
key_ =
fmt::format("{}:{}@{}", filename.substr(filename.find_last_of('/') + 1), location.line(), location.func());
Profiling::Instance()->TimeBegin(key_);
}
inline ~FunctionSampler() { Profiling::Instance()->TimeEnd(key_); }
private:
std::string key_;
};
}// namespace sled
#endif// SLED_PROFILING_PROFILING_H

72
src/profiling/profiling.cc Normal file
View File

@ -0,0 +1,72 @@
#include "sled/profiling/profiling.h"
#include <uprofile.h>
namespace sled {
Profiling *
Profiling::Instance()
{
static Profiling instance;
return &instance;
}
bool
Profiling::Start(const std::string &file)
{
uprofile::start(file.c_str());
started_ = true;
return true;
}
void
Profiling::Stop()
{
uprofile::stop();
}
void
Profiling::TimeBegin(const std::string &name)
{
if (!started_) return;
uprofile::timeBegin(name);
}
void
Profiling::TimeEnd(const std::string &name)
{
if (!started_) return;
uprofile::timeEnd(name);
}
void
Profiling::StartProcessMemoryMonitoring(int period_ms)
{
if (!started_) return;
uprofile::startProcessMemoryMonitoring(period_ms);
}
void
Profiling::StartSystemMemoryMonitoring(int period_ms)
{
if (!started_) return;
uprofile::startSystemMemoryMonitoring(period_ms);
}
void
Profiling::StartCPUUsageMonitoring(int period_ms)
{
if (!started_) return;
uprofile::startCPUUsageMonitoring(period_ms);
}
void
Profiling::GetSystemMemory(int &total_mem, int &available_mem, int &free_mem)
{
uprofile::getSystemMemory(total_mem, available_mem, free_mem);
}
void
Profiling::GetProcessMemory(int &rss, int &shared)
{
uprofile::getProcessMemory(rss, shared);
}
}// namespace sled

20
src/profiling/profiling_test.cc Normal file
View File

@ -0,0 +1,20 @@
#include <gtest/gtest.h>
#include <sled/log/log.h>
#include <sled/profiling/profiling.h>
TEST(Profiling, GetProcessMemory)
{
int rss, shared;
sled::Profiling::GetProcessMemory(rss, shared);
EXPECT_GT(rss, 0);
EXPECT_GE(shared, 0);
}
TEST(Profiling, GetSystemMemory)
{
int total_mem, available_mem, free_mem;
sled::Profiling::GetSystemMemory(total_mem, available_mem, free_mem);
EXPECT_GE(total_mem, 0);
EXPECT_GE(available_mem, 0);
EXPECT_GE(free_mem, 0);
}