packages/iSulad
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

bugfix for runc and cri

Browse Source 
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
(cherry picked from commit 9c3acba9915c23718ae8a806daa49022a73756eb)
This commit is contained in:
zhangxiaoyu 2023-04-24 10:31:51 +08:00 committed by openeuler-sync-bot
parent df0dae6811
commit 5ec852595b
21 changed files with 3592 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

72
0027-modifying-cpurt-file-permissions.patch Normal file
View File

@ -0,0 +1,72 @@
From eb46344fd8b7d42e6268353bfc801a1a9c8cb9a3 Mon Sep 17 00:00:00 2001
From: songbuhuang <544824346@qq.com>
Date: Thu, 9 Mar 2023 19:38:09 +0800
Subject: [PATCH 27/46] modifying cpurt file permissions
Signed-off-by: songbuhuang <544824346@qq.com>
---
src/common/constants.h | 4 ++++
src/daemon/common/sysinfo.c | 3 +--
src/daemon/executor/container_cb/execution.c | 3 ++-
3 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/src/common/constants.h b/src/common/constants.h
index 93a069de..1a4cb7c4 100644
--- a/src/common/constants.h
+++ b/src/common/constants.h
@@ -22,6 +22,10 @@ extern "C" {
/* mode of file and directory */
+#define DEFAULT_CGROUP_FILE_MODE 0644
+
+#define DEFAULT_CGROUP_DIR_MODE 0755
+
#define DEFAULT_SECURE_FILE_MODE 0640
#define DEFAULT_SECURE_DIRECTORY_MODE 0750
diff --git a/src/daemon/common/sysinfo.c b/src/daemon/common/sysinfo.c
index 8ad92b2a..cb02bee3 100644
--- a/src/daemon/common/sysinfo.c
+++ b/src/daemon/common/sysinfo.c
@@ -24,6 +24,7 @@
#include <linux/magic.h>
#include <sys/stat.h>
+#include "constants.h"
#include "err_msg.h"
#include "isula_libutils/log.h"
#include "utils.h"
@@ -71,8 +72,6 @@
#define CGROUP_MOUNTPOINT "/sys/fs/cgroup"
#define CGROUP_ISULAD_PATH CGROUP_MOUNTPOINT"/isulad"
-#define DEFAULT_CGROUP_DIR_MODE 0755
-#define DEFAULT_CGROUP_FILE_MODE 0644
#define CGROUP2_CONTROLLERS_PATH CGROUP_MOUNTPOINT"/cgroup.controllers"
#define CGROUP2_SUBTREE_CONTROLLER_PATH CGROUP_MOUNTPOINT"/cgroup.subtree_control"
#define CGROUP2_CPUSET_CPUS_EFFECTIVE_PATH CGROUP_MOUNTPOINT"/cpuset.cpus.effective"
diff --git a/src/daemon/executor/container_cb/execution.c b/src/daemon/executor/container_cb/execution.c
index 92c34b09..130bdaa4 100644
--- a/src/daemon/executor/container_cb/execution.c
+++ b/src/daemon/executor/container_cb/execution.c
@@ -46,6 +46,7 @@
#include "isulad_config.h"
#include "specs_api.h"
#include "container_api.h"
+#include "constants.h"
#include "execution_extend.h"
#include "execution_information.h"
#include "execution_stream.h"
@@ -318,7 +319,7 @@ static int maybe_create_cpu_realtime_file(int64_t value, const char *file, const
return 0;
}
- ret = util_mkdir_p(path, CONFIG_DIRECTORY_MODE);
+ ret = util_mkdir_p(path, DEFAULT_CGROUP_DIR_MODE);
if (ret != 0) {
ERROR("Failed to mkdir: %s", path);
return -1;
--
2.25.1

492
0028-add-design-docs-for-cri-manager.patch Normal file
View File

File diff suppressed because one or more lines are too long

43
0029-improve-check-of-process-failure.patch Normal file
View File

@ -0,0 +1,43 @@
From 5a6112a35daa7229ffb03d0dbb1df0bd1dba3469 Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Thu, 16 Mar 2023 19:16:30 +0800
Subject: [PATCH 29/46] improve check of process failure
1. fix docs error;
2. check error to decrease Unnecessary process;
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
docs/cri_pod_manager_design.md | 2 +-
src/daemon/entry/cri/cri_container_manager_service_impl.cc | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs/cri_pod_manager_design.md b/docs/cri_pod_manager_design.md
index aa7ac165..91d74f5a 100644
--- a/docs/cri_pod_manager_design.md
+++ b/docs/cri_pod_manager_design.md
@@ -263,7 +263,7 @@ PodSandboxManagerServiceImpl::PodSandboxStatus(const std::string &podSandboxID,
}
```
-详细的代码走读，代码流程图在 [StopPod代码走读中](#stoppodsandbox) 中进行了详细的分析，可以参考对应的部分。
+详细的代码走读，代码流程图在 [StopPod代码走读](#stoppodsandbox) 中进行了详细的分析，可以参考对应的部分。
## 通用接口
diff --git a/src/daemon/entry/cri/cri_container_manager_service_impl.cc b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
index 93b939c9..0fc1884f 100644
--- a/src/daemon/entry/cri/cri_container_manager_service_impl.cc
+++ b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
@@ -730,7 +730,7 @@ void ContainerManagerServiceImpl::PackContainerStatsAttributes(
container->mutable_attributes()->set_id(id);
auto status = ContainerStatus(std::string(id), error);
- if (status == nullptr) {
+ if (error.NotEmpty()) {
return;
}
--
2.25.1

251
0030-support-isula-update-when-runtime-is-runc.patch Normal file
View File

@ -0,0 +1,251 @@
From 0752a4324e7a8f54e4ebe5efb403221388b483d3 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Mon, 6 Mar 2023 11:31:56 +0800
Subject: [PATCH 30/46] support isula update when runtime is runc
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
.../executor/container_cb/execution_extend.c | 1 +
src/daemon/modules/api/runtime_api.h | 1 +
.../modules/runtime/isula/isula_rt_ops.c | 167 +++++++++++++++++-
3 files changed, 163 insertions(+), 6 deletions(-)
diff --git a/src/daemon/executor/container_cb/execution_extend.c b/src/daemon/executor/container_cb/execution_extend.c
index b0da705e..58303f80 100644
--- a/src/daemon/executor/container_cb/execution_extend.c
+++ b/src/daemon/executor/container_cb/execution_extend.c
@@ -1129,6 +1129,7 @@ static int do_update_resources(const container_update_request *request, containe
if (container_is_running(cont->state)) {
params.rootpath = cont->root_path;
params.hostconfig = hostconfig;
+ params.state = cont->state_path;
if (runtime_update(id, cont->runtime, &params)) {
ERROR("Update container %s failed", id);
ret = -1;
diff --git a/src/daemon/modules/api/runtime_api.h b/src/daemon/modules/api/runtime_api.h
index c5e05ca2..a8dfdeae 100644
--- a/src/daemon/modules/api/runtime_api.h
+++ b/src/daemon/modules/api/runtime_api.h
@@ -164,6 +164,7 @@ typedef struct _rt_attach_params_t {
typedef struct _rt_update_params_t {
const char *rootpath;
const host_config *hostconfig;
+ const char *state;
} rt_update_params_t;
typedef struct _rt_listpids_params_t {
diff --git a/src/daemon/modules/runtime/isula/isula_rt_ops.c b/src/daemon/modules/runtime/isula/isula_rt_ops.c
index 51a72c4f..4553fa90 100644
--- a/src/daemon/modules/runtime/isula/isula_rt_ops.c
+++ b/src/daemon/modules/runtime/isula/isula_rt_ops.c
@@ -39,6 +39,7 @@
#include "constants.h"
#include "isula_libutils/shim_client_process_state.h"
#include "isula_libutils/shim_client_runtime_stats.h"
+#include "isula_libutils/shim_client_cgroup_resources.h"
#include "isula_libutils/oci_runtime_state.h"
#include "isulad_config.h"
#include "utils_string.h"
@@ -54,6 +55,9 @@
#define RESIZE_DATA_SIZE 100
#define PID_WAIT_TIME 120
+// file name formats of cgroup resources json
+#define RESOURCE_FNAME_FORMATS "%s/resources.json"
+
// handle string from stderr output.
typedef int(*handle_output_callback_t)(const char *output);
@@ -725,18 +729,18 @@ static int shim_create(bool fg, const char *id, const char *workdir, const char
runtime_exec_param_dump(params);
if (snprintf(fpid, sizeof(fpid), "%s/shim-pid", workdir) < 0) {
- ERROR("failed make shim-pid full path");
+ ERROR("Failed make shim-pid full path");
return -1;
}
if (pipe2(exec_fd, O_CLOEXEC) != 0) {
- ERROR("failed to create pipe for shim create");
+ ERROR("Failed to create pipe for shim create");
return -1;
}
pid = fork();
if (pid < 0) {
- ERROR("failed fork for shim parent %s", strerror(errno));
+ ERROR("Failed fork for shim parent %s", strerror(errno));
close(exec_fd[0]);
close(exec_fd[1]);
return -1;
@@ -1256,13 +1260,164 @@ int rt_isula_attach(const char *id, const char *runtime, const rt_attach_params_
return -1;
}
-int rt_isula_update(const char *id, const char *runtime, const rt_update_params_t *params)
+static int to_engine_resources(const host_config *hostconfig, shim_client_cgroup_resources *cr)
+{
+ uint64_t period = 0;
+ int64_t quota = 0;
+
+ if (hostconfig == NULL || cr == NULL) {
+ return -1;
+ }
+
+ cr->block_io = util_common_calloc_s(sizeof(shim_client_cgroup_resources_block_io));
+ if (cr->block_io == NULL) {
+ ERROR("Out of memory");
+ return -1;
+ }
+
+ cr->cpu = util_common_calloc_s(sizeof(shim_client_cgroup_resources_cpu));
+ if (cr->cpu == NULL) {
+ ERROR("Out of memory");
+ return -1;
+ }
+
+ cr->memory = util_common_calloc_s(sizeof(shim_client_cgroup_resources_memory));
+ if (cr->memory == NULL) {
+ ERROR("Out of memory");
+ return -1;
+ }
+
+ cr->block_io->weight = hostconfig->blkio_weight;
+ cr->cpu->shares = (uint64_t)hostconfig->cpu_shares;
+ cr->cpu->period = (uint64_t)hostconfig->cpu_period;
+ cr->cpu->quota = hostconfig->cpu_quota;
+ cr->cpu->cpus = util_strdup_s(hostconfig->cpuset_cpus);
+ cr->cpu->mems = util_strdup_s(hostconfig->cpuset_mems);
+ cr->memory->limit = (uint64_t)hostconfig->memory;
+ cr->memory->swap = (uint64_t)hostconfig->memory_swap;
+ cr->memory->reservation = (uint64_t)hostconfig->memory_reservation;
+ cr->memory->kernel = (uint64_t)hostconfig->kernel_memory;
+ cr->cpu->realtime_period = hostconfig->cpu_realtime_period;
+ cr->cpu->realtime_runtime = hostconfig->cpu_realtime_runtime;
+
+ // when --cpus=n is set, nano_cpus = n * 1e9.
+ if (hostconfig->nano_cpus > 0) {
+ // in the case, period will be set to the default value of 100000(0.1s).
+ period = (uint64_t)(100 * Time_Milli / Time_Micro);
+ // set quota = period * n, in order to let container process fully occupy n cpus.
+ if ((hostconfig->nano_cpus / 1e9) > (INT64_MAX / (int64_t)period)) {
+ ERROR("Overflow of quota");
+ return -1;
+ }
+ quota = hostconfig->nano_cpus / 1e9 * (int64_t)period;
+ cr->cpu->period = period;
+ cr->cpu->quota = quota;
+ }
+
+ return 0;
+}
+
+static int create_resources_json_file(const char *workdir, const shim_client_cgroup_resources *cr, char *fname,
+ size_t fname_size)
{
- ERROR("isula update not support on isulad-shim");
- isulad_set_error_message("isula update not support on isulad-shim");
+ struct parser_context ctx = { OPT_GEN_SIMPLIFY, 0 };
+ parser_error perr = NULL;
+ char *data = NULL;
+ int retcode = 0;
+ int nret = 0;
+
+ nret = snprintf(fname, fname_size, RESOURCE_FNAME_FORMATS, workdir);
+ if (nret < 0 || (size_t)nret >= fname_size) {
+ ERROR("Failed make resources.json full path");
+ return -1;
+ }
+
+ data = shim_client_cgroup_resources_generate_json(cr, &ctx, &perr);
+ if (data == NULL) {
+ retcode = -1;
+ ERROR("Failed generate json for resources.json error=%s", perr);
+ goto out;
+ }
+
+ if (util_write_file(fname, data, strlen(data), DEFAULT_SECURE_FILE_MODE) != 0) {
+ retcode = -1;
+ ERROR("Failed write resources.json");
+ goto out;
+ }
+
+out:
+ UTIL_FREE_AND_SET_NULL(perr);
+ UTIL_FREE_AND_SET_NULL(data);
+
+ return retcode;
+}
+
+// show std error msg, always return -1.
+static int show_stderr(const char *err)
+{
+ isulad_set_error_message(err);
return -1;
}
+int rt_isula_update(const char *id, const char *runtime, const rt_update_params_t *params)
+{
+ int ret = 0;
+ char workdir[PATH_MAX] = { 0 };
+ char resources_fname[PATH_MAX] = { 0 };
+ const char *opts[2] = { 0 };
+ shim_client_cgroup_resources *cr = NULL;
+
+ if (id == NULL || runtime == NULL || params == NULL || params->state == NULL || strlen(params->state) == 0) {
+ ERROR("Nullptr arguments not allowed");
+ return -1;
+ }
+
+ ret = snprintf(workdir, sizeof(workdir), "%s/%s/update", params->state, id);
+ if (ret < 0 || (size_t)ret >= sizeof(workdir)) {
+ ERROR("Failed join update full path");
+ return -1;
+ }
+
+ ret = util_mkdir_p(workdir, DEFAULT_SECURE_DIRECTORY_MODE);
+ if (ret < 0) {
+ ERROR("Failed mkdir update workdir %s", workdir);
+ return ret;
+ }
+
+ cr = util_common_calloc_s(sizeof(shim_client_cgroup_resources));
+ if (cr == NULL) {
+ ERROR("Out of memory");
+ goto del_out;
+ }
+
+ ret = to_engine_resources(params->hostconfig, cr);
+ if (ret < 0) {
+ ERROR("Failed to get resources for update");
+ goto del_out;
+ }
+
+ ret = create_resources_json_file(workdir, cr, resources_fname, sizeof(resources_fname));
+ if (ret != 0) {
+ ERROR("%s: failed create update json file", id);
+ goto del_out;
+ }
+
+ opts[0] = "--resources";
+ opts[1] = resources_fname;
+
+ if (runtime_call_simple(workdir, runtime, "update", opts, 2, id, show_stderr) != 0) {
+ ERROR("Call runtime update id failed");
+ ret = -1;
+ }
+
+del_out:
+ if (util_recursive_rmdir(workdir, 0)) {
+ ERROR("Rmdir %s failed", workdir);
+ }
+ free_shim_client_cgroup_resources(cr);
+ return ret;
+}
+
int rt_isula_pause(const char *id, const char *runtime, const rt_pause_params_t *params)
{
char workdir[PATH_MAX] = { 0 };
--
2.25.1

80
0031-when-calling-runc-start-unset-NOTIFY_-SOCKET.patch Normal file
View File

@ -0,0 +1,80 @@
From dd36a6031c3f25171d34c9f8cc483af01a4dace1 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Mon, 20 Mar 2023 14:49:07 +0800
Subject: [PATCH 31/46] when calling runc start, unset NOTIFY_ SOCKET
Signed-off-by: sailorvii <chenw66@chinaunicom.cn>
---
src/daemon/modules/runtime/isula/isula_rt_ops.c | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/src/daemon/modules/runtime/isula/isula_rt_ops.c b/src/daemon/modules/runtime/isula/isula_rt_ops.c
index 4553fa90..e974964a 100644
--- a/src/daemon/modules/runtime/isula/isula_rt_ops.c
+++ b/src/daemon/modules/runtime/isula/isula_rt_ops.c
@@ -469,6 +469,12 @@ static void runtime_exec_func(void *arg)
_exit(EXIT_FAILURE);
}
+ // clear NOTIFY_SOCKET from the env to adapt runc start
+ if (strcmp(rei->subcmd, "start") == 0 && unsetenv("NOTIFY_SOCKET") != 0) {
+ dprintf(STDERR_FILENO, "unset env NOTIFY_SOCKET failed %s", strerror(errno));
+ _exit(EXIT_FAILURE);
+ }
+
execvp(rei->cmd, rei->params);
dprintf(STDERR_FILENO, "exec %s %s %s failed", rei->cmd, rei->subcmd, rei->id);
_exit(EXIT_FAILURE);
@@ -941,7 +947,7 @@ int rt_isula_start(const char *id, const char *runtime, const rt_start_params_t
char shim_pid_file_name[PATH_MAX] = { 0 };
pid_t pid = 0;
pid_t shim_pid = -1;
- int ret = 0;
+ int ret = -1;
int splice_ret = 0;
proc_t *proc = NULL;
proc_t *p_proc = NULL;
@@ -963,28 +969,24 @@ int rt_isula_start(const char *id, const char *runtime, const rt_start_params_t
pid = get_container_process_pid(workdir);
if (pid < 0) {
- ret = -1;
ERROR("%s: failed wait init pid", id);
goto out;
}
file_read_int(shim_pid_file_name, &shim_pid);
if (shim_pid < 0) {
- ret = -1;
ERROR("%s: failed to read isulad shim pid", id);
goto out;
}
proc = util_get_process_proc_info(pid);
if (proc == NULL) {
- ret = -1;
ERROR("%s: failed to read pidinfo", id);
goto out;
}
p_proc = util_get_process_proc_info(shim_pid);
if (p_proc == NULL) {
- ret = -1;
ERROR("%s: failed to read isulad shim pidinfo", id);
goto out;
}
@@ -996,10 +998,10 @@ int rt_isula_start(const char *id, const char *runtime, const rt_start_params_t
if (runtime_call_simple(workdir, runtime, "start", NULL, 0, id, NULL) != 0) {
ERROR("call runtime start id failed");
- ret = -1;
goto out;
}
+ ret = 0;
out:
if (ret != 0) {
show_shim_runtime_errlog(workdir);
--
2.25.1

850
0032-add-CRI-container-design-doc.patch Normal file
View File

File diff suppressed because one or more lines are too long

74
0033-fix-util_getgrent_r-overflow.patch Normal file
View File

File diff suppressed because one or more lines are too long

44
0034-modify-the-return-value-of-the-util_waitpid_with_tim.patch Normal file
View File

@ -0,0 +1,44 @@
From be989b6beff994f1b06e885ef766f01a6a3c0969 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Mon, 20 Mar 2023 14:17:00 +0800
Subject: [PATCH 34/46] modify the return value of the
util_waitpid_with_timeout to status
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
src/utils/cutils/utils.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/src/utils/cutils/utils.c b/src/utils/cutils/utils.c
index 7f36d019..3cede76a 100644
--- a/src/utils/cutils/utils.c
+++ b/src/utils/cutils/utils.c
@@ -321,6 +321,7 @@ rep:
int util_waitpid_with_timeout(pid_t pid, const int64_t timeout, handle_timeout_callback_t cb)
{
int nret = 0;
+ int st;
time_t start_time = time(NULL);
time_t end_time;
double interval;
@@ -330,7 +331,7 @@ int util_waitpid_with_timeout(pid_t pid, const int64_t timeout, handle_timeout_c
}
for (;;) {
- nret = waitpid(pid, NULL, WNOHANG);
+ nret = waitpid(pid, &st, WNOHANG);
if (nret == pid) {
break;
}
@@ -349,7 +350,7 @@ int util_waitpid_with_timeout(pid_t pid, const int64_t timeout, handle_timeout_c
// sleep some time instead to avoid cpu full running and then retry.
usleep(100);
}
- return 0;
+ return st;
}
int util_wait_for_pid_status(pid_t pid)
--
2.25.1

39
0035-fix-inspect-data-memleak.patch Normal file
View File

@ -0,0 +1,39 @@
From d4f524ef82ffdf2aed4847fa25795894a7bb7eda Mon Sep 17 00:00:00 2001
From: zhangxiaoyu <zhangxiaoyu58@huawei.com>
Date: Tue, 28 Mar 2023 10:14:22 +0800
Subject: [PATCH 35/46] fix inspect data memleak
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
---
src/daemon/entry/cri/cni_network_plugin.cc | 2 +-
src/daemon/entry/cri/cri_container_manager_service_impl.cc | 1 +
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/daemon/entry/cri/cni_network_plugin.cc b/src/daemon/entry/cri/cni_network_plugin.cc
index 02e75ffe..8cad0126 100644
--- a/src/daemon/entry/cri/cni_network_plugin.cc
+++ b/src/daemon/entry/cri/cni_network_plugin.cc
@@ -478,7 +478,7 @@ auto CniNetworkPlugin::GetNetNS(const std::string &podSandboxID, Errors &err) ->
container_inspect *inspect_data = CRIHelpers::InspectContainer(podSandboxID, err, false);
if (inspect_data == nullptr) {
- goto cleanup;
+ return result;
}
if (inspect_data->state->pid == 0) {
err.Errorf("cannot find network namespace for the terminated container %s", podSandboxID.c_str());
diff --git a/src/daemon/entry/cri/cri_container_manager_service_impl.cc b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
index 0fc1884f..d406496e 100644
--- a/src/daemon/entry/cri/cri_container_manager_service_impl.cc
+++ b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
@@ -1100,6 +1100,7 @@ ContainerManagerServiceImpl::ContainerStatus(const std::string &containerID, Err
ContainerStatusPtr contStatus(new (std::nothrow) runtime::v1alpha2::ContainerStatus);
if (contStatus == nullptr) {
error.SetError("Out of memory");
+ free_container_inspect(inspect);
return nullptr;
}
--
2.25.1

193
0036-containers-in-same-sandbox-should-have-same-process-.patch Normal file
View File

@ -0,0 +1,193 @@
From 12b64931a90594e837e1f3dc9a0dde5292bb73b5 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Sat, 8 Apr 2023 15:42:30 +0800
Subject: [PATCH 36/46] containers in same sandbox should have same process
labels
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
.../cri/cri_container_manager_service_impl.cc | 71 ++++++++++++++++++-
.../cri/cri_container_manager_service_impl.h | 5 +-
src/daemon/entry/cri/cri_helpers.cc | 31 ++++++++
src/daemon/entry/cri/cri_helpers.h | 3 +
4 files changed, 107 insertions(+), 3 deletions(-)
diff --git a/src/daemon/entry/cri/cri_container_manager_service_impl.cc b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
index d406496e..a64d222c 100644
--- a/src/daemon/entry/cri/cri_container_manager_service_impl.cc
+++ b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
@@ -126,8 +126,66 @@ auto ContainerManagerServiceImpl::PackCreateContainerHostConfigSecurityContext(
return 0;
}
+auto ContainerManagerServiceImpl::DoUsePodLevelSELinuxConfig(const runtime::v1alpha2::ContainerConfig &containerConfig,
+ host_config *hostconfig, const std::string &realPodSandboxID, Errors &error) -> int
+{
+ int ret = -1;
+ size_t newSize = 0;
+ size_t oldSize = 0;
+ container_inspect *inspect = nullptr;
+ std::vector<std::string> selinuxLabelOpts;
+ char **tmp_security_opt = nullptr;
+ std::string tmp_str;
+
+ inspect = CRIHelpers::InspectContainer(realPodSandboxID, error, true);
+ if (error.NotEmpty()) {
+ return -1;
+ }
+
+ if (inspect->process_label == nullptr) {
+ ret = 0;
+ goto cleanup;
+ }
+
+ tmp_str = std::string(inspect->process_label);
+ selinuxLabelOpts = CRIHelpers::GetSELinuxLabelOpts(tmp_str, error);
+ if (error.NotEmpty()) {
+ ERROR("Failed to get SELinuxLabelOpts for container %s", containerConfig.metadata().name().c_str());
+ goto cleanup;
+ }
+ if (selinuxLabelOpts.empty()) {
+ error.Errorf("SElinuxLabelOpts for container %s is empty", containerConfig.metadata().name().c_str());
+ goto cleanup;
+ }
+ if (selinuxLabelOpts.size() > (SIZE_MAX / sizeof(char *)) - hostconfig->security_opt_len) {
+ ERROR("Out of memory");
+ error.Errorf("Out of memory");
+ goto cleanup;
+ }
+ newSize = (hostconfig->security_opt_len + selinuxLabelOpts.size()) * sizeof(char *);
+ oldSize = hostconfig->security_opt_len * sizeof(char *);
+ ret = util_mem_realloc((void **)(&tmp_security_opt), newSize, (void *)hostconfig->security_opt, oldSize);
+ if (ret != 0) {
+ ERROR("Out of memory");
+ error.Errorf("Out of memory");
+ goto cleanup;
+ }
+ hostconfig->security_opt = tmp_security_opt;
+ for (const auto &securityOpt : selinuxLabelOpts) {
+ hostconfig->security_opt[hostconfig->security_opt_len] = util_strdup_s(securityOpt.c_str());
+ hostconfig->security_opt_len++;
+ }
+
+cleanup:
+ free_container_inspect(inspect);
+ return ret;
+}
+
+
+
auto ContainerManagerServiceImpl::GenerateCreateContainerHostConfig(
- const runtime::v1alpha2::ContainerConfig &containerConfig, Errors &error) -> host_config *
+ const runtime::v1alpha2::ContainerConfig &containerConfig,
+ const std::string &realPodSandboxID, Errors &error) -> host_config *
{
host_config *hostconfig = (host_config *)util_common_calloc_s(sizeof(host_config));
if (hostconfig == nullptr) {
@@ -159,6 +217,15 @@ auto ContainerManagerServiceImpl::GenerateCreateContainerHostConfig(
goto cleanup;
}
+ // If selinux label is not specified in container config, use pod level SELinux config
+ if (!containerConfig.linux().has_security_context() ||
+ !containerConfig.linux().security_context().has_selinux_options()) {
+ if (DoUsePodLevelSELinuxConfig(containerConfig, hostconfig, realPodSandboxID, error) != 0) {
+ error.SetError("Failed to security context to host config");
+ goto cleanup;
+ }
+ }
+
return hostconfig;
cleanup:
@@ -367,7 +434,7 @@ ContainerManagerServiceImpl::GenerateCreateContainerRequest(const std::string &r
container_config *custom_config { nullptr };
- host_config *hostconfig = GenerateCreateContainerHostConfig(containerConfig, error);
+ host_config *hostconfig = GenerateCreateContainerHostConfig(containerConfig, realPodSandboxID, error);
if (error.NotEmpty()) {
goto cleanup;
}
diff --git a/src/daemon/entry/cri/cri_container_manager_service_impl.h b/src/daemon/entry/cri/cri_container_manager_service_impl.h
index 6ac1cbc4..dcb2cd44 100644
--- a/src/daemon/entry/cri/cri_container_manager_service_impl.h
+++ b/src/daemon/entry/cri/cri_container_manager_service_impl.h
@@ -83,7 +83,7 @@ private:
const std::string &podSandboxRuntime,
Errors &error) -> container_create_request *;
auto GenerateCreateContainerHostConfig(const runtime::v1alpha2::ContainerConfig &containerConfig,
- Errors &error) -> host_config *;
+ const std::string &realPodSandboxID, Errors &error) -> host_config *;
auto GenerateCreateContainerCustomConfig(const std::string &containerName, const std::string &realPodSandboxID,
const runtime::v1alpha2::ContainerConfig &containerConfig,
const runtime::v1alpha2::PodSandboxConfig &podSandboxConfig,
@@ -92,6 +92,9 @@ private:
host_config *hostconfig, Errors &error) -> int;
auto PackCreateContainerHostConfigSecurityContext(const runtime::v1alpha2::ContainerConfig &containerConfig,
host_config *hostconfig, Errors &error) -> int;
+ auto DoUsePodLevelSELinuxConfig(const runtime::v1alpha2::ContainerConfig &containerConfig,
+ host_config *hostconfig,
+ const std::string &realPodSandboxID, Errors &error) -> int;
void MakeContainerConfig(const runtime::v1alpha2::ContainerConfig &config, container_config *cConfig,
Errors &error);
void CreateContainerLogSymlink(const std::string &containerID, Errors &error);
diff --git a/src/daemon/entry/cri/cri_helpers.cc b/src/daemon/entry/cri/cri_helpers.cc
index 711196ba..2bc6bed7 100644
--- a/src/daemon/entry/cri/cri_helpers.cc
+++ b/src/daemon/entry/cri/cri_helpers.cc
@@ -619,6 +619,37 @@ auto GetSecurityOpts(const std::string &seccompProfile, const char &separator, E
return seccompSecurityOpts;
}
+auto GetSELinuxLabelOpts(const std::string &selinuxLabel, Errors &error)
+-> std::vector<std::string>
+{
+ // security Opt Separator Change Version : k8s v1.23.0 (Corresponds to docker 1.11.x)
+ // New version '=' , old version ':', iSulad cri is based on v18.09, so iSulad cri use new version separator
+ const char securityOptSep { '=' };
+ // LabeSep is consistent with the separator used when parsing labels
+ const char labeSep { ':' };
+ std::vector<iSuladOpt> selinuxOpts { };
+ char **labelArr = nullptr;
+ size_t labelArrLen = 0;
+ std::vector<std::string> opts = {"user", "role", "type", "level"};
+ std::vector<std::string> vect;
+
+ labelArr = util_string_split_n(selinuxLabel.c_str(), labeSep, 4);
+ if (labelArr == nullptr) {
+ error.Errorf("Invalid selinux label: %s", selinuxLabel.c_str());
+ return vect;
+ }
+
+ labelArrLen = util_array_len((const char **)labelArr);
+ for (size_t i {}; i < labelArrLen; i++) {
+ iSuladOpt tmp = { "label", opts[i] + std::string(1, labeSep) + std::string(labelArr[i]), "" };
+ selinuxOpts.push_back(tmp);
+ }
+
+ util_free_array(labelArr);
+
+ return fmtiSuladOpts(selinuxOpts, securityOptSep);
+}
+
auto CreateCheckpoint(CRI::PodSandboxCheckpoint &checkpoint, Errors &error) -> std::string
{
cri_checkpoint *criCheckpoint { nullptr };
diff --git a/src/daemon/entry/cri/cri_helpers.h b/src/daemon/entry/cri/cri_helpers.h
index d50759ad..4ef227d7 100644
--- a/src/daemon/entry/cri/cri_helpers.h
+++ b/src/daemon/entry/cri/cri_helpers.h
@@ -127,6 +127,9 @@ auto ToIsuladContainerStatus(const runtime::v1alpha2::ContainerStateValue &state
auto GetSecurityOpts(const std::string &seccompProfile, const char &separator, Errors &error)
-> std::vector<std::string>;
+auto GetSELinuxLabelOpts(const std::string &selinuxLabel, Errors &error)
+-> std::vector<std::string>;
+
auto CreateCheckpoint(CRI::PodSandboxCheckpoint &checkpoint, Errors &error) -> std::string;
void GetCheckpoint(const std::string &jsonCheckPoint, CRI::PodSandboxCheckpoint &checkpoint, Errors &error);
--
2.25.1

434
0037-clean-container-process-after-execSync-timeout-exit.patch Normal file
View File

@ -0,0 +1,434 @@
From 53935dc36121b10975e047438ae8135ffa702920 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Tue, 4 Apr 2023 16:30:54 +0800
Subject: [PATCH 37/46] clean container process after execSync timeout exit
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
src/cmd/isulad-shim/common.c | 30 ++++
src/cmd/isulad-shim/common.h | 11 ++
src/cmd/isulad-shim/main.c | 15 +-
src/cmd/isulad-shim/process.c | 158 +++++++++++++-----
src/cmd/isulad-shim/process.h | 2 +-
.../modules/runtime/isula/isula_rt_ops.c | 40 ++---
6 files changed, 188 insertions(+), 68 deletions(-)
diff --git a/src/cmd/isulad-shim/common.c b/src/cmd/isulad-shim/common.c
index 0c345187..f188da1e 100644
--- a/src/cmd/isulad-shim/common.c
+++ b/src/cmd/isulad-shim/common.c
@@ -26,6 +26,7 @@
#include <sys/stat.h>
#include <stdbool.h>
#include <stdarg.h>
+#include <limits.h>
int set_fd_no_inherited(int fd)
{
@@ -316,3 +317,32 @@ int open_no_inherit(const char *path, int flag, mode_t mode)
return fd;
}
+
+static bool is_invalid_error_str(const char *err_str, const char *numstr)
+{
+ return err_str == NULL || err_str == numstr || *err_str != '\0';
+}
+
+int shim_util_safe_uint64(const char *numstr, uint64_t *converted)
+{
+ char *err_str = NULL;
+ uint64_t ull;
+
+ if (numstr == NULL || converted == NULL) {
+ return -EINVAL;
+ }
+
+ errno = 0;
+ ull = strtoull(numstr, &err_str, 0);
+ if (errno > 0) {
+ return -errno;
+ }
+
+ if (is_invalid_error_str(err_str, numstr)) {
+ return -EINVAL;
+ }
+
+ *converted = (uint64_t)ull;
+ return 0;
+}
+
diff --git a/src/cmd/isulad-shim/common.h b/src/cmd/isulad-shim/common.h
index d06c5256..91808295 100644
--- a/src/cmd/isulad-shim/common.h
+++ b/src/cmd/isulad-shim/common.h
@@ -19,6 +19,7 @@
#include <stdbool.h>
#include <stddef.h>
#include <sys/types.h>
+#include <stdint.h>
#ifdef __cplusplus
extern "C" {
@@ -31,6 +32,14 @@ extern "C" {
#define SHIM_ERR (-1)
#define SHIM_ERR_WAIT (-2)
#define SHIM_ERR_NOT_REQUIRED (-3)
+#define SHIM_ERR_TIMEOUT (-4)
+
+// common exit code is defined in stdlib.h
+// EXIT_FAILURE 1 : Failing exit status.
+// EXIT_SUCCESS 0 : Successful exit status.
+// custom shim exit code
+// SHIM_EXIT_TIMEOUT 2: Container process timeout exit code
+#define SHIM_EXIT_TIMEOUT 2
#define INFO_MSG "info"
#define WARN_MSG "warn"
@@ -68,6 +77,8 @@ void close_fd(int *pfd);
int open_no_inherit(const char *path, int flag, mode_t mode);
+int shim_util_safe_uint64(const char *numstr, uint64_t *converted);
+
#ifdef __cplusplus
}
#endif
diff --git a/src/cmd/isulad-shim/main.c b/src/cmd/isulad-shim/main.c
index eedd8fda..68e99e53 100644
--- a/src/cmd/isulad-shim/main.c
+++ b/src/cmd/isulad-shim/main.c
@@ -62,7 +62,8 @@ static int set_subreaper()
return SHIM_OK;
}
-static int parse_args(int argc, char **argv, char **cid, char **bundle, char **rt_name, char **log_level)
+static int parse_args(int argc, char **argv, char **cid, char **bundle, char **rt_name, char **log_level,
+ uint64_t *timeout)
{
if (argc < 4) {
return SHIM_ERR;
@@ -82,6 +83,12 @@ static int parse_args(int argc, char **argv, char **cid, char **bundle, char **r
}
}
+ if (argc > 5) {
+ if (shim_util_safe_uint64(strdup(argv[5]), timeout) != 0) {
+ return SHIM_ERR;
+ }
+ }
+
return SHIM_OK;
}
@@ -99,6 +106,8 @@ int main(int argc, char **argv)
int efd = -1;
process_t *p = NULL;
pthread_t tid_accept;
+ // execSync timeout
+ uint64_t timeout = 0;
g_log_fd = open_no_inherit(SHIM_LOG_NAME, O_CREAT | O_WRONLY | O_APPEND | O_SYNC, 0640);
if (g_log_fd < 0) {
@@ -117,7 +126,7 @@ int main(int argc, char **argv)
exit(EXIT_FAILURE);
}
- ret = parse_args(argc, argv, &container_id, &bundle, &rt_name, &log_level);
+ ret = parse_args(argc, argv, &container_id, &bundle, &rt_name, &log_level, &timeout);
if (ret != SHIM_OK) {
write_message(g_log_fd, ERR_MSG, "parse args failed:%d", ret);
exit(EXIT_FAILURE);
@@ -167,5 +176,5 @@ int main(int argc, char **argv)
released_timeout_exit();
- return process_signal_handle_routine(p, tid_accept);
+ return process_signal_handle_routine(p, tid_accept, timeout);
}
diff --git a/src/cmd/isulad-shim/process.c b/src/cmd/isulad-shim/process.c
index 5222629c..02609911 100644
--- a/src/cmd/isulad-shim/process.c
+++ b/src/cmd/isulad-shim/process.c
@@ -1213,69 +1213,145 @@ static int try_wait_all_child(void)
return 1;
}
-int process_signal_handle_routine(process_t *p, const pthread_t tid_accept)
+static int waitpid_with_timeout(int ctr_pid, int *status, const int64_t timeout)
{
- int ret = SHIM_ERR;
- bool exit_shim = false;
int nret = 0;
- int i;
- struct timespec ts;
+ time_t start_time = time(NULL);
+ time_t end_time;
+ double interval;
+ int st;
for (;;) {
- int status;
- ret = reap_container(p->ctr_pid, &status);
+ nret = waitpid(-1, &st, WNOHANG);
+ if (nret == ctr_pid) {
+ break;
+ }
+ end_time = time(NULL);
+ interval = difftime(end_time, start_time);
+ if (nret == 0 && interval >= timeout) {
+ return SHIM_ERR_TIMEOUT;
+ }
+ // sleep some time instead to avoid cpu full running and then retry.
+ usleep(1000);
+ }
+
+ if (WIFSIGNALED(st)) {
+ *status = EXIT_SIGNAL_OFFSET + WTERMSIG(st);
+ } else {
+ *status = WEXITSTATUS(st);
+ }
+
+ if (*status == CONTAINER_ACTION_REBOOT) {
+ nret = setenv("CONTAINER_ACTION", "reboot", 1);
+ if (nret != SHIM_OK) {
+ write_message(g_log_fd, WARN_MSG, "set reboot action failed:%d", SHIM_SYS_ERR(errno));
+ }
+ } else if (*status == CONTAINER_ACTION_SHUTDOWN) {
+ nret = setenv("CONTAINER_ACTION", "shutdown", 1);
+ if (nret != SHIM_OK) {
+ write_message(g_log_fd, WARN_MSG, "set shutdown action failed:%d", SHIM_SYS_ERR(errno));
+ }
+ }
+ return SHIM_OK;
+}
+
+/*
+ * If timeout <= 0, blocking wait in reap_container.
+ * If timeout > 0, non-blocking wait pid with timeout.
+ */
+static int wait_container_process_with_timeout(process_t *p, const unsigned int timeout, int *status)
+{
+ int ret = SHIM_ERR;
+
+ if (timeout > 0) {
+ return waitpid_with_timeout(p->ctr_pid, status, timeout);
+ }
+
+ for (;;) {
+ ret = reap_container(p->ctr_pid, status);
if (ret == SHIM_OK) {
- exit_shim = true;
- if (status == CONTAINER_ACTION_REBOOT) {
+ if (*status == CONTAINER_ACTION_REBOOT) {
ret = setenv("CONTAINER_ACTION", "reboot", 1);
if (ret != SHIM_OK) {
write_message(g_log_fd, WARN_MSG, "set reboot action failed:%d", SHIM_SYS_ERR(errno));
}
- } else if (status == CONTAINER_ACTION_SHUTDOWN) {
+ } else if (*status == CONTAINER_ACTION_SHUTDOWN) {
ret = setenv("CONTAINER_ACTION", "shutdown", 1);
if (ret != SHIM_OK) {
write_message(g_log_fd, WARN_MSG, "set shutdown action failed:%d", SHIM_SYS_ERR(errno));
}
}
- } else if (ret == SHIM_ERR_WAIT) {
+ return SHIM_OK;
+ }
+
+ if (ret == SHIM_ERR_WAIT) {
/* avoid thread entering the infinite loop */
usleep(1000);
+ }
+
+ if (ret == SHIM_ERR) {
+ // if the child process is not expected, retry.
continue;
}
- if (exit_shim) {
- process_kill_all(p);
+ }
- // wait atmost 120 seconds
- DO_RETRY_CALL(120, 1000000, nret, try_wait_all_child);
- if (nret != 0) {
- write_message(g_log_fd, ERR_MSG, "Failed to wait all child after 120 seconds");
- }
+}
- process_delete(p);
- if (p->exit_fd > 0) {
- (void)write_nointr(p->exit_fd, &status, sizeof(int));
- }
- // wait for task_console_accept thread termination. In order to make sure that
- // the io_copy connection is established and io_thread is not used by multiple threads.
- if (p->state->terminal) {
- if (clock_gettime(CLOCK_REALTIME, &ts) == -1) {
- write_message(g_log_fd, ERR_MSG, "Failed to get realtime");
- nret = pthread_join(tid_accept, NULL);
- } else {
- // Set the maximum waiting time to 60s to prevent stuck.
- ts.tv_sec += 60;
- nret = pthread_timedjoin_np(tid_accept, NULL, &ts);
- }
+int process_signal_handle_routine(process_t *p, const pthread_t tid_accept, const unsigned int timeout)
+{
+ int i;
+ int nret = 0;
+ int ret = 0;
+ int status = 0;
+ struct timespec ts;
- if (nret != 0) {
- write_message(g_log_fd, ERR_MSG, "Failed to join task_console_accept thread");
- }
- }
+ ret = wait_container_process_with_timeout(p, timeout, &status);
+ if (ret == SHIM_ERR_TIMEOUT) {
+ // kill container process to ensure process_kill_all effective
+ nret = kill(p->ctr_pid, SIGKILL);
+ if (nret < 0 && errno != ESRCH) {
+ write_message(g_log_fd, ERR_MSG, "Can not kill process (pid=%d) with SIGKILL", p->ctr_pid);
+ exit(EXIT_FAILURE);
+ }
+ }
- for (i = 0; i < 3; i++) {
- destroy_io_thread(p, i);
- }
- return status;
+ process_kill_all(p);
+
+ // wait atmost 120 seconds
+ DO_RETRY_CALL(120, 1000000, nret, try_wait_all_child);
+ if (nret != 0) {
+ write_message(g_log_fd, ERR_MSG, "Failed to wait all child after 120 seconds");
+ }
+
+ process_delete(p);
+ if (p->exit_fd > 0) {
+ (void)write_nointr(p->exit_fd, &status, sizeof(int));
+ }
+ // wait for task_console_accept thread termination. In order to make sure that
+ // the io_copy connection is established and io_thread is not used by multiple threads.
+ if (p->state->terminal) {
+ if (clock_gettime(CLOCK_REALTIME, &ts) == -1) {
+ write_message(g_log_fd, ERR_MSG, "Failed to get realtime");
+ nret = pthread_join(tid_accept, NULL);
+ } else {
+ // Set the maximum waiting time to 60s to prevent stuck.
+ ts.tv_sec += 60;
+ nret = pthread_timedjoin_np(tid_accept, NULL, &ts);
}
+
+ if (nret != 0) {
+ write_message(g_log_fd, ERR_MSG, "Failed to join task_console_accept thread");
+ }
+ }
+
+ for (i = 0; i < 3; i++) {
+ destroy_io_thread(p, i);
}
+
+ if (ret == SHIM_ERR_TIMEOUT) {
+ write_message(g_log_fd, INFO_MSG, "Wait %d timeout", p->ctr_pid);
+ exit(SHIM_EXIT_TIMEOUT);
+ }
+ return status;
+
}
diff --git a/src/cmd/isulad-shim/process.h b/src/cmd/isulad-shim/process.h
index 66820f68..7e3259e8 100644
--- a/src/cmd/isulad-shim/process.h
+++ b/src/cmd/isulad-shim/process.h
@@ -97,7 +97,7 @@ process_t* new_process(char *id, char *bundle, char *runtime);
int open_io(process_t *p, pthread_t *tid_accept);
int process_io_init(process_t *p);
int create_process(process_t *p);
-int process_signal_handle_routine(process_t *p, const pthread_t tid_accept);
+int process_signal_handle_routine(process_t *p, const pthread_t tid_accept, const unsigned int timeout);
#ifdef __cplusplus
}
diff --git a/src/daemon/modules/runtime/isula/isula_rt_ops.c b/src/daemon/modules/runtime/isula/isula_rt_ops.c
index e974964a..5a01b8c6 100644
--- a/src/daemon/modules/runtime/isula/isula_rt_ops.c
+++ b/src/daemon/modules/runtime/isula/isula_rt_ops.c
@@ -54,6 +54,7 @@
#define SHIM_LOG_SIZE ((BUFSIZ - 100) / 2)
#define RESIZE_DATA_SIZE 100
#define PID_WAIT_TIME 120
+#define SHIM_EXIT_TIMEOUT 2
// file name formats of cgroup resources json
#define RESOURCE_FNAME_FORMATS "%s/resources.json"
@@ -692,27 +693,6 @@ static int status_to_exit_code(int status)
return exit_code;
}
-static int try_wait_pid(pid_t pid)
-{
- if (waitpid(pid, NULL, WNOHANG) == pid) {
- return 0;
- }
-
- return 1;
-}
-
-static void kill_and_show_err(pid_t pid)
-{
- int nret = 0;
- kill(pid, SIGKILL);
- // wait atmost 0.5 seconds
- DO_RETRY_CALL(5, 100000, nret, try_wait_pid, pid);
- if (nret != 0) {
- WARN("Fail to wait isulad-shim");
- }
- isulad_set_error_message("Exec container error;exec timeout");
-}
-
static int shim_create(bool fg, const char *id, const char *workdir, const char *bundle, const char *runtime_cmd,
int *exit_code, const int64_t timeout)
{
@@ -731,7 +711,14 @@ static int shim_create(bool fg, const char *id, const char *workdir, const char
params[i++] = bundle;
params[i++] = runtime_cmd;
params[i++] = "info";
- params[i++] = "2m0s";
+ // execSync timeout
+ if (timeout > 0) {
+ params[i] = util_int_to_string(timeout);
+ if (params[i] == NULL) {
+ ERROR("Failed to convert execSync timeout %ld to string", timeout);
+ return -1;
+ }
+ }
runtime_exec_param_dump(params);
if (snprintf(fpid, sizeof(fpid), "%s/shim-pid", workdir) < 0) {
@@ -805,7 +792,7 @@ realexec:
goto out;
}
- status = util_waitpid_with_timeout(pid, timeout, kill_and_show_err);
+ status = util_wait_for_pid_status(pid);
if (status < 0) {
ERROR("failed wait shim-parent %d exit %s", pid, strerror(errno));
ret = -1;
@@ -1204,6 +1191,13 @@ int rt_isula_exec(const char *id, const char *runtime, const rt_exec_params_t *p
goto errlog_out;
}
+ if (*exit_code == SHIM_EXIT_TIMEOUT) {
+ ret = -1;
+ isulad_set_error_message("Exec container error;exec timeout");
+ ERROR("isulad-shim %d exit for execing timeout", pid);
+ goto errlog_out;
+ }
+
pid = get_container_process_pid(workdir);
if (pid < 0) {
ERROR("%s: failed get exec process id", workdir);
--
2.25.1

346
0038-support-to-config-selinux-label-in-cri.patch Normal file
View File

@ -0,0 +1,346 @@
From d231a8e5f8ea1c8a5584fdc8baa579b18d1b6a19 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Wed, 29 Mar 2023 03:26:45 +0800
Subject: [PATCH 38/46] support to config selinux label in cri
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
.../cri/cri_container_manager_service_impl.cc | 47 +++++----
.../cri/cri_container_manager_service_impl.h | 1 +
src/daemon/entry/cri/cri_helpers.cc | 95 ++++++++++++++++++-
src/daemon/entry/cri/cri_helpers.h | 17 +++-
.../cri_pod_sandbox_manager_service_impl.cc | 47 ++++-----
5 files changed, 149 insertions(+), 58 deletions(-)
diff --git a/src/daemon/entry/cri/cri_container_manager_service_impl.cc b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
index a64d222c..6278512f 100644
--- a/src/daemon/entry/cri/cri_container_manager_service_impl.cc
+++ b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
@@ -98,30 +98,21 @@ auto ContainerManagerServiceImpl::PackCreateContainerHostConfigSecurityContext(
// security Opt Separator Change Version : k8s v1.23.0 (Corresponds to docker 1.11.x)
// New version '=' , old version ':', iSulad cri is based on v18.09, so iSulad cri use new version separator
const char securityOptSep { '=' };
- std::vector<std::string> securityOpts = CRIHelpers::GetSecurityOpts(
- containerConfig.linux().security_context().seccomp_profile_path(), securityOptSep, error);
+ const ::runtime::v1alpha2::LinuxContainerSecurityContext &context = containerConfig.linux().security_context();
+ CRIHelpers::commonSecurityContext commonContext = {
+ .hasSELinuxOption = context.has_selinux_options(),
+ .selinuxOption = context.selinux_options(),
+ .seccompProfile = context.seccomp_profile_path(),
+ };
+ std::vector<std::string> securityOpts = CRIHelpers::GetSecurityOpts(commonContext, securityOptSep, error);
if (error.NotEmpty()) {
- error.Errorf("failed to generate security options for container %s", containerConfig.metadata().name().c_str());
+ error.Errorf("Failed to generate security options for container %s", containerConfig.metadata().name().c_str());
return -1;
}
- if (!securityOpts.empty()) {
- char **tmp_security_opt = nullptr;
- if (securityOpts.size() > (SIZE_MAX / sizeof(char *)) - hostconfig->security_opt_len) {
- error.Errorf("Out of memory");
- return -1;
- }
- size_t newSize = (hostconfig->security_opt_len + securityOpts.size()) * sizeof(char *);
- size_t oldSize = hostconfig->security_opt_len * sizeof(char *);
- int ret = util_mem_realloc((void **)(&tmp_security_opt), newSize, (void *)hostconfig->security_opt, oldSize);
- if (ret != 0) {
- error.Errorf("Out of memory");
- return -1;
- }
- hostconfig->security_opt = tmp_security_opt;
- for (const auto &securityOpt : securityOpts) {
- hostconfig->security_opt[hostconfig->security_opt_len] = util_strdup_s(securityOpt.c_str());
- hostconfig->security_opt_len++;
- }
+ CRIHelpers::AddSecurityOptsToHostConfig(securityOpts, hostconfig, error);
+ if (error.NotEmpty()) {
+ error.Errorf("Failed to add securityOpts to hostconfig for container %s", containerConfig.metadata().name().c_str());
+ return -1;
}
return 0;
}
@@ -148,7 +139,7 @@ auto ContainerManagerServiceImpl::DoUsePodLevelSELinuxConfig(const runtime::v1al
}
tmp_str = std::string(inspect->process_label);
- selinuxLabelOpts = CRIHelpers::GetSELinuxLabelOpts(tmp_str, error);
+ selinuxLabelOpts = CRIHelpers::GetPodSELinuxLabelOpts(tmp_str, error);
if (error.NotEmpty()) {
ERROR("Failed to get SELinuxLabelOpts for container %s", containerConfig.metadata().name().c_str());
goto cleanup;
@@ -181,7 +172,14 @@ cleanup:
return ret;
}
-
+auto ContainerManagerServiceImpl::IsSELinuxLabelEmpty(const ::runtime::v1alpha2::SELinuxOption &selinuxOption) -> bool
+{
+ if (selinuxOption.user().length() == 0 && selinuxOption.role().length() == 0 && selinuxOption.type().length() == 0 &&
+ selinuxOption.level().length() == 0) {
+ return true;
+ }
+ return false;
+}
auto ContainerManagerServiceImpl::GenerateCreateContainerHostConfig(
const runtime::v1alpha2::ContainerConfig &containerConfig,
@@ -219,7 +217,8 @@ auto ContainerManagerServiceImpl::GenerateCreateContainerHostConfig(
// If selinux label is not specified in container config, use pod level SELinux config
if (!containerConfig.linux().has_security_context() ||
- !containerConfig.linux().security_context().has_selinux_options()) {
+ !containerConfig.linux().security_context().has_selinux_options() ||
+ IsSELinuxLabelEmpty(containerConfig.linux().security_context().selinux_options())) {
if (DoUsePodLevelSELinuxConfig(containerConfig, hostconfig, realPodSandboxID, error) != 0) {
error.SetError("Failed to security context to host config");
goto cleanup;
diff --git a/src/daemon/entry/cri/cri_container_manager_service_impl.h b/src/daemon/entry/cri/cri_container_manager_service_impl.h
index dcb2cd44..ad53a65f 100644
--- a/src/daemon/entry/cri/cri_container_manager_service_impl.h
+++ b/src/daemon/entry/cri/cri_container_manager_service_impl.h
@@ -84,6 +84,7 @@ private:
Errors &error) -> container_create_request *;
auto GenerateCreateContainerHostConfig(const runtime::v1alpha2::ContainerConfig &containerConfig,
const std::string &realPodSandboxID, Errors &error) -> host_config *;
+ auto IsSELinuxLabelEmpty(const ::runtime::v1alpha2::SELinuxOption &selinuxOption) -> bool;
auto GenerateCreateContainerCustomConfig(const std::string &containerName, const std::string &realPodSandboxID,
const runtime::v1alpha2::ContainerConfig &containerConfig,
const runtime::v1alpha2::PodSandboxConfig &podSandboxConfig,
diff --git a/src/daemon/entry/cri/cri_helpers.cc b/src/daemon/entry/cri/cri_helpers.cc
index 2bc6bed7..ec14d0b2 100644
--- a/src/daemon/entry/cri/cri_helpers.cc
+++ b/src/daemon/entry/cri/cri_helpers.cc
@@ -68,6 +68,8 @@ const std::string Constants::CNI_MUTL_NET_EXTENSION_KEY { "extension.network.kub
const std::string Constants::CNI_MUTL_NET_EXTENSION_ARGS_KEY { "CNI_MUTLINET_EXTENSION" };
const std::string Constants::CNI_ARGS_EXTENSION_PREFIX_KEY { "extension.network.kubernetes.io/cniargs/" };
const std::string Constants::IMAGE_NAME_ANNOTATION_KEY { "io.kubernetes.cri.image-name" };
+// Usually, the format of level is "s0:c60,c525" or "s0-s0:c40.c23"
+const std::string Constants::SELINUX_LABEL_LEVEL_PATTERN { "^s[0-9](-s[0-9])?(:c[0-9]{1,4}(\\.c[0-9]{1,4})?(,c[0-9]{1,4}(\\.c[0-9]{1,4})?)*)?$" };
const char *InternalLabelKeys[] = { CRIHelpers::Constants::CONTAINER_TYPE_LABEL_KEY.c_str(),
CRIHelpers::Constants::CONTAINER_LOGPATH_LABEL_KEY.c_str(),
@@ -598,6 +600,34 @@ auto GetSeccompiSuladOpts(const std::string &seccompProfile, Errors &error) -> s
return ret;
}
+
+auto GetSelinuxiSuladOpts(const ::runtime::v1alpha2::SELinuxOption &selinux, Errors &error)-> std::vector<iSuladOpt>
+{
+ std::vector<iSuladOpt> selinuxOpts { };
+ // LabeSep is consistent with the separator used when parsing labels
+ const char labeSep { ':' };
+
+ if (selinux.level().length() != 0 &&
+ util_reg_match(CRIHelpers::Constants::SELINUX_LABEL_LEVEL_PATTERN.c_str(), selinux.level().c_str()) != 0) {
+ error.Errorf("The format of 'level' %s is not correct", selinux.level().c_str());
+ return selinuxOpts;
+ }
+
+ if (selinux.user().length() > 0) {
+ selinuxOpts.push_back({ "label", std::string("user") + std::string(1, labeSep) + selinux.user(), "" });
+ }
+ if (selinux.role().length() > 0) {
+ selinuxOpts.push_back({ "label", std::string("role") + std::string(1, labeSep) + selinux.role(), "" });
+ }
+ if (selinux.type().length() > 0) {
+ selinuxOpts.push_back({ "label", std::string("type") + std::string(1, labeSep) + selinux.type(), "" });
+ }
+ if (selinux.level().length() > 0) {
+ selinuxOpts.push_back({ "label", std::string("level") + std::string(1, labeSep) + selinux.level(), "" });
+ }
+ return selinuxOpts;
+}
+
auto GetSeccompSecurityOpts(const std::string &seccompProfile, const char &separator, Errors &error)
-> std::vector<std::string>
{
@@ -609,17 +639,44 @@ auto GetSeccompSecurityOpts(const std::string &seccompProfile, const char &separ
return fmtiSuladOpts(seccompOpts, separator);
}
-auto GetSecurityOpts(const std::string &seccompProfile, const char &separator, Errors &error)
+auto GetSELinuxLabelOpts(const bool hasSELinuxOption, const ::runtime::v1alpha2::SELinuxOption &selinux,
+ const char &separator, Errors &error)
-> std::vector<std::string>
{
- std::vector<std::string> seccompSecurityOpts = GetSeccompSecurityOpts(seccompProfile, separator, error);
+ if (!hasSELinuxOption) {
+ return std::vector<std::string>();
+ }
+
+ std::vector<iSuladOpt> selinuxOpts = GetSelinuxiSuladOpts(selinux, error);
if (error.NotEmpty()) {
- error.Errorf("failed to generate seccomp security options for container: %s", error.GetMessage().c_str());
+ return std::vector<std::string>();
}
- return seccompSecurityOpts;
+
+ return fmtiSuladOpts(selinuxOpts, separator);
}
-auto GetSELinuxLabelOpts(const std::string &selinuxLabel, Errors &error)
+auto GetSecurityOpts(const commonSecurityContext &context, const char &separator, Errors &error)
+-> std::vector<std::string>
+{
+ std::vector<std::string> securityOpts;
+ std::vector<std::string> seccompSecurityOpts = GetSeccompSecurityOpts(context.seccompProfile, separator, error);
+ if (error.NotEmpty()) {
+ error.Errorf("Failed to generate seccomp security options for container: %s", error.GetMessage().c_str());
+ return securityOpts;
+ }
+
+ std::vector<std::string> selinuxOpts = CRIHelpers::GetSELinuxLabelOpts(context.hasSELinuxOption,
+ context.selinuxOption, separator, error);
+ if (error.NotEmpty()) {
+ error.Errorf("Failed to generate SELinuxLabel options for container %s", error.GetMessage().c_str());
+ return securityOpts;
+ }
+ securityOpts.insert(securityOpts.end(), seccompSecurityOpts.begin(), seccompSecurityOpts.end());
+ securityOpts.insert(securityOpts.end(), selinuxOpts.begin(), selinuxOpts.end());
+ return securityOpts;
+}
+
+auto GetPodSELinuxLabelOpts(const std::string &selinuxLabel, Errors &error)
-> std::vector<std::string>
{
// security Opt Separator Change Version : k8s v1.23.0 (Corresponds to docker 1.11.x)
@@ -650,6 +707,34 @@ auto GetSELinuxLabelOpts(const std::string &selinuxLabel, Errors &error)
return fmtiSuladOpts(selinuxOpts, securityOptSep);
}
+void AddSecurityOptsToHostConfig(std::vector<std::string> &securityOpts, host_config *hostconfig, Errors &error)
+{
+ if (securityOpts.empty()) {
+ return;
+ }
+
+ char **tmp_security_opt = nullptr;
+ if (securityOpts.size() > (SIZE_MAX / sizeof(char *)) - hostconfig->security_opt_len) {
+ error.Errorf("Too many securityOpts");
+ ERROR("Too many securityOpts");
+ return;
+ }
+ size_t newSize = (hostconfig->security_opt_len + securityOpts.size()) * sizeof(char *);
+ size_t oldSize = hostconfig->security_opt_len * sizeof(char *);
+ int ret = util_mem_realloc((void **)(&tmp_security_opt), newSize, (void *)hostconfig->security_opt, oldSize);
+ if (ret != 0) {
+ error.Errorf("Out of memory");
+ ERROR("Out of memory");
+ return;
+ }
+ hostconfig->security_opt = tmp_security_opt;
+ for (const auto &securityOpt : securityOpts) {
+ hostconfig->security_opt[hostconfig->security_opt_len] = util_strdup_s(securityOpt.c_str());
+ hostconfig->security_opt_len++;
+ }
+
+}
+
auto CreateCheckpoint(CRI::PodSandboxCheckpoint &checkpoint, Errors &error) -> std::string
{
cri_checkpoint *criCheckpoint { nullptr };
diff --git a/src/daemon/entry/cri/cri_helpers.h b/src/daemon/entry/cri/cri_helpers.h
index 4ef227d7..a5d6cf61 100644
--- a/src/daemon/entry/cri/cri_helpers.h
+++ b/src/daemon/entry/cri/cri_helpers.h
@@ -67,10 +67,17 @@ public:
static const std::string CNI_MUTL_NET_EXTENSION_KEY;
static const std::string CNI_MUTL_NET_EXTENSION_ARGS_KEY;
static const std::string CNI_ARGS_EXTENSION_PREFIX_KEY;
+ static const std::string SELINUX_LABEL_LEVEL_PATTERN;
static const std::string IMAGE_NAME_ANNOTATION_KEY;
};
+struct commonSecurityContext {
+ const bool hasSELinuxOption;
+ const ::runtime::v1alpha2::SELinuxOption selinuxOption;
+ const std::string seccompProfile;
+};
+
auto GetDefaultSandboxImage(Errors &err) -> std::string;
auto MakeLabels(const google::protobuf::Map<std::string, std::string> &mapLabels, Errors &error)
@@ -124,12 +131,18 @@ auto ValidateCheckpointKey(const std::string &key, Errors &error) -> bool;
auto ToIsuladContainerStatus(const runtime::v1alpha2::ContainerStateValue &state) -> std::string;
-auto GetSecurityOpts(const std::string &seccompProfile, const char &separator, Errors &error)
+auto GetSELinuxLabelOpts(const bool hasSELinuxOption, const ::runtime::v1alpha2::SELinuxOption &selinux,
+ const char &separator, Errors &error)
-> std::vector<std::string>;
-auto GetSELinuxLabelOpts(const std::string &selinuxLabel, Errors &error)
+auto GetSecurityOpts(const commonSecurityContext &context, const char &separator, Errors &error)
-> std::vector<std::string>;
+auto GetPodSELinuxLabelOpts(const std::string &selinuxLabel, Errors &error)
+-> std::vector<std::string>;
+
+void AddSecurityOptsToHostConfig(std::vector<std::string> &securityOpts, host_config *hostconfig, Errors &error);
+
auto CreateCheckpoint(CRI::PodSandboxCheckpoint &checkpoint, Errors &error) -> std::string;
void GetCheckpoint(const std::string &jsonCheckPoint, CRI::PodSandboxCheckpoint &checkpoint, Errors &error);
diff --git a/src/daemon/entry/cri/cri_pod_sandbox_manager_service_impl.cc b/src/daemon/entry/cri/cri_pod_sandbox_manager_service_impl.cc
index 7ff545db..1cb3254d 100644
--- a/src/daemon/entry/cri/cri_pod_sandbox_manager_service_impl.cc
+++ b/src/daemon/entry/cri/cri_pod_sandbox_manager_service_impl.cc
@@ -197,34 +197,27 @@ void PodSandboxManagerServiceImpl::MakeSandboxIsuladConfig(const runtime::v1alph
const char securityOptSep = '=';
// Security Opts
- if (c.linux().has_security_context()) {
- std::vector<std::string> securityOpts =
- CRIHelpers::GetSecurityOpts(c.linux().security_context().seccomp_profile_path(), securityOptSep, error);
- if (error.NotEmpty()) {
- error.Errorf("failed to generate security options for sandbox %s: %s",
- c.metadata().name().c_str(), error.GetMessage().c_str());
- return;
- }
- if (!securityOpts.empty()) {
- char **tmp_security_opt = nullptr;
+ if (!c.linux().has_security_context()) {
+ return;
+ }
- if (securityOpts.size() > (SIZE_MAX / sizeof(char *)) - hc->security_opt_len) {
- error.Errorf("Out of memory");
- return;
- }
- size_t newSize = (hc->security_opt_len + securityOpts.size()) * sizeof(char *);
- size_t oldSize = hc->security_opt_len * sizeof(char *);
- int ret = util_mem_realloc((void **)(&tmp_security_opt), newSize, (void *)hc->security_opt, oldSize);
- if (ret != 0) {
- error.Errorf("Out of memory");
- return;
- }
- hc->security_opt = tmp_security_opt;
- for (const auto &securityOpt : securityOpts) {
- hc->security_opt[hc->security_opt_len] = util_strdup_s(securityOpt.c_str());
- hc->security_opt_len++;
- }
- }
+ CRIHelpers::commonSecurityContext commonContext = {
+ .hasSELinuxOption = c.linux().security_context().has_selinux_options(),
+ .selinuxOption = c.linux().security_context().selinux_options(),
+ .seccompProfile = c.linux().security_context().seccomp_profile_path(),
+ };
+
+ std::vector<std::string> securityOpts = CRIHelpers::GetSecurityOpts(commonContext, securityOptSep, error);
+ if (error.NotEmpty()) {
+ error.Errorf("Failed to generate security options for sandbox %s: %s",
+ c.metadata().name().c_str(), error.GetMessage().c_str());
+ return;
+ }
+ CRIHelpers::AddSecurityOptsToHostConfig(securityOpts, hc, error);
+ if (error.NotEmpty()) {
+ error.Errorf("Failed to add securityOpts to hostconfig for sandbox %s: %s", c.metadata().name().c_str(),
+ error.GetMessage().c_str());
+ return;
}
}
--
2.25.1

94
0039-add-files_limit-to-oci-spec.patch Normal file
View File

@ -0,0 +1,94 @@
From 6b5a0a345ae767cc245d1008377b8489acf2c0c7 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Sat, 8 Apr 2023 14:50:20 +0800
Subject: [PATCH 39/46] add files_limit to oci spec
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
src/daemon/modules/spec/specs.c | 49 +++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)
diff --git a/src/daemon/modules/spec/specs.c b/src/daemon/modules/spec/specs.c
index f32ff911..ad6d01d2 100644
--- a/src/daemon/modules/spec/specs.c
+++ b/src/daemon/modules/spec/specs.c
@@ -890,6 +890,27 @@ static int make_sure_oci_spec_linux_resources_pids(oci_runtime_spec *oci_spec)
return 0;
}
+static int make_sure_oci_spec_linux_resources_files(oci_runtime_spec *oci_spec)
+{
+ int ret = 0;
+
+ ret = make_sure_oci_spec_linux_resources(oci_spec);
+ if (ret < 0) {
+ return -1;
+ }
+
+ if (oci_spec->linux->resources->files != NULL) {
+ return 0;
+ }
+
+ oci_spec->linux->resources->files = util_common_calloc_s(sizeof(defs_resources_files));
+ if (oci_spec->linux->resources->files == NULL) {
+ ERROR("Out of memory");
+ return -1;
+ }
+ return 0;
+}
+
static int merge_pids_limit(oci_runtime_spec *oci_spec, int64_t pids_limit)
{
int ret = 0;
@@ -905,6 +926,20 @@ out:
return ret;
}
+static int merge_files_limit(oci_runtime_spec *oci_spec, int64_t files_limit)
+{
+ int ret = 0;
+
+ ret = make_sure_oci_spec_linux_resources_files(oci_spec);
+ if (ret < 0) {
+ ERROR("Failed to merge files limit");
+ return ret;
+ }
+
+ oci_spec->linux->resources->files->limit = files_limit;
+ return ret;
+}
+
static int merge_hostname(oci_runtime_spec *oci_spec, const host_config *host_spec, container_config *container_spec)
{
free(oci_spec->hostname);
@@ -1251,6 +1286,15 @@ out:
return ret;
}
+static int merge_conf_files_limit(oci_runtime_spec *oci_spec, const host_config *host_spec)
+{
+ if (host_spec->files_limit == 0) {
+ return 0;
+ }
+
+ return merge_files_limit(oci_spec, host_spec->files_limit);
+}
+
int merge_conf_cgroup(oci_runtime_spec *oci_spec, const host_config *host_spec)
{
int ret = 0;
@@ -1290,6 +1334,11 @@ int merge_conf_cgroup(oci_runtime_spec *oci_spec, const host_config *host_spec)
goto out;
}
+ ret = merge_conf_files_limit(oci_spec, host_spec);
+ if (ret != 0) {
+ goto out;
+ }
+
out:
return ret;
}
--
2.25.1

25
0040-support-setting-pod-to-privilege.patch Normal file
View File

@ -0,0 +1,25 @@
From 28ee9b5b9e09e0b9e4eaf87356af900d0e380ff0 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Thu, 30 Mar 2023 06:15:56 +0800
Subject: [PATCH 40/46] support setting pod to privilege
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
src/daemon/entry/cri/cri_security_context.cc | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/daemon/entry/cri/cri_security_context.cc b/src/daemon/entry/cri/cri_security_context.cc
index 1cd9287d..b2843626 100644
--- a/src/daemon/entry/cri/cri_security_context.cc
+++ b/src/daemon/entry/cri/cri_security_context.cc
@@ -209,6 +209,7 @@ void ApplySandboxSecurityContext(const runtime::v1alpha2::LinuxPodSandboxConfig
}
if (lc.has_security_context()) {
const runtime::v1alpha2::LinuxSandboxSecurityContext &old = lc.security_context();
+ sc->set_privileged(old.privileged());
if (old.has_run_as_user()) {
*sc->mutable_run_as_user() = old.run_as_user();
}
--
2.25.1

76
0041-add-hugepage_limit.patch Normal file
View File

@ -0,0 +1,76 @@
From 1a4e5174a9abcc83c9ace0cf7cabbdaf03697ae3 Mon Sep 17 00:00:00 2001
From: shijiaqi1 <jiaqi@isrc.iscas.ac.cn>
Date: Wed, 8 Feb 2023 13:31:36 +0800
Subject: [PATCH 41/46] add hugepage_limit
---
.../cri/cri_container_manager_service_impl.cc | 19 +++++++++++++++++
src/daemon/entry/cri/cri_helpers.cc | 21 ++++++++++++++++++-
2 files changed, 39 insertions(+), 1 deletion(-)
diff --git a/src/daemon/entry/cri/cri_container_manager_service_impl.cc b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
index 6278512f..5398c088 100644
--- a/src/daemon/entry/cri/cri_container_manager_service_impl.cc
+++ b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
@@ -1226,6 +1226,25 @@ void ContainerManagerServiceImpl::UpdateContainerResources(const std::string &co
if (!resources.cpuset_mems().empty()) {
hostconfig->cpuset_mems = util_strdup_s(resources.cpuset_mems().c_str());
}
+ if (resources.hugepage_limits_size() != 0) {
+ hostconfig->hugetlbs = (host_config_hugetlbs_element **)util_smart_calloc_s(
+ sizeof(host_config_hugetlbs_element *), resources.hugepage_limits_size());
+ if (hostconfig->hugetlbs == nullptr) {
+ error.SetError("Out of memory");
+ return;
+ }
+ for (int i = 0; i < resources.hugepage_limits_size(); i++) {
+ hostconfig->hugetlbs[i] =
+ (host_config_hugetlbs_element *)util_common_calloc_s(sizeof(host_config_hugetlbs_element));
+ if (hostconfig->hugetlbs[i] == nullptr) {
+ error.SetError("Out of memory");
+ goto cleanup;
+ }
+ hostconfig->hugetlbs[i]->page_size = util_strdup_s(resources.hugepage_limits(i).page_size().c_str());
+ hostconfig->hugetlbs[i]->limit = resources.hugepage_limits(i).limit();
+ hostconfig->hugetlbs_len++;
+ }
+ }
request->host_config = host_config_generate_json(hostconfig, &ctx, &perror);
if (request->host_config == nullptr) {
diff --git a/src/daemon/entry/cri/cri_helpers.cc b/src/daemon/entry/cri/cri_helpers.cc
index ec14d0b2..e588b6c4 100644
--- a/src/daemon/entry/cri/cri_helpers.cc
+++ b/src/daemon/entry/cri/cri_helpers.cc
@@ -447,8 +447,27 @@ void UpdateCreateConfig(container_config *createConfig, host_config *hc,
hc->cpuset_mems = util_strdup_s(rOpts.cpuset_mems().c_str());
}
hc->oom_score_adj = rOpts.oom_score_adj();
- }
+ if (rOpts.hugepage_limits_size() != 0) {
+ hc->hugetlbs = (host_config_hugetlbs_element **)util_smart_calloc_s(sizeof(host_config_hugetlbs_element *),
+ rOpts.hugepage_limits_size());
+ if (hc->hugetlbs == nullptr) {
+ error.SetError("Out of memory");
+ return;
+ }
+ for (int i = 0; i < rOpts.hugepage_limits_size(); i++) {
+ hc->hugetlbs[i] =
+ (host_config_hugetlbs_element *)util_common_calloc_s(sizeof(host_config_hugetlbs_element));
+ if (hc->hugetlbs[i] == nullptr) {
+ error.SetError("Out of memory");
+ return;
+ }
+ hc->hugetlbs[i]->page_size = util_strdup_s(rOpts.hugepage_limits(i).page_size().c_str());
+ hc->hugetlbs[i]->limit = rOpts.hugepage_limits(i).limit();
+ hc->hugetlbs_len++;
+ }
+ }
+ }
createConfig->open_stdin = config.stdin();
createConfig->tty = config.tty();
}
--
2.25.1

232
0042-add-effective-and-permitted-type-of-cap-to-oci-spec.patch Normal file
View File

@ -0,0 +1,232 @@
From 8e1bd51183eeb2ca2713b85afd52b12e584bc7a8 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Wed, 12 Apr 2023 21:19:56 +0800
Subject: [PATCH 42/46] add effective and permitted type of cap to oci spec
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
src/contrib/config/config.json | 28 ++++++
.../config/systemcontainer_config.json | 28 ++++++
src/daemon/modules/spec/specs_security.c | 88 ++++++++++++++-----
3 files changed, 120 insertions(+), 24 deletions(-)
diff --git a/src/contrib/config/config.json b/src/contrib/config/config.json
index f84f3394..9070a893 100644
--- a/src/contrib/config/config.json
+++ b/src/contrib/config/config.json
@@ -36,10 +36,38 @@
"CAP_AUDIT_WRITE"
],
"effective": [
+ "CAP_CHOWN",
+ "CAP_DAC_OVERRIDE",
+ "CAP_FSETID",
+ "CAP_FOWNER",
+ "CAP_MKNOD",
+ "CAP_NET_RAW",
+ "CAP_SETGID",
+ "CAP_SETUID",
+ "CAP_SETFCAP",
+ "CAP_SETPCAP",
+ "CAP_NET_BIND_SERVICE",
+ "CAP_SYS_CHROOT",
+ "CAP_KILL",
+ "CAP_AUDIT_WRITE"
],
"inheritable": [
],
"permitted": [
+ "CAP_CHOWN",
+ "CAP_DAC_OVERRIDE",
+ "CAP_FSETID",
+ "CAP_FOWNER",
+ "CAP_MKNOD",
+ "CAP_NET_RAW",
+ "CAP_SETGID",
+ "CAP_SETUID",
+ "CAP_SETFCAP",
+ "CAP_SETPCAP",
+ "CAP_NET_BIND_SERVICE",
+ "CAP_SYS_CHROOT",
+ "CAP_KILL",
+ "CAP_AUDIT_WRITE"
],
"ambient": [
]
diff --git a/src/contrib/config/systemcontainer_config.json b/src/contrib/config/systemcontainer_config.json
index 8ebce8c6..9169956e 100644
--- a/src/contrib/config/systemcontainer_config.json
+++ b/src/contrib/config/systemcontainer_config.json
@@ -36,10 +36,38 @@
"CAP_AUDIT_WRITE"
],
"effective": [
+ "CAP_CHOWN",
+ "CAP_DAC_OVERRIDE",
+ "CAP_FSETID",
+ "CAP_FOWNER",
+ "CAP_MKNOD",
+ "CAP_NET_RAW",
+ "CAP_SETGID",
+ "CAP_SETUID",
+ "CAP_SETFCAP",
+ "CAP_SETPCAP",
+ "CAP_NET_BIND_SERVICE",
+ "CAP_SYS_CHROOT",
+ "CAP_KILL",
+ "CAP_AUDIT_WRITE"
],
"inheritable": [
],
"permitted": [
+ "CAP_CHOWN",
+ "CAP_DAC_OVERRIDE",
+ "CAP_FSETID",
+ "CAP_FOWNER",
+ "CAP_MKNOD",
+ "CAP_NET_RAW",
+ "CAP_SETGID",
+ "CAP_SETUID",
+ "CAP_SETFCAP",
+ "CAP_SETPCAP",
+ "CAP_NET_BIND_SERVICE",
+ "CAP_SYS_CHROOT",
+ "CAP_KILL",
+ "CAP_AUDIT_WRITE"
],
"ambient": [
]
diff --git a/src/daemon/modules/spec/specs_security.c b/src/daemon/modules/spec/specs_security.c
index 62f67082..d4884097 100644
--- a/src/daemon/modules/spec/specs_security.c
+++ b/src/daemon/modules/spec/specs_security.c
@@ -261,36 +261,80 @@ free_out:
return ret;
}
+// tweak_all_type_capabilities can tweak all support type capabilities by adding or dropping capabilities
+// based on the basic capabilities.
+static int tweak_all_type_capabilities(defs_process_capabilities **caps, const char **adds, size_t adds_len,
+ const char **drops, size_t drops_len)
+{
+ int ret = 0;
+ ret = tweak_capabilities(&((*caps)->bounding), &((*caps)->bounding_len), adds, adds_len, drops, drops_len);
+ if (ret != 0) {
+ ERROR("Failed to tweak bounding capabilities");
+ return -1;
+ }
+
+ ret = tweak_capabilities(&((*caps)->permitted), &((*caps)->permitted_len), adds, adds_len, drops, drops_len);
+ if (ret != 0) {
+ ERROR("Failed to tweak permitted capabilities");
+ return -1;
+ }
+
+ ret = tweak_capabilities(&((*caps)->effective), &((*caps)->effective_len), adds, adds_len, drops, drops_len);
+ if (ret != 0) {
+ ERROR("Failed to tweak effective capabilities");
+ return -1;
+ }
+ return ret;
+}
+
+static void clear_caps(defs_process_capabilities **caps)
+{
+ util_free_array_by_len((*caps)->bounding, (*caps)->bounding_len);
+ (*caps)->bounding_len = 0;
+ (*caps)->bounding = NULL;
+
+ util_free_array_by_len((*caps)->permitted, (*caps)->permitted_len);
+ (*caps)->permitted_len = 0;
+ (*caps)->permitted = NULL;
+
+ util_free_array_by_len((*caps)->effective, (*caps)->effective_len);
+ (*caps)->effective_len = 0;
+ (*caps)->effective = NULL;
+}
+
int refill_oci_process_capabilities(defs_process_capabilities **caps, const char **src_caps, size_t src_caps_len)
{
int ret = 0;
- size_t i = 0;
if (*caps == NULL) {
*caps = util_common_calloc_s(sizeof(defs_process_capabilities));
if (*caps == NULL) {
- ret = -1;
- goto out;
+ return -1;
}
}
- if ((*caps)->bounding != NULL) {
- // free current capabilities
- for (i = 0; i < ((*caps)->bounding_len); i++) {
- free((*caps)->bounding[i]);
- (*caps)->bounding[i] = NULL;
- }
- free((*caps)->bounding);
- (*caps)->bounding = NULL;
- }
- (*caps)->bounding_len = 0;
+ // clear current capabilities
+ clear_caps(caps);
// copy capabilities
ret = copy_capabilities(&((*caps)->bounding), &((*caps)->bounding_len), src_caps, src_caps_len);
if (ret != 0) {
- ERROR("Failed to copy all capabilities");
+ ERROR("Failed to copy all bounding capabilities");
+ return -1;
}
-out:
+
+ ret = copy_capabilities(&((*caps)->permitted), &((*caps)->permitted_len), src_caps, src_caps_len);
+ if (ret != 0) {
+ ERROR("Failed to copy all permitted capabilities");
+ return -1;
+ }
+
+ ret = copy_capabilities(&((*caps)->effective), &((*caps)->effective_len), src_caps, src_caps_len);
+ if (ret != 0) {
+ ERROR("Failed to copy all effective capabilities");
+ return -1;
+ }
+
return ret;
}
@@ -823,25 +867,21 @@ int merge_caps(oci_runtime_spec *oci_spec, const char **adds, size_t adds_len, c
ret = make_sure_oci_spec_process_capabilities(oci_spec);
if (ret < 0) {
- goto out;
+ return ret;
}
if (adds_len > LIST_SIZE_MAX || drops_len > LIST_SIZE_MAX) {
ERROR("Too many capabilities to add or drop, the limit is %lld", LIST_SIZE_MAX);
isulad_set_error_message("Too many capabilities to add or drop, the limit is %d", LIST_SIZE_MAX);
- ret = -1;
- goto out;
+ return -1;
}
- ret = tweak_capabilities(&oci_spec->process->capabilities->bounding, &oci_spec->process->capabilities->bounding_len,
- adds, adds_len, drops, drops_len);
+ ret = tweak_all_type_capabilities(&oci_spec->process->capabilities, adds, adds_len, drops, drops_len);
if (ret != 0) {
- ERROR("Failed to tweak capabilities");
- ret = -1;
- goto out;
+ ERROR("Failed to tweak all type capabilities");
+ return -1;
}
-out:
return ret;
}
--
2.25.1

30
0043-isulad-shim-fix-log-loss-bug.patch Normal file
View File

@ -0,0 +1,30 @@
From 795f9f56bd33ed382ae1e68300247ef651d77864 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Sun, 16 Apr 2023 19:57:48 +1400
Subject: [PATCH 43/46] isulad-shim: fix log loss bug
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
src/cmd/isulad-shim/process.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/src/cmd/isulad-shim/process.c b/src/cmd/isulad-shim/process.c
index 02609911..a676e7ce 100644
--- a/src/cmd/isulad-shim/process.c
+++ b/src/cmd/isulad-shim/process.c
@@ -1348,6 +1348,12 @@ int process_signal_handle_routine(process_t *p, const pthread_t tid_accept, cons
destroy_io_thread(p, i);
}
+ if (!p->state->exec) {
+ // if log did not contain "/n", print remaind container log when exit isulad-shim
+ shim_write_container_log_file(p->terminal, STDID_OUT, NULL, 0);
+ shim_write_container_log_file(p->terminal, STDID_ERR, NULL, 0);
+ }
+
if (ret == SHIM_ERR_TIMEOUT) {
write_message(g_log_fd, INFO_MSG, "Wait %d timeout", p->ctr_pid);
exit(SHIM_EXIT_TIMEOUT);
--
2.25.1

69
0044-remove-unused-func.patch Normal file
View File

@ -0,0 +1,69 @@
From 2b3a35ec1e0e3afd090618f2120fd8e756c4c0c6 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Wed, 19 Apr 2023 07:22:42 +0800
Subject: [PATCH 44/46] remove unused func
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
.../modules/runtime/isula/isula_rt_ops.c | 31 ++-----------------
1 file changed, 2 insertions(+), 29 deletions(-)
diff --git a/src/daemon/modules/runtime/isula/isula_rt_ops.c b/src/daemon/modules/runtime/isula/isula_rt_ops.c
index 5a01b8c6..3a6269a1 100644
--- a/src/daemon/modules/runtime/isula/isula_rt_ops.c
+++ b/src/daemon/modules/runtime/isula/isula_rt_ops.c
@@ -1078,28 +1078,6 @@ int rt_isula_rm(const char *id, const char *runtime, const rt_rm_params_t *param
return 0;
}
-static char *try_generate_exec_id()
-{
- char *id = NULL;
-
- id = util_common_calloc_s(sizeof(char) * (CONTAINER_EXEC_ID_MAX_LEN + 1));
- if (id == NULL) {
- ERROR("Out of memory");
- return NULL;
- }
-
- if (util_generate_random_str(id, (size_t)CONTAINER_EXEC_ID_MAX_LEN) != 0) {
- ERROR("Generate id failed");
- goto err_out;
- }
-
- return id;
-
-err_out:
- free(id);
- return NULL;
-}
-
static bool fg_exec(const rt_exec_params_t *params)
{
if (params->console_fifos[0] != NULL || params->console_fifos[1] != NULL || params->console_fifos[2] != NULL) {
@@ -1122,7 +1100,7 @@ int rt_isula_exec(const char *id, const char *runtime, const rt_exec_params_t *p
int pid = 0;
shim_client_process_state p = { 0 };
- if (id == NULL || runtime == NULL || params == NULL || exit_code == NULL) {
+ if (id == NULL || runtime == NULL || params == NULL || exit_code == NULL || params->suffix == NULL) {
ERROR("nullptr arguments not allowed");
return -1;
}
@@ -1135,12 +1113,7 @@ int rt_isula_exec(const char *id, const char *runtime, const rt_exec_params_t *p
return -1;
}
- if (params->suffix != NULL) {
- exec_id = util_strdup_s(params->suffix);
- } else {
- exec_id = try_generate_exec_id();
- }
-
+ exec_id = util_strdup_s(params->suffix);
if (exec_id == NULL) {
ERROR("Out of memory or generate exec id failed");
return -1;
--
2.25.1

32
0045-if-the-exit-code-in-the-response-of-execSync-is-not-.patch Normal file
View File

@ -0,0 +1,32 @@
From 26c4a702ad9c919d8413b64f498d58b13375bbda Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Tue, 18 Apr 2023 17:30:03 +0800
Subject: [PATCH 45/46] if the exit code in the response of execSync is not
zero, an error will be returned in cri
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
src/daemon/entry/connect/grpc/runtime_runtime_service.cc | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/src/daemon/entry/connect/grpc/runtime_runtime_service.cc b/src/daemon/entry/connect/grpc/runtime_runtime_service.cc
index 451eeeef..4cc5a4ee 100644
--- a/src/daemon/entry/connect/grpc/runtime_runtime_service.cc
+++ b/src/daemon/entry/connect/grpc/runtime_runtime_service.cc
@@ -289,6 +289,13 @@ grpc::Status RuntimeRuntimeServiceImpl::ExecSync(grpc::ServerContext *context,
return grpc::Status(grpc::StatusCode::UNKNOWN, error.GetMessage());
}
+ if (reply->exit_code() != 0) {
+ ERROR("Object: CRI, Type: Sync exec in container: %s with exit code: %d", request->container_id().c_str(),
+ reply->exit_code());
+ error.SetError(reply->stderr());
+ return grpc::Status(grpc::StatusCode::UNKNOWN, error.GetMessage());
+ }
+
WARN("Event: {Object: CRI, Type: sync execed Container: %s}", request->container_id().c_str());
return grpc::Status::OK;
--
2.25.1

89
0046-free-timeout-when-shim_create-finished.patch Normal file
View File

@ -0,0 +1,89 @@
From 4dfc94f2beb816eb2e26ede07e803a230405b193 Mon Sep 17 00:00:00 2001
From: zhongtao <zhongtao17@huawei.com>
Date: Thu, 20 Apr 2023 13:28:26 +0800
Subject: [PATCH 46/46] free timeout when shim_create finished
Signed-off-by: zhongtao <zhongtao17@huawei.com>
---
.../modules/runtime/isula/isula_rt_ops.c | 30 +++++++++++++------
1 file changed, 21 insertions(+), 9 deletions(-)
diff --git a/src/daemon/modules/runtime/isula/isula_rt_ops.c b/src/daemon/modules/runtime/isula/isula_rt_ops.c
index 3a6269a1..ceaf464e 100644
--- a/src/daemon/modules/runtime/isula/isula_rt_ops.c
+++ b/src/daemon/modules/runtime/isula/isula_rt_ops.c
@@ -694,7 +694,7 @@ static int status_to_exit_code(int status)
}
static int shim_create(bool fg, const char *id, const char *workdir, const char *bundle, const char *runtime_cmd,
- int *exit_code, const int64_t timeout)
+ int *exit_code, const char* timeout)
{
pid_t pid = 0;
int exec_fd[2] = { -1, -1 };
@@ -712,12 +712,8 @@ static int shim_create(bool fg, const char *id, const char *workdir, const char
params[i++] = runtime_cmd;
params[i++] = "info";
// execSync timeout
- if (timeout > 0) {
- params[i] = util_int_to_string(timeout);
- if (params[i] == NULL) {
- ERROR("Failed to convert execSync timeout %ld to string", timeout);
- return -1;
- }
+ if (timeout != NULL) {
+ params[i++] = timeout;
}
runtime_exec_param_dump(params);
@@ -917,7 +913,7 @@ int rt_isula_create(const char *id, const char *runtime, const rt_create_params_
}
get_runtime_cmd(runtime, &cmd);
- ret = shim_create(false, id, workdir, params->bundle, cmd, NULL, -1);
+ ret = shim_create(false, id, workdir, params->bundle, cmd, NULL, NULL);
if (ret != 0) {
runtime_call_delete_force(workdir, runtime, id);
ERROR("%s: failed create shim process", id);
@@ -1099,6 +1095,7 @@ int rt_isula_exec(const char *id, const char *runtime, const rt_exec_params_t *p
char bundle[PATH_MAX] = { 0 };
int pid = 0;
shim_client_process_state p = { 0 };
+ char *timeout = NULL;
if (id == NULL || runtime == NULL || params == NULL || exit_code == NULL || params->suffix == NULL) {
ERROR("nullptr arguments not allowed");
@@ -1158,7 +1155,18 @@ int rt_isula_exec(const char *id, const char *runtime, const rt_exec_params_t *p
}
get_runtime_cmd(runtime, &cmd);
- ret = shim_create(fg_exec(params), id, workdir, bundle, cmd, exit_code, params->timeout);
+
+ // execSync timeout
+ if (params->timeout > 0) {
+ timeout = util_int_to_string(params->timeout);
+ if (timeout == NULL) {
+ ERROR("Failed to convert execSync timeout %ld to string", params->timeout);
+ ret = -1;
+ goto del_out;
+ }
+ }
+
+ ret = shim_create(fg_exec(params), id, workdir, bundle, cmd, exit_code, timeout);
if (ret != 0) {
ERROR("%s: failed create shim process for exec %s", id, exec_id);
goto errlog_out;
@@ -1183,6 +1191,10 @@ errlog_out:
show_shim_runtime_errlog(workdir);
}
+ if (timeout != NULL) {
+ free(timeout);
+ }
+
del_out:
if (util_recursive_rmdir(workdir, 0)) {
ERROR("rmdir %s failed", workdir);
--
2.25.1

28
iSulad.spec
View File

@ -1,5 +1,5 @@
%global _version 2.0.18
%global _release 4
%global _release 5
%global is_systemd 1
%global enable_shimv2 1
%global is_embedded 1
@ -39,6 +39,26 @@ Patch0023: 0023-change-sleep-to-usleep-to-avoid-lossing-of-accuracy.patch
Patch0024: 0024-adapt-to-repo-of-openeuler-url-changed.patch
Patch0025: 0025-modify-sleep-time.patch
Patch0026: 0026-change-goto-branch.patch
Patch0027: 0027-modifying-cpurt-file-permissions.patch
Patch0028: 0028-add-design-docs-for-cri-manager.patch
Patch0029: 0029-improve-check-of-process-failure.patch
Patch0030: 0030-support-isula-update-when-runtime-is-runc.patch
Patch0031: 0031-when-calling-runc-start-unset-NOTIFY_-SOCKET.patch
Patch0032: 0032-add-CRI-container-design-doc.patch
Patch0033: 0033-fix-util_getgrent_r-overflow.patch
Patch0034: 0034-modify-the-return-value-of-the-util_waitpid_with_tim.patch
Patch0035: 0035-fix-inspect-data-memleak.patch
Patch0036: 0036-containers-in-same-sandbox-should-have-same-process-.patch
Patch0037: 0037-clean-container-process-after-execSync-timeout-exit.patch
Patch0038: 0038-support-to-config-selinux-label-in-cri.patch
Patch0039: 0039-add-files_limit-to-oci-spec.patch
Patch0040: 0040-support-setting-pod-to-privilege.patch
Patch0041: 0041-add-hugepage_limit.patch
Patch0042: 0042-add-effective-and-permitted-type-of-cap-to-oci-spec.patch
Patch0043: 0043-isulad-shim-fix-log-loss-bug.patch
Patch0044: 0044-remove-unused-func.patch
Patch0045: 0045-if-the-exit-code-in-the-response-of-execSync-is-not-.patch
Patch0046: 0046-free-timeout-when-shim_create-finished.patch
%ifarch x86_64 aarch64
Provides: libhttpclient.so()(64bit)
@ -283,6 +303,12 @@ fi
%endif
%changelog
* Mon Apr 24 2023 zhangxiaoyu <zhangxiaoyu58@huawei.com> - 2.0.18-5
- Type: bugfix
- ID: NA
- SUG: NA
- DESC: bugfix for runc and cri
* Wed Mar 09 2023 zhangxiaoyu <zhangxiaoyu58@huawei.com> - 2.0.18-4
- Type: bugfix
- ID: NA