Linux: sysstat；pidstat
简介
通过systemd提供的timer机制，定时收集stat数据写道/var/log/sa/目录；每天一个sa日志文件。同时每天23点53分，会将sa二进制文件解析成文本文件放到sar文件中。每天的零点7分，重新生成一个新文件。
遇到的问题
日志文件很大
每天产生300M的日志文件。
原因是，我们创建了2048个vlan的网络接口。而在sa中关于网络接口的统计数据是：
00:46:40 IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s %ifutil
00:50:29 enp0s3 0.08 0.13 0.03 0.01 0.00 0.00 0.00 0.00
00:50:29 lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
00:46:40 IFACE rxerr/s txerr/s coll/s rxdrop/s txdrop/s txcarr/s rxfram/s rxfifo/s txfifo/s
00:50:29 enp0s3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
00:50:29 lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
软件包含有的文件
# rpm -ql sysstat-10.1.5-19.el7.x86_64
/etc/cron.d/sysstat
/etc/sysconfig/sysstat
/etc/sysconfig/sysstat.ioconf
/usr/bin/cifsiostat
/usr/bin/iostat
/usr/bin/mpstat
/usr/bin/nfsiostat-sysstat
/usr/bin/pidstat
/usr/bin/sadf
/usr/bin/sar
/usr/bin/tapestat
/usr/lib/systemd/system/sysstat.service
/usr/lib64/sa
/usr/lib64/sa/sa1
/usr/lib64/sa/sa2
/usr/lib64/sa/sadc
/usr/share/doc/sysstat-10.1.5
/usr/share/doc/sysstat-10.1.5/CHANGES
/usr/share/doc/sysstat-10.1.5/COPYING
/usr/share/doc/sysstat-10.1.5/CREDITS
/usr/share/doc/sysstat-10.1.5/FAQ
/usr/share/doc/sysstat-10.1.5/README
/usr/share/doc/sysstat-10.1.5/sysstat-10.1.5.lsm
/usr/share/man/man1/cifsiostat.1.gz
/usr/share/man/man1/iostat.1.gz
/usr/share/man/man1/mpstat.1.gz
/usr/share/man/man1/nfsiostat-sysstat.1.gz
/usr/share/man/man1/pidstat.1.gz
/usr/share/man/man1/sadf.1.gz
/usr/share/man/man1/sar.1.gz
/usr/share/man/man1/tapestat.1.gz
/usr/share/man/man5/sysstat.5.gz
/usr/share/man/man8/sa1.8.gz
/usr/share/man/man8/sa2.8.gz
/usr/share/man/man8/sadc.8.gz
/var/log/sa
任务
# more /etc/cron.d/sysstat
# Run system activity accounting tool every 10 minutes
*/10 * * * * root /usr/lib64/sa/sa1 1 1
# 0 * * * * root /usr/lib64/sa/sa1 600 6 &
# Generate a daily summary of process accounting at 23:53
53 23 * * * root /usr/lib64/sa/sa2 -A
每天23点53开始对sa数据进行统计；放到sar文件中。system state report；
配置
# cat /etc/sysconfig/sysstat
# sysstat-10.1.5 configuration file.
# How long to keep log files (in days).
# If value is greater than 28, then log files are kept in
# multiple directories, one for each month.
HISTORY=28
# Compress (using gzip or bzip2) sa and sar files older than (in days):
COMPRESSAFTER=31
# Parameters for the system activity data collector (see sadc manual page)
# which are used for the generation of log files.
SADC_OPTIONS="-S DISK"
# Compression program to use.
ZIP="bzip2"
服务
sysstat.service
[root@rhel-cleanmodules ~]# cat /usr/lib/systemd/system/sysstat.service
[Unit]
Description=Resets System Activity Logs
[Service]
Type=oneshot // 只是一次允许
RemainAfterExit=yes 如果进程退出，也认为服务是活的。
User=root
ExecStart=/usr/lib64/sa/sa1 --boot
[Install]
WantedBy=multi-user.target
Also=sysstat-collect.timer
Also=sysstat-summary.timer (man systemd.unit)
Also=
Additional units to install/deinstall when this unit is installed/deinstalled. If the user requests installation/deinstallation of a
unit with this option configured, systemctl enable and systemctl disable will automatically install/uninstall units listed in this
option as well.
This option may be used more than once, or a space-separated list of unit names may be given.
sysstat-collect.timer 和 sysstat-collect.service
[root@rhel-cleanmodules man1]# systemctl status sysstat-collect.timer
● sysstat-collect.timer - Run system activity accounting tool every 10 minutes
Loaded: loaded (/usr/lib/systemd/system/sysstat-collect.timer; enabled; vendor preset: disabled)
Active: active (waiting) since Mon 2021-10-11 17:02:42 EDT; 35min ago
Trigger: Mon 2021-10-11 17:40:00 EDT; 2min 2s left
每十分钟一次
sysstat-summary.timer 和 sysstat-summary.service
timer，启动之后，过期就会主动调用对应的service。
每天一次
/usr/lib64/sa/sa1
#@(#) sysstat-11.7.3
#@(#) sa1: Collect and store binary data in system activity data file.
#
# Set default value for some variables.
# Used only if ${SYSCONFIG_DIR}/sysstat doesn't exist!
HISTORY=0
SADC_OPTIONS=""
SA_DIR=/var/log/sa
SYSCONFIG_DIR=/etc/sysconfig
umask 0022
[ -r ${SYSCONFIG_DIR}/sysstat ] && . ${SYSCONFIG_DIR}/sysstat
[ -d ${SA_DIR} ] || SA_DIR=/var/log/sa
if [ ${HISTORY} -gt 28 ]
then
SADC_OPTIONS="${SADC_OPTIONS} -D"
fi
ENDIR=/usr/lib64/sa
cd ${ENDIR}
[ "$1" = "--boot" ] && shift && BOOT=y || BOOT=n
if [ $# = 0 ] && [ "${BOOT}" = "n" ]
then
# Note: Stats are written at the end of previous file *and* at the
# beginning of the new one (when there is a file rotation) only if
# outfile has been specified as '-' on the command line...
exec ${ENDIR}/sadc -F -L ${SADC_OPTIONS} 1 1 ${SA_DIR}
else
exec ${ENDIR}/sadc -F -L ${SADC_OPTIONS} $* ${SA_DIR}
fi
/usr/lib64/sa/sa2 -A
这个负责压缩日志文件，同时产生新的sa日志文件。
sadc
-F The creation of outfile will be forced. If the file already exists and has a format unknown to sadc then it will be truncated. This
may be useful for daily data files created by an older version of sadc and whose format is no longer compatible with current one.
pidstat
查询Linux线程的统计数据
#pidstat -w 3 10 > /tmp/pidstat.out
10:15:24 AM UID PID cswch/s nvcswch/s Command
10:15:27 AM 0 1 162656.7 16656.7 systemd
10:15:27 AM 0 9 165451.04 15451.04 ksoftirqd/0
-C comm
Display only tasks whose command name includes the string comm. This string can be a regular expression.
-R Report realtime priority and scheduling policy information.
The following values may be displayed:
UID The real user identification number of the task being monitored.
USER The name of the real user owning the task being monitored.
PID The identification number of the task being monitored.
prio The realtime priority of the task being monitored.
policy The scheduling policy of the task being monitored.
Command The command name of the task.
-r Report page faults and memory utilization.
When reporting statistics for individual tasks, the following values may be displayed:
minflt/s Total number of minor faults the task has made per second, those which have not required loading a memory page from disk.
majflt/s Total number of major faults the task has made per second, those which have required loading a memory page from disk.
VSZ Virtual Size: The virtual memory usage of entire task in kilobytes.
RSS Resident Set Size: The non-swapped physical memory used by the task in kilobytes.
%MEM The tasks’s currently used share of available physical memory.
When reporting global statistics for tasks and all their children, the following values may be displayed:
minflt-nr Total number of minor faults made by the task and all its children, and collected during the interval of time.
majflt-nr Total number of major faults made by the task and all its children, and collected during the interval of time.
-w Report task switching activity (kernels 2.6.23 and later only).
The following values may be displayed:
cswch/s Total number of voluntary context switches the task made per second. A voluntary context switch occurs when a task blocks because it requires a resource that is unavailable.
nvcswch/s Total number of non voluntary context switches the task made per second. A involuntary context switch takes place when a task executes for the duration of its time slice and then is forced to relinquish the processor.