GBase 8a在V95版本不同列、不同行数文件加载性能测试对比

发表于2021年2月4日2021年5月8日作者 laozizhu

本文针对10-1000列的表，10-1000万行数据(int),进行加载性能测试，验证加载性能，提供用户项目参考。

相同列数，不同数据库版本，开启事务影响等，请参考 8a针对宽表1000列的小文件加载不同版本配置性能测试

目录导航

测试结果汇总

在笔记本虚拟机单节点4核4G内存的GBase 8a V952开始事务的环境下，常规的20-50列的表，10万行以内，也能在5秒以内完成；低于1万行的数据，即使1000列，也能在10秒内完成。

行数	10列	20列	50列	100列	200列	500列	1000列
10	0.28	0.39	0.32	0.31	0.41	0.58	0.67
100	0.42	0.33	0.37	0.32	0.46	0.67	0.89
1000	0.31	0.28	0.45	0.56	0.68	0.97	1.69
10000	0.39	0.51	0.75	1.07	1.57	3.92	8.63
100000	0.95	1.11	2.26	4.30	8.36	28.30	64.61
1000000	2.91	5.08	11.96	25.14
10000000	23.53

环境

笔记本虚拟机，4核4G内存。

集群规模

单节点集群

[gbase@gbase_rh7_001 ~]$ gcadmin showcluster vc vc01
CLUSTER STATE:         ACTIVE
VIRTUAL CLUSTER MODE:  NORMAL

==========================================================
|           GBASE VIRTUAL CLUSTER INFORMATION            |
==========================================================
|    VcName    | DistributionId |        comment         |
----------------------------------------------------------
|     vc01     |       1        | first vc with one node |
----------------------------------------------------------
=========================================================================================================
|                                 VIRTUAL CLUSTER DATA NODE INFORMATION                                 |
=========================================================================================================
| NodeName |                IpAddress                 | DistributionId | gnode | syncserver | DataState |
---------------------------------------------------------------------------------------------------------
|  node1   |                10.0.2.101                |       1        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------

1 data node

硬件

CPU

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 142
model name      : Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz
stepping        : 9
cpu MHz         : 2712.004
cache size      : 3072 KB
physical id     : 0

磁盘

SSD

内存

[gbase@gbase_rh7_001 ~]$ free -m
              total        used        free      shared  buff/cache   available
Mem:           3790        2816         522         139         451          79
Swap:          4095         642        3453
[gbase@gbase_rh7_001 ~]$

测试数据

生成方法，请参考 GBase 8a针对宽表1000列的小文件加载性能测试

[root@gbase_rh7_001 gbase]# ll -h 10c
total 600M
-rw-r--r-- 1 root root 540M Feb  3 16:27 10_10000000.txt
-rw-r--r-- 1 root root  54M Feb  3 14:23 10_1000000.txt
-rw-r--r-- 1 root root 5.4M Feb  3 13:59 10_100000.txt
-rw-r--r-- 1 root root 553K Feb  3 13:58 10_10000.txt
-rw-r--r-- 1 root root  56K Feb  3 13:58 10_1000.txt
-rw-r--r-- 1 root root 5.5K Feb  3 13:58 10_100.txt
-rw-r--r-- 1 root root  562 Feb  3 13:57 10_10.txt
[root@gbase_rh7_001 gbase]# ll -h 20c
total 120M
-rw-rw-r-- 1 gbase gbase 108M Feb  3 15:28 20_1000000.txt
-rw-rw-r-- 1 gbase gbase  11M Feb  3 15:17 20_100000.txt
-rw-rw-r-- 1 gbase gbase 1.1M Feb  3 15:16 20_10000.txt
-rw-rw-r-- 1 gbase gbase 111K Feb  3 15:16 20_1000.txt
-rw-rw-r-- 1 gbase gbase  12K Feb  3 15:16 20_100.txt
-rw-rw-r-- 1 gbase gbase 1.2K Feb  3 15:16 20_10.txt
[root@gbase_rh7_001 gbase]# ll -h 50c
total 300M
-rw-rw-r-- 1 gbase gbase 270M Feb  3 16:06 50_1000000.txt
-rw-rw-r-- 1 gbase gbase  27M Feb  3 15:34 50_100000.txt
-rw-rw-r-- 1 gbase gbase 2.7M Feb  3 15:30 50_10000.txt
-rw-rw-r-- 1 gbase gbase 277K Feb  3 15:29 50_1000.txt
-rw-rw-r-- 1 gbase gbase  28K Feb  3 15:29 50_100.txt
-rw-rw-r-- 1 gbase gbase 2.8K Feb  3 15:29 50_10.txt
[root@gbase_rh7_001 gbase]# ll -h 100c
total 600M
-rw-r--r-- 1 root root 540M Feb  3 16:14 100_1000000.txt
-rw-r--r-- 1 root root  54M Feb  3 14:07 100_100000.txt
-rw-r--r-- 1 root root 5.4M Feb  3 14:00 100_10000.txt
-rw-r--r-- 1 root root 553K Feb  3 13:59 100_1000.txt
-rw-r--r-- 1 root root  56K Feb  3 13:59 100_100.txt
-rw-r--r-- 1 root root 5.6K Feb  3 13:59 100_10.txt
[root@gbase_rh7_001 gbase]#

[root@gbase_rh7_001 gbase]# ll -h 200c
total 120M
-rw-r--r-- 1 root root 108M Feb  4 09:17 200_100000.txt
-rw-r--r-- 1 root root  11M Feb  4 08:57 200_10000.txt
-rw-r--r-- 1 root root 1.1M Feb  4 08:55 200_1000.txt
-rw-r--r-- 1 root root 111K Feb  4 08:54 200_100.txt
-rw-r--r-- 1 root root  12K Feb  4 08:54 200_10.txt
[root@gbase_rh7_001 gbase]# ll -h 500c
total 300M
-rw-r--r-- 1 root root 270M Feb  4 09:46 500_100000.txt
-rw-r--r-- 1 root root  27M Feb  4 09:01 500_10000.txt
-rw-r--r-- 1 root root 2.7M Feb  4 08:56 500_1000.txt
-rw-r--r-- 1 root root 277K Feb  4 08:56 500_100.txt
-rw-r--r-- 1 root root  28K Feb  4 08:56 500_10.txt
[root@gbase_rh7_001 gbase]# ll -h 1000c
total 600M
-rw-rw-r--  1 gbase gbase 5.4M Feb  2 15:01 1000.txt
-rw-rw-r--  1 gbase gbase 553K Feb  2 14:59 100.txt
-rw-rw-r--  1 gbase gbase  56K Feb  2 14:59 10.txt
-rw-rw-r--  1 gbase gbase 540M Feb  2 16:20 10w.txt
-rw-------. 1 gbase gbase  108 Dec 23 10:13 1.txt
-rw-rw-r--  1 gbase gbase  55M Feb  2 15:03 1w.txt
[root@gbase_rh7_001 gbase]#

测试记录

行数	10列	20列	50列	100列	200列	500列	1000列
10	00:00.24	00:00.37	00:00.31	00:00.30	00:00.64	00:00.48	00:00.50
10	00:00.29	00:00.29	00:00.46	00:00.33	00:00.37	00:00.47	00:00.80
10	00:00.30	00:00.23	00:00.29	00:00.29	00:00.35	00:00.62	00:00.61
10	00:00.26	00:00.29	00:00.26	00:00.30	00:00.35	00:00.92	00:00.80
10	00:00.29	00:00.76	00:00.28	00:00.31	00:00.36	00:00.40	00:00.62
100	00:00.32	00:00.39	00:00.28	00:00.34	00:00.35	00:00.49	00:00.73
100	00:00.32	00:00.43	00:00.17	00:00.28	00:00.40	00:00.98	00:00.69
100	00:00.33	00:00.27	00:00.77	00:00.34	00:00.88	00:00.89	00:01.19
100	00:00.31	00:00.31	00:00.30	00:00.33	00:00.32	00:00.48	00:00.67
100	00:00.83	00:00.25	00:00.31	00:00.33	00:00.34	00:00.52	00:01.19
1000	00:00.26	00:00.26	00:00.83	00:00.61	00:00.41	00:00.96	00:01.59
1000	00:00.41	00:00.32	00:00.38	00:00.79	00:01.06	00:00.98	00:01.60
1000	00:00.27	00:00.28	00:00.34	00:00.42	00:00.38	00:00.98	00:01.59
1000	00:00.30	00:00.28	00:00.37	00:00.38	00:01.02	00:00.97	00:01.57
1000	00:00.31	00:00.28	00:00.34	00:00.59	00:00.54	00:00.96	00:02.09
10000	00:00.36	00:00.74	00:00.77	00:00.97	00:01.54	00:03.74	00:08.90
10000	00:00.39	00:00.46	00:00.81	00:01.09	00:01.52	00:03.77	00:08.50
10000	00:00.38	00:00.46	00:00.75	00:01.08	00:01.58	00:03.87	00:08.56
10000	00:00.40	00:00.43	00:00.73	00:01.10	00:01.59	00:03.93	00:08.71
10000	00:00.42	00:00.47	00:00.70	00:01.10	00:01.60	00:04.31	00:08.49
100000	00:01.08	00:01.39	00:02.69	00:05.09	00:10.10	00:39.96	01:26.38
100000	00:00.88	00:01.02	00:02.39	00:04.46	00:07.95	00:24.21	00:55.95
100000	00:01.05	00:01.02	00:02.14	00:03.95	00:07.11	00:25.73	00:57.75
100000	00:00.87	00:01.08	00:02.06	00:04.35	00:08.12	00:27.10	01:06.35
100000	00:00.89	00:01.02	00:02.02	00:03.67	00:08.52	00:24.49	00:56.62
1000000	00:02.86	00:04.98	00:11.93	00:24.35
1000000	00:02.92	00:05.19	00:12.06	00:28.14
1000000	00:02.86	00:05.24	00:11.88	00:24.26
1000000	00:02.65	00:04.90	00:11.85	00:24.53
1000000	00:03.28	00:05.10	00:12.08	00:24.44
10000000	00:23.89
10000000	00:24.27
10000000	00:23.19
10000000	00:23.01
10000000	00:23.29

小文件10列加载V9_4核CPU_开启事务记录下载

小文件20列加载V9_4核CPU_开启事务记录下载

小文件50列加载V9_4核CPU_开启事务记录下载

小文件100列加载V9_4核CPU_开启事务记录下载

小文件200列加载V9_4核CPU_开启事务记录下载

小文件500列加载V9_4核CPU_开启事务记录下载

小文件1000列加载V9_4核CPU_开启事务记录下载

Post Views: 446

2024年5月
一	二	三	四	五	六	日
		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31

测试结果汇总

环境

集群规模

硬件

CPU

磁盘

内存

测试数据

测试记录

相关文章: