南大通用GBase 8a在V95版本不同列、不同行数文件加载性能测试对比

本文针对10-1000列的表,10-1000万行数据(int),进行加载性能测试,验证加载性能,提供用户项目参考。

相同列数,不同数据库版本,开启事务影响等,请参考 8a针对宽表1000列的小文件加载不同版本配置性能测试

测试结果汇总

在笔记本虚拟机单节点4核4G内存的GBase 8a V952开始事务的环境下,常规的20-50列的表,10万行以内,也能在5秒以内完成;低于1万行的数据,即使1000列,也能在10秒内完成。

行数10列20列50列100列200列500列1000列
100.280.390.320.310.410.580.67
1000.420.330.370.320.460.670.89
10000.310.280.450.560.680.971.69
100000.390.510.751.071.573.928.63
1000000.951.112.264.308.3628.3064.61
10000002.915.0811.9625.14   
1000000023.53      

环境

笔记本虚拟机,4核4G内存。

集群规模

单节点集群

[gbase@gbase_rh7_001 ~]$ gcadmin showcluster vc vc01
CLUSTER STATE:         ACTIVE
VIRTUAL CLUSTER MODE:  NORMAL

==========================================================
|           GBASE VIRTUAL CLUSTER INFORMATION            |
==========================================================
|    VcName    | DistributionId |        comment         |
----------------------------------------------------------
|     vc01     |       1        | first vc with one node |
----------------------------------------------------------
=========================================================================================================
|                                 VIRTUAL CLUSTER DATA NODE INFORMATION                                 |
=========================================================================================================
| NodeName |                IpAddress                 | DistributionId | gnode | syncserver | DataState |
---------------------------------------------------------------------------------------------------------
|  node1   |                10.0.2.101                |       1        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------

1 data node

硬件

CPU

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 142
model name      : Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz
stepping        : 9
cpu MHz         : 2712.004
cache size      : 3072 KB
physical id     : 0

磁盘

SSD

内存

[gbase@gbase_rh7_001 ~]$ free -m
              total        used        free      shared  buff/cache   available
Mem:           3790        2816         522         139         451          79
Swap:          4095         642        3453
[gbase@gbase_rh7_001 ~]$

测试数据

生成方法,请参考 GBase 8a针对宽表1000列的小文件加载性能测试

[root@gbase_rh7_001 gbase]# ll -h 10c
total 600M
-rw-r--r-- 1 root root 540M Feb  3 16:27 10_10000000.txt
-rw-r--r-- 1 root root  54M Feb  3 14:23 10_1000000.txt
-rw-r--r-- 1 root root 5.4M Feb  3 13:59 10_100000.txt
-rw-r--r-- 1 root root 553K Feb  3 13:58 10_10000.txt
-rw-r--r-- 1 root root  56K Feb  3 13:58 10_1000.txt
-rw-r--r-- 1 root root 5.5K Feb  3 13:58 10_100.txt
-rw-r--r-- 1 root root  562 Feb  3 13:57 10_10.txt
[root@gbase_rh7_001 gbase]# ll -h 20c
total 120M
-rw-rw-r-- 1 gbase gbase 108M Feb  3 15:28 20_1000000.txt
-rw-rw-r-- 1 gbase gbase  11M Feb  3 15:17 20_100000.txt
-rw-rw-r-- 1 gbase gbase 1.1M Feb  3 15:16 20_10000.txt
-rw-rw-r-- 1 gbase gbase 111K Feb  3 15:16 20_1000.txt
-rw-rw-r-- 1 gbase gbase  12K Feb  3 15:16 20_100.txt
-rw-rw-r-- 1 gbase gbase 1.2K Feb  3 15:16 20_10.txt
[root@gbase_rh7_001 gbase]# ll -h 50c
total 300M
-rw-rw-r-- 1 gbase gbase 270M Feb  3 16:06 50_1000000.txt
-rw-rw-r-- 1 gbase gbase  27M Feb  3 15:34 50_100000.txt
-rw-rw-r-- 1 gbase gbase 2.7M Feb  3 15:30 50_10000.txt
-rw-rw-r-- 1 gbase gbase 277K Feb  3 15:29 50_1000.txt
-rw-rw-r-- 1 gbase gbase  28K Feb  3 15:29 50_100.txt
-rw-rw-r-- 1 gbase gbase 2.8K Feb  3 15:29 50_10.txt
[root@gbase_rh7_001 gbase]# ll -h 100c
total 600M
-rw-r--r-- 1 root root 540M Feb  3 16:14 100_1000000.txt
-rw-r--r-- 1 root root  54M Feb  3 14:07 100_100000.txt
-rw-r--r-- 1 root root 5.4M Feb  3 14:00 100_10000.txt
-rw-r--r-- 1 root root 553K Feb  3 13:59 100_1000.txt
-rw-r--r-- 1 root root  56K Feb  3 13:59 100_100.txt
-rw-r--r-- 1 root root 5.6K Feb  3 13:59 100_10.txt
[root@gbase_rh7_001 gbase]#

[root@gbase_rh7_001 gbase]# ll -h 200c
total 120M
-rw-r--r-- 1 root root 108M Feb  4 09:17 200_100000.txt
-rw-r--r-- 1 root root  11M Feb  4 08:57 200_10000.txt
-rw-r--r-- 1 root root 1.1M Feb  4 08:55 200_1000.txt
-rw-r--r-- 1 root root 111K Feb  4 08:54 200_100.txt
-rw-r--r-- 1 root root  12K Feb  4 08:54 200_10.txt
[root@gbase_rh7_001 gbase]# ll -h 500c
total 300M
-rw-r--r-- 1 root root 270M Feb  4 09:46 500_100000.txt
-rw-r--r-- 1 root root  27M Feb  4 09:01 500_10000.txt
-rw-r--r-- 1 root root 2.7M Feb  4 08:56 500_1000.txt
-rw-r--r-- 1 root root 277K Feb  4 08:56 500_100.txt
-rw-r--r-- 1 root root  28K Feb  4 08:56 500_10.txt
[root@gbase_rh7_001 gbase]# ll -h 1000c
total 600M
-rw-rw-r--  1 gbase gbase 5.4M Feb  2 15:01 1000.txt
-rw-rw-r--  1 gbase gbase 553K Feb  2 14:59 100.txt
-rw-rw-r--  1 gbase gbase  56K Feb  2 14:59 10.txt
-rw-rw-r--  1 gbase gbase 540M Feb  2 16:20 10w.txt
-rw-------. 1 gbase gbase  108 Dec 23 10:13 1.txt
-rw-rw-r--  1 gbase gbase  55M Feb  2 15:03 1w.txt
[root@gbase_rh7_001 gbase]#

测试记录

行数10列20列50列100列200列500列1000列
1000:00.2400:00.3700:00.3100:00.3000:00.6400:00.4800:00.50
1000:00.2900:00.2900:00.4600:00.3300:00.3700:00.4700:00.80
1000:00.3000:00.2300:00.2900:00.2900:00.3500:00.6200:00.61
1000:00.2600:00.2900:00.2600:00.3000:00.3500:00.9200:00.80
1000:00.2900:00.7600:00.2800:00.3100:00.3600:00.4000:00.62
10000:00.3200:00.3900:00.2800:00.3400:00.3500:00.4900:00.73
10000:00.3200:00.4300:00.1700:00.2800:00.4000:00.9800:00.69
10000:00.3300:00.2700:00.7700:00.3400:00.8800:00.8900:01.19
10000:00.3100:00.3100:00.3000:00.3300:00.3200:00.4800:00.67
10000:00.8300:00.2500:00.3100:00.3300:00.3400:00.5200:01.19
100000:00.2600:00.2600:00.8300:00.6100:00.4100:00.9600:01.59
100000:00.4100:00.3200:00.3800:00.7900:01.0600:00.9800:01.60
100000:00.2700:00.2800:00.3400:00.4200:00.3800:00.9800:01.59
100000:00.3000:00.2800:00.3700:00.3800:01.0200:00.9700:01.57
100000:00.3100:00.2800:00.3400:00.5900:00.5400:00.9600:02.09
1000000:00.3600:00.7400:00.7700:00.9700:01.5400:03.7400:08.90
1000000:00.3900:00.4600:00.8100:01.0900:01.5200:03.7700:08.50
1000000:00.3800:00.4600:00.7500:01.0800:01.5800:03.8700:08.56
1000000:00.4000:00.4300:00.7300:01.1000:01.5900:03.9300:08.71
1000000:00.4200:00.4700:00.7000:01.1000:01.6000:04.3100:08.49
10000000:01.0800:01.3900:02.6900:05.0900:10.1000:39.9601:26.38
10000000:00.8800:01.0200:02.3900:04.4600:07.9500:24.2100:55.95
10000000:01.0500:01.0200:02.1400:03.9500:07.1100:25.7300:57.75
10000000:00.8700:01.0800:02.0600:04.3500:08.1200:27.1001:06.35
10000000:00.8900:01.0200:02.0200:03.6700:08.5200:24.4900:56.62
100000000:02.8600:04.9800:11.9300:24.35   
100000000:02.9200:05.1900:12.0600:28.14   
100000000:02.8600:05.2400:11.8800:24.26   
100000000:02.6500:04.9000:11.8500:24.53   
100000000:03.2800:05.1000:12.0800:24.44   
1000000000:23.89      
1000000000:24.27      
1000000000:23.19      
1000000000:23.01      
1000000000:23.29