This post is a continuation of the Swift RESAR saga. Last time I presented the design of RESAR Swift using MySQL. I would now like to present performance results for this implementation.
Swift MySQL Database Construction
The following shows MySQL database construction times for a given number of disk devices. Each disk device was partitioned into 3 disklets. A Reliability Group was also created for each disk device. Each Reliability Group consisted of 3 disklets.
stream count = 100
total database time = 0:00:12.663257
write time per stream = 0.12663257
stream count = 1,000
total database time = 0:02:03.366211
write time per stream = 0.123366211
stream count = 10,000
total database time = 0:20:43.474188
write time per stream = 0.1243474188
stream count = 100,000
total database time = 3:29:22.296089
write time per stream = 0.12562296089
stream count = 1,000,000
total database time = 1 day, 10:33:13.599063
write time per stream = 0.124393599063
So for a cloud cluster of 1 million devices, MySQL database construction time was (on average) 0.12 seconds for a single device and reliability group. It required over 34 hours to create the entire database of 1 million devices and 1 million reliability groups.
The following shows MySQL database query times for the tables.
Query Device Table by (ID)
average time = 0.00038575144
Query Device Table by (HostName, DeviceName)
average time = 0.00045933403
Query Disklet Table by (ID)
average time = 0.00028206711
Query Disklet Table by (DeviceID)
average time = 0.00031838307
Query ReliabilityGroup Table by (ID)
average time = 0.00027287826
Query ReliabilityGroupsDisklets Table by (ID)
average time = 0.000272622203333
Query ReliabilityGroupsDisklets Table by (ReliabilityGroupID)
average time = 0.00035928513
Query ReliabilityGroupsDisklets Table by (DiskletID)
average time = 0.000347662563333
Query ReliabilityGroupsDisklets Table by (DeviceID)
average time = 0.00038096357
So the minimum query time was 0.000272622203333 seconds and the maximum was
0.00045933403 seconds. We sincerely hoped that this database creation time was excessive and that the second RESAR Swift approach would greatly improve database performance.
So I have presented performance results for the implementation Swift RESAR MySQL
implementation. In my next post, I will present a better implementation. But first I’ll have to determine why the MySQL implementation is so slow. Once we have determined the performance bottlenecks, then we can work on a solution.