近红外光谱多核并行谱区选择任务调度策略研究

Task Scheduling Strategies of Parallel Near Infrared Spectral Region Selection on Multi Core and Its Application

摘要:

在近红外光谱的谱区选择算法中，滑动窗口偏最小二乘法具有较高的预测精度，但是其程序运行时间很长。在不改变算法预测精度的前提下，首先以分段间隔偏最小二乘法为基础研究了顺序分配法、等间距法和排序法等3种任务调度策略对并行算法性能的影响。在这3种任务调度策略中，排序法具有较好的负载均衡性和较高的并行效率。然后在16核云计算平台下将排序法分配策略分别应用于组合分段偏最小二乘法、反向分段偏最小二乘法和滑动窗口偏最小二乘法等谱区的并行选择算法中。经实验测试，在单核串行算法下反向分段偏最小二乘法和滑动窗口偏最小二乘法的程序运行时间分别为9.22h和55.51h，在排序法分配策略下采用2核并行算法时其程序运行时间分别缩短为4.98h和29.03h，分别节省了45.99%和47.70%的程序运行时间。实验结果表明：当考虑并行效率和计算成本时，在1～16核的并行算法中，选用2核并行执行以上4种谱区选择算法都具有最高的并行效率和性价比。

Abstract:

The moving window partial least squares (mwPLS) has higher prediction accuracy in the wavelength selection of near infrared spectroscopy, but the runtime of mwPLS is very long on single core serial algorithm. In order to reduce running time for wavelength selection, the task scheduling strategies on multi core were investigated. Taking interval partial least squares (iPLS) as example on the premise of not changing the prediction accuracy of the serial algorithm, the sequential distribution algorithm (SDA), equal space allocation algorithm (ESAA) and sorting allocation algorithm (SAA) were presented to study the effect of task scheduling strategy on the performance of parallel algorithm. The SAA had the best load balance and the highest parallel efficiency among the three task scheduling strategies. Then, based on the 16 cores cloud computing platform, the SAA was applied to the parallel wavelength selection algorithm of synergy interval partial least squares (siPLS), backward interval partial least squares (biPLS) and mwPLS. Comparing with single core serial algorithm, two cores parallel wavelength selection of biPLS and mwPLS with SAA reduced the runtime from 9.22h and 55.51h to 4.98h and 29.03h, and totally 45.99% and 47.70% runtime of biPLS and mwPLS were saved, respectively. The experimental results showed that when considering the parallel efficiency and computational cost condition, the two cores parallel algorithm for the four spectral region selection algorithms had the highest parallel efficiency and cost performance among the 1~16 core parallel algorithm.