Java线程(十一)：Fork/Join-Java并行计算框架

版权声明：本文为博主原创文章，转载请注明出处http://blog.csdn.net/ghsau。

并行计算在处处都有大数据的今天已经不是一个新鲜的词汇了，现在已经有单机多核甚至多机集群并行计算，注意，这里说的是并行，而不是并发。严格的将，并行是指系统内有多个任务同时执行，而并发是指系统内有多个任务同时存在，不同的任务按时间分片的方式切换执行，由于切换的时间很短，给人的感觉好像是在同时执行。 

Java在JDK7之后加入了并行计算的框架Fork/Join，可以解决我们系统中大数据计算的性能问题。Fork/Join采用的是分治法，Fork是将一个大任务拆分成若干个子任务，子任务分别去计算，而Join是获取到子任务的计算结果，然后合并，这个是递归的过程。子任务被分配到不同的核上执行时，效率最高。伪代码如下：

<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">Result solve(Problem problem) {
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (problem is small)
directly solve problem
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">else</span> {
<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">split</span> problem <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">into</span> independent parts
fork <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">new</span> subtasks <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">to</span> solve <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">each</span> part
join all subtasks
compose <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">result</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">from</span> subresults
}
}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li></ul>

Fork/Join框架的核心类是ForkJoinPool，它能够接收一个ForkJoinTask，并得到计算结果。ForkJoinTask有两个子类，RecursiveTask（有返回值）和RecursiveAction（无返回结果），我们自己定义任务时，只需选择这两个类继承即可。类图如下： 




 

下面来看一个实例：计算一个超大数组所有元素的和。代码如下：

<code class="hljs cs has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">import java.util.Arrays;
import java.util.Random;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveTask;

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">/**
* @author: shuang.gao  Date: 2015/7/14 Time: 8:16
*/</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">class</span> SumTask extends RecursiveTask<Integer> {

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> final <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span> serialVersionUID = -<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6196480027075657316</span>L;

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> final <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> THRESHOLD = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">500000</span>;

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[] array;

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> low;

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> high;

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-title" style="box-sizing: border-box;">SumTask</span>(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[] array, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> low, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> high) {
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">this</span>.array = array;
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">this</span>.low = low;
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">this</span>.high = high;
}

@Override
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">protected</span> Integer <span class="hljs-title" style="box-sizing: border-box;">compute</span>() {
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> sum = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>;
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (high - low <= THRESHOLD) {
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 小于阈值则直接计算</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> (<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> i = low; i < high; i++) {
sum += array[i];
}
} <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">else</span> {
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 1. 一个大任务分割成两个子任务</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> mid = (low + high) >>> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>;
SumTask left = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> SumTask(array, low, mid);
SumTask right = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> SumTask(array, mid + <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, high);

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 2. 分别计算</span>
left.fork();
right.fork();

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 3. 合并结果</span>
sum = left.<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">join</span>() + right.<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">join</span>();
}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> sum;
}

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">main</span>(String[] args) throws ExecutionException, InterruptedException {
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[] array = genArray(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1000000</span>);

System.<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">out</span>.println(Arrays.toString(array));

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 1. 创建任务</span>
SumTask sumTask = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> SumTask(array, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, array.length - <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>);

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span> begin = System.currentTimeMillis();

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 2. 创建线程池</span>
ForkJoinPool forkJoinPool = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> ForkJoinPool();

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 3. 提交任务到线程池</span>
forkJoinPool.submit(sumTask);

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 4. 获取结果</span>
Integer result = sumTask.<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">get</span>();

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span> end = System.currentTimeMillis();

System.<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">out</span>.println(String.format(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"结果 %s 耗时 %sms"</span>, result, end - begin));
}

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[] <span class="hljs-title" style="box-sizing: border-box;">genArray</span>(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> size) {
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[] array = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[size];
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> (<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> i = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>; i < size; i++) {
array[i] = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> Random().nextLong();
}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> array;
}
}
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li></ul>

我们通过调整阈值（THRESHOLD），可以发现耗时是不一样的。实际应用中，如果需要分割的任务大小是固定的，可以经过测试，得到最佳阈值；如果大小不是固定的，就需要设计一个可伸缩的算法，来动态计算出阈值。如果子任务很多，效率并不一定会高。 

未完待续。。。

参考资料

http://gee.cs.oswego.edu/dl/papers/fj.pdf 
https://docs.oracle.com/javase/tutorial/essential/concurrency/forkjoin.html 
https://www.ibm.com/developerworks/cn/java/j-lo-forkjoin/ 
http://www.ibm.com/developerworks/cn/java/j-jtp11137.html

本文来自：高爽|Coder，原文地址：http://blog.csdn.net/ghsau/article/details/46287769，转载请注明。