Presentation Language / Library co-evolution in Java SE 8
The big new language feature of Java SE 8 may be closures, but the real power will be in the library abstractions they enable. In this talk, we'll look at the new language features for Java SE 8 (lambda expressions, method references, and extension methods), and the upgrades that are under development in the Collections framework to take advantage of these new features. We'll explore how these language features enable the development of more powerful libraries, and how these library improvements in turn enable more expressive and less error-prone programs.
Published on: 2011-12-20T07:31:46.000Z
Channel: Devoxx'11 (all)
Tags: Lambda closures
Speakers:
Brian Goetz
Brian Goetz has been a professional software developer for more than twenty years. Brian is the author of the very successful 'Java Concurrency in Practice', and has written over 75 articles on Java development. He is one of the primary members of the Java Community Process JSR 166 Expert Group (Concurrency Utilities), and has served on numerous other JCP Expert Groups. Brian is a Sr. Staff Engineer at Sun Microsystems.
PDF: slides.pdf
Slides:
untitled
Language / Library / VM co-evolution in Java SE 8
Language / Library / VM co-evolution in Java SE 8
Brian Goetz Java Language Architect, Oracle Corporation
Thursday, November 17, 2011
Disclaimer
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle's products remains at the sole discretion of Oracle.
Thursday, November 17, 2011
Agenda
Agenda
· · · · · · Background Lambda Expressions Library Evolution Virtual Extension Methods Evolving Collections Wrap-up
Thursday, November 17, 2011
Background
BACKGROUND
Thursday, November 17, 2011
Where are we?
Where are we?
· · · Multicore hardware is now the default
· Moore's law delivering more cores, not faster cores
We must learn to write software that parallelizes gracefully All components of the Java SE platform language, libraries, and VM are co-evolving to meet the challenge
Thursday, November 17, 2011
Where are we going?
Where are we going?
· Goal: easy-to-use parallel libraries for Java
· Libraries can hide a host of complex concerns (task scheduling, thread management, load balancing)
· Goal: reduce conceptual and syntactic gap between serial and parallel expressions of the same computation
· · Right now, the serial code and the parallel code for a given computation don't look anything like each other Fork-join (added in Java SE 7) is a good start, but not enough
Thursday, November 17, 2011
It's all about the libraries
It's all about the libraries
· Most of the time, we should prefer to evolve the programming model through libraries
· · · · Time to market can evolve libraries faster than language Decentralized more library developers than language developers Risk easier to change libraries, more practical to experiment Impact language changes require coordinated changes to multiple compilers, IDEs, and other tools
· But sometimes we reach the limits of what is practical to express in libraries, and need some help from the language
Thursday, November 17, 2011
The problem: external iteration
The problem: external iteration
List students = ... double highestScore = 0.0; for (Student s : students) { if (s.gradYear == 2011) { if (s.score > highestScore) { highestScore = s.score; } } }
· · ·
Client controls iteration Inherently serial: iterate from beginning to end Not thread-safe because business logic is stateful (mutable accumulator variable)
Thursday, November 17, 2011
Internal iteration with inner classes
Internal iteration with inner classes
SomeCoolList students = ... double highestScore = students.filter(new Predicate() { public boolean op(Student s) { return s.getGradYear() == 2011; } }).map(new Mapper() { public Double extract(Student s) { return s.getScore(); } }).max();
· · · · ·
Iteration, filtering and accumulation are handled by the library Not inherently serial traversal may be done in parallel Traversal may be done lazily, so one pass rather than three Thread-safe because business logic is stateless in the client But...high barrier to use
· · Syntactically ugly Have to switch libraries
Thursday, November 17, 2011
Internal iteration with lambdas
Internal iteration with lambdas
SomeCoolList students = ... double highestScore = students.filter(Student s -> s.getGradYear() == 2011) .map(Student s -> s.getScore()) .max();
· · · · · ·
More readable More abstract Less error-prone No reliance on mutable state Easier to make parallel How are we going to get there?
Thursday, November 17, 2011
Why closures for Java?
Why closures for Java?
· Provide libraries a path to multicore
· · · · · Today, developer's primary tool for computing over aggregates is the for loop which is fundamentally serial Parallel-friendly APIs need internal iteration Internal iteration needs a concise code-as-data mechanism Closures are useful for all kinds of libraries, serial or parallel Enable a higher degree of cooperation between libraries and client code Java is the lone holdout among mainstream OO languages at this point to not have closures Adding closures to Java is no longer a radical idea
· Empower library developers
· It's about time!
· ·
Thursday, November 17, 2011
The real challenge: library evolution
The real challenge: library evolution
· Adding closures is a big language change
· · If Java had closures from day 1, our APIs would definitely look different So adding closures now makes our aging APIs show their age even more
· Most important APIs (Collections) are based on interfaces
· Can't add to interfaces without breaking source compatibility
· Adding closures to Java, but not upgrading the APIs to use them, would be silly · Therefore we also need better mechanisms for library evolution
· Lots of options here each with pros and cons
Thursday, November 17, 2011
Brief inspiration diversion
Brief inspiration diversion
Thursday, November 17, 2011
Brief inspiration diversion
Brief inspiration diversion
Thursday, November 17, 2011
Brief inspiration diversion
Brief inspiration diversion
Thursday, November 17, 2011
Brief inspiration diversion
Brief inspiration diversion
Thursday, November 17, 2011
Lambda Expressions
LAMBDA EXPRESSIONS
(AND RELATED FEATURES)
Thursday, November 17, 2011
Lambda Expressions
Lambda Expressions
· Lambda expressions are anonymous functions
· Like a method, has a typed argument list, a return type, a set of thrown exceptions, and a body
double highestScore = students.filter(Student s -> s.getGradYear() == 2011) .map(Student s -> s.getScore()) .max();
Thursday, November 17, 2011
What is the type of a lambda expression?
What is the type of a lambda expression?
· For years, we've used single-method interfaces to represent functions and callbacks
· · · Define: a functional interface is an interface with one method Functional interfaces are identified structurally The type of a lambda expression will be a functional interface
{ boolean compare(T x, T y); } { boolean accept(File x); } { void run(); } { T call(); }
interface Comparator interface FileFilter interface Runnable interface Callable
interface DirectoryStream.Filter { boolean accept(T x); } interface ActionListener { void actionPerformed(...); }
Thursday, November 17, 2011
Target typing
Target typing
· A lambda expression is a way to create an instance of a functional interface
· · Which functional interface? Infer from context! Works both in assignment and method invocation contexts · Can use casts if needed to resolve ambiguity
Comparator c = new Comparator() { public int compare(String x, String y) { return x.length() - y.length(); } };
Comparator c = (String x, String y) -> x.length() - y.length();
Thursday, November 17, 2011
Local variable capture
Local variable capture
· Lambda expressions can refer to effectively final local variables from the enclosing scope
· · Effectively final means that the variable meets the requirements for final variables (e.g., assigned once), even if not explicitly declared final This is a form of type inference
void expire(File root, long before) { ... root.listFiles(File p -> p.lastModified() <= before); ... }
Thursday, November 17, 2011
Lexical scoping
Lexical scoping
· The meaning of names are the same inside the lambda as outside
· · Including `this' refers to the enclosing object, not the lambda itself Think of `this' as a final predefined local
class SessionManager { long before = ...; void expire(File root) { ... // refers to `this.before', just like outside the lambda root.listFiles(File p -> checkExpiry(p.lastModified(), before)); } boolean checkExpiry(long time, long expiry) { ... } }
Thursday, November 17, 2011
Improved Type Inference
Improved Type Inference
· Compiler can often infer parameter types in a lambda expression
Collections.sort(ls, (String x, String y) -> x.length() - y.length());
Collections.sort(ls, (x, y) -> x.length() - y.length());
· ·
Parameter types in the lambda expression are inferred based on the target functional interface's method signature Fully statically typed no dynamic typing here · Type inference is "more typing with less typing" · First appeared in Java 5.0 generic methods · Extended in Diamond feature from Project Coin · Further extended in Project Lambda
Thursday, November 17, 2011
Method References
Method References
· Method references let us reuse a method as a lambda expression
FileFilter x = new FileFilter() { public boolean accept(File f) { return f.canRead(); } };
Thursday, November 17, 2011
Method References
Method References
· Method references let us reuse a method as a lambda expression
FileFilter x = new FileFilter() { public boolean accept(File f) { return f.canRead(); } };
FileFilter x = (File f) -> f.canRead();
Thursday, November 17, 2011
Method References
Method References
· Method references let us reuse a method as a lambda expression
FileFilter x = new FileFilter() { public boolean accept(File f) { return f.canRead(); } };
FileFilter x = (File f) -> f.canRead();
FileFilter x = File::canRead;
Thursday, November 17, 2011
Putting it all together
Putting it all together
With a little help from the libraries
· With some minor library improvements, we can make common idioms more expressive, reliable, and compact
Collections.sort(people, new Comparator() { public int compare(Person x, Person y) { return x.getLastName().compareTo(y.getLastName()); } });
Thursday, November 17, 2011
Putting it all together
Putting it all together
With a little help from the libraries
· With some minor library improvements, we can make common idioms more expressive, reliable, and compact
Collections.sort(people, new Comparator() { public int compare(Person x, Person y) { return x.getLastName().compareTo(y.getLastName()); } });
Collections.sort(people, (Person x, Person y) -> x.getLastName().compareTo(y.getLastName()) });
Thursday, November 17, 2011
Putting it all together
Putting it all together
With a little help from the libraries
· With some minor library improvements, we can make common idioms more expressive, reliable, and compact
Collections.sort(people, new Comparator() { public int compare(Person x, Person y) { return x.getLastName().compareTo(y.getLastName()); } });
Collections.sort(people, comparing(Person p -> p.getLastName()) );
Thursday, November 17, 2011
Putting it all together
Putting it all together
With a little help from the libraries
· With some minor library improvements, we can make common idioms more expressive, reliable, and compact
Collections.sort(people, new Comparator() { public int compare(Person x, Person y) { return x.getLastName().compareTo(y.getLastName()); } });
Collections.sort(people, comparing(p -> p.getLastName()));
Thursday, November 17, 2011
Putting it all together
Putting it all together
With a little help from the libraries
· With some minor library improvements, we can make common idioms more expressive, reliable, and compact
Collections.sort(people, new Comparator() { public int compare(Person x, Person y) { return x.getLastName().compareTo(y.getLastName()); } });
Collections.sort(people, comparing(Person::getLastName));
Thursday, November 17, 2011
Putting it all together
Putting it all together
With a little help from the libraries
· With some minor library improvements, we can make common idioms more expressive, reliable, and compact
Collections.sort(people, new Comparator() { public int compare(Person x, Person y) { return x.getLastName().compareTo(y.getLastName()); } });
people.sort(comparing(Person::getLastName));
Thursday, November 17, 2011
Putting it all together
Putting it all together
With a little help from the libraries
· With some minor library improvements, we can make common idioms more expressive, reliable, and compact
Collections.sort(people, new Comparator() { public int compare(Person x, Person y) { return x.getLastName().compareTo(y.getLastName()); } }); More concise More abstract More reuse More object-oriented
people.sort(comparing(Person::getLastName));
Thursday, November 17, 2011
Library Evolution
LIBRARY EVOLUTION
Thursday, November 17, 2011
Where are we trying to get to?
Where are we trying to get to?
· What we want: aggregate operations on collections
· · · New methods on Collections that allow for bulk operations Such as: filter, map, reduce, forEach, sort Which can run in parallel
int heaviestBlueBlock = blocks.filter(b -> b.getColor() == BLUE) .map(Block::getWeight) .reduce(0, Integer::max);
· How are we going to get there?
· · Can't add new methods to interfaces without modifying all implementations Can't necessarily find or control all implementations
Thursday, November 17, 2011
API evolution is a first-class problem
API evolution is a first-class problem
· Interfaces are a double-edged sword
· Once published, cannot add to them without breaking existing implementations The older an API gets, the more obvious the decay We're a victim of our own success; Java has lots of old APIs Lots of bad choices here · Let the API stagnate · Try and replace it in entirety every few years! · Nail bags on the side (e.g., Collections.sort())
· Fundamental problem: can't evolve interface-based APIs
· · ·
· Key Principle: burden of API evolution should fall to implementors, not users
· Solutions that require users to permanently cruft up their code to use new features are undesirable
Thursday, November 17, 2011
Library-based evolution options
Library-based evolution options
1. 2. Status quo: add more static methods to Collections helper class
· Not very object-oriented
Don't evolve existing collections, but add some new parallel implementation classes
· · Doesn't do anything for serial programming model Far removed from existing collections Unrealistic the Collections types permeate the library Huge design space, still stuck with VM limitations (32-bit arrays, erasure) Hurts everyone who follows best-practice style advice Code riddled with casts Ties users to implementation instead of interface contracts More casts and conversions Where does this lead to? NewSuperImprovedList3?
3. 4.
Build "Collections 2"
· · · · ·
Add aggregate operations to concrete classes
5.
Add new parent interfaces (SuperList) with aggregate ops
· ·
Thursday, November 17, 2011
Language-based evolution options
Language-based evolution options
1. Static (C#-style) extension methods
· · · · Proven, but limited Unproven, conflicts with existing modularity design Unproven Might be a nice idea if we were starting over
2. Interface versioning 3. Expanders, Classboxes 4. Traits 5. Virtual extension methods
Thursday, November 17, 2011
Static extension methods in C#
Static extension methods in C#
· C# has static extension methods
· · Makes a static method call look like an instance call Rewrites calls at compile time from list.sort(args) to Collections.sort(list, args)
· Simple to implement, no VM changes · Limitations
· · · · · · Classes don't know about their extension methods · So cannot provide a "better" implementation Not reflectively discoverable No covariant overrides Brittle if default changes, clients have to be recompiled Poor interaction with existing instance methods of same name Not very object-oriented
Thursday, November 17, 2011
Solution: virtual extension methods
Solution: virtual extension methods
· Virtual extension methods are specified in the interface · The sort method below is an extension method
· · · · From caller's perspective, an ordinary interface method Default is only used when implementation classes do not provide a body for the extension method Implementation classes can provide a better version, or not "If you cannot afford an implementation of forEach, one will be provided for you at no charge."
· List provides a default implementation
· Drawback: requires VM support
interface List { // existing methods, plus void sort(Comparator super T> cmp) default { Collections.sort(this, cmp); }; }
Thursday, November 17, 2011
Virtual Extension Methods
VIRTUAL EXTENSION METHODS
Thursday, November 17, 2011
Virtual extension methods
Virtual extension methods
· Gack, is this multiple inheritance in Java?
· · · · Yes, but Java already has multiple inheritance of types This adds multiple inheritance of behavior too But not state, which is where most of the trouble is Though can still be a source of complexity due to separate compilation and dynamic linking
· API evolution may be the primary motivator, but useful as an inheritance mechanism in itself
· Most of the way there to stateless traits...shhhh
Thursday, November 17, 2011
Method inheritance from interfaces
Method inheritance from interfaces
· Since a class can now inherit behavior from multiple supertypes, how do we choose? · Rule #1: treat inheritance of behavior from classes and interfaces separately
· · If there is an inheritance conflict between a superclass and a superinterface, the superclass takes priority Defaults only come into play if there is no declaration for that method in the superclass chain
· Rule #2: Conflicts between multiple interfaces are resolved using subtyping
· More specific interfaces take priority
· Conflicts can also be resolved explicitly · Invocation is resolved to a default if there is a unique, most-specific default-providing interface
· Otherwise, user can implement method explicitly
Thursday, November 17, 2011
Method resolution
Method resolution
Pruning less specific interfaces
· If interface B extends A, then B is more specific than A
· If both A and B provide a default, we remove A from consideration because B is more specific
· The fact that C declares Collection as an immediate supertype is irrelevant
· Set is more specific and also provides a default, so it wins over Collection
interface Collection { public Collection filter(Predicate p) default ...; } interface Set extends Collection { public Set filter(Predicate p) default ...; } class D implements Set { ... } class C extends D implements Collection { ... }
Thursday, November 17, 2011
Method resolution
Method resolution
Handling diamond-shaped inheritance · Diamonds do not pose a problem · When analyzing D, there is a unique, most specific default-providing interface A · Therefore D inherits its implementation of m() from A
· Diamonds are a problem for state inheritance, not behavior
interface A { void m() default ...; } interface B extends A { } interface C extends A { } class D implements B, C { ... }
Thursday, November 17, 2011
Method resolution
Method resolution
Explicit disambiguation · If ambiguous multiple inheritance is detected at compile-time, the programmer must provide a nonabstract method
interface A { void m() default ...; } interface B { void m() default ...; } class C implements A, B { /* required */ public void m() { A.super.m(); } }
· New syntax X.super.m() to invoke the default inherited from immediate superinterface X
Thursday, November 17, 2011
Extension methods are a VM feature
Extension methods are a VM feature
· Might be possible to implement extension methods entirely as a language feature, but this would create many downstream problems
· · Brittleness Binary compatibility
· In reality, everything else about inheritance is a VM feature
· · · Trying to implement otherwise would cause visible seams Built into behavior of invocation bytecodes Bonus: available to other JVM languages too
· VM evolving to meet the needs of the language
Thursday, November 17, 2011
Example: optional methods
Example: optional methods
· Virtual extension methods can reduce the implementation burden (and boilerplate)
interface Iterator { boolean hasNext(); T next(); void remove() default { throw new UnsupportedOperationException(); }; }
· Most implementations of Iterator don't provide a useful remove() method
· So why make the developer declare it?
Thursday, November 17, 2011
Example: retrofits
Example: retrofits
· JDK 1.0 had Enumeration; later replaced with Iterator
· · APIs that return Enumeration became second-class citizens So let's retrofit Enumeration to implement Iterator
interface Enumeration extends Iterator { boolean hasMoreElements(); E nextElement(); boolean hasNext() default { return hasMoreElements(); } E next() default { return nextElement(); } void remove() default { throw new UnsupportedOperationException(); } }
Thursday, November 17, 2011
Example: simple extensions
Example: simple extensions
interface Collection { removeAll(Predicate super E> p) default { ... }; } interface List { void sort(Comparator super E> c) default { ... }; } interface Reader { void eachLine(Block block) default { ... }; } collection.removeAll(s -> s.length() > 20); collection.removeAll(String::isEmpty); list.sort(comparing(Person::getLastName).reverse()); reader.eachLine(s -> { System.out.println(s); });
Thursday, November 17, 2011
Extension methods are virtual
Extension methods are virtual
interface List { void sort(Comparator super E> c) default Collections.sort; void parallelSort(Comparator super E> c) default Collections.sort; // can't do much better } class ArrayList implements List { public void parallelSort(Comparator super E> c) { ForkJoinUtils.parallelSort(myArray, offset, len); } }
Thursday, November 17, 2011
Compatibility goals
Compatibility goals
· The whole point of extension methods is being able to compatibly evolve APIs
· Motivated by the fact that we've got some APIs in serious need of evolution Source compatibility Binary compatibility
· Compatibility has multiple faces
· ·
· The key operation we care about is adding new methods with defaults to existing interfaces
· Without necessarily recompiling the implementation class
· Also care about adding defaults to existing methods, and changing defaults on existing extension methods
· Removals of most kinds are unlikely to be compatible
Thursday, November 17, 2011
Evolving Collections
EVOLVING COLLECTIONS
Thursday, November 17, 2011
Remember what we wanted to write?
Remember what we wanted to write?
List students = ... double highestScore = students.filter(s -> s.getGradYear() == 2011) .map(s -> s.getScore()) .reduce(0.0, Integer::max);
· Need to add new methods to Collection / List · Lots of design choices here
· · · · Eager vs lazy In-place vs create-new Serial vs parallel Where in the hierarchy to put new methods
Thursday, November 17, 2011
Attractive target: Iterable
Attractive target: Iterable
· Add streaming operations to Iterable
· · · All collections get these for free Default easily implemented in terms of iterator() Similar to Scala's Traversable, CLR's IReadOnlyList
public interface Iterable { Iterator iterator(); boolean isEmpty() void forEach(Block super T> block) Iterable filter(Predicate super T> predicate) Iterable map(Mapper super T, ? extends U> mapper) T reduce(T base, Operator reducer) Iterable sorted(Comparator super T> comp) > C into(C collection) // and more... } default default default default default default ...; ...; ...; ...; ...; ...;
default ...;
Thursday, November 17, 2011
Eager vs lazy
Eager vs lazy
· New methods are a mix of lazy and eager
· · Naturally lazy operations: filter, map, cumulate Naturally eager operations: reduce, forEach, into
· Many useful operations can be represented as pipelines of source-lazy-lazy-eager
· · · collection-filter-map-reduce array-map-sorted-foreach No new abstractions for LazyCollection, LazyList, etc
· The laziness is mostly invisible
Thursday, November 17, 2011
Attractive target: Iterable
Attractive target: Iterable
· Add streaming operations to Iterable
· · · All collections get these for free Default easily implemented in terms of iterator() Similar to Scala's Traversable, CLR's IReadOnlyList
public interface Iterable { Iterator iterator(); boolean isEmpty() void forEach(Block super T> block) Iterable filter(Predicate super T> predicate) Iterable map(Mapper super T, ? extends U> mapper) T reduce(T base, Operator reducer) Iterable sorted(Comparator super T> comp) > C into(C collection) // and more... } default default default default default default ...; ...; ...; ...; ...; ...;
default ...;
Thursday, November 17, 2011
Eager vs lazy
Eager vs lazy
· New methods are a mix of lazy and eager
· · Naturally lazy operations: filter, map, cumulate Naturally eager operations: reduce, forEach, into
· Many useful operations can be represented as pipelines of source-lazy-lazy-eager
· · · collection-filter-map-reduce array-map-sorted-foreach No new abstractions for LazyCollection, LazyList, etc
· The laziness is mostly invisible
Thursday, November 17, 2011
Going parallel
Going parallel
· A collection knows how to iterate itself
· · · By adding to Iterable, all Collections types now expose bulk data operations Without changing the Collections implementations! Individual classes can provide better implementations · Can add direct-access versions to ArrayList · Or, create new RandomAccessList as new parent of ArrayList, and put extensions there What is the parallel equivalent of Iterable?
· Can we do the same for parallel algorithms?
·
Thursday, November 17, 2011
Going parallel
Going parallel
· Many problems yield to "divide-and-conquer" recursive decomposition
· · Break down problem into subproblems, solve in parallel, and combine results Subproblems broken down recursively until small enough for solution by sequential algorithm
· JDK 7 provides general-purpose Fork/Join framework based on work stealing
· · Fork-join parallelism is powerful and efficient But, want to protect users from having to actually write forkjoin code
Thursday, November 17, 2011
Recursive decomposition
Recursive decomposition
· Recursive decomposition + work stealing offer a way to write portable performant parallel algorithms
· Can parallelize many sequential algorithms this way
Result solve(Problem p) { if (p.size() < SMALL_ENOUGH) return p.solveSequentially(); else { Problem left = p.extractLeftHalf(); Problem right = p.extractRightHalf(); IN_PARALLEL { solve(left); solve(right); } return combine(left, right); } }
Thursday, November 17, 2011
Work stealing
Work stealing
· Fork-join is implemented using work-stealing
· · · · · · · · Create a pool of worker threads Each worker thread maintains a private work queue (deque) When forking, worker push new tasks on queue When waiting or idle, pop a task off queue and execute it If worker's queue is empty, steals work off another's queue With no central coordination With little scheduling overhead With minimal synchronization costs · Queue access is almost never contended · Stealing infrequent in practice just enough stealing to distribute work around the pool
· Result: reasonable load balancing
Thursday, November 17, 2011
Parallel High Score Finder with Fork/ Join
Parallel High Score Finder with Fork/ Join
class ScoreProblem { final List students; final int size; ScoreProblem(List ls) { this.students = ls; this.size = this.students.size(); } public double solveSequentially() { double highestScore = 0.0; for (Student s : students) { if (s.gradYear == 2011) { if (s.score > highestScore) { highestScore = s.score; } } } return highestScore; } public ScoreProblem subproblem(int start, int end) { return new ScoreProblem(students.subList(start, end)); } }
ForkJoinExecutor pool = new ForkJoinPool(nThreads); ScoreFinder finder = new ScoreFinder(problem); ForkJoinExecutor pool = new ForkJoinPool(nThreads); pool.invoke(finder); ScoreFinder finder = new ScoreFinder(problem); pool.invoke(finder); class ScoreFinder extends RecursiveAction { private final ScoreProblem problem; class ScoreFinder extends RecursiveAction { double highestScore; private final ScoreProblem problem; double highestScore; protected void compute() { if (problem.size < THRESHOLD) protected void compute() { highestScore = problem.solveSequentially(); if (problem.size < THRESHOLD) else { highestScore = problem.solveSequentially(); int m = problem.size / 2; else { ScoreFinder left, right; int m==new ScoreFinder(problem.subproblem(0, m)) problem.size / 2; left ScoreFinder left, right; right = new ScoreFinder(problem.subproblem(m, problem.size)); left = new ScoreFinder(problem.subproblem(0, m)) forkJoin(left, right); right = new = Math.max(left.highestScore, highestScoreScoreFinder(problem.subproblem(m, problem.size)); forkJoin(left, right); right.highestScore); } highestScore = Math.max(left.highestScore, right.highestScore); } } } } }
Thursday, November 17, 2011
Going parallel
Going parallel
· Fork/Join relies on the client to decompose the problem
· · · · Must configure thread pool Must choose sequential vs. parallel threshold Must determine decomposition approach (2-way, n-way...) Fork/Join is effective, but is really "parallelism assembly language"
· More powerful (and object-oriented) for the collection to decompose itself! · Iterable embodies internal iteration, so let's define a mechanism to embody parallel internal iteration
Thursday, November 17, 2011
Iterable and Spliterable
Iterable and Spliterable
· The parallel equivalent of Iterable is ... Spliterable
· · · · A Spliterable can be decomposed into two smaller chunks · Small chunks can be iterated sequentially Most data structures (arrays, trees, maps) admit a natural means of subdividing themselves Default implementations work in terms of Spliterable Add parallel() method to collections to dispense a Spliterable
public interface Spliterable { Iterator iterator(); Spliterable left(); Spliterable right(); Iterable sequential(); } // plus extension methods
Thursday, November 17, 2011
Explicit but unobtrusive parallelism
Explicit but unobtrusive parallelism
List students = new ArrayList<>(...); ... double highestScore = students.parallel() .filter(s -> s.getGradYear() == 2011) .map(s -> s.getScore()) .reduce(0.0, Integer::max);
· · · ·
More readable Better abstraction No reliance on mutable state Runs in parallel
· · Implementation fuses three operations into a single parallel pass Works on any data structure that knows how to subdivide itself
Thursday, November 17, 2011
Spliterable
Spliterable
· Methods on Spliterable are analogous to Iterable
· Individual collections can always override default implementation
public interface Spliterable { Iterator iterator(); Spliterable left(); Spliterable right(); Iterable sequential(); boolean isEmpty() void forEach(Block super T> block) Spliterable filter(Predicate super T> predicate) Spliterable map(Mapper super T, ? extends U> mapper) T reduce(T base, Operator reducer) Spliterable sorted(Comparator super T> comp) > C into(C collection) // and more... } default default default default default default ...; ...; ...; ...; ...; ...;
default ...;
Thursday, November 17, 2011
Example: a simple query
Example: a simple query
Problem: Given a personal music library, get the set of albums for which at least one track is highly-rated
class Library { Set albums; Set allAlbums() { return albums; } Set favoriteAlbums() { // TODO } }
class Album { String title; List