Using two streams in Java lambda to compute covari

问题:

Let's say I have two arrays of double. I've been experimenting with Stream from Java 8. I think I've understood the main ideas but then I realised that I'm not sure how to manipulate two Streams at the same time.

For example, I want to calculate the covariance of both arrays.

public class foo {

    public static double mean(double[] xs) {
        return Arrays.stream(xs).average().getAsDouble();
}

    public static void main(String[] args) {
        double[] xs = {1, 2, 3, 4, 5, 6, 7, 8, 9};
        double[] ys = {1517.93, 1757.78, 1981.1, 2215.73, 2942.66, 3558.32, 4063.91, 4521.16, 5101.76, 5234.12};

        System.out.println("Mean of xs: " + mean(xs));
        double xs_sumDeviation = Arrays.stream(xs)
            .boxed()
            .mapToDouble(d -> d.doubleValue() - mean(xs))
            .sum();
       // Covariance
        double covXY = Arrays.stream(xs, ys)
            .mapToDouble(x,y -> {
                  double numerator = (x-mean(xs)* (y-mean(ys);
                  double denominator = Math.sqrt((x-mean(xs)* (x-mean(xs));
                  return numerator / denominator;
             })
            .sum();

    }
}

Thank you for suggestions.

Attempt 1.

public static double covariance(double[] xs, double[] ys) {
    double xmean = mean(xs);
    double ymean = mean(ys);
    double numerator = IntStream.range(0, Math.min(xs.length, ys.length))
            .parallel()
            .mapToDouble(i -> (xs[i] - xmean) * (ys[i] - ymean))
            .sum();
    double denominator = Math.sqrt(IntStream.range(0, xs.length)
            .parallel()
            .mapToDouble(i -> (xs[i] - xmean) * (xs[i] - xmean))
            .sum());
    return numerator / denominator;

回答1:

In other programming languages, there is some kind of zip function, that takes several iterables, and returns an iterator that aggregates elements from each of the iterables. See for example the function zip in the Python Library.

Although it would be possible to make a similar function in Java, it's hard to implement it in such a way, that it supports efficient parallel execution. However, there is a commonly used pattern in Java, that is a bit different. In your case, it might look as follows:

public static double covariance(double[] xs, double[] ys) {
    double xmean = mean(xs);
    double ymean = mean(ys);
    return IntStream.range(0, Math.min(xs.length, ys.length))
        .parallel()
        .mapToDouble(i -> {
                double numerator = (xs[i] - xmean) * (ys[i] - ymean);
                double denominator = ...;
                return numerator / denominator;
            })
        .sum();
}

Instead of combining two streams, you create an IntStream with all indexes, and you access the elements of the different collections by index. That works pretty well as long as the collections support random access operations.

回答2:

Other FP languages have the zip operation, which makes a stream of pairs from two streams of elements. This has not been made available in Java.

In case of arrays, however, you can easily make a stream of array indices and let your functions close over the arrays, accessing them by the index parameter.

I should also warn you that in this line

.mapToDouble(d -> d.doubleValue() - mean(xs))

you are making your program complexity O(n²) because mean is recalculated at each step. You should precalculate that and close over the result in the lambda.