Why R calculations are inconsistent in my virtual

2019-02-27 01:24发布

问题:

I'm trying to build a new virtual machine with R and the follow packages below running as a R server to my calculations.

    #this is how I install my R-packages
    function install_packages(){
        folder='dir.create(Sys.getenv("R_LIBS_USER"), showWarnings = FALSE, recursive = TRUE)'
        packages='install.packages(c("Rserve","fArma","fGarch","tseries","MASS","lattice","gtools","gmodels","gplots","HiddenMarkov", "xts", "PerformanceAnalytics"), Sys.getenv("R_LIBS_USER"), repos = "http://cran.rstudio.com")'

        echo "$folder" >> ./install_packages.R
        echo "$packages" >> ./install_packages.R

        sudo /usr/bin/R CMD BATCH install_packages.R
        rm -f ./install_packages.R
    }

If I make a call (using mvn clean package) from my host machine to this new virtual machine, it gives me a strange error in my calculations:

Running com.company.documentengine.statistics.JensensAlphaTest
Oct 28, 2015 2:17:45 PM com.company.documentengine.toolbox.util.DatabaseConnection connectToDB
INFO: PostgreSQL JDBC Driver Registered
Oct 28, 2015 2:17:45 PM com.company.documentengine.toolbox.util.DatabaseConnection connectToDB
INFO: test Database connection confirmed for user postgres
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 17.971 sec <<< FAILURE! - in com.company.documentengine.statistics.JensensAlphaTest
testCalculate(com.company.documentengine.statistics.JensensAlphaTest)  Time elapsed: 8.821 sec  <<< FAILURE!
java.lang.AssertionError: Calculation wrong. expected:<0.039801296645998546> but was:<NaN>
    at org.junit.Assert.fail(Assert.java:88)
    at org.junit.Assert.failNotEquals(Assert.java:834)
    at org.junit.Assert.assertEquals(Assert.java:553)
    at com.company.documentengine.statistics.JensensAlphaTest.testCalculate(JensensAlphaTest.java:40)

Now, if I make the same call but from new virtual machine to my host machine (which also has all these packages installed), everything works.

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.company.documentengine.statistics.JensensAlphaTest
Oct 28, 2015 1:23:13 PM com.company.documentengine.toolbox.util.DatabaseConnection connectToDB
INFO: PostgreSQL JDBC Driver Registered
Oct 28, 2015 1:23:13 PM com.company.documentengine.toolbox.util.DatabaseConnection connectToDB
INFO: test Database connection confirmed for user postgres
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.465 sec - in com.company.documentengine.statistics.JensensAlphaTest

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21.423s
[INFO] Finished at: Wed Oct 28 13:23:20 UTC 2015
[INFO] Final Memory: 18M/362M
[INFO] ------------------------------------------------------------------------

I'm really confuse about this, can anyone please give me some suggestion/idea, please!

EDIT

I tried to debug my test to see where I'm making the mistake, but still no clue. Now I know at least that my problem is with ... look my debug comparison please. And this is the comparison to all my packages used in both cases.

Java Code

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = {TestContext.class})
@ActiveProfiles(profiles = {"test"})
public class JensensAlphaTest {

    @Autowired
    private TestSeriesManager testSeriesManager;

    @Test
    public void testCalculate() throws Exception {
        PriceSeries<PriceSeriesDatum> dax = testSeriesManager.getDax();
        PriceSeries<PriceSeriesDatum> sDax = testSeriesManager.getSDax();
        InterestRateSeries<InterestRateDatum> euribor = testSeriesManager.getEuribor();

        LocalDate asOfDate = LocalDate.of(2014, 10, 1);
        JensensAlpha jensensAlpha = new JensensAlpha(dax, sDax, euribor, asOfDate);

        double eps = 1e-15;
        /* here is the inconsistent part */
        double actualValue = jensensAlpha.calculate(Period.SINCE_INCEPTION, ReturnsType.DAILY_DISCRETE);
        double expectedValue = 0.039801296645998546;
        assertEquals("Calculation wrong.", expectedValue, actualValue, eps);
    }

}

This is the method called:

public double calculate(Period period, ReturnsType returnsType) {

NavigableMap<LocalDate, Double> returnSeries = returnsType.getReturnSeries(series);
NavigableMap<LocalDate, Double> returnBenchmark = returnsType.getReturnSeries(benchmark);
NavigableMap<LocalDate, Double> returnRiskFree = returnsType.getReturnSeries(riskFree);

LocalDate startDate = period.getStartDate(returnSeries);

NavigableMap<LocalDate, Double> cutReturnSeries = StatisticsUtils.getMapSince(startDate, returnSeries);

NavigableMap<LocalDate, Double> cutBenchmarkReturnSeries;
NavigableMap<LocalDate, Double> cutRiskFreeReturnSeries;
try {
    cutBenchmarkReturnSeries = StatisticsUtils.getMapSince(startDate, returnBenchmark);
    cutRiskFreeReturnSeries = StatisticsUtils.getMapSince(startDate, returnRiskFree);
} catch (IllegalArgumentException e) {
    throw new NotEnoughDataException(
            "This error can occur when the price series is short (only a few returns), so the benchmark is not"
                    + " updated for the taken first date of the series.", e);
}

REXPS4[] inputClasses =
        {RexpParser.createREXPS4Class(cutReturnSeries), RexpParser.createREXPS4Class(cutBenchmarkReturnSeries),
                RexpParser.createREXPS4Class(cutRiskFreeReturnSeries)};
RScript script = RScript.fromFileName("JensensAlpha.R");
REXPS4 resultClass = script.execute(inputClasses);

try {
    return resultClass.getAttribute("value").asDouble();
} catch (REXPMismatchException e) {
    throw new RScriptException("Exception while getting results from the R script.", e);
}

}

And the execute method:

@Override
    public REXPS4 execute(REXPS4[] inputClasses) {

        RConnection c = RConnectionSingleton.INSTANCE.getRConnection();

        try {

            int inputClassNumber = 1;
            for (REXPS4 inputClass : inputClasses) {

                c.assign("inputClass" + inputClassNumber, inputClass);
                inputClassNumber++;

            }

            c.eval(code);
            /* the resultClass is wrong only when I connect to my vm */ 
            return (REXPS4) c.get("resultClass", null, true);

        } catch (REngineException e) {
            throw new ScriptExecutionException("Exception while trying to execute the RScript.", e);
        }

    }

回答1:

Just to let you know what happened with my issue. The problem was the TIMEZONE. I don't know exactly why, but R or some of the packages that we use for our calculations requires the timezone to be the same.

I'm located in Germany (timezone CET which is +1 UTC) and I was setting my virtual machine to use UTC and thus the problem. Oh man, I'm really glad to solve this issue (3 days straight working on this!) but now everything is fine! A huge thank you to my colleague @Ralf for the tip!

Other issue related:1, 2, 3. I hope this helps someone! :)