Compiling selenium webdriver source - buck issues

Question

Been spending yesterday compiling the open source Appium tool, and today Selenium.

Between the projects, on Windows, I've had to install different versions of the JDK, ruby, python 2.7 (I had 3.6), git, ffmpeg, opencv4nodejs,nodejs, npm, choco and a variety of other random tools and utilities. cmake. Bundletool. Ant.

Anyway, I've gotten further than I've gotten before, but now for the last few hours I'm stuck.

go.bat clean executes fine go.bat <anything else> seems to fail, whether its used with test_java, test_htmlunit, build, test_firefox and more.

However, I've not been able to track down where in the miles of rake/buck stuff the actual error is coming from.

An example:

go.bat test_java

Output:

$ ./go.bat test_java

C:\Users\mayom\selenium>java -Xmx4096m -XX:MetaspaceSize=1024m -XX:ReservedCodeCacheSize=512m -client -jar third_party/jruby/jruby-complete.jar -X-C -S rake test_java python buckw kill python buckw build --stamp-build=detect --deep //java/client/test/org/openqa/selenium/atoms:test Starting new Buck daemon... PARSING BUCK FILES: FINISHED IN 1.7s CREATING ACTION GRAPH: FINISHED IN 0.4s BUILT 145/223 JOBS 1.2s //javascript/webdriver:key BUILT 146/223 JOBS 4.1s //java/client/src/org/openqa/selenium/remote:remote-gen-manifest BUILT 147/223 JOBS 4.8s //third_party/java/jetty:jetty-gen-manifest BUILT 148/223 JOBS 4.5s //java/client/src/org/openqa/selenium:core-gen-manifest BUILT 149/223 JOBS 1.9s //java/client/src/org/openqa/selenium/remote:remote-lib-build-info BUILT 150/223 JOBS 1.3s //java/client/src/org/openqa/selenium:core-lib-build-info BUILT 151/223 JOBS 2.7s //third_party/java/jetty:jetty-lib-build-info BUILT 181/223 JOBS 1.3s //java/client/src/org/openqa/selenium/remote:remote-lib-build-info#class-abi BUILT 185/223 JOBS 1.3s //third_party/java/jetty:jetty-lib-build-info#class-abi BUILT 189/223 JOBS 0.2s //javascript/atoms:events BUILT 197/223 JOBS 0.8s //javascript/atoms:inject BUILT 198/223 JOBS 1.4s //java/client/src/org/openqa/selenium:core-lib-build-info#class-abi BUILT 199/223 JOBS 0.8s //javascript/atoms:locators BUILT 200/223 JOBS 0.9s //javascript/atoms:html5 BUILT 201/223 JOBS 0.5s //javascript/atoms:devices BUILT 206/223 JOBS 0.4s //javascript/atoms:action BUILT 207/223 JOBS 0.2s //javascript/webdriver/atoms:atoms-lib STILL BUILDING //third_party/java/jetty:move-services (20 seconds) STILL BUILDING //javascript/atoms/fragments:execute_script (20 seconds) STILL BUILDING //javascript/webdriver/atoms:inputs (20 seconds)

C:\Users\mayom\selenium\buck-out\gen\third_party\java\jetty\move-services__srcs>md tmp && cd tmp && jar xf C:\Users\mayom\selenium\buck-out\gen\third_party\java\jetty\bundle-jars.jar && del META-INF/services/* && echo org.seleniumhq.jetty9.http.Http1FieldPreEncoder
1>META-INF\services\org.seleniumhq.jetty9.http.HttpFieldPreEncoder && jar cvf C:\Users\mayom\selenium\buck-out\gen\third_party\java\jetty\move-services\moved-bundle.jar *

Parameter format not correct - "services".

STILL BUILDING //javascript/atoms/fragments:execute_script (40 seconds) STILL BUILDING //javascript/webdriver/atoms:inputs (40 seconds) BUILT 209/223 JOBS 45.2s //javascript/atoms/fragments:execute_script BUILT 210/223 JOBS 50.8s //javascript/webdriver/atoms:inputs DOWNLOADED 0 ARTIFACTS, 0.00 BYTES, 100.0% CACHE MISS BUILDING: FINISHED IN 1m 2.4s (100%) 210/223 JOBS, 20 UPDATED BUILD FAILED Command failed with exit code 1. stderr: Parameter format not correct - "services".

When running <genrule>.
When building rule //third_party/java/jetty:move-services. go aborted! python buckw build --stamp-build=detect --deep

//java/client/test/org/openqa/selenium/atoms:test failed with exit code: 1 C:/Users/mayom/selenium/rake-tasks/buck.rb:52:in block in buck_cmd' C:/Users/mayom/selenium/rake-tasks/buck.rb:36:inbuck_cmd' C:/Users/mayom/selenium/rake-tasks/buck.rb:227:in `block in (root)' Tasks: TOP => test_java => //java/client/test/org/openqa/selenium/atoms:test:run => //java/client/test/org/openqa/selenium/atoms:test (See full trace by running task with --trace)

And if I run it with --trace:

$ ./go.bat test_java --trace

C:\Users\mayom\selenium>java -Xmx4096m -XX:MetaspaceSize=1024m -XX:ReservedCodeCacheSize=512m -client -jar third_party/jruby/jruby-complete.jar -X-C -S rake test_java --trace Parsing cpp/build.desc Parsing cpp/iedriverserver/build.desc Parsing javascript/build.desc Parsing javascript/android-atoms/build.desc Parsing javascript/atoms/build.desc Parsing javascript/atoms/fragments/build.desc Parsing javascript/chrome-driver/build.desc Parsing javascript/ie-driver/build.desc Parsing javascript/jsunit/build.desc Parsing javascript/node/build.desc Parsing javascript/remote/build.desc Parsing javascript/selenium-atoms/build.desc Parsing javascript/selenium-core/build.desc Parsing javascript/webdriver/build.desc Parsing javascript/webdriver/atoms/build.desc Parsing javascript/webdriver/atoms/fragments/build.desc Parsing javascript/webdriver/atoms/fragments/inject/build.desc Parsing javascript/webdriver/atoms/inject/build.desc Parsing py/build.desc Parsing rb/build.desc Parsing third_party/closure/build.desc Parsing third_party/firebug/build.desc Parsing third_party/jruby/build.desc Parsing third_party/js/sizzle/build.desc Parsing third_party/js/wgxpath/build.desc ** Invoke test_java (first_time) ** Invoke //java/client/test/org/openqa/selenium/atoms:test:run (first_time) ** Invoke //java/client/test/org/openqa/selenium/atoms:test (first_time) ** Execute //java/client/test/org/openqa/selenium/atoms:test python buckw kill python buckw build --stamp-build=detect --deep //java/client/test/org/openqa/selenium/atoms:test Starting new Buck daemon... PARSING BUCK FILES: FINISHED IN 1.7s CREATING ACTION GRAPH: FINISHED IN 0.5s BUILT 146/223 JOBS 3.9s //java/client/src/org/openqa/selenium/remote:remote-gen-manifest BUILT 147/223 JOBS 4.2s //java/client/src/org/openqa/selenium:core-gen-manifest BUILT 148/223 JOBS 4.4s //third_party/java/jetty:jetty-gen-manifest BUILT 155/223 JOBS 0.8s //java/client/src/org/openqa/selenium:core-lib-build-info#class-abi BUILT 179/223 JOBS 2.5s //third_party/java/jetty:jetty-lib-build-info BUILT 187/223 JOBS 2.8s //java/client/src/org/openqa/selenium/remote:remote-lib-build-info BUILT 189/223 JOBS 0.8s //java/client/test/org/openqa/selenium/atoms:execute_script BUILT 191/223 JOBS 1.4s //third_party/java/jetty:jetty-lib-build-info#class-abi BUILT 195/223 JOBS 2.7s //java/client/src/org/openqa/selenium:core-lib-build-info BUILT 206/223 JOBS 0.7s //java/client/test/org/openqa/selenium/atoms:atoms_inputs BUILT 207/223 JOBS 1.2s //java/client/src/org/openqa/selenium/remote:remote-lib-build-info#class-abi

C:\Users\mayom\selenium\buck-out\gen\third_party\java\jetty\move-services__srcs>md tmp && cd tmp && jar xf C:\Users\mayom\selenium\buck-out\gen\third_party\java\jetty\bundle-jars.jar && del META-INF/services/* && echo org.seleniumhq.jetty9.http.Http1FieldPreEncoder
1>META-INF\services\org.seleniumhq.jetty9.http.HttpFieldPreEncoder && jar cvf C:\Users\mayom\selenium\buck-out\gen\third_party\java\jetty\move-services\moved-bundle.jar *

Parameter format not correct - "services".

DOWNLOADED 0 ARTIFACTS, 0.00 BYTES, 100.0% CACHE MISS BUILDING: FINISHED IN 14.2s (100%) 212/223 JOBS, 12 UPDATED BUILD FAILED Command failed with exit code 1. stderr: Parameter format not correct - "services".

When running <genrule>.
When building rule //third_party/java/jetty:move-services. Exception `RuntimeError' at

C:/Users/mayom/selenium/rake-tasks/buck.rb:52 - python buckw build --stamp-build=detect --deep //java/client/test/org/openqa/selenium/atoms:test failed with exit code: 1 Exception #<Class:#<RuntimeError:0x1ee40b5c>>' at C:/Users/mayom/selenium/rake-tasks/buck.rb:52 - python buckw build --stamp-build=detect --deep //java/client/test/org/openqa/selenium/atoms:test failed with exit code: 1 Exception#>' at C:/Users/mayom/selenium/rake-tasks/buck.rb:52 - python buckw build --stamp-build=detect --deep //java/client/test/org/openqa/selenium/atoms:test failed with exit code: 1 Exception #<Class:#<RuntimeError:0x1ee40b5c>>' at C:/Users/mayom/selenium/rake-tasks/buck.rb:52 - python buckw build --stamp-build=detect --deep //java/client/test/org/openqa/selenium/atoms:test failed with exit code: 1 go aborted! python buckw build --stamp-build=detect --deep //java/client/test/org/openqa/selenium/atoms:test failed with exit code: 1 C:/Users/mayom/selenium/rake-tasks/buck.rb:52:inblock in buck_cmd' org/jruby/RubyIO.java:4902:in popen3' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby/open3_windows.rb:74:in popen3' C:/Users/mayom/selenium/rake-tasks/buck.rb:36:in buck_cmd' C:/Users/mayom/selenium/rake-tasks/buck.rb:227:inblock in (root)' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:240:in block in execute' org/jruby/RubyArray.java:1735:ineach' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:235:in execute' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:179:inblock in invoke_with_call_chain' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/monitor.rb:214:in mon_synchronize' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:172:in invoke_with_call_chain' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:201:in block in invoke_prerequisites' org/jruby/RubyArray.java:1735:ineach' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:199:in invoke_prerequisites' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:178:inblock in invoke_with_call_chain' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/monitor.rb:214:in mon_synchronize' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:172:in invoke_with_call_chain' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:201:in block in invoke_prerequisites' org/jruby/RubyArray.java:1735:ineach' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:199:in invoke_prerequisites' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:178:inblock in invoke_with_call_chain' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/monitor.rb:214:in mon_synchronize' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:172:in invoke_with_call_chain' uri:classloader:/gems/rake-10.5.0/lib/rake/task.rb:165:in invoke' uri:classloader:/gems/rake-10.5.0/lib/rake/application.rb:150:in invoke_task' uri:classloader:/gems/rake-10.5.0/lib/rake/application.rb:106:in block in top_level' org/jruby/RubyArray.java:1735:ineach' uri:classloader:/gems/rake-10.5.0/lib/rake/application.rb:106:in block in top_level' uri:classloader:/gems/rake-10.5.0/lib/rake/application.rb:115:in run_with_threads' uri:classloader:/gems/rake-10.5.0/lib/rake/application.rb:100:in top_level' uri:classloader:/gems/rake-10.5.0/lib/rake/application.rb:78:inblock in run' uri:classloader:/gems/rake-10.5.0/lib/rake/application.rb:176:in standard_exception_handling' uri:classloader:/gems/rake-10.5.0/lib/rake/application.rb:75:inrun' uri:classloader:/gems/rake-10.5.0/bin/rake:33:in <main>' org/jruby/RubyKernel.java:973:inload' uri:classloader:/META-INF/jruby.home/bin/rake:1:in `' Tasks: TOP => test_java => //java/client/test/org/openqa/selenium/atoms:test:run => //java/client/test/org/openqa/selenium/atoms:test

The parameter 'services'? I can't find for the life of me where that complaint is coming from. Would appreciate anyone with a more indepth knowledge of rake/buck etc than myself.