Smooth Scrolling for absolute positioned contents

I am using this Page scroller.js to give smooth scroll effect to my page. but it won't work.

Issue for this was that I was targeting the div which had position:absolute;. I still need this div to have position absolute and also have the smooth scroll work.

Here is my Html

<div id="wrapper">
<nav id="globalNav">
  <h1><a href="#"><img src="img/logo.png" alt="SHISEIDO" /></a></h1>
  <ul class="subMenu">
    <li><img src="img/seperator.png" alt=""></li>
    <li><a href="#works">TOP</a></li>
    <li><a href="#concept">CONCEPT</a></li>
    <li><a href="#contact">CONTACT</a></li>
    <li><img src="img/seperator.png" alt=""></li>
    <li><img src="img/seperator.png" alt=""></li>
    <li><p class="copyright">Copyright © KOOGEN 2015<br>All rights reserved.</p></li>
  </ul>
  <div id="fbFeed">
    <p></p>
  </div>
</nav>

<div class="col-sm-8" id="left">
  <div class="set">
  </div><!-- set -->
</div>

<div class="col-sm-4" id="right">
    <div id="inner1">
      <div id="works"></div>
      <div id="concept"></div>
      <div id="contact"></div> 
    </div><!-- inner -->
</div>


</div><!-- #wrapper -->

So, #left #right divs have position absolute to create 3 column layout. I added anchor link to my globalNav so that when I click, contents inside #right div will scroll smoothly.

heres my css added to #left and #right divs.

      #left {
    position: absolute;
    top: 0px;
    bottom: 0;
    left: 218px;
    width: 186px;
    overflow-y: scroll;
    background-color: rgba(255, 255, 255, 0.45);
    padding-left: 6px;
  }
 #left .set{
  padding-top: 230px;
 }
  #right {
    position: absolute;
    top: 0;
    bottom: 0;
    right: 0;
    overflow-y: scroll;
    width: 67.5%;
  }
 #right #works{
  padding-top: 230px;
 }
 #right .inner{
  position: relative;
 }

Any help would be much appreciated! Thank you for your time!

UPDATE: This is the fiddle. http://jsfiddle.net/hirohito/Ls3jwpm9/

It works when the window's width is very narrow, but when I stretch it out to the ideal window size, smooth scroll stop working.

you can use for smooth scroll

    $(document).on('click', 'a[href^="#"]', function(e) {
        var id = $(this).attr('href');
        var $id = $(id);
        if ($id.size() === 0) {
        return;
        }
        e.preventDefault();
        var pos = $(id).offset().top;
        pos = pos-95;   
        $('body, html').animate({scrollTop: pos});
    });