Few rigidbody cause Bullet Physics slowly

I'm currently doing the job for intergrating physics engine, Bullet Physics, into my graphics engine, Before that, I implemented the easy collision system with SAP and Narrowphase algorithm, the cost of time was 3ms for SAP and Narrowphase with about 300 objects.

Because of some bugs of my algorithm, I decided to change to the real physics engine, Bullet Physics. So I followed the tutorial by official articles. When I thought I know how to implement in my graphics engine, and the output screen becomes 3 fps.

It seems to be my problem on what I understand. So I make a real simple example to reproduce the lag what I encountered.

btBroadphaseInterface* broadphase = new btDbvtBroadphase();

btDefaultCollisionConfiguration* collisionConfiguration = new btDefaultCollisionConfiguration();
btCollisionDispatcher* dispatcher = new btCollisionDispatcher(collisionConfiguration);

btSequentialImpulseConstraintSolver* solver = new btSequentialImpulseConstraintSolver;

btDiscreteDynamicsWorld* dynamicsWorld = new btDiscreteDynamicsWorld(dispatcher, broadphase, solver, collisionConfiguration);

dynamicsWorld->setGravity(btVector3(0, -10, 0));


btCollisionShape* groundShape = new btStaticPlaneShape(btVector3(0, 1, 0), 1);

btCollisionShape* fallShape = new btSphereShape(1);


btDefaultMotionState* groundMotionState = new btDefaultMotionState(btTransform(btQuaternion(0, 0, 0, 1), btVector3(0, -1, 0)));
btRigidBody::btRigidBodyConstructionInfo
    groundRigidBodyCI(0, groundMotionState, groundShape, btVector3(0, 0, 0));
btRigidBody* groundRigidBody = new btRigidBody(groundRigidBodyCI);
dynamicsWorld->addRigidBody(groundRigidBody);


btDefaultMotionState* fallMotionState =
    new btDefaultMotionState(btTransform(btQuaternion(0, 0, 0, 1), btVector3(0, 50, 0)));
btScalar mass = 1;
btVector3 fallInertia(0, 0, 0);
fallShape->calculateLocalInertia(mass, fallInertia);
btRigidBody::btRigidBodyConstructionInfo fallRigidBodyCI(mass, fallMotionState, fallShape, fallInertia);
btRigidBody* fallRigidBody = new btRigidBody(fallRigidBodyCI);
btRigidBody** fallRigidBodies = new btRigidBody*[300];

for (int i = 0; i < 300; i++)
{
    fallRigidBodies[i] = new btRigidBody(fallRigidBodyCI);
    dynamicsWorld->addRigidBody(fallRigidBodies[i]);
}


for (int i = 0; i < 1000; i++) {
    Debug::StartMeasureNumber(10); // my time measurement function & measurement id
    dynamicsWorld->stepSimulation(1 / 60.f, 10);
    Debug::EndMeasureNumber(10); // this will report the time elapsed.
    btTransform trans;
    fallRigidBody->getMotionState()->getWorldTransform(trans);

    //std::cout << "sphere height: " << trans.getOrigin().getY() << std::endl;
}

dynamicsWorld->removeRigidBody(fallRigidBody);
delete fallRigidBody->getMotionState();
delete fallRigidBody;

dynamicsWorld->removeRigidBody(groundRigidBody);
delete groundRigidBody->getMotionState();
delete groundRigidBody;


delete fallShape;

delete groundShape;


delete dynamicsWorld;
delete solver;
delete collisionConfiguration;
delete dispatcher;
delete broadphase;

In the above code, I just modified the Hello World tutorial in the last of the page. The code that produce extremelty slow on stepSimulation. The different what I did is to add 300 rigid bodies to dynamicsWorld. Also provide the debug information below.

1138ms,634ms,386ms,297ms,247ms,217ms,211ms,192ms,175ms,163ms,156ms,149ms 147ms,147ms,137ms,137ms,133ms,126ms,128ms,123ms,126ms,127ms,119ms,119ms,115ms 116ms,114ms,114ms,114ms,118ms,120ms,108ms,107ms,107ms,109ms,103ms,105ms,102ms 115ms,106ms,102ms,99ms,99ms,96ms,94ms,93ms,93ms,97ms,94ms,94ms,89ms,90ms,89ms 90ms,90ms,87ms,87ms,84ms,85ms,86ms,92ms,88ms,84ms,85ms,83ms,110ms,86ms,84ms 83ms,85ms,82ms,89ms,80ms,80ms,77ms,76ms,81ms,75ms,78ms,79ms,75ms,77ms,78ms, 76ms,78ms,79ms,75ms,77ms,74ms,74ms,73ms,72ms,78ms,72ms,71ms,72ms,73ms,73ms, 77ms,77ms,71ms,70ms,71ms,68ms,71ms,71ms,73ms,69ms,68ms,67ms,67ms,66ms,68ms 71ms,74ms,66ms,66ms,65ms,65ms,66ms,67ms,64ms,65ms,63ms,66ms,64ms,65ms,63ms 67ms,64ms,63ms,62ms,66ms,63ms,61ms,63ms,62ms,64ms,61ms,63ms,61ms,61ms,64ms 65ms,61ms,63ms,65ms,63ms,62ms,61ms,60ms,61ms,63ms,60ms,61ms,61ms,62ms,60ms, 62ms,65ms,60ms,61ms

Before 35th loop, it is extreme slow, and for the time being after that, it is going to be stable on 60 ms. But this is also slow for a graphics loop cycle to handle, So where am I understand wrong in the Hello World tutorial? I need someone to help me out :(

This question is sloved by BDL at the comments section. Just build the project to release version, and stepSimulation is run in 0ms!