Dynamic first observation: Need to put a variable

Question

I couldn't find anything on google.

I have a data set and need to iterate it in chunks, first chunk would be: firstobs = 1 obs = 30000 second chunk would be: firstobs = 30001 obs = 60000 third: firstobs = 60001 obs = 90000

and so on...

Is it possible? How do I write such thing?

Edited

I have this data, working on several million records:

 data _null_;
        set all_include_Stornos ;
        IF TREATMENT_IND = 1 AND DDS_ROW_IND NE 1 THEN DO;
        CALL EXECUTE ('%STORNO_TKUFA ('||POLICY_RK||');');
        CALL EXECUTE ('%UPDATE (STORNO_TKUFA_CUMULATE);');
        END;
        IF TREATMENT_IND in (4) AND DDS_ROW_IND NE 1  THEN DO;

            CALL EXECUTE ('%HAKPAA ('||POLICY_RK||','||POLICY_VERSION||');');
            call execute ('%UPDATE(HAKPAA_CUMULATE);');


        END;

        IF TREATMENT_IND  = 5  AND DDS_ROW_IND NE 1  and count_bitul_no <2  or (count_bitul_no >1 and max_bitul_ver = policy_verSion ) THEN DO;
            CALL EXECUTE ('%BITUL ('||POLICY_RK||','||POLICY_VERSION||');');
            CALL EXECUTE ('%UPDATE(BITUL_CUMULATE);');

        END;

        IF TREATMENT_IND = 6 AND DDS_ROW_IND NE 1  THEN DO;
        CALL EXECUTE ('%LAST_STATE ('||POLICY_RK||','||POLICY_VERSION||');'); 
        CALL EXECUTE ('%UPDATE (LAST_STATE_calc);');
        END;

        IF DDS_ROW_IND NE 1 and ((PREV_TREATMENT_IND = 4 AND TREATMENT_IND NOT IN (1,2,5)) or treatment_ind = 3) THEN DO;
            CALL EXECUTE ('%HAFSHARA ('||POLICY_RK||','||POLICY_VERSION||');');
            CALL EXECUTE ('%UPDATE (HAFSHARA_CUMULATE);');
        END;

            IF TREATMENT_IND = 2 AND POLICY_VERSION - 1 = max_bitul_ver AND DDS_ROW_IND NE 1  THEN DO;
        CALL EXECUTE ('%STORNO_BITUL ('||POLICY_RK||','||POLICY_VERSION||');');
        CALL EXECUTE ('%UPDATE (STORNO_BITUL_CUMULATE);');
        END;
RUN; 

As recommended by forum's members, because of the way Call Executed work, I need to split the data into chunks, so the program won't collapsed , as it did till now,

so I thought of a loop taking firstobs and increment it by 30K on each and every iteration.

Problem is, I couldn't get firstobs to work with a variable so far

Answer 1

Write a macro to do what you want.

%macro loop(max,by);
%do i=%eval(1+&by) %to &max %by &by;

data _null_;
set all_include_Stornos(firstobs=%eval(&i-&by) obs=&i);
...
run;
%end;
%mend;
%loop(90001,30000)

Edit: Realized I should explain how fisrtobs= and obs= work. Basically, it starts at record "firstobs" and ends at "obs". So you use a macro loop to go from 1 to 90001 by 30000. This way you only process 30k records at a time. You can always change the "by" value in the macro if you need to reduce the chunk size.