Talk:How-To Improve Large String Performance Using Buffers

If the final destination for the data (BUFFER in this example) is a file, I tend to use WRITESEQ APPEND to directly send the data there. In my application it would be unusual to store a 12MB record in a hashed file..

The opposite problem is having a large string that you need to process. I convert any delimiters to @AM or @VM, then use the REMOVE trick to make stepping through the string orders of magnitude faster. E.g.

READ BUFFER FROM ..
LOOP
  REMOVE NEXT.VAL FROM BUFFER SETTING MORE
WHILE MORE DO
  GOSUB PROCESS.NEXT.VAL
REPEAT

instead of

READ BUFFER FROM ..
MAX=DCOUNT(BUFFER,@AM)
FOR I=1 TO MAX
  NEXT.VAL=BUFFER<I>
  GOSUB PROCESS.NEXT.VAL
NEXT I